How to map rotate, pan, zoom to manipulate controls

Question

I am wondering if there is a way to restrict the motion of the rotate mouse cursor in when moving around a 3d plot.

For example, I want the y direction of the cursor to correspond to pulling the z axis toward or away from me (the camera), and the x direction to rotate the graphic around the z axis.

I can do something similar using,

Manipulate[
 Graphics3D[{Sphere[{1, 0, 0}, 0.05], Sphere[{0, 0, 0}, 0.2],
 {Arrow[{{0, 0, 0}, {1.1, 0, 0}}], Text["x", {1.2, 0, 0}], 
  Arrow[{{0, 0, 0}, {0, 1.1, 0}}], Text["y", {0, 1.2, 0}],
  Arrow[{{0, 0, 0}, {0, 0, 1.1}}], Text["z", {0, 0, 1.2}]}},
 ViewVector -> {40 {Cos[a] Sin[b], Sin[a] Sin[b], Cos[b]}, {0, 0, 0}},
 ViewAngle -> 4 Degree ,
SphericalRegion -> True,
Boxed -> False
],
{{a, 0.3}, -2 \[Pi], 2 \[Pi]},
{{b, 1.347}, -2 \[Pi], 2 \[Pi]}
]

where sliding $a$ rotates the graphic around the z axis (or rather the viewpoint) and sliding $b$ pulls the z axis in or out.

Edit to make description clearer

The 3 dimensional ViewVector seems to be changed somehow by the 2 dimensional "double arrow Rotation" cursor when clicking and dragging on the plot.

I am not even sure how this is possible, but is it possible to restrict it so that only 2 of the 3 Euler angles (e.g. rotation about the z axis and inclination of the x-y plane) are changed?

The third Euler angle is natively controlled by the "additional rotation" cursor, which is available when you put the cursor in the corner.