Seems like it should not move as you describe.
1. If you move it to the left and then move it back to the right, does the bolt click into position? (An indication that it was not quite seated i the first place which is unlikely)
2. If you cock the hammer all the way back so the pistol goes into battery, when you release your thumb pressure from the hammer does the hammer move forward more than you experience on other revolvers? An indication that the trigger sear is a tiny bit short.
I think that as long a the revolver remains in battery indexed properly it does not seem like much of a problem. But if the cylinder moves to the left with very little effort, with the implication that it would move out of battery slightly just because you move the pistol or change it's position (unlikely) it seems like you might try a new trigger.
My reading of history convinces me that most bad government results from too much government. Thomas Jefferson