Like many of the previous people here have said, it sounds like recoil anticipation. I don't know if you're a novice or experienced shooter, but it's nothing to be ashamed of.
Don't laugh, but I've been working on taming a 9 mm. I've shot everything from a Ruger MK II to S&W 29 with loads ranging from pea shooter to spicy habanero and had no problem. Granted, this is not all in day one. I worked my way up.
My little Keltec PF-9 is its own whole little animal. Light weight and small grip area make it tough to hold on to. Anticipating the sharp recoil, I initially adopted the strangle hold grip style, and have been trying to correct myself ever since.
.40 S&W is not that bad in the recoil department. Ms. Pax has it right, throw some dummy rounds in and see if your front sight takes a nose dive. If that's the case, you're tensing up in the wrist anticipating recoil. No more than a firm grip on the gun is required.
The best cure I can offer is spend some quality time on the range with your pistol. DO NOT go for marathon sessions! Initially, shoot no more than a couple boxes of ammo per trip. Fatigue and frustration can set in quickly.
I've taken my problem pistol and whatever .22 I felt like bringing out on many occasion. I would spend some time with the firearm I needed to work on my marksmanship issues with, then relax and plink away the rest of the afternoon. Shooting is supposed to be fun and relaxing. If you're having a hard time with one, switch to something else and come back to it later.
Keep at it, marksmanship comes with perseverance.