Remember that the chart above is for right handed shooters. Reverse it side to side for us left shooters.
It sounds as if you ar not getting enough finger on the trigger and are pushing sideways. This can also be caused by pulling the trigger with all of your fingers instead of just the trigger finger. Another possible cause is not having the backstrap of the gun deep enough into the palm of your hand.
Have someone look over your shoulder as you fire and watch the fron sight. Once he has established the point at which you shift the tip of the barrel, have him start watching your hand, standing at the palm side of your shooting hand. This might pinpoint the muscle movement that is pulling the POA down and right.
Armed and Safe: Not just a theory
If it time to bury them, it is time to dig them up.
Remember, "Behind every blade of grass."