Don't tap so hard!
Sorry.
Were I having that problem, after I got done cussing, I'd scribe an index mark in the bluing of the sight and the barrel. The point of a very sharp knife will work. I'd tap what I thought was "good enough" and then shoot. The index mark allows going back halfway.
When you're done, color the index mark with a black Pentel or use a Q-Tip with a dab of cold blue.