While shooter skill is obviously important, the best shooters want the best tools. (If you have to dig a ditch, a backhoe beats a teaspoon every time!)
The traditional single action (not a DA revolver fired SA) has several drawbacks as a target revolver. First is the long hammer fall. Not only does it allow more time for the gun to drift off target, but the heavy hammer makes the striking impact higher, jarring the gun off target.
The second major concern is trigger pull. The traditional SA (though not the new Rugers) has three hammer notches, and the trigger pull has to be heavy enough that the trigger/sear will clear both the "safety" and half-cock (loading) notches as the hammer falls. That means that the very light trigger pulls used for target work are unobtainable with a traditional SA lockwork.
Other drawbacks, such as grip shape, which affects recovery in rapid fire, and (often) poor sights, can be overcome, though not always easily.