Your point is one of common discussion.
Some argue that single-shot target actions (due to the lack of a magwell cutout) are stiffer, flex less under recoil forces, and are therefore more consistent (accurate)..
Same goes for long vs. short action....short, = stiffer, less flex...
I have seen more than one discussion where a home "smith" wrecked an action by over-torquing with an action wrench in an attempt to remove a stubborn barrel and visibly bent it out of shape- and then "straightened" it back out with supposedly no ill effects...
There's no end to the hair-splitting, down to whether corncob or walnut media gives the shiniest brass (who really cares) in our hobby. Oh well, what else do we have to talk about?