I've got four of the durned things, in .223, .30-30, .308, and .45-70. The .223 and the .308 are Ultra's the other two are skinny barrel rifles. They all shoot where they're looking, although I've had problems finding a good cast-bullet load for the .30-30.
The Handi is problematic getting the best accuracy off the bench, because of the way that the forend connects to the barrel. Lots of us put an 0-ring over the stud to float the barrel and that seems to tighten groups somewhat. When benching the rifle, be sure to maintain the same contact spot on the front bag. My trick is to slide the rifle forward until the front bag contacts the trigger guard. This puts the contact point at about the hinge and seems to help in load development. Once the load is developed, get away from the bench and shoot the rifle as usual.