If you are shooting just for the fun of it, then what difference does the group size make?
Fast does not mean inaccurate, it just means a wider envelope of acceptability. Nor does fast mean not using the sights. With proper training, you can learn to do Mozambique drills (from the holster, 2 shots in the body, 1 in the head - 7yds) in 1.25 - 1.5 seconds and see your sights.
If you want to drive tacks, you will need a different approach.
I'm not sure if you are asking for advice or just asking if 5 - 6 inch groups are acceptable. In a perfect world you should be able to do a "Bill drill" in under 2 seconds. That is an IPSC target at 7 yards, starting with hands at sides, draw and fire 6 rounds in less than 2 seconds. They all must be "A" zone hits (the "A" zone is about 7x11" iirc) to count. If you are shooting your groups that fast (or near that fast), then yes they are acceptable.