You did not indicate what type of pistol you are shooting. The pistol grip is important. Most of the molded grips with finger cutouts create problems if your hands are not in the 25 percentile. I find them to be a hindrance as they do not fit my hand. I have fat short fingers and large palms.
Develop a rhythm in your shooting. Do everything in the same order until it comes second nature. Trust your grip and muscle memory. As long as you keep the sites aligned you are going to hit the target. Lon Wiggers used to warm up by firing a 22 at 25 yds and never look at the target and never get out of the 9 ring.
Speed comes with confidence in yourself and your equipment. As you learn your pistol you will find that you do not need the sites out to say 20 yards.
My pistol instructors SSG Schongert and later SFC Reed both beat the following into my head. Shooting is 90% mental and 10% muscle.