Something that can help speed this up is sticking with the mininum number of weapons groups as possible.
Just Sig autos, or Glock autos or Only S&W revolvers or only Ruger revolvers.
That way the motions you go through are the same and you don't fumble, try to flick a safety or forget to, drop a hammer when you meant to release a slide, etc.
Personally, Glock handguns, AR based rifles and Benelli shotguns are what I focus on.
It is actually kind of funny, when I pick up someone elses gun I will do things and people will ask. "What are you doing, dork?" because I am going through my "standard" motions for that particular platform before I realize it is a BPS shotgun or a revolver instead of a Glock because I start to do them automatically.