Quite often a delay in the firing of a flintlock can be traced to the shooters technique, but not always. Some shooters over prime and cover the touch hole. This gives it a fuse effect as the powder has to burn through the hole into the main charge. A small priming charge placed at the outer side of the pan will give a nice flash and ignite the main charge faster generally.
A well tuned lock and proper flash hole placement above the charge height help a lot too. A nicely tuned flintlock is a joy to operate.