Reliable and fast ignition from a flintlock has more to do with through cleaning and loading than it does with the priming powder. If fffg is working for you then you are doing a good job. IMHO after shooting many rounds through flintlocks ffffg makes a difference. The ignition time is noticeably quicker.
In general I agree with the comment that it is not uncommon that old timers used the same powder as in the barrel. Looking through “The Kentucky Rifle Hunting Pouch” by Madison Grant most of the flintlock rifle pouches only have one horn. But some have 2, the same book has a separate chapter about priming horns. This leads me to believe that some and likely many hunting rifles used priming powder.
Take care and best regards,