You cannot usually pick the fight you are going to be in. The fight is going to be what the fight is going to be. If I were up against BG's toting single action six guns I would be fine with one, however, this aint Dodge city.
As romantic and quaint as it may be you will be at a serious disadvantage if you ever find your self in a gun fight so armed.
You may have seen a non trained amateur attempting to reload a double action revolver, never confuse that with what a good shooter can accomplish. There are also plenty of guys around that shoot J frames pretty good.
My rifle and pistol are tools, I am the weapon.