Your interpretation of our handgun law is correct, and you are well served by leaving the cap & ball at home. If however you are under the immediate supervision and control of a permit holder or LEO, you would be permitted to fire their sidearms. Enforcement, however, varies greatly from one jurisdiction to the next. Better safe than sorry.
You cannot handle or shoot a handgun in NYS, legally, unless you have a valid NYS pistol permit. I'm an NRA instructor in New York State and we cannot allow any student to handle or shoot a handgun as part of the safety training. All gun stores are supposed to ask to see your pistol permit before they can even show you a handgun (let you handle it). Your cap and ball handgun is illegal if you have the components to load and shoot it as stated above.