NYS gun laws can be/are far too restrictive and complicated. I started my career as an LEO before my 21st birthday (at that time, minimum age was 18, don't know if that has changed). Small town, part-time Officers only. Full peace/police authority on duty and off. MPTC qualified, annual re-qual at the PD's range, etc. All officers had to provide their own sidearms, meeting the Dept. criteria.
Although I was a "Full-Fledged Sworn Officer of the Law", I had to have my uncle purchase my sidearm for me, because an FFL holder could not sell handguns to LEO and CCW holders under the age of 21. This, I believe, remains the same, although permit applicants must now be 21 years or older.
Your interpretation of our handgun law is correct, and you are well served by leaving the cap & ball at home. If however you are under the immediate supervision and control of a permit holder or LEO, you would be permitted to fire their sidearms. Enforcement, however, varies greatly from one jurisdiction to the next. Better safe than sorry.