At one time, I had one of the CVA "Squirrel Rifles" - a short little percussion rifle in .32. I found a used one that was a left handed rifle and had a scope on it (I had to start shooting left handed due to eye problems). It was a great little rifle and perfect for squirrels and smaller varmints. It was accurate and just plain fun to shoot. I haven't hunted squirrels in a number of years so i sold it to a fellow who was left handed and wanted it in the worst way so as least someone is getting some good use of it.
I still have a .36 H & A under hammer that would work as well if I was accurate enough to make head shots (which I'm not! lol). I used to shoot once a month with a fellow that had a H & A under hammer in .32. It was amusing as he carried his shooting supplies in a musket cap tin - caps, balls, patches and used a small flask for his powder. He could outshoot all of us with it. I still haven't ruled out the idea of having a .32 barrel made up for my H & A to interchange with the .36 barrel. In the long run, I think you'd enjoy a .32 for squirrels . . . not a whole lot different than a .22 but a whole lot more fun! Good luck!
If a pair of '51 Navies were good enough for Billy Hickok, then a single Navy on my right hip is good enough for me . . . besides . . . I'm probably only half as good as he was anyways. Hiram's Rangers Badge #63