Start with a .22lr
For starters, get a .22lr. You can get a nice one for the cost of 10 boxes of centerfire ammo. You can learn a lot on it before turning to a pricier package. It's basically a free gun.
Especially at 25 yards!
For bolts, CZ-455 or -452 for $400-$500, or Savage Mk II for around $200. Accuracy is similar, most find the fit and finish a little nicer on the CZ.
Semi-auto, 10/22 as suggested for around $220 on up, or Marlin 795 for $150 or so. The Ruger has more after-market parts, but there's a good selection of the relevant parts for the Marlin. Many people find the Marlin to be more accurate out of the box.
Lever action - Henry has the smoothest action. Marlin 39a the most history, possibly more collectable. I like the straight stock Mountie, personally.
Edit: aftermarket adjustable peep sights available for all of these.
"Jeez, man, what's another word for 'stupid?'"
"Tactical." -Tom Servo
Marlin 795, 795ss, 39A Mountie, CZ-455 American, CZ-452 Scout, CZ-75 Kadet, AOM160 M1 Carbine, USGI M1 Carbine, M1 Garand, Dan Wesson PM7 1911