I fired an 8mm Mauser in my grandfathers basement when I was kid didn't know any better (He should have). I couldn't hear anything for several hours and my ears rang for several days.
We never wore hearing protection as kids out plinking, but I wouldn't think of shooting without protection these days. Deer hunting is the one exception.
Just remember, when you pull the trigger, the bullets come out going very, very fast. So make sure to keep the weapon pointing away from you.