The '06 can be a kicker, depending on how its stocked, the fit, and the weight of the rifle and your bullet weight choice. We have (my sis actually) a S$W 1500 in the family that is downright unpleasant with 180's. And I have a CZ Mannlicher that, even hand loaded down with 180's and 200's which it seems to like best, will get your undivided attention.
I'd encourage you stay with 150;s, they will have less recoil, ol' man Newton ya'know. I've had good experience w/ cutting recoil on kickers by getting a premium pad installed like the "Kick-eeze". One on the CZ tamed it down to be tolerable, and another on my short barreled 12ga 3" mag turkey gun helped it as well. Any of the space rubber pads should help. A final option may be to find some of the reduced recoil 'o6 loads commecially, or have a trusted friend work up a reduced load for your rifle. I've done that several times for folks w/ '06 and .270 as well, for beginners.
Do a lot of dryfiring, and limit your bench time. Shooting from field positions seems to reduce the kick, and will make you a better overall shot in the long run.