Cobra. What you're doing is perfectly OK. Most, if not all of my light load shooting is with cast bullets in 30-30, .308 Win. and 30-06. Bullet weights run from 115 gr. to 200 gr. and loads are usually with one of the 2400s, 4227, and one of the 4895s. A 190 gr. (Lyman #311644) gas checked bullet with 17.0 gr. of 2400 will hit hard enough at 300 meters to knock down pig silhouettes. Have to raise the sights way up though. A neat gallery load is 5.0 gr. of Unique in either the 30-30. .308 or 30-06. A quarter square of toilet paper tamped against the powder to hold it in place is about right. Great for tin can plinking or small game to maybe 50 yards. Sounds like a kids cap gun and recoil is, for all practical purposes, non-existant.
My all time favorite loads thoght are Lyman's #311644 with 25.0 gr. of IMR or H-4895 in the .308 Win. and 27.0 gr. in the 30-06. I haven't done much with the 06 lately, but that .308 load in my rifles have given 1.5 MOA at 200 yards and 2.5 MOA at 300 yards. I see no reason why a 30-06 wouldn't be capable of doing the same with minor load adjustment.
Also, you can get some Winchester bulk 150 gr. Power points and use 25.0 gr. of one of the 4895s. My wife likes that load in her .308 for practice. Just enough recoil so that you know you're shooting, but not enough to cause pain. If you try the cast bullets, be sure they are sized to .310".
COMPROMISE IS NOT AN OPTION!