It doesn't seem like a .25-06 is the cartridge you really need. It's neither a great target round nor a great woods cartridge. Maybe you should look into another rifle that offers the kind of cartridge you really want to shoot/hunt with?
You already have good hunting cartridges, so why not look into a dedicated varmint or target round and a rifle that will complement it?
If I can't be with the rifle I love, I love the rifle I'm with!!