I don't know that there is ever "too much" as far as terminal ballistics - on target, etc. May be that the & mag is "too much" for what you intend to do. (& as an aside, kudos for "no brush-busting" - a misnomer in my mind).
Far as that goes, I wouldn't choose a 7 mag for what you stated, just because it isn't "needed" for what you intend.
.41/.44 mag (or equivalent) in the pistol class to the 30/30 to 7-08 (class) would do you just as well as the 7 mag. Anything in between and +&- a hair will serve you just fine.
'Course, your opening statement "just for the heck of it" really does say it all, huh?
That's enough IMHO, but, you can do "it all" with less (but only if you want to
).
BTW, the 7 mag kicks a bit more than the 30-06 in my experience - YMMV. & I pretty much did away with my 30-06 & started using a Rem M7 .308 just because I didn't "need" the
30-06 power for elk in the same scenario you presented for deer.