I was under the impression that the first cold/clean/dry bore shot is really the most important, being that's what it will be on the first shot of a hunt. ? Made sense when it was suggested
Ultimately, up to you. My gunsmith agrees with the fouled bore theory as well. He does not clean the barrel until the accuracy starts to be affected. He will then clean it, and go shoot 2-3 rounds to get it fouled and shooting where it should again.
Same could apply to hunting. Zero your rifle, clean, shoot 2 shots, go hunting. Clean after you get your freezer filled, or the season is over.