Kinda the same reason track stars don't wear Timberlands in the 500 meter.
We are all human and looking for a edge. Lighter rifles make it easier to go farther and harder with less lower back pain at the end of the day.
Accuracy is equal between a mid weight and fly weight gun providing you are using a high quality rifles.. Its the shooter that makes the difference. As mentioned, you only are going to shoot 1 or 2 shots so even a ultra lightweight barrel doesn't matter with heat. I have carried ALOT of rifles over the rough stuff but setting up a fly weight mountain gun that shoots accurate is about as good as it gets. Engineering and better build materials has given us light weight rifles that can do it all. That wasn't the case even 20 years ago.
This is my "no compromise" fly weight rig. Serious glass included and still at the 6lbs weight. Shoots 1/4" moa with factory 308 ammo.