If you feel that strongly that it wouldn't matter, why not just take yours and loosen it up?
Why would I do anything? What's to be achieved?
If my analysis of how the kit locks up and gets it's rigidity is correct, the type of change you suggest wouldn't LOOSEN IT UP or otherwise affect accuracy. It would just be removing metal for no reason...
My current Kadet Kit was not my first Kadet Kit. An earlier Kadet Kit (bought from the same local deal) would NOT drop in that same gun.
We didn't know, back then -- late 1990's -- that some fitting might be required; the materials that came with the kit then didn't address fitting, as is now the case. (The dealer had their gunsmith look at it, and he didn't know what to do.)
The dealer sent the Kit back to CZ and got another; that is the one I still use today. The only change I've made to it was to install the newer design firing pin, which CZ sent, free of charge.
The replacement unit dropped right in, and with the kit installed and the slide stop in place, it seems to be a very tightly-fitted assembly. You can't move things around as you might expect. The slide stop and the front of the frame seems to holds thing pretty securely in place. (In fact, if there was any slop in the barrel positioning, I think we'd see some feeding issues, or shaving of bullets, and that doesn't seem to happen.)
Your analysis would suggest that all "loose" assemblies -- like mine -- should have material added to the lugs to improve accuracy. Have you heard of anyone doing that? I haven't. Perhaps that would help, but it generally doesn't seen necessary. And, if my analysis is correct, it would be a waste of effort.
Perhaps I'm wrong -- but somebody will have to use more than theoretical arguments to make/prove their point.
My experience would suggest that there is a good bit of variance in the dimensions of the barrel lugs of the Kadet Kits, which probably aren't made on CNC equipment like the CZ frames.