The .204 cases, even if ejected as fast as you can manage, are in the chamber longer and transfer more heat to the chamber/barrel.
The .223 cases, on the other hand, are violently expelled from the AR just milliseconds after the bullet leaves the barrel. There is no time for the chamber to soak up the heat in the brass case.
Arguments could be made about different powders acting differently, as well; but the actual affect would be trivial.
"Such is the strange way that man works -- first he virtually destroys a species and then does everything in his power to restore it."