You are going to hear this sooner or later, so why not sooner. The recoil from a .357 will be much greater than a .38 spc of same barrel length. You can overcome that, for the most part, by practicing with .38s in your .357 and loading it with the magnums for carry.
But one more thing...the .357, in addition to added recoil, has a lot of muzzle flip (understatement), and noise, and flash, compared to the .38. Something to consider if your plan is to defend yourself in close quarters. Someone mentioned on this forum discharging the magnum inside a car without ear protection at night to get a feel for what he could expect under actual conditions. I'm not saying I recommend you do that, but it does make you consider the subject more thoroughly. Btw, I believe the guy who did that claimed to be essentially deaf and blind for several minutes afterwards. It always comes down to the same question doesnt it?
"What do you want it for??"