..I've heard mixed reviews on the 22-250 to be honest....
The .22-250 is THE long range varmint round. Any mixed reviews that you have heard are in error
Since a .223 is too loud for you the .22-250 is going to be off the menu anyways as it is louder than the .223 rem.
A .22 hornet or even a .17hmr rimfire would be nice options. There are a ton of bolts from savage and marlin in the .17hmr and they are all good and affordable.
Not too many manufacturers make a lot of models in the .22 hornet but it is a good mild mannered varmint round.