I can certainly see the benefit in what you are contemplating but I personally wouldn't do it. There can be benefit in variety and just the fun of it. Currently, I also have only 5 handguns but I intend to correct that before long. Now, 1 thing I do keep the same is that I don't have a manual safety on anything I carry but that still leaves a lot of options.
"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
- Benjamin Franklin