As above, friction wears gun finishes, and ANY holster provides enough friction that the finish will wear off.
Every time you put the gun in the holster the friction abrades the finish.
As for wiping the gun down with cold blue, keep in mind that cold blue can actually cause rust, and often leaves a permanent blotch in the original bluing. It also tends to remove the original bluing if used often enough.
Cold blue is best used as a touch-up on small scratches or worn areas, not as a regular treatment.
Also as above, if you don't want the finish to wear, never put the gun in a holster. In fact, it's best to not even handle the gun.
Guns wear if handled or holstered, and there's no way around it.
Regularly using cold blue is not good.