Before spending any money, attend some matches near you as a spectator. Bring hearing and eye protection. Talk to the competitors and officials and tell them you are interested in getting into the sport. They will be happy to give advice and you can see what equipment they are using. It will help you make a more informed decision on what class or classes you want to compete in and what equipment to get.
If you have a range nearby that rents guns, try some different ones to get a feel for what you like.
I shot my first USPSA match with a ported EAA Witness in 9mm I already owned, a cheap holster and some borrowed mag pouches. The porting put me in Open class and the 9mm made for minor scoring, but nobody had more fun than me and I learned a lot. No, I didn't win the match, but neither was I dead last.