If the bullets in the already-loaded ammo LOOK the same (from the case mouth to the bullet's nose) then the only real concern is whetheer they are longer inside the case, where you can't see.
But, if they are longer inside the case, then they should weigh enough more than the correct bullets that you should be able to detect the problem by weighing the completed rounds.
So, I don't see any need to pull the bullets that look OK from the outside in the rounds that you have already completed.
However, I would appreciate it if you would post the brand of .40 S&W 155 grain bullets that you purchased.
If there are problems with this manufacturer, it would be helpful for us to know it so that we can be more careful with situations that would be harder to detect, like the .363" bullets in a 9mm box that hounddawg suggested. Or, .338" bullets in a box of 8mm rifle bullets.