The patch serves several purposes.
- It seals the bore when using an undersized ball
- A lubricated patch keeps the fouling soft
- It allows easier loading since you use an undersized ball
- It grips the rifling (and the ball) and allows it to 'take' the rifling and spin the ball for improved accuracy
Without a patch, the ball has no lube, if bore size it will require to be pounded all the way down to the powder charge and very likely it will get stuck. If undersized, it will just rattle down the bore when fired. Accuracy will be poor and power will be be reduced due to blow by.