Apples, oranges, pear, grapefruit, or whatever, they still do more damage as a result of the increased energy they are able to transfer effectively to the target.
Besides, you don't have to get to 2500 to 3500 fps to see a difference and some common handguns exceed the 1300 fps you mentioned. Case in point, 357 magnum vs 45 ACP. If the difference in velocity and thus difference in energy didn't matter then the 45 would be a much more potent cartridge. Does anyone honestly believe it is? I can tell you this, a 45 Colt and 454 Casull shoot the same diameter bullet and neither shoots 2500 fps but the difference on game is like night and day. With the Colt, I can shoot cottontails and eat right up to the bullet hole. With the Casull, there isn't much left of a jackrabbit. A 45-70 at 1700-1800 fps with a soft nosed bullet will leave a huge exit wound on a hog. There are plenty of loads for the 357 magnum, 357 Sig, 10mm, and a few others that will do 1500+ fps from a full size handgun. These handguns also aren't shooting a little 22 caliber bullet.
Here is another point. Lets compare 380 and 9mm. Because of it's increased energy, the 9mm's bullets can be made to expand more and still penetrate to a sufficient depth. That is benefit of it's increased energy.
The bottom line is that energy is the potential to do work. In this case the work is damage to the target. As I said earlier, IF you can transfer that energy effectively to the target then the round with more energy will be more effective. That point is not even debatable. What is debatable is which loads in which calibers in which guns actually transfer their energy effectively. I have already said that all else being equal, I would bet on the larger bullet transferring it's energy more effectively but all else is not always equal.
Here is another example that may help make my point a little clearer. A baseball will have more energy than a fillet knife and it will transfer 100% of it to whatever it hits but the filet knife will transfer it's energy much more effectively. Effective, as I am using it doesn't simply mean energy transferred to the target but energy transferred to the target as damage.
"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
- Benjamin Franklin