45 vs 9mm interesting test

[stagpanther]

I rest my case.

[Double Naught Spy]

So my point is just that after 100 years of these two calibers being in use most major military and police are using the 9mm today. That is anecdotal data that does not support the claim that the 45 has more stopping power than the 9mm.

You mean like how the FBI when from primarily .38 spl to 10mm to .40 cal and back to 9mm again? A buddy of mine was a firearms instructor for Dallas PD. He has said that moving to the 9mm was a great deal because people passed their qualification tests more readily. Plus, it was much cheaper to shoot 9mm than .45 acp. Now they say 9mm is as good as any other service caliber. In short, 9mm is overall one of the most versatile of calibers in terms of performance, recoil management, and costs - but it should be pointed out that they aren't talking about ball ammo.

[Pumpkin]

In a locked breech, service size pistol, why isn’t the.380 even better than the 9mm using all the measurable criteria that makes the 9mm looks so good?
If it’s because it doesn’t meet FBI requirements (to some, all that matters), I’m pretty sure one of the ammunition manufacturers could design a 380 load that would meet them or come close.

[Shadow9mm]

Originally Posted by Pumpkin

In a locked breech, service size pistol, why isn’t the.380 even better than the 9mm using all the measurable criteria that makes the 9mm looks so good?
If it’s because it doesn’t meet FBI requirements (to some, all that matters), I’m pretty sure one of the ammunition manufacturers could design a 380 load that would meet them or come close.

Your forgetting the most important part, 380 is more expensive than 9mm. While 380 might pass a specific agencies criteria, if they can get 9mm cheaper they will never pick 380.

[Pumpkin]

Was the 40S&W cheaper than the 9mm when it replaced it, no it was not.

[Webleymkv]

Originally posted by Koda94
The way I read that actually supports my position that most military and police agencies are using the 9mm today, as anecdotal evidence the 45 does not have more stopping power.

There are any number of reasons that a police or military organization might choose a given cartridge besides stopping power. Weight, cost, capacity, performance against intermediate barriers, politics, and conformity amongst allied powers all come into play.

[Pumpkin]

Agreed

[Double Naught Spy]

There are any number of reasons that a police or military organization might choose a given cartridge besides stopping power. Weight, cost, capacity, performance against intermediate barriers, politics, and conformity amongst allied powers all come into play.

Right!

And where do police and military organizations shop? Lowest bidder. Do they always buy what is best for their people? Of course not. Lethality and effectiveness are not necessarily their main goal so long as it is good enough given all the other parameters they have to weigh in order to outfit a force of people.

[Webleymkv]

Originally posted by JohnKSa
That overstates the case. Without a study we don't even know if the difference is significant. And, in fact, the data we do have suggests very strongly that it is not--as does the fact that there is no general consensus on the topic in spite of years of trying to find one.

Herin lies the crux of our disagreement: what data do we really have? The only two studies I'm aware of are Marshall/Sanow which has credibility issues and Ellifritz which is so broad that it's not useful. Without those two studies, we're down to laboratory testing which shows measurable differences but cannot predict the significance of those differences and anecdotal reports which, as I've explained, are often contradictory and unreliable. If there is other data you're aware of please share as I'd be extremely interested to see it.

Ok, then, let's look at the why. WHY is reliable data exceedingly difficult to get? WHY is it so critical to have pristine data to find a difference in outcome of that difference in outcome is significant?

Reliable data is difficult to get because most of the data we have comes from police reports. Police aren't in the data and statistics business, they're in the closing cases business. Data that might be very helpful to us like the type of bullet that's used might very well be omitted from the report if the officer doesn't deem it relevant to the facts of the case. Quite simply, good data is elusive because most of the people collecting it aren't using it for the same purposes we are.

We only need large number of data points when we're looking for very small differences. We only need pristine data when we're looking for very small differences. If the difference we're looking for is having a significant effect on outcomes, it's easy to find, it doesn't take a lot of data points and the data can be pretty noisy.

But the only credible data we have is so noisy that it's unusable. Making no distinction in the calculations between FMJ and JHP bullets is akin to a case study on the health effects of eating beef making no distinction between beef liver and Big Mac's. The problem isn't that we need copious amounts of data but rather that we have basically no useable data at all.

It's essentially a tautology--the simplest kind of reasoning. If you can't find significant effects due to caliber differences in real-world shootings then the effect due to caliber differences in real-world shootings is not significant.

I fail to see how we can draw any conclusions one way or the other given that we have basically no useful data to draw from. This is akin to saying that Julius Caesar definitely wasn't allergic to grapefruit because if he was, someone would've proved it by now. Well of course you can't prove he was, we don't know if he ever even ate grapefruit much less what reaction he might've had to it.

[Koda94]

Originally Posted by Double Naught Spy

You mean like how the FBI when from primarily .38 spl to 10mm to .40 cal and back to 9mm again? A buddy of mine was a firearms instructor for Dallas PD. He has said that moving to the 9mm was a great deal because people passed their qualification tests more readily. Plus, it was much cheaper to shoot 9mm than .45 acp. Now they say 9mm is as good as any other service caliber. In short, 9mm is overall one of the most versatile of calibers in terms of performance, recoil management, and costs - but it should be pointed out that they aren't talking about ball ammo.

If I recall correctly the old story of the FBIs quest to find the right caliber ended up showing it wasnt a power factor but a marksmanship factor, which the 9mm typically has less recoil. (There was also the issue of simply being outgunned by semi auto longarms but thats another topic. )
I'm not aware of what kind of projectile shape they consider in any service pistol, that seems to be a wash as whatever they pick is simply what they use... ie, comparing 9mm FMJ to 45 FMJ tests have still never favored either bullet.

Back to the FBIs quest, my understanding is various police agencies have experimented with different calibers too. I don't think any agency is immune to wondering if something can be better and in fact quite normal. There are also people with their own biases that are in charge at times that project their biases onto those decisions. Over the years I'm sure we all have (note: I dont work in law enforcement...) met someone who's a police officer, or was in the military, or maybe even a sniper etc. who's given me wrong gun advice or insists "his gun choice" is "best" because of his experiences. I remember the stories of soldiers complaining the 9mm was too weak when the military switched to the Beretta, when in reality it was the open top slide letting sand in desert environments was the real complaint. People often don't have the ability to separate their bias from the facts, and that happens with experienced shooters as well. The FBI went on a quest after a disastrous shootout looking to shift blame onto the caliber, and in the end learned the 9mm was just fine.

[Koda94]

Originally Posted by Webleymkv

There are any number of reasons that a police or military organization might choose a given cartridge besides stopping power. Weight, cost, capacity, performance against intermediate barriers, politics, and conformity amongst allied powers all come into play.

I'll fully admit this is my own opinion but using discernment I just cant see any agency picking a new caliber that performs less stopping power wise. Historically anytime theres been questions on a cartridges stopping power agencies have went for more power.

When any agency picks a new cartridge they will certainly factor in other things like weight, cost, capacity etc. thats no surprise at all but I just cant believe they would chose those elements over less stopping power.

[Shadow9mm]

Originally Posted by Pumpkin

Was the 40S&W cheaper than the 9mm when it replaced it, no it was not.

There are other factors to that. From what i understand a lot of law enforcement agencies went away from 9mm after both the north hollywood and the fbi in the miami incidents. Baseed on those many agencies determined that 9mm was inadequate and switched to larger or more powerful calibers, 357 sig, 40, 45.

But that brings up and interesting point though, 9mm was at one point, deemed to be inadequate for law enforcement use . It is only in the last few years with advances in bullet and gunpowder technology that it has again been deemed adequate again.

45 has always been considered adequate.

[Webleymkv]

Originally posted by Koda94
I'll fully admit this is my own opinion but using discernment I just cant see any agency picking a new caliber that performs less stopping power wise. Historically anytime theres been questions on a cartridges stopping power agencies have went for more power.

When any agency picks a new cartridge they will certainly factor in other things like weight, cost, capacity etc. thats no surprise at all but I just cant believe they would chose those elements over less stopping power.

And yet there are many cases in which they did exactly that. The .357 Magnum was at the height of its reputation as a "manstopper" in the 1980's when police agencies began abandoning it for 9mm. The so-called "Treasury Load" .38 Special +P+ was devised as a way to get around the political incorrectness of carrying .357 Magnum ammunition. The NYPD, which is the largest police agency in the country mandated FMJ and LRN ammunition for their officers until 1998 despite much more effective JHP loadings having been available for many years. The British military in the 1920's went away from .455 Webley to the anemic .38/200. The Imperial Russian Army traded in their .44 Russian revolvers for the anemic 7.62 Nagant and later the Red Army phased out their 7.62x25 Tokarevs for the substantially less powerful 9x18 Makarov.

[Koda94]

Originally Posted by Webleymkv

And yet there are many cases in which they did exactly that. The .357 Magnum was at the height of its reputation as a "manstopper" in the 1980's when police agencies began abandoning it for 9mm. The so-called "Treasury Load" .38 Special +P+ was devised as a way to get around the political incorrectness of carrying .357 Magnum ammunition. The NYPD, which is the largest police agency in the country mandated FMJ and LRN ammunition for their officers until 1998 despite much more effective JHP loadings having been available for many years. The British military in the 1920's went away from .455 Webley to the anemic .38/200. The Imperial Russian Army traded in their .44 Russian revolvers for the anemic 7.62 Nagant and later the Red Army phased out their 7.62x25 Tokarevs for the substantially less powerful 9x18 Makarov.

Im not surprised there are some entities that have made different choices, including stepping down in power. Its my understanding that the majority of police agencies in the US are using 9mm though. I dont know, but my guess is thats what the NYPD is using today?
I'm not familiar with British or Russian calibers, though Id say if the Russians have went backwards in power then Im not sayin a word... :P My impression is they dont make their choices in the best interests of their soldiers either.

A lot of this is fun discussion but not the subject though. I still haven't seen any evidence the 45 has more stopping power than the 9mm.

[Shadow9mm]

Originally Posted by Koda94

Im not surprised there are some entities that have made different choices, including stepping down in power. Its my understanding that the majority of police agencies in the US are using 9mm though. I dont know, but my guess is thats what the NYPD is using today?
I'm not familiar with British or Russian calibers, though Id say if the Russians have went backwards in power then Im not sayin a word... :P My impression is they dont make their choices in the best interests of their soldiers either.

A lot of this is fun discussion but not the subject though. I still haven't seen any evidence the 45 has more stopping power than the 9mm.

What do you consider to be stopping power, how do you define it?

[Koda94]

Originally Posted by Shadow9mm

What do you consider to be stopping power, how do you define it?

Im asking the same question. Im not the one making the claim the 45 is "better" and am just waiting for the evidence.

[Webleymkv]

NYPD began transitioning from .38 Special S&W revolvers to 9mm Glock pistols in 1993 and still issues 9mm Glocks today. IIRC their current duty load is Speer 124 gr +P Gold Dot for 9mm and, for the few officers with grandfathered revolvers, Speer 135 gr +P Gold Dot in .38 Special. As for an agency switching from 9mm to .45 ACP, the Indiana State Police switched from the Glock 17 9mm to the Sig P227 in
45 ACP in 2013. I'm not sure if they still issue the P227, however, as it is no longer in production although I do know it was still the issue pistol as late as 2019.

[44 AMP]

When any agency picks a new cartridge they will certainly factor in other things like weight, cost, capacity etc. thats no surprise at all but I just cant believe they would chose those elements over less stopping power.

Why not??

Yes, that is a serious question.

Remember that there is a significant difference between administrators making decisions based on what they believe is best for their organization as a whole, and you and I deciding on what is best for us.

History is full of examples where "superior" rounds were replaced with inferior ones, because the lesser rounds still did the job and were cheaper. Handguns don't win wars, or win battles, though they can play an important role in personal combat, they are not the soldiers primary arms, and the military doesn't consider them as such nor give them priority.

The Police do give them more priority, but the decisions of what the police use is still made by administrators, many of whom have no personal experience of shots fired in anger and who's priorities are what is "good enough" (in their opinions) and what is cheapest.

History has taught (those willing to learn) that it is NOT the effectiveness of individual rounds that wins in combat, it is firepower that wins. The Military learned this for good during WWII, and even the police have come to adopt the concept, which is why they now mostly use semi autos.

The police didn't "abandon" the .357 for the 9mm, they "abandoned" the REVOLVER for the higher capacity semi auto, because the perception was they were being "outgunned" by the criminals in terms of firepower.

On the other hand, for private citizens, it IS the effectiveness of the individual rounds that matters most. We don't operate in groups, with others for support, there's no other squad members or a partner with us, no dedicated back up on call, no arty or air strikes we can call in for support, with extremely rare exceptions, it is just us, alone, as individuals against those who would do us harm.

If you don't think that makes a difference in what we choose and why we choose it, compared to what the military and police have chosen for them, then we're not living in the same world.

[Koda94]

Originally Posted by 44 AMP

Why not??

Yes, that is a serious question.

Remember that there is a significant difference between administrators making decisions based on what they believe is best for their organization as a whole, and you and I deciding on what is best for us.

History is full of examples where "superior" rounds were replaced with inferior ones, because the lesser rounds still did the job and were cheaper. Handguns don't win wars, or win battles, though they can play an important role in personal combat, they are not the soldiers primary arms, and the military doesn't consider them as such nor give them priority.

The Police do give them more priority, but the decisions of what the police use is still made by administrators, many of whom have no personal experience of shots fired in anger and who's priorities are what is "good enough" (in their opinions) and what is cheapest.

History has taught (those willing to learn) that it is NOT the effectiveness of individual rounds that wins in combat, it is firepower that wins. The Military learned this for good during WWII, and even the police have come to adopt the concept, which is why they now mostly use semi autos.

The police didn't "abandon" the .357 for the 9mm, they "abandoned" the REVOLVER for the higher capacity semi auto, because the perception was they were being "outgunned" by the criminals in terms of firepower.

On the other hand, for private citizens, it IS the effectiveness of the individual rounds that matters most. We don't operate in groups, with others for support, there's no other squad members or a partner with us, no dedicated back up on call, no arty or air strikes we can call in for support, with extremely rare exceptions, it is just us, alone, as individuals against those who would do us harm.

If you don't think that makes a difference in what we choose and why we choose it, compared to what the military and police have chosen for them, then we're not living in the same world.

Then the lesser rounds were not inferior.
If the people, soldiers and officers, who actually use these rounds did not trust the rounds issued to them there would be protest.

If its the firepower that wins then certain the 9mm has more firepower than the 45.

For private citizens, the civilian market has traditionally followed the military or police trends. Do not 9mm sales exceed 45 these days? But certainly its not mandatory, lots of people carry larger calibers by their own choice.

It feels like the conversation is focusing too much on the 9mm and going circular. Though I do appreciate learning more of the history and opinions on how agencies choose their cartridges for issue and certainly comparing the attributes of each cartridge is appropriate.

But the subject is if the 45 is better. Still no evidence provided. Maybe we should put a definition on what "better" is for this topic?

[Webleymkv]

Originally posted by Koda,94
Then the lesser rounds were not inferior.
If the people, soldiers and officers, who actually use these rounds did not trust the rounds issued to them there would be protest.

If its the firepower that wins then certain the 9mm has more firepower than the 45.

You expressed disbelief that a police or military would intentionally pick a round with less "stopping power," to me that means round for round effectiveness on the target. If capacity is brought into the equation then we're talking about attributes other than "stopping power" factoring into the decision which is the whole point several of us have been trying to make.

For private citizens, the civilian market has traditionally followed the military or police trends. Do not 9mm sales exceed 45 these days? But certainly its not mandatory, lots of people carry larger calibers by their own choice.

Again you're conflating popularity and effectiveness. 9mm is less expensive and can be had in smaller and lighter guns than .45 both of which might influence a buyer's decision. .22LR is more popular than any other caliber worldwide, but few would argue that it's the most effective defensive cartridge.

But the subject is if the 45 is better. Still no evidence provided. Maybe we should put a definition on what "better" is for this topic?

I can't speak for everyone, but I never claimed that the .45 was better, only that we have insufficient data to make a determination.

[Koda94]

@Webleymkv
I believe we are on the same page. To clarify its the OP thats claiming its better.

My references to "stopping power" is only because I'm guessing thats what the OP means by "better". I'm trying to stick only to the differences between the only those two rounds "stopping power" (whatever that is) round vs round. Other attributes like capacity etc. certainly apply but my guess is the OP is only referring to the stopping power. So thats the only reason Ive referred to stopping power.

[zeke]

When someone says there no proof that 45 acp is more effective, better etc,; the natural response is where is the proof that it isn't?

One gripe about gelatin testers is they seem overly concerned about max diameter of the expanded bullet and associate the final dia with the max dia.

[JohnKSa]

But that brings up and interesting point though, 9mm was at one point, deemed to be inadequate for law enforcement use . It is only in the last few years with advances in bullet and gunpowder technology that it has again been deemed adequate again.

That's one take.

Another take is that the FBI, eager to come up with a reason for the Miami fiasco that didn't involve any criticism of policy and tactics blamed the whole thing on the performance of one single round. They came up with a testing protocol that justified this approach and switched calibers in response to the results of the testing protocol. They stuck with it until advances in ammunition engineering allowed them to switch back to 9mm with the blessings of their protocol.

Two questions:

1. Did they do that in spite of the fact that they had lots of real world data showing that .40S&W was better than 9mm, or because they couldn't prove superiority one way or the other?

2. Why did they even come up with the protocol in the first place when they could just look at the real world shooting data. Oh, wait, their expert told them that it would essentially be impossible to choose a winner based on real world shooting data--so they HAD to come up with some other way of picking a winner. So the real question is this: Does their testing protocol actually choose winners in the real world? Or does it only allow the FBI to justify choices made for other reasons. Like the choice made after Miami and the recent choice to move back to 9mm.

In a locked breech, service size pistol, why isn’t the.380 even better than the 9mm using all the measurable criteria that makes the 9mm looks so good?

If you look at Ellifritz's data, it suggests that you're not giving up much by choosing to use it. That said, it won't meet the FBI's pass/fail threshold and it struggles making sufficient penetration numbers to consistently reach the vitals with expanding ammo.

Another interesting data point. Many years ago, the FBI published the results of their expansion penetration testing online. It had a wide variety of loads and calibers with the test results for all of them. I was still thinking that it would be possible to name a winner based on gel testing and so I number-crunched all of them. Took every single load and turned the expansion/penetration numbers into an artificial "wound volume" and then, for fun, averaged all the numbers for each caliber to come up with an average "wound volume" score and expressed that "wound volume" as a percentage of a 180lb human. I rounded the numbers to make them easier to compare and found that all the service pistol caliber scores rounded to the same non-zero value--0.1% of the total body weight. Which was sort of an interesting result.

The .380ACP, when rounded to the same level scored a zero.

The Imperial Russian Army traded in their .44 Russian revolvers for the anemic 7.62 Nagant and later the Red Army phased out their 7.62x25 Tokarevs for the substantially less powerful 9x18 Makarov.

The fact that the Russians have, in times past, had access to a huge number of civilians that they wished to, and did, in fact, kill, often using firearms to do so, is something to think about in the context of what cartridges they use. I'm not stating a formal hypothesis or making any sort of claim. It's just thought-provoking.

Herin lies the crux of our disagreement: what data do we really have?

There is no disagreement. Let's look at what I initially said:

"As far as effectiveness in real-world gunfights goes, to date, no one has been able to show that within the general service pistol caliber performance class, there is any benefit to using one caliber over another."

I agree with you that there's no data out there that shows the real-world superiority of one caliber in the service pistol class over another one in the same class.

Reliable data is difficult to get because most of the data we have comes from police reports. Police aren't in the data and statistics business, they're in the closing cases business. Data that might be very helpful to us like the type of bullet that's used might very well be omitted from the report if the officer doesn't deem it relevant to the facts of the case. Quite simply, good data is elusive because most of the people collecting it aren't using it for the same purposes we are.

Summation: "Because it's critical that we have very good data to see the difference."

And I ask the question again. WHY does the data have to be so good if this difference is significant? Are you saying that we have to dig down to the actual type of bullet used before we expect to see a difference? So we can't just compare caliber to caliber, we have to get much finer grained than that to see the difference?

If difference due to caliber doesn't show up, why would difference due to specific loadings or bullets within caliber show up?

If this difference is really doing something practical for defenders, why is it so hard to see? And what could it possibly be doing for them if we can't even detect it?

But the only credible data we have is so noisy that it's unusable.

Again, why does the data have to be so good if this difference we're looking for is significant? If we have to get lots of clean data to show the difference, what is the practical benefit to a single defender whose shooting isn't tightly controlled?

How much real-world data do we have to throw away to get the results we want? What results do we want? How will we know if we got the results because of the data we threw away or because it's really a difference that's in the data?

I fail to see how we can draw any conclusions one way or the other given that we have basically no useful data to draw from.

Again, there is no disagreement. Here's what I said:

"Without a study we don't even know if the difference is significant."

"Until someone can show a detectable difference, we really need to consider that any of the following possibilities could be true. 1) Heavier calibers in the service pistol class create a very small disadvantage to the defender. or 2) Lighter calibers in the service pistol class create a very small disadvantage to the defender. or 3) Choices within the service pistol class do not provide any benefit or disadvantage to the defender."

The problem is that people are assuming not only that there is a difference, but that it is significant and that they know which calibers it favors.

The inescapable fact remains that no one has even been able to demonstrate that the difference is there with real-world data. Unlike your ancient grapefruit example, there is real-world shooting data to look at to answer the question. People and organizations have tried to answer the question for years, and yet there is still no answer.

So, should we assume that this difference no one can find actually exists? Sure, we can assume it exists but that it's just very difficult to detect.

Should we assume that it's a significant effect? That's definitely problematic. The idea that this difference we want to be helping us in significant ways is only detectable when using really great data and careful techniques is, to say the least, not confidence inspiring.

Should we assume which caliber(s) will be the winner if it does turn out that it's a significant effect? That makes no sense at all. We can't even detect it, let along prove it's significant--how can we call winners?

Im not the one making the claim the 45 is "better" and am just waiting for the evidence.

Well said. I've been waiting for it for decades--others much longer.

I'm not going to say there's no difference due to terminal performance differences due to caliber choice in the service pistols, but I feel very confident in saying that if there is one, that's not going to be what saves you if you get in a jam and have to use your self-defense gun to bail you out.

People who want to assume that it's making a real-world significant difference are entitled to their opinion. But they should probably at least try to come up with a solid explanation for why no one has been able to use data from real world shootings to show this real-world significant difference that they claim exists.

[Webleymkv]

Originally posted by JohnKSa
And I ask the question again. WHY does the data have to be
so good if this difference is significant? Are you saying that we have to dig down to the actual type of bullet used before we expect to see a difference? So we can't just compare caliber to caliber, we have to get much finer grained than that to see the difference?

If difference due to caliber doesn't show up, why would difference due to specific loadings or bullets within caliber show up?

If this difference is really doing something practical for defenders, why is it so hard to see? And what could it possibly be doing for them if we can't even detect it?
Quote:
But the only credible data we have is so noisy that it's unusable.
Again, why does the data have to be so good if this difference we're looking for is significant? If we have to get lots of clean data to show the difference, what is the practical benefit to a single defender whose shooting isn't tightly controlled?

How much real-world data do we have to throw away to get the results we want? What results do we want? How will we know if we got the results because of the data we threw away or because it's really a difference that's in the data?

The main problem with the Ellifritz study as it relates to this discussion is that he lumps expanding and non-expanding bullets together and in some cases, lumps completely different cartridges together. FMJ and JHP bullets in the sorts of calibers we're talking about perform so drastically different that lumping them together is likely to skew the entire data set and not give a realistic representation of either. While the Marshall/Sanow method of identifying the exact make and model of bullet would've been nice, I realize that that's probably not practical given the data Ellifritz had at his disposal. The bare minimum for his data to be useful, in my opinion, would have been to at least separate the data for expanding and non-expanding bullets so that the performance, or lack thereof, of one isn't skewing the data for the other.

The inescapable fact remains that no one has even been able to demonstrate that the difference is there with real-world data. Unlike your ancient grapefruit example, there is real-world shooting data to look at to answer the question. People and organizations have tried to answer the question for years, and yet there is still no answer.

Aside from Marshall/Sanow which has too many credibility issues and Ellifritz which lumps too many variables together, what other data do we have? As I keep repeating, I'm unaware of any other study of real world shootings besides these two. If you have another source of data please share it.

You say that no one has been able to prove a significant difference between the effectiveness of common service calibers despite years of trying. My question is who besides Marshall, Sanow, and Ellifritz has been trying by studying real world shootings? The point I'm trying to make is that there might be no discernable difference, there might be a discernable but still small difference, or there might be a drastic difference but we have no way of knowing because we have no useful data to draw conclusions from.

[JohnKSa]

The main problem with the Ellifritz study as it relates to this discussion is that he lumps expanding and non-expanding bullets together ...

One could argue that as long as he does this with all the calibers in his comparison, it should all even out. That's if the difference is actually due to caliber and not to bullet selection.

But maybe the best observation is that if the difference is too small to see if one doesn't separate the data out by bullet type, then it sounds like we've just found one of the variables that is important enough to obscure differences due to caliber. One of many, it would seem. Not a very convincing point if the goal is to prove one caliber is significantly better than another.

While the Marshall/Sanow method of identifying the exact make and model of bullet would've been nice, I realize that that's probably not practical given the data Ellifritz had at his disposal.

Why does it make sense to assume that Ellifritz only has access to data that is poorer than what M&S had? Given that he's operating decades farther into the future and during the information era where data is more available and more transportable than ever, just the opposite assumption would seem to be more reasonable. Of course one of the accusations against M&S was that they manufactured data.

The bare minimum for his data to be useful, in my opinion, would have been to at least separate the data for expanding and non-expanding bullets so that the performance, or lack thereof, of one isn't skewing the data for the other.

Get with him and see if he's willing to re-run his numbers with your suggestions.

The question is, what if doing that still doesn't show a difference? Will you be satisfied then, or will you have a reason to dismiss the new results? How will you know when you've got a valid result--when it confirms your current opinion?