Chronograph numbers

[Aguila Blanca]

Originally Posted by GeauxTide

Have used 10 for 40 years. 5 shot strings are not statistically significant for SD.

Thank you.

[74A95]

What is the issue with statistically significant SD?

https://en.wikipedia.org/wiki/Standard_deviation

In science, it is common to report both the standard deviation of the data (as a summary statistic) and the standard error of the estimate (as a measure of potential error in the findings). By convention, only effects more than two standard errors away from a null expectation are considered "statistically significant", a safeguard against spurious conclusion that are really due to random sampling error.

Could you clarify what/why a statistically significant SD means, and why it matters? Thanks.

[Aguila Blanca]

Originally Posted by 74A95

Could you clarify what/why a statistically significant SD means, and why it matters? Thanks.

Nope, I can't.

I took one class in statistics, well over 50 years ago, and never encountered the term again until I bought a chronograph and found that Standard Deviation is one of the numbers that chronographs spit out. I don't know just what it means or what value it has to most shooters, but I reckon the chronograph makers wouldn't put SD in there if some people didn't find that data point to be useful.

My concern is that we have devices that report a number for SD. For those who care about that number, it would then make sense to want to use a large enough sample for the number to be statistically significant. I don't know what that sample size is, so I hoped someone who knows more about statistics than I could tell me.

Let me offer an example:

Take a 5-shot string. Values are 100, 98, 99, 97, and 80.

Average is 94.8. Extreme spread is 20. SD is 8.35.


Now take a 10-shot string. Values are 100, 97, 98, 97, 99, 98, 99, 97, 99, and 80.

No value in either string is below 97, except for the last, which would be the outlier.

In the second string, the average is higher: 96.4. This is to be expected, since the sample size has added five additional values that are all higher than the average for the first group. In both groups, the extreme spread is the same: 20.

But for the second group, the SD is 5.85. That's a big difference from 8.35. So, it's clear that the sample size affects the result, but at what point does enough become enough? If a sample of 10 is better than a sample of 5, then a sample of 100 is probably better than a sample of 10. But where does one draw the line and say, "This sample is large enough for the SD value to mean something"?

[WESHOOT2]

Fair.


During development I used at least twenty rds fired to determine if the load was worth pursuing.
Before offering the load I would normally use a MINIMUM of 100 rds fired to increase the data.
I often used multiple launch platforms during the testing.

But, as always, I used the stated criteria for SD relevance.

[74A95]

Originally Posted by Aguila Blanca

But for the second group, the SD is 5.85. That's a big difference from 8.35. So, it's clear that the sample size affects the result, but at what point does enough become enough? If a sample of 10 is better than a sample of 5, then a sample of 100 is probably better than a sample of 10. But where does one draw the line and say, "This sample is large enough for the SD value to mean something"?

I think you're trying to apply a meaning to the SD which does not exist.

from the previous link;

" . . . the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range."

You don't pick a sample size to best reflect or to reach a certain SD because a certain SD is not the goal. The SD simply is a calculated value that tells you something about your data - " . . . the amount of variation or dispersion of a set of values". That's it. Nothing more.

[scatterbrain]

For what ever reason, the lower SD my offered loads have, the tighter the group size and the fewer "flyers" I have. The SD may have had nothing to do with it.

[Aguila Blanca]

Originally Posted by 74A95

I think you're trying to apply a meaning to the SD which does not exist.

No, I'm not.

Originally Posted by 74A95

You don't pick a sample size to best reflect or to reach a certain SD because a certain SD is not the goal. The SD simply is a calculated value that tells you something about your data - " . . . the amount of variation or dispersion of a set of values". That's it. Nothing more.

I never even hinted at trying to pick a sample size to attain a certain SD. I asked what minimum sample size is necessary for the SD to be statistically significant (or "valid," or "useful").

To attempt stating the utility of the SD to shooters in everyday language (which is the only language I know when we get into probability and statistics), consider the data we can collect from a report on a batch of commercial ammo. Someone test fires a certain number number of rounds, and they throw some numbers at us. What do those numbers tell us?

Maximum and minimum velocity are self-explanatory, but if they are moderately far apart we want to know more about what happens in between.

Average: I hope we all know how the average is calculated, but unless we have the velocity for every shot in the sample set, the average doesn't tell us much. Using my numbers from above, say the max is 100 and the min is 80. Now let's say we have an average of 90. There are many ways that could happen: in a 10-shot sample we could have five at 100 and five at 80; we could have one at 100, four at 95, four at 85, and one at 80; we could have three at 100, two at 95, two at 85, and three at 80; and there are many other distributions of the ten velocity readings that would average out to 90. So we want something else.

What are we looking for in ammunition? Consistency. Absent other factors, the more consistent the muzzle velocity, the better the odds are of getting small (consistent) groups. A lower SD is the best indicator that the next round will probably be pretty close in velocity to the previous round. And -- at least to a point -- as the sample size increases in number, the SD becomes a better indicator of consistency.

But, while shooting 100 rounds would give a SD that's probably very representative of the entire population (i.e. the total run of that batch of ammunition), in today's market of components (especially primers) being unobtainable it's not realistic for a home reloader to shoot 100-round strings to develop the SD for his/her ammo. We want to use up as little ammo as possible, but enough for the results to be meaningful.

The question then becomes: How many are enough?

[WESHOOT2]

20 rds is 'enough', if one cannot afford more.



I was able to buy 2,000 Federal Small Pistol Match primers recently, at a local gunshop, for just over $203.
The last 1,000 I'd bought last year were $55.
I became complacent.




I still have a fair supply of powder......

[74A95]

I'm not saying that SD is not useful, but it has rather limited usefulness. Consistency matters if your loading for power factor, which means you're trying to find a load or velocity that will prevent you from going lower than the minimum speed for power factor and you're loading to be barely above that.

But SD does not predict group size in a typical handgun at typical handgun shooting distance (see the link in post #2), and it's even mentioned in that article that, "Even the Army Marksmanship Unit recognizes that, “Extremely uniform velocities alone are ‘not’ a reliable predictor of accuracy!"" They should know. It might, and can be useful for very long distance shooting.

But since consistency is not correlated with accuracy, one should not bet on that to predict group size. From that article, which is supported by data and statistical analysis, "The real proof of accuracy is how well they group, and not the consistency of their speed. Chronographs measure velocity, and targets measure accuracy. Trust the target, not the chronograph."

You don't need a chronograph when testing accuracy. Only targets.

[WESHOOT2]

You don't need a chronograph when testing accuracy. Only targets.

Fact.

[Aguila Blanca]

74A95 --

I asked a simple question. Please stop trying to drag the discussion off the topic.

[74A95]

Originally Posted by Aguila Blanca

74A95 --

I asked a simple question. Please stop trying to drag the discussion off the topic.

?????

Then the answer to your question is - there is no sample size in which the SD becomes statistically significant.

[Aguila Blanca]

Originally Posted by 74A05

Then the answer to your question is - there is no sample size in which the SD becomes statistically significant.

If that were true, standard deviation would not exist.

Please stop derailing MY thread. It's a simple question. If you don't know the answer, don't post. If you have no interest in standard deviation for your shooting, skip this thread and move on to some other topic.

[74A95]

Originally Posted by Aguila Blanca

If that were true, standard deviation would not exist.

Please stop derailing MY thread. It's a simple question. If you don't know the answer, don't post. If you have no interest in standard deviation for your shooting, skip this thread and move on to some other topic.

By your own admission, you don't know what it means. Your question of when SD reaches statistical significance proves you don't know what SD is for.

I've had more than one statistics class, in college and graduate school and applied that in a science career. Take the hint: I probably know more about what SD is and what it's used for than you do.

As noted at the Wiki page, " the standard deviation is a measure of the amount of variation or dispersion of a set of values." That's it. It's simply a "descriptive" statistic. It's just a measure of variation. So is standard error. So is extreme spread. And a bunch of others. You're trying to make SD more than it is, which you don't understand.

SD, as a measurement of error, is used for hypothesis testing to make statistical comparisons, such as in t-tests, ANOVAs and so on.

As for the question of how many shots are required for the SD to be meaningful? There is no answer to that because the question is nonsensical. SD has squat to do with determining sample size.

Standard deviation is not what you use to determine how many shots in your sample. Period. That's what you should tell the person who emailed you.

What you can do is look at the conventions used in shooting. 10 shots is generally a good enough value to have reasonable confidence that the average (mean) calculated from those velocities will be a reasonable representative of the 'population' average.

If you want to get a better feel for selecting sample size and what that means, see the Wiki page. https://en.wikipedia.org/wiki/Sample_size_determination. Here is the important message about sample size, "The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample."

Selecting sample size is NOT about how many data points are required to make the SD magical. Sample size is about being a reasonable representation of the population you're measuring. See the underlined part.

Clear as mud?

[berettaprofessor]

The answer to "how many?" is always 42.

[stagpanther]

"Statistics don't prove anything, they suggest correlation for decision-making" the immortal words from my professor who taught advanced statistics.

[hounddawg]

I have found that chrono stats can vary from group to group with the same load within any given session depending on my hold. While the smallest groups almost invariably have the lowest ES/SD the consistency of my technique is a lot more important than the ammo even at mid and long range. Always remember that people were shooting tiny groups long before the first shooting chrono was available. It's what on paper that counts, don't obsess over numbers.

[pgdion]

In a nutshell, statistically significant means it actually matters. This hi ges on a couple of things. First that the amount of change is actually important. I knew I guy that ripped the headliner out of his car to save weight so it would go faster. 5 lbs in a 3500 car is not significant. The other part is that there is enough data (data points) that you can be sure your numbers are accurate. 5 doesn't do it, 7 neither from a statistics standpoint. When I've done component testing, the statisticians want to see sizes of 15, 29, 50, and up. They want enough to generate a good bell curve and prove the data is normal. That said, 10 usually gives pretty decent set. You might get by with less (like 5) if they really stack on top of each other.

BTW- said friend, he actually ruined what was a mint condition 1980 Cameron turning it into a "race" car. And it's not even all that fast.

[hounddawg]

BTW- said friend, he actually ruined what was a mint condition 1980 Cameron turning it into a "race" car. And it's not even all that fast.


pgdion Today 09:30 AM

Car freak here so I gotta ask, what the heck is a Cameron, you mean a Camaro?

apologies for off topic post

[RC20]

Short answer is SD is meaningless for the purpose of accuracy. What its good for not a clue (I was lucky to manage Algebra). Average is important FPS wise as is the ES (which throws average into the dumpster if its way off, ie, you have a quality control issue).

I have had some very accurate loads that did not exhibit a low SD. And I have had single digit SD loads that had mediocre accuracy. Bet you have, too. It's just one more piece of information.

Yep
Ok, I am shooting up some 270 factory ammo that does not shoot worth diddly in a hunting gun (I have a 270 target barrel as I have been gifted with gobs of 270 bullets and its fun)

Now I am curious on what the variance in accuracy is out of the Target gun (the Sako 270 hunting guns hates factory)

Well the factory in the 270 target pretty well sucks as well. Ok, however, some Winchester Coated plastic tip nickle I gets a nice 3/4 inch group out of (which is the best I can do with the Sako 270 and hand loads though that beats the 2.5 inch factory groups all to hell).

So, 5 shot average is 3140 ES: 46 SD: 19.7.

The previous group was 150 gr Sierra Bullets in a factory load, 2 inch group

Avg: 2927 ES: 21 SD: 8.8

So clearly the 150 gr rounds were hugely better (via SD) even though it was by far the worst group.

Me thinks SD is irrelevant, as I had one shot in another group that was really wild, but it also was 200 FPS lower than the lowest of the other 4. Mmmm me thinks, kind of sucky quality control and I bet it was not the SD!

From what I can see, if your ES is in the 50 range, you done good quality controls wise and see where the holes in the target are.

Now, If I had an ES of 100 and I had a 1/4 group I would be in hog heaven as well. Seems to be for sure under 60-70 for ES and the hell with SD.

[pgdion]

@ hounddog .... It's a Camaro after Google spell checks it for you. No car guys at Google I guess. ������

[pgdion]

I should add ..... SD is an indication of how likely your data is going to deviate from the average (mean). It's based on typical data fitting a bell curve. Think of how many data points you need to start to shape a bell. It's a lot more than 10. Even 15 is weak. That doesn't mean you need 15 points for good data though. If you have 5 to 10 points and they are all very close, you can be pretty sure the average of those points will give you good results. The SD on that few of points will be pretty meaningless though.

[Shadow9mm]

Originally Posted by RC20

Short answer is SD is meaningless for the purpose of accuracy. What its good for not a clue (I was lucky to manage Algebra). Average is important FPS wise as is the ES (which throws average into the dumpster if its way off, ie, you have a quality control issue).



Yep
Ok, I am shooting up some 270 factory ammo that does not shoot worth diddly in a hunting gun (I have a 270 target barrel as I have been gifted with gobs of 270 bullets and its fun)

Now I am curious on what the variance in accuracy is out of the Target gun (the Sako 270 hunting guns hates factory)

Well the factory in the 270 target pretty well sucks as well. Ok, however, some Winchester Coated plastic tip nickle I gets a nice 3/4 inch group out of (which is the best I can do with the Sako 270 and hand loads though that beats the 2.5 inch factory groups all to hell).

So, 5 shot average is 3140 ES: 46 SD: 19.7.

The previous group was 150 gr Sierra Bullets in a factory load, 2 inch group

Avg: 2927 ES: 21 SD: 8.8

So clearly the 150 gr rounds were hugely better (via SD) even though it was by far the worst group.

Me thinks SD is irrelevant, as I had one shot in another group that was really wild, but it also was 200 FPS lower than the lowest of the other 4. Mmmm me thinks, kind of sucky quality control and I bet it was not the SD!

From what I can see, if your ES is in the 50 range, you done good quality controls wise and see where the holes in the target are.

Now, If I had an ES of 100 and I had a 1/4 group I would be in hog heaven as well. Seems to be for sure under 60-70 for ES and the hell with SD.

In my experience a good SD or es is important. However from an accuracy standpoint there are other things at play, depending on how deep down the rabbit hole you want to go. For me I dial in a good es/SD, then tune the seating depth. The ES/SD ensures consistency, and the seating depth tunes the group size.

I have always learned towards ES rather than SD personally as I understand the function itself better.

[GeauxTide]

Wow, this has gone NASA. In my EXPERIENCE of 40 years with a Oehler 33 and now a 35p, every 10 shot string I've ever shot that had a low extreme spread had a low standard deviation together with <MOA accuracy. Listed below are the calibers that I reload. An old Finance professor told me that "statistics are like bikinis, what they reveal is interesting, but what they conceal is vital"!

[hounddawg]

Put a barrel good tuner on a rifle and take a decent load and play with the tuner. Shoot 5 round groups , say 20 groups while adjusting the tuner. Group sizes will change dramatically yet the load is the same. Why?

Then take a load that shot sub 10 SD and sub 20 ES on a 5 round test group, wait a week shoot again. Is it still a sub 10 SD and sub 20 ES ?

Earlier in this thread I posted a pic of three 10 shot groups shot at 300 yards. First group had a ES of 168 and a vertical spread of 2.12 inches, another had a ES of 47 and a vertical of 1.452 inches , the third had a ES of 97 and a vertical of 3.56 inches. All 30 rounds were loaded in the same session and to the same BTO and powder charge weighed to within plus or minus .02 gns. So where is the correlation between ES and vertical spread here? How can a group with a 47 ES shoot a sub .5 MOA group at 300 ? Why did three groups loaded and shot at the same time have radically different FPS spreads?