Rancorous debate often accompanies the topic of selecting a self-defense cartridge. Everyone has an opinion and cites various authorities to bolster their positions.
Perhaps the most frequently used standard in the best caliber for self defense debate is the FBI ammunition test protocol. But, is the FBI testing method good for predicting effectiveness of ammunition? If so, does this ammo test hold the same relevancy to the general shooting public that it does for the federal law enforcement agency?
What is the FBI testing protocol?
Alternatively called the FBI penetration testing, the ballistics test set can help gauge law enforcement bullet terminal performance in a standardized medium. This allows for a bullet penetration test that is, in theory, repeatable and useful for establishing a known standard of performance in controlled conditions.
A standardized test allows for the direct comparison of bullet performance. Using this test, a law enforcement agency can compare things like hollow point bullet penetration and expansion between two different cartridges before selecting which ammunition to issue its officers.
This is advantageous for anyone who will have to justify the costs of duty ammo to a budget committee or agency head. Likewise, it is a practical place to start for someone looking for the best performing duty ammunition.
The testing itself requires multiple rounds of the ammo load to be shot into a specific mix of ballistic gelatin. Some of the tests are conducted with intermediate barriers placed between the shooter and gel. Bullet velocity, penetration, expansion and weight retention information is collected.
What are the specific ammunition tests?
As I mentioned previously, the FBI ammo testing requires the use of a standardized medium: ballistics gelatin. This ballistic gel mix is a 10% ordnance gelatin formulation that is chilled to form blocks. This formulation is used as the gelatin is similar in consistency to human muscle tissue.
For handgun load testing, bullets are fired into a ballistic gel block from a distance of 10′ and data is collected about how each load performs. There are six (6) different tests conducted by shooting ballistic gel. They are:
- bare gelatin, no barriers
- heavy clothing on gelatin, no barriers
- light clothing, steel barrier
- light clothing, wallboard barrier
- light clothing, plywood barrier
- light clothing, automobile glass barrier
Each of the barriers and clothing choices are made to simulate shooting circumstances that FBI agents have encountered or could reasonably expect to encounter during a lethal force confrontation. For the testing, each of the barriers and clothing articles are described in exacting detail to enhance the repeatability of the testing.
At the time I am writing this article, the heavy clothing is defined by the FBI protocol as four layers of clothing that might be encountered during winter conditions. The layers are:
- cotton t-shirt material (~5.25 oz/yard, 48 threads per inch)
- cotton material (~3.5 oz/yard, 80 threads per inch)
- Malden Mills Polartec 200 fleece
- cotton denim (~14.4 oz/yard, 50 threads per inch)
Light clothing is defined as two layers:
- cotton t-shirt material (~5.25 oz/yard, 48 threads per inch)
- cotton material (~3.5 oz/yard, 80 threads per inch)
Outside of the Polartec fleece, no other nylon or synthetic materials are used for the clothing portion of the testing.
In the FBI protocol, bullets that fail to penetrate 12″ or penetrate beyond 18″ are penalized. Bullets that expand and penetrate between 12″ and 18″ of gelatin are generally considered acceptable. There is a complex formula for analyzing performance for suitability in a law enforcement context that is beyond the scope of this article.
Performance Does Not Equal Effectiveness…Or Does it?
The FBI testing protocol quantifies a bullet’s terminal performance in a consistent medium. The tests are designed to be repeatable by anyone with the resources to purchase, mix and chill the gelatin. Results from the testing can then be used to compare one bullet to another in terms of its performance in the testing.
However:
The FBI tests do not predict bullet effectiveness.
More than a few of you will stop and reread the previous statement. This is because the FBI tests are frequently used as “evidence” that one round is better than another. Yet, this is simply not true.
The FBI tests only measure a bullet’s performance in a controlled setting and medium. You may be able to correlate performance “on the street” with performance in gelatin, but there is no direct relationship there. In other words, a bullet’s score on the FBI test does not equate to a certain level of performance in real life.
“I don’t think any test can be a predictor of performance other than ‘the bullets that seem to work well tend to meet these test protocols’,” said firearms expert Grant Cunningham. “Beyond that, it’s really generalization. For example, expanding bullets tend to be more effective than non-expanding bullets all other things being equal.”
According to Cunningham, people have misconstrued the FBI testing to support personal preferences in ammunition selection.
“This is why we get dogmatic assertions of bullet effectiveness, such as ‘If it doesn’t penetrate at least 12 inches, it’s not suitable for self defense’,” said Cunningham. “We also still hear predictions of ‘effectiveness’…based on those…measurements, which the tests were never designed to determine. The issue, then, isn’t the tests themselves…the problem is getting people to understand what the tests are and what they do (or don’t do.)”
Based on largely anecdotal data, the handgun ammunition that performs well in the FBI testing tends to perform well in violent encounters. Nevertheless, this is a correlation only and you cannot reasonably extract a predictive formula from it.
There have been a number of significant studies on real world performance of ammunition. Perhaps one of the most famous – and most controversial – was the long term look at handgun stopping power conducted by Evan Marshal and Edwin Sanow. More recently, Greg Ellifritz took a stab at compiling and analyzing data on firearm performance in the real world.
Though the details are different in all of the studies I have seen, the results tend to suggest that:
- there is a minimum level of cartridge power to see reliable performance from a handgun,
- expanding bullets tend to perform better than non-expanding bullets,
- long guns perform much better than handguns, and
- no previous study of actual shootings provides reliable data to make specific predictions on the performance of any specific cartridge.
None of the studies I’ve read are able to provide clear evidence that the FBI testing protocol is an accurate predictor of the elusive thing called “stopping power.” Unfortunately, the FBI testing protocol seems to be the best tool we have for studying potential in controlled conditions.
Is FBI penetration testing useful for self defense?
The FBI penetration test is specifically designed to measure the performance of law enforcement ammunition. It was not designed to test for the best self defense ammunition, nor have I ever seen any claims from the agency that it is a good test for measuring self defense loads.
Nevertheless, the FBI testing is the only widely adopted standard for measuring the performance of handgun ammunition. It is completely understandable why it is the de facto standard for self defense ammo performance even though it wasn’t designed to be it.
While the FBI ammunition test protocol is good, I don’t believe that it is ideal for self defense ammunition. Simply put, what works for law enforcement may not be good for an armed citizen protecting himself against attack.
Active Self Protection is a YouTube channel hosted by firearms instructor John Correia. Currently, the channel has more than 1,000 real-life self defense videos posted. Most of these depict armed citizens defending themselves against attacks.
Throughout these videos, several things become obvious as it relates to the differences between common self defense shootings and law enforcement shootings. While law enforcement may deal with barricaded subjects on a semi-regular basis, there are few instances where armed citizens are shooting through steel, walls and laminated auto glass.
From my own observations – both reviewing shootings caught on film and participating in law enforcement investigations – armed citizens are most frequently defending against attackers who are not shooting from cover.
If you accept that the idea that a bullet penetrating 10% ballistics gelatin to a depth of 12-18″ is more likely to stop a determined attacker than one that does not, then loads developed to perform well in all of the FBI testing will likely be adequate for self defense also.
However, since 2/3 of the current FBI test for handgun ammo is skewed toward defeating hard, intermediate barriers, and self defense shooters are not likely to need this kind of ammunition, does the FBI testing actually give us a good view of what makes the best self defense ammunition?
“[FBI tests three through eight] are unimportant in the private sector,” said Cunningham. “They may be important to people whose job is to arrest other people, which is where barrier penetration becomes an important characteristic. We don’t, and generally shouldn’t, need to worry about that.”
Do we need a better self defense ammunition test?
Maybe.
If the question is “do we need a better test that can reliably predict cartridge performance in the real world?” then the answer is most definitely yes. However, such a test seems to be beyond the community’s grasp. Such a test may never be developed.
However, if the question is merely “would private citizens be better served by another test instead of the FBI protocol?” the answer is “possibly.”
In my mind, the bias toward barrier penetration in the FBI protocol skews ammunition development away from bullet designs that might have the potential for better terminal performance. Several self defense experts I have spoken to hold a similar opinion.
Instead of six tests with a bias toward defeating hard barriers, I envision two tests. The basic two would look like this:
- bare gelatin – A basic test that acts as a baseline for bullet performance.
- clothing over gelatin – This would consist of a two or more layers of clothes such as a heavy sweatshirt cotton with a t-shirt cotton or a synthetic material similar to a Nike Dri-FIT underneath.
I would also like to see the use of a temperature stable testing medium. The FBI’s gelatin choice requires significant logistics complications which makes independent testing of ammunition difficult. If there is anything I’ve learned in the testing of flashlights, independent testing is exceptionally valuable for consumers.
Some flashlight manufacturers appear to game the specifications to create a performance impression that my testing shows isn’t true. I believe independent testing of ammunition is equally important.
Independent testing would allow for better consumer education about how well loads perform out of compact pistols. According to firearms trainer Paul Carlson:
If the average Joe is carrying a Glock 43, he needs to know that ammo performance may not replicate the test results from a duty size gun.
Clear Ballistics is a popular testing medium for many people conducting independent ammo testing. It does not need to be refrigerated and can be used in both cool and warm environments. While it is similar in performance to 10% ordnance gel used in the FBI testing, it is not the same. Testing done with Clear Ballistics should not be compared directly to FBI test results.
However, tests using Clear Ballistics can be compared to each other. A few years back, Lucky Gunner did a detailed analysis and test of popular self defense handgun loads using the Clear Ballistics testing medium. While the data is not suitable for direct comparison to any testing done in accordance to the FBI testing criteria, it serves as an excellent comparison between loads in the testing.
Perhaps just as importantly, Lucky Gunner published its testing standards. This allows other people to use the same standards with the Clear Ballistics product to replicate the testing. People testing the loads using the same criteria can help confirm the validity of the Lucky Gunner testing.
I am not advocating for Clear Ballistics, simply using it as the most obvious temperature stable alternative to the FBI’s gelatin preference.
If independent testing is not a concern, Cunningham may have the easiest approach.
“I don’t think we need new testing protocols. The problem is getting people to understand what the [FBI] tests are and what they do (or don’t do), developing a partial weighted score from the first two tests in the suite, and then getting everyone to agree to use them.”
Final Thoughts
I learned a long time ago to not accept conventional wisdom without verifying the information. Many people suggest that the FBI testing is the best way of identifying the optimal self defense ammunition. That may be true, but has anyone really developed an alternative evaluation method that could identify superior products for the armed citizen? Something that is measurable and not marketing hype?
As a starting point, the FBI testing protocol is a reasonable place to begin a search for quality self defense ammo. However, the standard was developed for users that are in different circumstances than most armed citizens who carry a gun for self defense.
A new set of standards that is designed for self defense – not law enforcement – should at least be considered. If developed, I believe the standards should use a temperature stable ballistic medium and test for the most common things an armed citizen may encounter. The tests should be designed for easy replication by all shooters with locally available materials.
What are your thoughts? Think I’ve missed the point, or hit on something worth exploring? Please leave your take in the comments section below. I know people can get passionate about calibers and stopping power – and that’s great – just keep things civil. I run a family friendly site and welcome all shooters.
Thanks to Federal, Hornady, Speer and Winchester for the use of their photos in this article.
37 replies on “FBI Ammunition Test Protocol & Relevancy to Self Defense”
Minimum 12″ penetration, I believe, came about due to the “failure” of the 9mm Silvertip to puncture the heart of one of the suspects in Miami after entering his right arm and crossing his torso to stop just short of the heart. Still, in my mind, pretty good performance for a 115 grain expanded JHP. I agree that the obsession with these tests is unrealistic. Like any ballistic testing, they are useful- to a point. So is water jug testing. Gunfights are not standardized. They will not conform therefore to test results. Nothing is a guarantee. The FBI’s big sin in my mind is ramming their opinion down the throats of every LE agency in the country despite a relative lack of gunfight experience. Why don’t we look to Border Patrol ( ICE) or DEA for additional insight into what works? Granted I’m no qualified expert.
With respect to ammunition evaluation, during my tenure there (1997 – 2012 at BRF), the FBI never “rammed their opinion down the throat of any LE Agency”. DATA and FBI selection criteria were all that were provided. Agencies were encouraged to make their own choices, based upon their circumstances. Agencies were told: “We will help you learn how to evaluate ammunition so you may make educated decisions.”
The closest we came to “recommending” a cartridge would be identifying what we used. This was only done if asked. Answering a question is factual. Many may have construed it as a recommendation but it was not.
As far as I am aware, the above continues to be the policy of the FBI BRF.
To pass the FBI protocol, a round has to pass all the tests, right? If so, it automatically passes those tests you’re saying are most relevant to the private citizen.
I don’t think there’s any real “weighting” here. If it passes the FBI protocol, then it passes the bare gel and clothing tests.
The only thing the FBI protocol would do is possibly reject rounds (for failing to pass the barrier tests), that might otherwise be suitable for private citizens not concerned with barriers. I’m willing to live with that.
If I can have a round that does BOTH (passes the non-barrier AND barrier tests), rather than simply passing the non-barrier tests, why wouldn’t I want that?
Of course terminal ballistics tests are meant to be predictive. Unfortunately, there is no way to meaningfully correlate lab penetration with real world effectiveness. In other words, lab results are predictive on the basis of reason, but supposed predictiveness cannot be demonstrated empirically. If only we could carry out executions by handgun firing squad.
I don’t think any of the barrier penetration tests are very relevant to self defense. As to which tests I think are, I go by the recommendations of the now disbanded international Wound Ballistics Association. The IWBA had a two-prong test:
– in bare, calibrated, 10% gelatin consistent expansion with an average penetration of 12.5 to 14.0 inches
– in calibrated, 10% gel covered with four layer of heavy denim consistent expansion with an average penetration of 13.0 to 16.0 inches.
Consistency of expansion is measured by the standard deviation of penetration depth — the smaller the standard deviation, the more consistent the expansion. In the IWBA’s draft assessment specifications they suggest a consistency goal of a standard deviation of 0.6 inch as being easily achievable with premium JHPs.
The two tests have equal weight — fail either one and the round in question is deemed inadequate.
I’ve never seen the FBI offer a rationale for relaxing the IWBA’s standards; thus, I’ll stick with the professional scientists and practitioners of the IWBA. After all, in this day and age the FBI has proven it can’t be trusted.
Develop a new standard for comparing handgun self-defense ammunition?
I think Lucky Gunner already *did* that. They developed a reasonable test protocol derived from the FBI methodology, then stepped up and did the damn work. While the rest of us continued to yammer about the topic; or went out to shoot one round from three brands in one caliber into a ballistic catch unlike any other test anywhere. If you’d rather have 8″ or 30″ of penetration, or you like variable instead of uniform expansion/penetration — fine; data’s there, pick yer poison. If a load performs similarly on the most similar FBI and LG tests, I’d consider it a bonus if it also does well on FBI hard barriers. But I’d agree it makes little sense to weight the hard barrier tests much for routine civilian defense use, or to blindly favor an agency round for personal use.
Might there be some better protocol or measure of relevance? Sure, why not? Cook something up; do the work; throw the results out for the world to view. That’s how science operates. But — if you want to buy a box of ammo this afternoon, or next month, or without putting a year of your life and half your bank account into it — the substantial FBI and LG sets of real data are already available as alternatives to advertisements and strongly held/weakly supported opinions.
My same reasoning accepts the work of Ellifritz as “close enough” to demonstrating that caliber barely matters. Have a reliable gun; have the gun with you; be able to run the gun under stress — that matters a lot.