There has been much discussion in recent years about the use of rapid microbiological methods (RMM) in food safety testing and, most recently, controversial debates about whether they are still needed. With laboratory outsourcing at a historic high and better shipping methods delivering samples to labs ever faster, are rapid test methods—especially those employed in-plant by food processors—still a necessary technology? Or is the ease of use and faster time to results still essential for better food safety management?

We wanted to explore this question further, so we conducted a survey in September of more than 250 food processors from around the world and conducted additional interviews. We asked how many processors are employing RMM, which test they are using, and why they selected that type.

Growth of Rapid Microbiological Methods

Readers of Food Safety Magazine will be familiar with the many formats of RMM as they have become widely used in food plants and in commercial laboratories. Tests such as PCR, immunoassay, ELISA, automated bioburden testing, and similar instrument-based RMM have become commonplace in food safety. The growth of these RMM, however, is a relatively recent phenomenon dating back perhaps 20–25 years. When first introduced to the market, RMM were seen as very attractive due to their time savings. Microbiology tests that used to take 5–7 days using traditional growth-based methods could now be completed in 2–3 days, saving time and money associated with holding times and inventory, thereby increasing the plant's production capacity and the efficiency of logistics. These benefits led to their widespread adoption.

Strategic Consulting has been publishing microbiology testing market data since 1996. During that time, we have chronicled the adoption and growth of RMM. As an example, 20 years ago, in 2003, analysis using rapid pathogen test methods accounted for less than one-third of all pathogen samples analyzed. Ten years later, in 2013, this proportion had increased to 43 percent. Now, in 2023, that figure has grown to well over 50 percent. This market share gain for RMM is even more remarkable considering that it occurred during a time of overall strong organic growth in pathogen testing volume, with the total number of tests doubling from 2003 to 2013 and doubling again from 2013 to 2023. No need to do the math; that amounts to strong adoption of rapid pathogen testing methods.

As RMM were first being adopted by food microbiology labs, those labs were more likely to be in a laboratory housed within a food plant. In 2013, for example, more than 90 percent of all (pathogen and non-pathogen) microbiology samples collected were analyzed at the plant where they were collected. Today, that proportion has fallen to less than 70 percent.

Rapid Growth of Outsourcing

Due to the well-known risks of working with pathogens in an in-plant food laboratory, there has been an even more dramatic shift to outsourcing for pathogen samples. In 2013, less than 17 percent of pathogen samples were sent to a lab outside of the plant. Today, the volume sent out to a commercial lab or company-owned central lab is more than 50 percent, on a global basis.

There are reasons for this difference. In previous Food Safety Insights columns, it has been reported that this trend toward outsourcing is much stronger in North America and Europe than in the rest of the world. In North America and Europe, 60–70 percent of pathogen samples are sent to an outside lab for analysis. One of the reasons cited for this phenomenon is the higher availability of qualified food labs in the U.S. and Europe versus other areas of the world. Many processors in other regions tell us that they simply do not have a suitable local lab. For example, I have interviewed food processors in island nations who have no commercial lab in their country. Global commercial lab infrastructure certainly is not uniform.

Another reason cited for the variance is that outsourcing is more expensive than in-house analysis. In the U.S. and Europe, an outsourced pathogen test may cost as much as five times the amount of in-house analysis on a per-sample basis (not considering instrumentation, capital, and labor costs). U.S. and European companies have confirmed to us that outsourcing does indeed cost more, but outsourcing is a strategy for risk reduction, not for cost savings. This strategy may be less affordable to food processors in other regions of the world.

Regardless of the many reasons for outsourcing, the result is a large reduction in the volume of samples being analyzed at in-plant labs. With a smaller volume, operating an in-plant lab becomes even more expensive. This has led many companies to outsource their non-pathogen microbiology testing, as well, leading some to close their plant labs completely.

This effect can be seen in the data we collected in our survey. As shown in Figure 1, 78 percent of the processors in the U.S. said that they do not have an in-plant microbiology lab. For those companies outside of the U.S., the data was very different, with 54 percent saying that they do operate an in-plant lab.

For those companies reporting that they have an in-plant lab, we also wanted to see what type of analysis they do and if they use RMM. Of those with a lab in the U.S., 87 percent said that they use RMM; for the international processors, that figure was 77 percent (Figure 2). As for the analytical targets, in the U.S., 38 percent said that they analyze only indicator organisms, 8 percent analyze only pathogens, and 54 percent said that they use their in-house lab for both pathogens and indicators. For the international processors, 23 percent reported indicators only, 13 percent reported pathogens, and 65 percent reported both (Figure 3).

These figures, especially for the U.S. labs, are somewhat misleading. First, very few (22 percent) of the U.S. processors reported having a lab for any kind of analysis, and only 8 percent of those reported using their labs for pathogen testing only. This amounts to a very small percentage—less than 2 percent—of the overall sample of respondents. Furthermore, of those who reported using their in-house lab for both indicators and pathogens, 100 percent of those said that they outsource many or most of their pathogen samples because their customers require results from an independent third-party lab. It is clear that these processors may want to retain the capability to conduct some pathogen testing in-house, but that capability is not being used for many or most samples.

International companies indicate that their in-plant labs have more capabilities for both indicator and pathogen samples—with 65 percent reporting that they analyze for both. Most of these processors also say that they outsource pathogen samples, but unlike the U.S. companies, they send out only a few samples, mostly for the purpose of verification and/or calibration of in-house tests. Others mentioned that they outsource overflow samples and test for unusual pathogens that they do not encounter frequently, or those for which they lack analytical capabilities.

Additional Questions to be Explored

With this understanding of the background to the growth and current use of RMM, in the next Food Safety Insights column, we will continue to explore this topic with more details on which methods are being used, food processors' views on what attributes of the tests are most important, how fast is fast enough, and what even faster tests would enable them to accomplish. We will also look at where the growth may continue, and whether RMM use in commercial labs will continue to drive growth or if a plateau is likely. Growth in regions outside of the U.S. and Europe will also be examined, along with whether RMM technology is evolving to the point where it may be acceptable to bring back the analysis to an in-plant lab.

Look for answers to those questions and others in the next edition of Food Safety Insights.