type: other

jurisdiction: US

text: b' U.S. Department of Agriculture\n Office of Inspector General\n\n\n\n\nFSIS Sampling Protocol for Testing Beef Trim\n for E. coli O157:H7\n\n\n\n\n Audit Report 24601-9-KC\n February 2011\n\x0c UNITED STATES DEPARTMENT OF AGRICULTURE\n OFFICE OF INSPECTOR GENERAL\n\n Washington, D.C. 20250\n\n\n\n\nDATE: February 24, 2011\n\nREPLY TO\nATTN OF: 24601-9-KC\n\nTO: Alfred V. Almanza\n Administrator\n Food Safety and Inspection Service\n\nATTN: William C. Smith\n Assistant Administrator\n Office of Program Evaluation, Enforcement and Review\n\nFROM: Gil H. Harden /s/\n Assistant Inspector General\n for Audit\n\nSUBJECT: FSIS Sampling Protocol for Testing Beef Trim for E. coli O157:H7 \xe2\x80\x93 Phase I\n\nThis report presents the results of the subject audit. Your written response to the draft report,\ndated January 28, 2011, is attached with excerpts and the Office of Inspector General\xe2\x80\x99s (OIG)\nposition incorporated into the Finding and relevant Recommendation sections of the report.\n\nWe agree with management decision on Recommendation 1 of the report. However, we are\nunable to accept management decision on Recommendations 2, 3 and 4. Documentation and/or\nactions needed to reach management decisions for these recommendations are described in the\nOIG position section of the report.\n\nIn accordance with Departmental Regulation 1720-1, please furnish a reply within 60 days\ndescribing the corrective action taken or planned and the timeframes for implementing the\nrecommendations for which management decision has not been reached. Please note that the\nregulation requires management decision to be reached on all recommendations within 6 months\nfrom report issuance, and final action to be taken within 1 year of each management decision to\nprevent being listed in the Department\xe2\x80\x99s annual Performance and Accountability Report. Please\nfollow your agency\xe2\x80\x99s internal procedures in forwarding documentation for final action to the\nOffice of the Chief Financial Officer.\n\nWe appreciate the courtesies and cooperation extended to us by members of your staff during\nthis audit.\n\x0cTable of Contents\n\nExecutive Summary .................................................................................................1\nBackground & Objectives .......................................................................................5\nBackground...............................................................................................................5\nObjectives ..................................................................................................................8\nSection 1: FSIS Needs to Improve Its Methodology to Test for E. coli\nO157:H7 Contamination .........................................................................................9\n Finding 1: FSIS Should Take Steps to Reevaluate and Solicit Public\n Comment on Redesigning Its Beef Trim Sampling Methodology in Order to\n Improve the Detection of E. coli O157:H7 Contamination ..............................9\n Recommendation 1 ......................................................................................13\n Recommendation 2 ......................................................................................13\n Recommendation 3 ......................................................................................15\n Recommendation 4 ......................................................................................16\nScope and Methodology .........................................................................................18\nAppendix A: Response to Congressional Request for Information Regarding\nFSIS\xe2\x80\x99 E. coli O157:H7 Testing ..............................................................................21\nAppendix B: Update to January 2008 Memorandum to Former Deputy\nSecretary Conner on E. coli Sampling and Testing ............................................28\nAbbreviations .........................................................................................................30\nAgency\xe2\x80\x99s Response .................................................................................................31\n\x0cReport Title and Number: FSIS Sampling Protocol for Testing Beef\nTrim for E. coli O157:H7 - 24601-9-KC\n\nExecutive Summary\n\nResults in Brief\n\nOn November 12, 2009, the Chairwoman of the House Appropriations Subcommittee on\nAgriculture, Rural Development, Food and Drug Administration, and Related Agencies wrote to\nthe U.S. Department of Agriculture\xe2\x80\x99s (USDA) Office of Inspector General (OIG) to express\nconcerns about the efficacy of the testing the Food Safety Inspection Service (FSIS) performs to\ndetect Escherichia coli (E. coli) O157:H7 in U.S. beef \xe2\x80\x9ctrim.\xe2\x80\x9d 1 A recent outbreak of\nE. coli O157:H7 contamination had resulted in at least two deaths and more than two dozen\nillnesses in 11 States, and the Congresswoman posed a number of questions about whether FSIS\xe2\x80\x99\nsampling protocol is statistically valid; how FSIS collects and analyzes samples; and how the\nagency and industry respond to test results.\n\nIn order to expedite this Congressional request, OIG divided its work into two phases. In this\nreport, we are presenting the results of Phase I of our review of FSIS\xe2\x80\x99 sampling method, our\ndirect answers to the Congresswoman\xe2\x80\x99s questions (see Appendix A), and our followup\nobservations pertaining to the memorandum we issued in January 2008 related to E. coli\nO157:H7 testing (see Appendix B). In Phase II, we will perform fieldwork at beef processing\nplants to expand upon information presented in this report and also to pursue issues such as\nwhether plants are testing for E. coli O157:H7 consistently and if FSIS personnel are following\nFSIS procedures for sampling and testing. 2\n\nAt beef slaughter and fabrication plants (processing plants) nationwide, FSIS inspectors sample\nbeef trim and then it is tested to detect E. coli O157:H7 contamination. When FSIS began\nperforming these tests on beef trim in 2007, it chose a sampling method that it believed to be the\nmost scientifically rigorous available: N-60. As part of N-60, FSIS inspectors take 60 small, thin\nsample pieces of exterior carcass material from a very large unit of trim, known as a \xe2\x80\x9clot,\xe2\x80\x9d and\nship those samples to designated FSIS laboratories to be tested for the presence of\nE. coli O157:H7.\n\nAccording to FSIS officials, N-60 is not designed to find E. coli O157:H7 in any given lot at a\nspecific statistical confidence, but to verify that E. coli O157:H7-preventing interventions at\nprocessing plants (such as carcass washes or steam cabinets) are working as intended, and to\n\n\n1\n Trim consists of the pieces of meat that are cut away to make sought after cuts more desirable. Trim is normally\nprocessed into ground beef.\n2\n OIG acknowledges that both FSIS and the industry perform N-60 E. coli O157:H7 pathogen testing. However, we\nhave not reviewed or examined industry\xe2\x80\x99s efforts in their N-60 sampling programs, and we do not intend this report\nto reflect on those efforts. This report is exclusively directed towards the FSIS N-60 sampling program in beef trim.\n\n\n\n Audit Report 24601-09-KC 1\n\x0cemphasize to the plants that they are being monitored. Food safety, from FSIS\xe2\x80\x99 perspective, is\nbest assured by interventions such as these because no method of statistical sampling and\ntesting can guarantee with absolute certainty that a particular lot of beef trim is entirely free\nfrom E. coli O157:H7 contamination. While OIG agrees that testing alone cannot suffice to\nensure that consumers are safe from a pathogen like E. coli O157:H7, we found that FSIS\xe2\x80\x99\nN-60 sampling method is not designed to yield the statistical precision that is reasonable for food\nsafety or to verify that plant controls or interventions are working as intended.\n\nThe dilemma facing FSIS is that as plants\xe2\x80\x99 controls and interventions become more effective\nat eliminating fecal material on carcasses, the presence (or prevalence rate) of E. coli O157:H7\ncontamination in beef trim becomes lower. Statistically, the lower the prevalence rate, the more\ndifficult the pathogen is to find and the more samples need to be taken to detect it. We found\nthat FSIS\xe2\x80\x99 current sampling model of collecting just 60 sample pieces may not provide a\nreasonable likelihood of finding E. coli O157:H7 contamination at a very low prevalence rate\ngiven the following problems:\n\n\xe2\x80\xa2 FSIS has not determined the rate at which E. coli O157:H7 can be found in beef trim, or its\n \xe2\x80\x9cprevalence rate,\xe2\x80\x9d even though the design of an adequate sampling method should begin with\n this information. FSIS should also determine how often the prevalence rate should be\n reassessed.\n\n\xe2\x80\xa2 FSIS has not evaluated or justified the contamination level for E. coli O157:H7 that is\n associated with its N-60 sample size and confidence level (essentially the amount of\n contaminated trim that might slip through N-60 without being detected)\xe2\x80\x94in an FSIS sample\n of N-60, there is a 5 percent probability of missing actual contamination that is present in\n 5 percent of a given lot. There is also a 55 percent probability of missing actual\n contamination that is present in 1 percent of a given lot\xe2\x80\x94in other words, if the contamination\n level is very low, FSIS is more likely to miss contamination than to detect it.\n\nGiven its shortcomings and the presumed low occurrence of this pathogen, the N-60 sample size\nand design may not be adequate for detecting E. coli O157:H7 in beef trim.\n\nFSIS officials have stated that they are reluctant to increase the number of pieces in samples\nbeyond N-60 because taking a large number of sample pieces is impractical due to the associated\ntime and cost to take a sample and limitations on sampling and testing resources. Yet the\nprobable scarcity of E. coli O157:H7 requires that more than 60 sample pieces be taken if\nFSIS is to obtain results which provide a higher degree of confidence in detecting the\nE. coli O157:H7 pathogen and signal an establishment\xe2\x80\x99s failure to execute effective interventions\nor process controls. To ease the burden on FSIS resources, the agency could designate a\nspecialized team (or individual) that would supplement onsite inspectors for the more intensive\nsample collections.\n\nOIG acknowledges that increasing FSIS\xe2\x80\x99 sample size would strain the agency\xe2\x80\x99s sampling\nand laboratory testing resources. For this reason, we are also recommending that FSIS move to\nan inspection system that will determine which processing plants are at a higher risk of\nE. coli O157:H7 contamination. This would enable the agency to focus its testing resources on\n\n\n Audit Report 24601-09-KC 2\n\x0cthose plants. We also believe that FSIS should take advantage of the fact that many plants,\nespecially the larger and more sophisticated ones, are independently performing hundreds of\ntheir own E. coli O157:H7 tests daily, some more rigorously than others. If FSIS could assure\nitself these plants\xe2\x80\x99 samples and tests for E. coli O157:H7 at least met FSIS standards, or even\nmore rigorous ones, then FSIS could utilize these plants\xe2\x80\x99 testing results to augment its own. The\nplants, after all, have compelling economic reasons to have an effective testing program since\nthey bear the costs of any recalls.\n\nOIG concludes that FSIS needs to take steps to thoroughly reevaluate its N-60 sampling program\nfor testing beef trim. The design should provide the American public a high degree of\nconfidence that FSIS tests are accurately identifying E. coli O157:H7 in order to detect an\nestablishment\xe2\x80\x99s failure to execute effective interventions or process controls and, ultimately,\nprevent the distribution and consumption of E. coli O157:H7-contaminated meat.\n\nRecommendation Summary\nDevelop and implement a plan with specific timeframes and milestones to prioritize and perform\nnecessary baseline studies of beef trim and ground beef in order to determine the estimated\nprevalence rate of E. coli O157:H7 for redesigning FSIS\xe2\x80\x99 sampling program. The plan should\ninclude how often the prevalence should be reassessed.\n\nRe-evaluate its sample parameters (size and confidence level) and redesign a sampling program\nfor E. coli O157:H7 in beef trim that provides higher confidence in FSIS accurately detecting\ncontaminated product in order to effectively verify process controls at beef processing plants\nacross the nation.\n\nThoroughly document the scientific support and rationale for FSIS\xe2\x80\x99 revised E. coli O157:H7\nsampling program design, including the contamination level that will be associated with the new\nsample parameters and how the estimated prevalence rate impacts the new design. Publish FSIS\xe2\x80\x99\nrevised E. coli O157:H7 sampling design, along with its support and design rationale, for public\ncomment, and consider any recommended changes before implementing the new sampling\nprogram.\n\nDevelop a detailed operational plan with specific timeframes and milestones to implement an\ninspection system that focuses E. coli O157:H7 sampling and testing resources primarily at\nplants that are likely to be of higher risk, and consider the use of specialized sample collection\nteams.\n\nAgency Response\nIn its January 28, 2011, written response to the official draft report, FSIS generally expressed\nagreement with our finding and recommendations. Also, included in the FSIS\xe2\x80\x99 response were\ngeneral comments regarding the agency\xe2\x80\x99s trim sampling program. We have incorporated the\nFSIS response in the Findings and Recommendations section of this report, along with our\ncomments in the applicable OIG Position sections. FSIS\xe2\x80\x99 response to the official draft is\nincluded in its entirety at the end of this report.\n\n Audit Report 24601-09-KC 3\n\x0cOIG Position\n\nWe concur with FSIS\xe2\x80\x99 proposed corrective actions and have accepted management decision for\nRecommendation 1. However, FSIS did not provide sufficient detail regarding their planned\ncorrective actions on Recommendations 2, 3 and 4 for us to accept management decisions for\nthese recommendations at the issuance of this report. We have provided our comments and what\nis required to reach management decision on each recommendation in the applicable OIG\nPosition sections.\n\n\n\n\n Audit Report 24601-09-KC 4\n\x0cBackground & Objectives\n\nBackground\nFSIS is the public health regulatory agency of USDA. As such, the agency protects consumers\nby ensuring that beef is safe, wholesome, and accurately labeled. Under the Federal Meat\nInspection Act, FSIS inspects all beef sold in interstate commerce to ensure that it meets U.S.\nfood safety standards.\n\nAccording to the Centers for Disease Control and Prevention, E. coli are a large and diverse\ngroup of bacteria. Most strains of E. coli are harmless, but other strains of E. coli, such as\nE. coli O157:H7, cause disease by making Shiga toxin. The symptoms of Shiga toxin poisoning\ncan include severe stomach cramps, diarrhea, and vomiting. Most people who consume beef\ncontaminated with E. coli O157:H7 will recover within 5 to 7 days; some infections are very\nmild, but others can be lethal. The Centers for Disease Control and Prevention estimate that\nE. coli O157:H7 caused about 70,000 cases of Shiga poisoning annually in the United States: in\n2009, 52 of these cases proved fatal. USDA\xe2\x80\x99s Economic Research Service estimates that the\ntotal costs, such as doctor bills and hospitalization costs, associated with consuming\nE. coli O157:H7-contaminated meat are about $405 million annually.\n\nIn 1994, the U.S. District Court held that the Department could reasonably consider\nE. coli O157:H7 an adulterant, and consequently that it could, as part of its statutory authority,\nregulate ground beef that might be contaminated. 3 FSIS relies on the authority provided by this\nstatute to test ground beef and trim. Although it has not codified that authority in Federal\nregulations, it has issued multiple agency directives. In March 2010, FSIS issued its most\nrecent version of Directive 10,010.1, entitled \xe2\x80\x9cMicrobiological Testing Program and Other\nVerification Activities for Escherichia coli O157:H7 in Raw Ground Beef Products and Raw\nGround Beef and Beef Patty Components.\xe2\x80\x9d This directive includes instructions to FSIS\ninspection personnel and other program investigators on sampling and other verification\nactivities for E. coli O157:H7 in raw beef products. In August 2008, FSIS drafted a\n\xe2\x80\x9cCompliance Guideline for Sampling Beef Trimmings for Escherichia coli O157:H7.\xe2\x80\x9d The\ndraft was published for comment, but has yet to be finalized.\n\nGround beef is made up of less tender and less popular cuts of beef. After slaughter, the cattle\ncarcass is cut into various intact cuts of meat (e.g., loin, ribs, roasts, and steaks). After the intact\ncuts are removed, the pieces of meat and fat that remain, known as \xe2\x80\x9cbeef trim,\xe2\x80\x9d are either ground\nor sold to processing plants for grinding into hamburger products. The large quantity of ground\nbeef produced in the United States\xe2\x80\x94using nearly 4 billion pounds of trim annually4\xe2\x80\x94is sold to a\n\n\n\n3\n Texas Food Industry, et al., v. Mike Espy, et al., Civ. No. A-94-CA-748 JN, United States District Court, West\nDistrict, Texas, Austin Division, December 13, 1994.\n4\n February 2008 FSIS report titled \xe2\x80\x9cRisk-based Sampling for Escherichia coli O157:H7 in Ground Beef and Beef\nTrim\xe2\x80\x9d, p. 7. The figure was an estimate of pounds of beef trim based on the estimated pounds of trim for each class\nof beef. The slaughter data was taken from 2005 eARDS data.\n\n\n Audit Report 24601-09-KC 5\n\x0cwide variety of customers, including grocery stores, restaurants, fast food establishments, and the\nNational School Lunch Program, for the most part.\n\nSince 1994, FSIS has tested ground beef for E. coli O157:H7; beginning in 2007, the agency\nexpanded its testing program to include testing beef trim. E. coli O157:H7 can contaminate beef\ntrim when fecal material from slaughtered cattle comes into contact with, and remains on, the\ncarcass. To prevent beef trim from becoming contaminated, plants apply interventions\xe2\x80\x94safety\ncontrols such as lactic acid sprays, carcass washes, or steam cabinets, which are intended to\ndecontaminate carcasses before they are cut into pieces or the meat is shipped from the plant.\nAlthough E. coli O157:H7 grows very slowly when it is refrigerated, cold, and even freezing\ntemperatures, do not destroy it. The actual dose necessary to infect a person is unknown, but\nmost scientists believe that a small amount of E. coli O157:H7 can cause serious illness and even\ndeath, especially in younger children. The only certain and practical way of destroying all\nE. coli O157:H7 in a serving of ground beef is to cook the product thoroughly to 160 degrees\nFahrenheit or hotter, internally.\n\nFSIS explained that its sampling and testing program for E. coli O157:H7, is not so much a way\nof guaranteeing that a given shipment of beef trim is free from contamination, but rather a way of\ndetecting breakdowns in plants\xe2\x80\x99 processing controls or applied interventions that would lead to\nunusually high levels of E. coli O157:H7 contamination, and also as a way of communicating to\nplants that they are being monitored. While FSIS inspectors may sample from large plants each\nmonth, they may test for E. coli O157:H7 in some small to medium-sized plants as little as one\nsample per quarter. It is important to recognize that no system of statistical analysis can arrive at\n100 percent certainty that any given lot of beef trim is free of E. coli O157:H7. Although\nrepeatedly testing a given lot improves the probability of finding E. coli O157:H7, it cannot\nguarantee that the sampled lot of beef is completely free from contamination.\n\nWhen FSIS began developing a system to test for E. coli O157:H7 on beef trim, it concluded that\na sampling system called N-60 was the best available. N-60 sampling is based on a sampling\nmethodology presented by the International Commission on Microbiological Specifications for\nFood which explains that the number 60 is a sufficiently rigorous sampling methodology for\nthose food-borne hazards that are severe (cause severe illness or death) and where the pathogen\xe2\x80\x99s\nenvironment may increase the hazard. 5\n\nThe N-60 sampling method requires an inspector to collect 60 pieces of beef trim from a\nproduction lot of beef cuts or trimmings that will be used for the making of ground beef. 6 The\npieces the inspector collects are small thin slices preferably from the trim cuts that were closest\nto the surface of the carcass since E. coli O157:H7 would not normally be located inside the\nanimal\xe2\x80\x99s muscles. N-60 sampling is resource intensive, often taking inspection program\npersonnel over an hour to collect a sample. The sample is shipped to a designated FSIS\nlaboratory where it is composited for testing to determine the presence of E. coli O157:H7.\n\n5\n In the proper conditions, E. coli O157:H7 can grow and become more numerous.\n6\n FSIS designates the size of each of the 60 pieces sliced from a trim cut should be 3 inches x 1 inch x 1/8 inch,\napproximately the size of a rubber eraser. The 60 pieces are composited and sent as one sample for testing at a FSIS\nlaboratory.\n\n\n Audit Report 24601-09-KC 6\n\x0cEstablishments sometimes do not hold product pending test results, as evidenced by recalls based\non E. coli O157:H7 positive test results. However, the sampled beef is generally held from\ncommerce until the results are returned. If E. coli O157:H7 is detected, then the sampled beef is\ndestroyed or used in cooked product where the contaminant will be destroyed. In 2010, FSIS\ntook steps to refine and improve its N-60 sampling method by (1) issuing more detailed guidance\nconcerning trim sampling, followup sampling, and tracing product back to the source of the\ncontamination; (2) producing an instructional video; and (3) improving its laboratory procedures.\n\nEach year, FSIS schedules approximately 4,400 sample collections of beef trim using\nN-60 samples. 7 Onsite FSIS inspectors collect the 60 sample pieces for the tests from\nindustry-defined \xe2\x80\x9clots.\xe2\x80\x9d A lot is a quantity of product produced from similar manufacturing\nconditions, product types, or time periods. A \xe2\x80\x9clot\xe2\x80\x9d of beef trimmings is typically 10,000 pounds\nor less depending on the number of units in the lot. 8 Beef trimmings are often collected and\ntransported in large bins that hold approximately 2,000 pounds of trim. The bins, or \xe2\x80\x9ccombos,\xe2\x80\x9d\ncan be sampled as individual units, grouped in a five combo lot, or grouped in other lot sizes that\nfit a company\xe2\x80\x99s manufacturing process.\n\nFSIS also adopted the N-60 method not only to improve the effectiveness of its sampling, but\nalso to encourage the regulated industry to implement the method. According to FSIS officials,\nthe agency has recommended, in multiple documents, that establishments conduct testing of their\nown for E. coli O157:H7, since plants have considerable economic incentive to develop effective\nsampling and testing systems. If E. coli O157:H7 is found in beef trim that has left the plant, the\nplant is likely to bear the expense of recalling the product from commerce. In 2008, FSIS held a\npublic meeting to discuss methods for industry sampling of beef trimmings and its related draft\ncompliance guide. The compliance guide discusses the use of N-60 as part of industry statistical\nprocess control programs. FSIS officials stated that encouraging industry to sample product for\npathogens is a goal of all FSIS sampling programs.\n\nLarge plants that sample product may collect frequent samples of N-60 for testing within a day\xe2\x80\x99s\nproduction in order to detect sharp spikes in E. coli O157:H7 contamination, indicating to a plant\nmanager that an intervention or process control is not working effectively and immediate\ncorrective action is needed. However, FSIS does not require that plants perform these tests, nor\ndoes it currently provide standards that plants must meet if they choose to test their own product.\nHazard Analysis and Critical Control Point (HACCP) regulations do, however, require that\nestablishments verify, through testing or other means, that their food safety systems work. FSIS\nmay also look at establishment\xe2\x80\x99s sampling and testing methods during its Food Safety\nAssessments. Plants are also not required to inform FSIS inspectors of positive test results, but\nthey are required to make the test results available for the inspector\xe2\x80\x99s review. FSIS inspectors\nare required to examine the test results at least weekly to determine whether plants took\nappropriate corrective actions.\n\n\n7\n The FSIS website publishes the number of collected product samples that are analyzed every year: 2319 in 2008,\n2728 in 2009, and 2492 as of November 21, 2010.\n8\n FSIS allows plants that produce trim the flexibility to determine lot size. We obtained this information from\nindustry sources.\n\n\n Audit Report 24601-09-KC 7\n\x0cIt is important to note that FSIS sees its N-60 sampling of beef trimmings as only one of a\nnumber of verification activities FSIS conducts regarding establishment process controls for\nE. coli O157:H7. Sample collection and analysis is one of the eight HACCP verification\nactivities listed under the regulations in 9 Code of Federal Regulations FR 417.8. HACCP\nrequires that all significant hazards with the products and production environment be identified\nand controlled. Therefore, FSIS sampling of beef trimmings works along with these and other\ninspector verification activities, including FSIS sampling of ground beef and other ground beef\ncomponents, as well as industry process controls and sampling, to reduce and detect E. coli\nO157:H7 in non-intact beef products. According to FSIS officials, their N-60 sampling of beef\ntrimmings needs to be viewed as a component of the totality of verification activities within an\nestablishment over time.\n\nObjectives\nOur objective in this phase of the audit was to follow up on matters reported in our\nmemorandum, \xe2\x80\x9cFood Safety and Inspection Service Sampling and Testing of E. coli,\xe2\x80\x9d issued\nJanuary 29, 2008, related to examining the adequacy and effectiveness of FSIS\xe2\x80\x99 N-60 sampling\nmethod.\n\n\n\n\n Audit Report 24601-09-KC 8\n\x0cSection 1: FSIS Needs to Improve Its Methodology to Test for\nE. coli O157:H7 Contamination\n\nFinding 1: FSIS Should Take Steps to Reevaluate and Solicit Public\nComment on Redesigning Its Beef Trim Sampling Methodology in Order to\nImprove the Detection of E. coli O157:H7 Contamination\nWe found that FSIS\xe2\x80\x99 N-60 sampling program is not adequately designed to yield the statistical\nprecision that is reasonable for food safety purposes or to verify that a plant\xe2\x80\x99s interventions and\nprocess controls are working as intended. FSIS\xe2\x80\x99 sampling dilemma is that given the likely low\noccurrence of E. coli O157:H7 in U.S. beef trim, FSIS must collect more than 60 pieces before it\ncan attain the precision that is reasonable for food safety purposes. With just 60 sample pieces,\nFSIS is currently more likely not to find E. coli O157:H7 than it is to find it in a sample of\nproduct with, for example, 1 percent contamination. 9 The problems with N-60 occurred because\nFSIS, from the outset, did not design its sampling program to achieve a specific level of\nstatistical precision. This was because FSIS perceived its sample and collection analysis as only\none component of its eight HACCP verification activities 10 and, also, as a method to encourage\nthe regulated industry to conduct testing of their own for E. coli O157:H7. We believe FSIS\nneeds to correct N-60\xe2\x80\x99s limitations by redesigning its sampling program to provide a higher\ndegree of confidence that its beef trim tests are accurately identifying contaminated product in\norder to detect an establishment\xe2\x80\x99s failure to execute effective interventions or process controls;\nthus, preventing the establishment from distributing E. coli O157:H7-contaminated meat.\n\nThe Federal Meat Inspection Act states that FSIS is to protect the health and welfare of\nconsumers by preventing the distribution of meat products that are unwholesome or adulterated.\nIn 1994, USDA won a case in the U.S. District Court, which upheld that that the Department\ncould reasonably consider E. coli O157:H7 an adulterant, and that consequently it could, as part\nof its statutory authority, regulate beef that might be contaminated. 11 Since the 1990s, FSIS has\nmaintained a testing program for raw ground beef, but starting in 2007, the agency began testing\nthe components of ground beef, most notably trim.\n\nFSIS adopted the N-60 sampling method because the agency believed that N-60 was the most\nrigorous method identified in the scientific literature, yet it did so without fully communicating,\nin advance, to the public and to Congressional decision-makers the shortcomings of this method.\nAccording to FSIS officials, the agency did not hold a public meeting on N-60 sampling until\n2008, although it has briefed industry and consumer groups multiple times since. Moreover,\n\n\n9\n At 1 percent contamination, N-60 yields only a 45 percent level of confidence that the adulterant would be found,\nwhich is far short of the level of confidence normally considered statistically desirable. Since this is a question of\nfood safety, we use 1 percent in our example.\n10\n 9 Code of Federal Regulations 417.8 proscribes eight activities that FSIS performs to verify a plant\xe2\x80\x99s HACCP,\nincluding sample collection and analysis.\n11\n Texas Food Industry, et al., v. Mike Espy, et al., Civ. No. A-94-CA-748 JN, United States District Court, West\nDistrict, Texas, Austin Division, December 13, 1994.\n\n\n Audit Report 24601-09-KC 9\n\x0cwhen FSIS adopted N-60, it did not take the steps necessary to adapt the sampling program so\nthat it would correctly identify contaminated product, such as determining the rate at which\nE. coli O157:H7 can be found in U.S. beef trim. Instead, it designated the number 60 as an ideal\nsample size in order to achieve a balance between collecting a representative sample while not\nover-burdening its inspection staff.\n\nFSIS officials, academics, and industry representatives all agree that the expected occurrence rate\nof E. coli O157:H7 in the U.S. beef trim supply is very low; however, FSIS has not formally\ndetermined the prevalence of E. coli O157:H7, which should have been the first step in\ndeveloping an effective sampling design. In 2008, FSIS did publish a baseline data collection\nprogram, which reported partial results indicating the positive rate for E. coli O157:H7 was\n0.68 percent. 12 However, the study also stated that this rate was \xe2\x80\x9conly a raw number and should\nnot be considered as the national prevalence.\xe2\x80\x9d FSIS did not put forward 0.68 percent as a rate of\nnational prevalence in part because the agency gave industry advance notice that testing was\ngoing to be performed, and then relied on samples from product that industry had already\ntested. 13 FSIS had intended to issue a prevalence rate based on this study, but had not done so at\nthe time of our fieldwork because of competing priorities that delayed completing the project.\n\nIn followup discussions during November and December, 2010, FSIS officials informed us they\nhad a draft document that established a national prevalence rate from the 2008 baseline study\nthat they planned to finalize and issue soon. They stressed that the national prevalence rate was\nnot based on current positive test results because the data for the baseline study was from 2007\nand 2008. They also pointed out that the testing data were based on N-60 sampling results, since\nthat has been the standard sampling method used by FSIS and the industry. We believe that any\nnational prevalence rate based on these data may already be outdated, particularly because many\nplants may have improved their intervention methods in recent years. We also question the\nreliability of N-60 sampling to support the rate of positive test results used in determining the\nnational prevalence rate. In Phase II, we will further review the methodology and scientific\nsupport used to compute the national prevalence rate FSIS issues.\n\nAdditionally, OIG found that FSIS\xe2\x80\x99 N-60 relies on sample parameters that are questionable from\na food safety standpoint. This is because FSIS has not evaluated or justified the contamination\nlevel for E. coli O157:H7 that is associated with its N-60 sample size and confidence level\xe2\x80\x94\nessentially, the amount of contaminated trim that might slip through N-60 without being\ndetected. For instance, in an FSIS sample of N-60, there is a 5 percent probability of missing\nactual contamination that is present in 5 percent of a given lot. There is also a 55 percent\nprobability of missing actual contamination that is present in 1 percent of a given lot\xe2\x80\x94in other\nwords, if the contamination level is very low, FSIS is more likely to miss contamination than to\ndetect it. Therefore, to maintain a high probability of detecting a pathogen with presumably as\n\n12\n In \xe2\x80\x9cNationwide Microbiological Baseline Data Collection Program for the Raw Ground Beef Component:\nDomestic Beef Trimmings,\xe2\x80\x9d FSIS tested 1,900 samples and found that 13 samples were positive, for a percent\npositive rate of 0.68 percent. In this description, each of the \xe2\x80\x9csamples\xe2\x80\x9d was a group of 60 pieces of trim.\n13\n In plants that conduct their own testing, the product that FSIS samples has already been tested and passed\n(approved for entry into commerce as raw beef trim) by industry, and that the FSIS test is not an independent test in\nwhich FSIS chooses a lot off the production line before industry has gotten a test result for it.\n\n\n Audit Report 24601-09-KC 10\n\x0clow a level of contamination as E. coli O157:H7, FSIS\xe2\x80\x99 sample size must increase. 14 For\nexample, if the contamination rate of 5 percent were lowered to a rate more reflective of the\noccurrence of E. coli O157:H7, such as 1.5 percent, then to achieve a 95 percent confidence level\nin detecting E. coli O157:H7, the sample size must increase substantially, even to perhaps N-200.\nOtherwise, if the sample size remains at 60, then the level of confidence in finding E. coli\nO157:H7 would drop from 95 percent to 60 percent. At a contamination rate of 1 percent, it\nwould drop to only 45 percent.\n\nOIG maintains that testing for E. coli O157:H7 without a well supported prevalence rate hinders\nthe design of an appropriate sample and that FSIS must determine how to detect a lethal\ncontaminant that is quite scarce, even if its sample size must rise above 60 pieces. We concluded\nthat N-60\xe2\x80\x99s limitations are severe enough that the use of this method of sampling for\nE. coli O157:H7 is not especially useful for verifying that plant controls or applied interventions\nare working as intended or providing assurance that establishments are not distributing\nE. coli O157:H7-contaminated meat for consumption.\n\nIn order to design a better sampling method for testing for E. coli O157:H7, FSIS must first\ndetermine the prevalence of E. coli O157:H7 in trim. 15 Ideally, FSIS would develop a process\nfor determining a baseline prevalence for E. coli O157:H7 in beef trim that is ongoing, relevant\nfrom year-to-year, and that takes into account seasonal variations due to increased summer heat,\nand other factors. The process should also include how often the prevalence rate should be\nreassessed. Additionally, in collaboration with Congress, the industry, and the public, FSIS\nshould specify and justify its sample parameters and the associated contamination level that is\nsuitable for food safety purposes. In order to develop the best possible sampling program and\nensure transparency, FSIS should publish its proposed sampling design for public comment\nbefore the agency implements any changes. If these steps are taken, then it is probable that the\nnumber of sample pieces the agency needs to collect will rise.\n\nDuring final processing of our report, FSIS released its \xe2\x80\x9cNational Prevalence Estimate of\nPathogens in Domestic Beef Manufacturing Trimmings (Trim) December 2005 \xe2\x80\x93 January 2007\xe2\x80\x9d,\ndated January 2011. In the report, FSIS estimated that the national prevalence of E. coli\nO157:H7 in beef trim for use in the manufacture of raw ground beef was 0.39 percent, with a 95\npercent confidence interval was between 0.05 percent and 0.73 percent. The 0.39 percent\nprevalence rate represents the occurrence of lots tested that showed positive results for E. coli\nO157:H7; it does not reflect the prevalence, or level, of contamination in a given lot of beef trim.\nTherefore, we question how useful this prevalence rate will actually be in determining the\nnumber of sample pieces to collect in redesigning FSIS\xe2\x80\x99 N-60 sampling program.\n\n\n\n\n14\n For any sample size, there is some probability of failing to detect contamination that is actually present in the lot.\nHowever, for a specified sample size, such as N-60, the probability of failing to detect actual contamination\nincreases as the percent of actual contamination decreases.\n15\n In September 2003, we made this type of recommendation in our audit of FSIS\xe2\x80\x99 Oversight of Production Process\nand Recall at a ConAgra Plant (Report 24601-02-KC, dated September 30, 2003).\n\n\n Audit Report 24601-09-KC 11\n\x0cOIG acknowledges that collecting additional sample pieces would strain FSIS\xe2\x80\x99 collection and\nlaboratory resources. 16 At present, FSIS estimates that it has the resources to perform about\n4,400 tests for N-60 samples yearly. To help mitigate the strain on FSIS\xe2\x80\x99 resources, OIG\nsuggests that FSIS consider moving to a more risk-based sampling and testing approach, similar\nto its ground beef sampling program. FSIS could develop profiles of high-risk plants based on\nfactors such as the number and quality of interventions at the plant; the volume of meat\nprocessed; the time of the year; the plant\xe2\x80\x99s number of noncompliances; the type, frequency, and\nquality of the E. coli O157:H7 sampling and testing being performed by the plant; and the\npercent of FSIS positive test results. FSIS should then direct its limited sampling and testing\nresources towards high-risk plants.\n\nThough sampling at higher levels may increase the workload on FSIS inspectors, we believe that\nby designating a specialized team (or individual) that would supplement onsite inspectors for\nthese more intensive sample collections, FSIS could provide the resources it needs to improve its\nsampling and testing program. FSIS is already considering this change. USDA is familiar with\nusing teams of this sort since the Agricultural Marketing Service is currently using contractors to\ncollect ground beef samples for the product it buys for the National School Lunch Program.\n\nAdditionally, some large plants are already testing hundreds of samples daily for\nE. coli O157:H7. Currently, FSIS has little direct assurance that these samples and tests are\nbeing performed according to its standards, or that the tests plants perform are similar to one\nanother. If FSIS could ensure that industry sampling and testing at least met FSIS standards, it\ncould potentially rely on industry\xe2\x80\x99s results, which would allow the agency to add industry\xe2\x80\x99s\nmuch greater testing capacity to its own relatively limited resources. Such an approach would be\nbeneficial because (1) it would help FSIS eliminate the bottleneck caused by its own resource\nlimitations; and (2) it would help FSIS monitor the immediate effectiveness of controls and\ninterventions in place at those plants. According to FSIS officials, direct and regular collection\nof industry test results by FSIS for its verification programs could raise several logistical and\nlegal issues, especially if the industry data were to be used for enforcement, which should first be\nconsidered. However, the agency could consider the confidence it places in a given plant\xe2\x80\x99s\nsampling and testing system as one of the factors in whether it considers the plant to be at high\nrisk or not, which would allow it to focus its own efforts on plants where industry testing does\nnot meet FSIS standards.\n\nIt is also important to recognize that as more effective control measures are adopted by the\nindustry, the prevalence of E. coli O157:H7 contamination should continually decrease.\nEventually, a point may be reached where product testing may be of limited benefit and greater\nbenefit may be achieved by shifting FSIS resources to comprehensive analysis of plants\xe2\x80\x99\nvalidation of their interventions and control systems.\n\nIn any event, OIG concludes that FSIS needs to take steps to thoroughly reevaluate its sampling\nand testing program. As long as the agency is performing tests for E. coli O157:H7\ncontamination, its sampling methodology should be providing the American public a high degree\n\n16\n FSIS officials stated that an inspector requires about 1 hour to collect and prepare an N-60 sample for shipment to\nFSIS\xe2\x80\x99 laboratory.\n\n\n Audit Report 24601-09-KC 12\n\x0cof confidence in the accuracy of its testing results to detect an establishment\xe2\x80\x99s failure to execute\neffective interventions or process controls and, ultimately, prevent the distribution and\nconsumption of meat contaminated with E. coli O157:H7.\n\nRecommendation 1\nDevelop a plan with specific timeframes and milestones to prioritize and perform necessary\nbaseline studies of beef trim and ground beef in order to determine the estimated prevalence rate\nof E. coli O157:H7 for redesigning FSIS\xe2\x80\x99 sampling program. The plan should include how often\nthe prevalence should be reassessed.\n\nAgency Response\nFSIS will develop a plan for prioritizing and performing E. coli O157:H7 baseline studies of\nbeef, including ground beef, trim and other ground beef components, for the purpose of\ninforming and improving overall FSIS verification programs for control of the pathogen. FSIS\nwill publish the plan for comment before it is made final.\n\nPlanned Completion Date: December 2011\n\nOIG Position\nWe accept management decision.\n\nRecommendation 2\nRe-evaluate its sample parameters (size and confidence level) and redesign a sampling program\nfor E. coli O157:H7 in beef trim that provides higher confidence in FSIS accurately detecting\ncontaminated product in order to effectively verify process controls at beef processing plants\nacross the nation.\n\nAgency Response\nFSIS will evaluate its sampling program in the context of prevalence data and inspection\nverification data. FSIS believes that effective verification of food safety systems must include\ninspection of an establishment\xe2\x80\x99s production process to ensure that controls are adequately\ndesigned and implemented to prevent, eliminate or reduce contamination with E. coli O157:H7.\nTo improve inspection program personnel verification of process controls for the reduction of\nE. coli O157:H7 at both beef slaughter and processing establishments, FSIS will be taking the\nfollowing actions during 2011. Together, these actions will give inspection program personnel\ncomprehensive, aggregate data on establishment process controls for E. coli O157:H7.\n\n \xe2\x80\xa2 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n\n Audit Report 24601-09-KC 13\n\x0c carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n\n \xe2\x80\xa2 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef\n or bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following a\n confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n\n \xe2\x80\xa2 FSIS will implement a new inspection procedure, the Hazard Analysis Verification\n (HAV), which will examine an establishment\xe2\x80\x99s hazard analysis and decision making\n documents. The HAV will serve as a screening process to identify issues of concern in\n the design of the establishments\xe2\x80\x99 food safety systems. HAVs will be performed at\n specified frequencies and when certain information prompts identify establishments that\n may have modified their HACCP system or may have failed to change their HACCP\n plan in response to a significant change in the establishments\xe2\x80\x99 process. A HAV will\n verify that the establishment\xe2\x80\x99s hazard analysis addresses the applicable food safety\n hazards for the process, product, and intended use. It will also verify that there is at least\n one CCP 17 for the hazards that are reasonably likely to occur and that there is support for\n any decisions that applicable hazards are not reasonably likely to occur. It will also\n verify that any prerequisite programs (such as an SOP 18, GMP 19, purchase specifications,\n etc.) which are intended to prevent the occurrence of the food safety hazard are\n adequately justified and properly implemented.\n\nPlanned Completion Date: December 2011\n\nOIG Position\nWe agree that FSIS would benefit from an evaluation of its sampling program in the context of\nprevalence data and inspection verification data. Further, we are pleased to see that the agency\nhas outlined specific plans to improve the inspection program personnel verification of process\ncontrols for the reduction of E. coli O157:H7 in establishments in 2011. However, the agency\nresponse does not provide clarification regarding how the planned notice, directive, and HAV\nprocess will specifically improve the agency\xe2\x80\x99s trim sampling program. Therefore, we cannot\nreach management decision. In addition, FSIS\xe2\x80\x99 planned procedures on HAVs should articulate\nwhat corrective or enforcement measures will be taken on deficiencies exposed by these HAVs.\n\n\n\n17\n Critical Control Point\n18\n Standard Operating Procedure\n19\n Good Manufacturing Practices\n\n\n Audit Report 24601-09-KC 14\n\x0cRecommendation 3\nThoroughly document the scientific support and rationale for FSIS\xe2\x80\x99 revised E. coli O157:H7\nsampling program design, including the contamination level that will be associated with the new\nsample parameters and how the estimated prevalence rate impacts the new design. Publish FSIS\xe2\x80\x99\nrevised E. coli O157:H7 sampling design with its support and rationale for public comment, and\nconsider any recommended changes before implementing the new sampling program.\n\nAgency Response\nFSIS will develop a plan for prioritizing and performing E. coli O157:H7 baseline studies of\nbeef, including ground beef, trim and other ground beef components, for the purpose of\ninforming and improving FSIS verification programs for control of the pathogen. FSIS will\npublish the plan for comment before it is made final.\n\nFSIS also will be taking the following actions in 2011 (as stated above in response to\nRecommendation #2) to improve inspection program personnel verification of food safety\nsystems for the prevention, elimination or reduction of E. coli O157:H7 at both raw beef\nslaughter and processing establishments. Together, these actions will give inspection program\npersonnel comprehensive, aggregate data on establishment process controls for E. coli O157:H7.\n\n \xe2\x80\xa2 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n\n \xe2\x80\xa2 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef or\n bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following a\n confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n\n \xe2\x80\xa2 FSIS will implement a new inspection procedure, the HAV, which will examine an\n establishment\xe2\x80\x99s hazard analysis and decision making documents. The HAV will serve as\n a screening process to identify issues of concern in the design of the establishments\xe2\x80\x99 food\n safety systems. HAVs will be performed at specified frequencies and when certain\n information prompts identify establishments that may have modified their HACCP\n system or may have failed to change their HACCP plan in response to a significant\n change in the establishments\xe2\x80\x99 process. A HAV will verify that the establishment\xe2\x80\x99s hazard\n analysis addresses the applicable food safety hazards for the process, product, and\n intended use. It will also verify that there is at least one CCP for the hazards that are\n reasonably likely to occur and that there is support for any decisions that applicable\n\n Audit Report 24601-09-KC 15\n\x0c hazards are not reasonably likely to occur. It will also verify that any prerequisite\n programs (such as an SOP, GMP, purchase specifications, etc.) which are intended to\n prevent the occurrence of the food safety hazard are adequately justified and properly\n implemented.\n\nPlanned Completion Date: December 2011\n\nOIG Position\nWe agree that FSIS would benefit from performing E. coli O157:H7 baseline studies for the\npurpose of informing and improving FSIS verification programs for control of the pathogen. We\nare also pleased to see that the agency has outlined specific plans to improve the inspection\nprogram personnel verification of process controls for the reduction of E. coli O157:H7 in\nestablishments in 2011. However, the agency response does not provide clarification regarding\nhow the planned baseline studies, notice, directive, and HAV process will specifically improve\nthe agency\xe2\x80\x99s trim sampling program or cited inspection activities. Therefore, we cannot reach\nmanagement decision. FSIS needs to document and publish for comment the scientific support\nand rationale for its revised E. coli O157:H7 sampling program design, including the\ncontamination level that will be associated with the new sample parameters and how the\nestimated prevalence rate impacts the new design and inspection procedures.\n\nRecommendation 4\nDevelop a detailed operational plan with specific timeframes and milestones to implement an\ninspection system that focuses E. coli O157:H7 sampling and testing resources primarily at\nplants that are likely to be of higher risk, and consider the use of specialized sample collection\nteams.\n\nAgency Response\nFSIS will evaluate its sampling program in the context of prevalence data and inspection\nverification data. FSIS believes that effective verification of food safety systems must include\ninspection of an establishment\xe2\x80\x99s production process to ensure that controls are adequately\ndesigned and implemented to prevent, eliminate or reduce contamination with E. coli O157:H7.\nTo improve inspection program personnel verification of process controls for the reduction of\nE. coli O157:H7 at both beef slaughter and processing establishments, FSIS will be taking the\nfollowing actions during 2011. Together, these actions will give inspection program personnel\ncomprehensive, aggregate data on establishment process controls for E. coli O157:H7.\n\n \xe2\x80\xa2 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n\n\n\n Audit Report 24601-09-KC 16\n\x0c \xe2\x80\xa2 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef\n or bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following a\n confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n\n \xe2\x80\xa2 FSIS will implement a new inspection procedure, the HAV, which will examine an\n establishment\xe2\x80\x99s hazard analysis and decision making documents. The HAV will serve as\n a screening process to identify issues of concern in the design of the establishments\xe2\x80\x99 food\n safety systems. HAVs will be performed at specified frequencies and when certain\n information prompts identify establishments that may have modified their HACCP\n system or may have failed to change their HACCP plan in response to a significant\n change in the establishments\xe2\x80\x99 process. A HAV will verify that the establishment\xe2\x80\x99s\n hazard analysis addresses the applicable food safety hazards for the process, product, and\n intended use. It will also verify that there is at least one CCP for the hazards that are\n reasonably likely to occur and that there is support for any decisions that applicable\n hazards are not reasonably likely to occur. It will also verify that any prerequisite\n programs (such as an SOP, GMP, purchase specifications, etc.) which are intended to\n prevent the occurrence of the food safety hazard are adequately justified and properly\n implemented.\n\nPlanned Completion Date: December 2011\n\nOIG Position\nWe agree that FSIS would benefit from an evaluation of its sampling program in the context of\nprevalence data and inspection verification data. We are also pleased to see that the agency has\noutlined specific plans to improve the inspection program personnel verification of process\ncontrols for the reduction of E. coli O157:H7 in establishments in 2011. However, the agency\nresponse does not provide clarification regarding how the planned notice, directive, and HAV\nprocess will specifically improve the agency\xe2\x80\x99s trim sampling program and inspection activities to\nfocus on plants that are likely to be of higher risk. Therefore, we cannot reach management\ndecision. FSIS needs to develop a detailed operational plan with specific timeframes and\nmilestones to implement an inspection system that focuses E. coli O157:H7 sampling and testing\nresources primarily at plants that are likely to be of higher risk and consider the use of\nspecialized sample collection teams.\n\n\n\n\n Audit Report 24601-09-KC 17\n\x0cScope and Methodology\nIn November 2009, because of concerns regarding the efficacy of E. coli O157:H7 testing of\nbeef trim products, the Chairwoman of the Subcommittee on Agriculture, Rural Development,\nFood and Drug Administration, and Related Agencies of the House Appropriations Committee\nrequested that USDA\xe2\x80\x99s Inspector General investigate the scientific merits and potential\nshortcomings of N-60 sampling design and application of the test results from N-60 samples of\nbeef trim products.\n\nBetween January and June 2010, we performed our audit at FSIS Headquarters in\nWashington, D.C., and the Eastern Laboratory in Athens, Georgia. In addition to working with\nFSIS officials, we spoke with representatives of two meat industry groups, officials from two\nconsumer groups, academics from four universities, and representatives of three other agencies\ninside USDA to gain perspective on the challenges facing FSIS when sampling and testing for\nE. coli O157:H7. In addition, we reviewed documentation from FSIS; international agencies\nsuch as the Food and Agriculture Organization of the United Nations, World Health\nOrganization, Codex Alimentarius, and the European Union\xe2\x80\x99s Food Standards Agency; and\nscientific literature from organizations devoted to food safety and illness prevention.\n\nFSIS Headquarters\n\nAt FSIS Headquarters, we interviewed the appropriate senior-level officials and determined the\nresponsibilities of the following divisions as they relate to E. coli O157:H7 sampling and testing:\n\n \xe2\x80\xa2 Policy and Program Development\xe2\x80\x94this division provides leadership in the identification\n of policy needs, develops policy solutions to address the intent and application of\n verification and enforcement policy in plant activities, and provides direct technical\n support to FSIS field personnel.\n\n \xe2\x80\xa2 Field Operations\xe2\x80\x94this division manages the national inspection and enforcement\n activities. Further, within this division we conducted additional interviews with\n representatives of the recall group that coordinates all recall activities.\n\n \xe2\x80\xa2 Program Evaluation, Enforcement, and Review\xe2\x80\x94this division assesses FSIS\xe2\x80\x99 program\n functions and operations. We interviewed representatives from the Program Evaluation\n and Improvement Staff who provide leadership and technical expertise in the area of\n program evaluation.\n\n \xe2\x80\xa2 Data Integration and Food Protection\xe2\x80\x94this division coordinates all emergency response,\n food defense, and data analysis activities within FSIS. We interviewed representatives\n of the Data Analysis and Integration Group, which is responsible for evaluating\n individual FSIS data streams, ensuring data analyses are consistent and of high quality,\n and conducting data analyses for agency decisions.\n\n \xe2\x80\xa2 Public Health Science\xe2\x80\x94this division provides expert scientific analysis, advice, data, and\n recommendations on all matters involving public health and science that are of concern\n\n Audit Report 24601-09-KC 18\n\x0c to FSIS. We spoke with officials at the Regulatory Field Services Laboratory in Athens,\n Georgia, who support FSIS\xe2\x80\x99 farm-to-table food safety strategies at three field\n laboratories which conduct scientific tests in the disciplines of chemistry, microbiology,\n and pathology.\n\nOthers Outside of USDA\n\nWe also interviewed the following groups, entities, and universities about E. coli O157:H7\nsampling, testing, and control:\n\n \xe2\x80\xa2 Beef industry groups\xe2\x80\x94one group represented U.S. cattle producers and the other\n represented companies that process red meat in the United States and their suppliers\n throughout America.\n\n \xe2\x80\xa2 Consumer advocates\xe2\x80\x94one consumer group is a non-profit organization that advocates for\n policies it believes will result in healthy, safe food and the other group is a non-profit\n public health group dedicated to preventing illness and death from food-borne pathogens.\n\n \xe2\x80\xa2 Academic sources\xe2\x80\x94professors with expertise in food safety were interviewed from the\n University of Nebraska-Lincoln, Kansas State University, Iowa State University, and\n Texas A&M University about E. coli O157:H7 issues currently facing the meat industry.\n\nOther Agencies Within USDA\n\nWe talked to the following USDA agencies about E. coli O157:H7 detection and control:\n\n \xe2\x80\xa2 The Agricultural Research Service conducts research to develop solutions to agricultural\n problems of high national priority, in part to ensure high quality food, ensure safe food,\n sustain a competitive agricultural economy, enhance natural resources, and provide\n economic opportunities for rural communities.\n\n \xe2\x80\xa2 The Agricultural Marketing Service develops quality grading standards for agricultural\n commodities, administers marketing regulatory programs, oversees marketing\n agreements and orders, and makes food purchases for Federal food assistance programs.\n\n \xe2\x80\xa2 The Animal and Plant Health Inspection Service is charged with protecting and\n promoting U.S. agricultural health, regulating genetically engineered organisms,\n administering the Animal Welfare Act, and carrying out wildlife management activities.\n\nWe conducted this performance audit in accordance with Government auditing standards. Those\nstandards require that we plan and perform the audit to obtain sufficient, appropriate evidence to\nprovide a reasonable basis for our findings and conclusions based on our audit objectives. We\nbelieve that the evidence obtained provides a reasonable basis for our findings and conclusions\nbased on our audit objectives.\n\n\n\n\n Audit Report 24601-09-KC 19\n\x0cDuring the course of our audit we did not verify information in any FSIS electronic information\nsystem, and make no representation regarding the adequacy of any agency computer systems or\nthe information generated from them.\n\n\n\n\n Audit Report 24601-09-KC 20\n\x0cAppendix A: Response to Congressional Request for Information\nRegarding FSIS\xe2\x80\x99 E. coli O157:H7 Testing\nThe audit request from the Chairwoman of the Subcommittee on Agriculture, Rural\nDevelopment, Food and Drug Administration, and Related Agencies of the House\nAppropriations Committee posed a number of questions. What follows are our responses in\nthese areas:\n\n 1. What is the prevalence of E. coli O157:H7 in domestic beef trim used for raw ground\n beef production at processing facilities in the United States? When and how was this\n estimate reached? How frequently will it be re-assessed?\n\n FSIS has not established a prevalence rate for E. coli O157:H7 in a given lot of beef trim.\n As discussed in the finding in this report, we are recommending that FSIS develop and\n implement a plan with specific timeframes and milestones to prioritize and perform\n necessary baseline studies of beef trim and ground beef in order to determine the\n estimated prevalence rate of E. coli O157:H7 for redesigning FSIS\xe2\x80\x99 sampling program.\n This plan should include how often the prevalence should be reassessed.\n\n During final processing of our report, FSIS released its \xe2\x80\x9cNational Prevalence Estimate of\n Pathogens in Domestic Beef Manufacturing Trimmings (Trim) December 2005 \xe2\x80\x93 January\n 2007\xe2\x80\x9d, dated January 2011. In the report, FSIS estimated that the national prevalence of\n E. coli O157:H7 in beef trim for use in the manufacture of raw ground beef was 0.39\n percent, with a 95 percent confidence interval was between 0.05 percent and 0.73\n percent. It also includes the methods used to calculate the pathogen estimates and\n provides additional information on the statistical procedures used in the study. The 0.39\n percent prevalence rate represents the occurrence of lots tested that showed positive\n results for E. coli O157:H7; it does not reflect the prevalence, or level, of contamination\n in a given lot of beef trim. Therefore, we question how useful this prevalence rate will\n actually be in determining the number of sample pieces to collect in redesigning FSIS\xe2\x80\x99 N-\n 60 sampling program.\n\n 2. What is the confidence level of the N-60 sampling method as currently performed? Is that\n level appropriate for ground beef [trim]?\n\n In FSIS\xe2\x80\x99 current sample of N-60, there is a 95 percent confidence level of detecting\n contamination when it is present in 5 percent of the lot. There is just a 45 percent\n confidence level of detecting E. coli O157:H7 contamination when it is present in only\n 1 percent of the lot\xe2\x80\x94in other words, if the contamination level is very low, FSIS is more\n likely to miss contamination than to detect it. Therefore, to maintain a high probability of\n detecting a pathogen with presumably as low a level of contamination as\n E. coli O157:H7, FSIS\xe2\x80\x99 sample size must increase. As discussed in the finding in this\n report, we are recommending that FSIS re-evaluate its sample parameters (size and\n confidence level) and redesign a sampling program for E. coli O157:H7 in beef trim that\n provides higher confidence in FSIS accurately detecting contaminated product in order to\n effectively verify process controls at beef processing plants across the nation.\n\n Audit Report 24601-09-KC 21\n\x0c 3. What is the definition of a product \xe2\x80\x9clot\xe2\x80\x9d used in the N-60 testing and how was this\n definition determined? How is this definition applied to individual processing\n establishments? Is this an appropriate application?\n\n FSIS allows processing plants to define a \xe2\x80\x9clot.\xe2\x80\x9d 20 In defining lots, plants may consider a\n wide range of factors, including (a) any scientific, statistically based sampling\n program that the establishment uses to distinguish between segments of production;\n (b) standard operating procedures or other prerequisite programs used to control\n E. coli O157:H7 cross-contamination between raw beef components during production;\n (c) processing interventions; and (d) beef trimmings and raw beef components or rework\n carried from one production period to another. In the past, plants were allowed to define\n a lot as the beef trim produced from one clean up to the next clean up\xe2\x80\x94often a full day\xe2\x80\x99s\n production. However, according to FSIS, clean up to clean up alone is no longer a\n supportable basis for defining a lot.\n\n According to FSIS officials, most large producers of beef trim define a lot as either one\n 2,000 pound combo-bin or multiple combo-bins (generally five combo-bins with a total\n weight of 10,000 pounds). 21 FSIS generally defines its sample lot to match the\n establishment\xe2\x80\x99s lot size. However, FSIS may permit an establishment that routinely\n samples its products under its own testing program to reduce its lot size on the day that\n FSIS conducts sampling, which allows it to hold less product until FSIS test results are\n returned. These plants must support their sampling program, define their lots\n microbiologically using sample data, and conduct robust sampling throughout the day to\n support lot sizes of less than one day\xe2\x80\x99s production.\n\n According to HACCP principles it is appropriate for companies to define their lot size\n because HACCP dictates that the establishments are themselves responsible for\n governing the factors that affect their production. Industry and FSIS officials are\n generally supportive of this principle. Under HACCP, the plant determines the lot size\n through its hazard analysis. Whatever the plant decides, FSIS expects that the plant uses\n a scientific basis to meet the premise of HACCP, and FSIS inspectors are expected to\n verify that they have done so.\n\n OIG agrees that a plant should use a scientific basis to support its defined lot size, and\n FSIS inspectors should verify it meets the premise of HACCP. We will observe and\n analyze this area further in our Phase II audit.\n\n\n\n\n20\n FSIS Directive 10,010.1, dated March 2010, states that \xe2\x80\x9c[t]he establishment is responsible for defining the\nsampled lot.\xe2\x80\x9d\n21\n A combo-bin for OIG\xe2\x80\x99s purposes is defined as a 48 inch x 40 inch x 40 inch size container of beef carcass\ntrimmings.\n\n Audit Report 24601-09-KC 22\n\x0c4. How does this definition affect the statistical viability of this test? How confident are we\n that lots testing negative are truly negative if the definition of \xe2\x80\x9clot\xe2\x80\x9d varies among\n establishments?\n\n The lot, from a statistical perspective, is the universe from which FSIS takes its 60\n samples. Since the lots are generally quite large, variability in their size has relatively\n little, if any, effect on the viability of testing. For example, if all other variables are\n constant, it makes little difference if one takes 60 samples from one 2,000 pound combo\n lot or a lot of five combos (total weight of 10,000 pounds).\n\n As discussed in our finding in this report, OIG questions the effectiveness of N-60\n sampling, but not due to problems with the size of the lot, or the universe. It is important\n to emphasize that no method of statistical sampling will result in absolute certainty that\n the lot is free of E. coli O157:H7 contamination, particularly since E. coli O157:H7 has\n such a low occurrence rate in final trim product and is not randomly dispersed throughout\n the lot.\n\n5. Are all samples collected, stored, shipped, and analyzed by trained FSIS employees?\n\n Trim samples are collected by employees from both FSIS and industry. FSIS personnel\n collect all of the pieces of trim FSIS needs for its regulatory samples and the industry\n employees collect their own trim samples for testing. FSIS officials stated that all its\n field employees including Inspectors in Charge (IIC), Frontline Supervisors (FLS) and\n Public Health Veterinarians (PHV) are trained to properly collect, store, and ship FSIS\n samples.\n\n FSIS\xe2\x80\x99 E. coli O157:H7 testing is performed at the agency\xe2\x80\x99s internationally accredited\n laboratories and the analysis is conducted by competent scientific personnel who have\n been certified to test for E. coli O157:H7.\n\n In Phase II of our audit, we will review how FSIS and the industry employees are trained\n to collect, store, ship, and analyze samples.\n\n6. Are laboratories instructed to reject samples that do not meet minimum quality\n standards? If so, what are those minimum quality standards? How is it assured each\n sample meets them?\n\n We confirmed that FSIS laboratories have developed internal procedures for discarding\n samples that do not meet minimum quality standards. When samples are received at the\n laboratory, to assure that each sample meets all the minimum quality standards, the\n laboratory technician inspects and documents the condition of the container, shipping\n seals, accompanying paperwork, and product. A temperature is then taken of the product\n and the sample information is entered into the automated laboratory tracking system,\n which includes any applicable information on why a sample was rejected. Laboratory\n procedures list over 30 reasons and sub-reasons why a sample might be rejected\n\n\n\nAudit Report 24601-09-KC 23\n\x0c (including temperature, target tissue not received, no form received with sample, sample\n security seal problem, and sample container leaking).\n\n In Phase II of our audit, we will review how the industry ensures its private labs are\n rejecting samples that do not meet minimum quality standards.\n\n 7. Are the minimum standards associated with sample collection, storage, shipment, and\n analysis adequate? What type of testing standards must be met by laboratories analyzing\n the samples?\n\n FSIS has provided direction to inspection personnel for such responsibilities as collecting\n and submitting samples and acting on a positive FSIS test result. FSIS laboratories must\n also follow procedures that outline the minimum standards that samples must meet and\n that address issues like test methods, the receipt of samples, logging samples in, shipping\n them, and training staff. During Phase II of our audit, we will examine whether FSIS\n inspection personnel are following FSIS sampling procedures and assess their adequacy.\n\n FSIS laboratories that analyze E. coli O157:H7 samples meet two accreditations. 22\n However, there is currently no legal or other requirement that FSIS laboratories obtain\n any accreditations. The first accreditation is by the International Standards Organization,\n which specifies the general requirements for a laboratory to demonstrate its competence\n to carry out tests. The requirements are applicable to all organizations that perform tests\n or calibrations. The second accreditation is the American Association for Laboratory\n Accreditation (A2LA), which is a public service membership organization dedicated to\n operating a nationwide, broad spectrum laboratory accreditation system. Further, we\n confirmed that A2LA conducts yearly external audits of FSIS\xe2\x80\x99 laboratories, and FSIS\n conducts its own internal audits to assure that the laboratories are following the\n accrediting body\xe2\x80\x99s, and the laboratory\xe2\x80\x99s, own policies and procedures.\n\n During Phase II of our audit, we will also review the standards industry follows\n associated with sample collection and laboratory analysis.\n\n 8. Because E. coli O157:H7 is not necessarily distributed homogeneously in a product, how\n did the agency decide on its current collection methodology? Are samples collected truly\n representative of the entire lot?\n\n FSIS officials realized that in selecting a robust trim sampling program they needed to\n consider the fact that E. coli O157:H7 is not homogeneously distributed. Research has\n shown that E. coli O157:H7 is most likely found on the surface tissue of the trim and\n does not reside in the inner tissue. Therefore, FSIS samples surface tissue in order to\n increase the likelihood that it will find the pathogen.\n\n Combo bins are often over three feet deep and due to concerns for the safety of their\n sample collectors, FSIS requires that inspectors sample only the top layers of trim. Based\n\n22\n \xe2\x80\x9cAccreditation\xe2\x80\x9d is a formal recognition of competence that a laboratory can perform specific tests.\n\n\n Audit Report 24601-09-KC 24\n\x0c on our discussions with agency and industry officials, as well as academics, we agree that\n so long as the bins are filled without prejudice, sampling from the top layers of trim\n should not impair the sample design. Samples gathered in this fashion can be considered\n as representative of the entire lot.\n\n 9. Has FSIS established a protocol for reassessing an establishment\xe2\x80\x99s HACCP plan based\n upon N-60 test results? What actions are taken at an establishment after a positive\n N-60 test result?\n\n FSIS is responsible for verifying that slaughter and processing establishments implement\n food safety systems that comply with HACCP regulations. FSIS has established\n procedures for what actions should be taken after an FSIS positive E. coli O157:H7 test,\n including reviewing the establishment\xe2\x80\x99s HACCP plan, by specially trained personnel\n during a Food Safety Assessment.\n\n After a positive FSIS test result, FSIS staffs are instructed to (1) collect supplier\n information; 23 (2) review establishment testing results; (3) verify disposition of affected\n product; (4) conduct followup sampling at the establishment and its suppliers; (5) conduct\n verification activities at the supplying establishment; (6) conduct a Food Safety\n Assessment 24 at the establishment, and (7) take enforcement actions if warranted.\n\n HACCP also requires that after a positive FSIS or industry test result, the establishment\n should use its own testing information to identify failures in its controls, take appropriate\n corrective actions, and reassess its HACCP plan, if applicable. For example, after a\n positive test result at one establishment, it was determined that an antimicrobial\n intervention had inadvertently been turned off and needed to be restarted. However,\n according to FSIS officials, there are circumstances in 9 Code of Federal Regulations\n 417.3 that if an establishment has already addressed E. coli O157:H7 in their HACCP\n plan, they may not have to reassess their plan in response to a positive test result.\n\n During Phase II of our audit, we will review how FSIS and the industry use positive N-60\n test results.\n\n 10. What is the implication of a positive N-60 test result on the individual lot tested? On\n other lots from the same establishment produced on the same date?\n\n\n\n\n23\n With the issuance of FSIS Notice 58-10, FSIS is now collecting supplier information at the time of sample\ncollection.\n24\n A comprehensive Food Safety Assessment considers all food safety aspects that relate to the establishment and its\nproducts, the nature and source of all materials received, the establishment\xe2\x80\x99s processes, and the environment of the\nestablishment. The Food Safety Assessment is designed to examine the validity of the Sanitation Standard\nOperating Procedures, pre-requisite programs, testing programs, and any other programs that constitute the\nestablishment\xe2\x80\x99s HACCP system.\n\n\n Audit Report 24601-09-KC 25\n\x0c FSIS requires lots with a positive FSIS or industry test result for E. coli O157:H7 to be\n destroyed or used in a product that has a validated lethality treatment, such as cooking. 25\n\n The implication for other lots produced at the same establishment and on the same date\n depends on the establishment\xe2\x80\x99s operations and materials used. When ground beef is\n produced, it may be manufactured from trim and other ground beef components obtained\n from one or multiple suppliers. If product from one lot was found positive for\n E. coli O157:H7, HACCP regulations require a plant to provide a scientific basis to\n justify why any raw ground product produced from these source materials should not be\n considered adulterated. If the plant cannot demonstrate how its final production lots are\n scientifically independent, then all final product lots produced using the same source\n materials would be implicated and subject to cooking, discard, or recall.\n\n 11. How are the FSIS N-60 test results and the establishment\xe2\x80\x99s N-60 test results correlated?\n How do N-60 testing protocols differ between FSIS and the industry?\n\n According to agency procedures, when FSIS finds product to be E. coli O157:H7\n positive, the inspection staff issues a noncompliance record, which requires the\n establishment to review its control procedures and take corrective actions. The only time\n a noncompliance report is not issued when FSIS finds a positive is when the company\n simultaneously conducted a test and its test also showed a positive result, in which case\n the product should have been destroyed or fully cooked. Establishments with a program\n to routinely divert all FSIS-tested product to cooking would not receive a noncompliance\n report, as well. Further, the inspectors are to ensure that the product was disposed of\n properly.\n\n In contrast, FSIS procedures do not allow for prompt correlation of plant testing results\n with FSIS test results or observation of disposition of E. coli O157:H7 positive product.\n According to FSIS officials, establishments are required to create a record of the positive\n in their HACCP system, in which FSIS has full access. However, plants are not required\n to notify FSIS of positive test results when they occur. FSIS procedures only require\n company test results be made available to FSIS inspectors, and that the results be\n reviewed by inspectors on a weekly basis.\n\n FSIS has established N-60 testing protocols for its inspectors in FSIS Directive 10,010.1.\n However, FSIS has not defined what constitutes agency-approved N-60 sampling\n methods and has no requirement that industry follow any specific methodologies or\n standards for collection, storage, or shipping. During Phase II of our audit, we will\n review how industry sampling and testing protocols vary among plants and differ from\n FSIS standards.\n\n 12. What are the implications of the USDA label associated with a negative N-60 test?\n\n\n\n25\n Directive 10,010.1, March 2010 notes this requirement in FSIS tested product.\n\n\n Audit Report 24601-09-KC 26\n\x0c The purpose of providing a USDA-approved testing label on product containers would be\n to give receiving establishments information regarding the testing of raw ground beef\n components for the presence of E. coli O157:H7. Industry and FSIS have differing\n thoughts on the issues surrounding labeling of N-60 trim. Industry officials believe that\n this label will provide receiving establishments with necessary information regarding the\n testing of raw ground beef components in lieu of a Certificate of Analysis (COA) when a\n COA does not accompany tested products sold through a distributor. FSIS officials\n believe grinders would assume that there is no risk in any labeled trim they are buying,\n and FSIS does not want to encourage this assumption since grinders have an obligation to\n sell final product that is not contaminated with E. coli O157:H7.\n\nDuring Phase II of our audit, we will review if and how industry is currently labeling FSIS N-60\ntested trim.\n\n\n\n\n Audit Report 24601-09-KC 27\n\x0cAppendix B: Update to January 2008 Memorandum to Former\nDeputy Secretary Conner on E. coli Sampling and Testing\nWe followed up on matters reported in our memorandum, \xe2\x80\x9cFood Safety and Inspection Service\nSampling and Testing of E. coli,\xe2\x80\x9d issued January 29, 2008, related to examining the adequacy\nand effectiveness of FSIS\xe2\x80\x99 N-60 sampling method. In the memorandum, we reported that\n(1) FSIS needs to gather a more representative sample; (2) FSIS needs to minimize or eliminate\ndiscarded samples; and (3) FSIS needs to reduce turnaround time on laboratory results. To\naddress our objective, we conducted interviews with USDA and non-USDA officials and\nreviewed FSIS documents and data produced since our January 29, 2008, letter.\n\nAccording to industry experts, academics, and USDA officials, large producers have moved\ntowards taking multiple samples from the same ground beef lot at intervals during the day, some\nas often as every 15 minutes. In addition, in July 2010, the Agricultural Marketing Service\nstrengthened its National School Lunch Program purchase specifications for ground beef to\ninclude increasing microbiological sampling frequency for finished products to every 15 minutes\nand instituting additional rejection criteria for source trimmings. We believe that these changes\nshould result in plants gathering a more representative sample, but we will not reach any\nconclusions until we have an opportunity to visit plants during Phase II of our audit.\n\nFSIS has also taken steps to minimize the discarding of samples by documenting the reasons for\ndiscards and sending a monthly report to the districts for a followup review. An analysis of\ndiscard rates conducted by OIG for calendar years 2008, 2009, and part of 2010 found that there\nhas been a significant decrease in the number of samples discarded by FSIS\xe2\x80\x99 laboratories. 26 In\naddition, officials expressed confidence that the new sample reservation system in FSIS\xe2\x80\x99 Public\nHealth Information System (PHIS) will correct the problem of uncollected samples. When PHIS\nis implemented, FSIS needs to monitor the system to assure that PHIS will minimize or eliminate\ndiscard samples and help the agency to reach its E. coli O157:H7 sampling goals.\n\nFSIS continues to experience delays in returning testing results to plants. We will followup in\nthe next phase of this audit to determine whether FSIS or industry have identified any potential\nsolutions for reducing these delays at this time.\n\nOverall, FSIS sees its testing program as a means for ensuring that plants take adequate safety\nprecautions. From the agency\xe2\x80\x99s perspective, in order to ensure food safety, plants must\nimplement effective and supportable HACCP systems including microbiological intervention. In\naddition, consumers must be educated about properly cooking ground beef. During the course of\nour review, we found that FSIS has made a number of improvements to its E. coli O157:H7\nsampling and testing program to accomplish these intended program objectives, including such\nthings as:\n\n\n\n26\n Our analysis shows that for the discard categories selected by OIG, there was a 40 percent drop in discards from\n2008 to 2009 levels (865 to 516 discards) and indications of a deeper drop in 2010 (showing only 132 discards as of\nMay 2010).\n\n\n Audit Report 24601-09-KC 28\n\x0c (1) Updating guidance concerning trim sampling, followup sampling, tracing product back to\n the source of the contamination, and conducting food safety assessments;\n\n (2) Analyzing the results of controls used by industry to control E. coli O157:H7 in beef\n operations (based in part on this information, FSIS issued a best practices document);\n\n (3) Participating in the President\xe2\x80\x99s Food Safety Working Group, which is advising on how to\n improve the U.S. food safety system for the twenty-first century; and\n\n (4) Expanding its mode of food safety outreach to include Twitter, Facebook, and YouTube.\n The FSIS twitter feed, USDAFoodSafety, grew from 751 followers in 2009 to more than\n 55,000 in 2010.\n\nDuring the course of our review we also identified that the industry is researching technology to\nreduce E. coli O157:H7 contamination to include developing more effective interventions to\nreduce or eliminate E. coli O157:H7 and developing vaccines to control E. coli O157:H7 in\nliving animals.\n\nWe will assess the impact of these improvements in Phase II of our audit while performing steps\nto (1) observe the collection of beef trim samples by FSIS and employees at slaughter and\nprocessing establishments; (2) analyze how the beef industry\xe2\x80\x99s sampling and testing protocols\nvary among plants and compare to FSIS standards; (3) review the quality of private laboratory\nservices provided to the beef industry for testing and discarding samples; and (4) examine how\ntest results are used by FSIS and the beef industry to improve food safety.\n\n\n\n\n Audit Report 24601-09-KC 29\n\x0cAbbreviations\n\nA2LA .......................... American Association for Laboratory Accreditation\nCCP ............................. Critical Control Point\nCOA ............................ Certificate of Analysis\nFLS .............................. Front Line Supervisor\nGMP ............................ Good Manufacturing Practices\nFSIS............................. Food Safety and Inspection Service\nHACCP ....................... Hazard Analysis and Critical Control Points\nHAV ............................ Hazard Analysis Verification\nIIC ............................... Inspector-In-Charge\nPHIS ............................ Public Health Information System\nPHV............................. Public Health Veterinarian\nOIG ............................. Office of Inspector General\nSOP ............................. Standard Operating Procedure\nUSDA.......................... United States Department of Agriculture\n\n\n\n\n Audit Report 24601-09-KC 30\n\x0cAgency\xe2\x80\x99s Response\n\n\n\n\n USDA\xe2\x80\x99S\n\n\n\nFOOD SAFETY AND INSPECTION SERVICE\n\n\n\n RESPONSE TO AUDIT REPORT\n\n\n\n\n Audit Report 24601-09-KC 31\n\x0c United States Food Safety Washington, D.C.\n Department of and Inspection 20250\n Agriculture Service\n\n\n TO: Gil Harden\n Assistant Inspector General for Audit\n Office of Inspector General\n\n FROM: Alfred V. Almanza /s/ January 28, 2011\n Administrator\n Food Safety and Inspection Service\n\nSUBJECT: Office of Inspector General (OIG) Official Draft Audit Report \xe2\x80\x93 Food Safety and\n Inspection Service N-60 Testing Protocol for E. coli O157:H7 \xe2\x80\x93 Phase I, Report\n number 24601-09-KC\n\n\nThe Food Safety and Inspection Service (FSIS) has reviewed this report and responded to each\nof the recommendations. We also have provided some general comments on the audit report\nfindings, which clarify our proposed actions in response to the recommendations. We\nappreciate OIG\xe2\x80\x99s efforts to accurately characterize the N-60 sampling program and to provide\ncreative recommendations that give FSIS the latitude needed to improve it.\n\nGeneral Comments\n\nAs noted in the report, sample collection and analysis is only one of the HACCP verification\nactivities provided for under the regulations in 9 CFR 417. And specifically, FSIS sees N-60\nsampling of beef trim as only one of a number of verification activities that FSIS conducts\nregarding establishment process controls for E. coli O157:H7. FSIS sampling of beef trim works\nalong with inspection and other verification activities, including FSIS sampling of ground beef\nand other ground beef components and the review of establishment testing results, to detect\nand reduce E. coli O157:H7 in beef products.\n\nIn response to the Congressional requestor, the audit report focuses on the statistical design\nand outcome of discrete N-60 sampling events. We do not dispute the statistics presented in\nthe report on this subject, but emphasize, as stated above, that a single N-60 sampling result is\nnot viewed apart from other verification activities. Note that along with sampling and carcass-\nby-carcass inspection, more than 500,000 inspection procedures were performed in CY2010 at\nroughly 650 slaughter establishments that would also be subject to trim sampling. These\ninspection procedures, performed daily at slaughter establishments, play an important role in\nensuring establishments are producing safe and wholesome products.\n\nFurther, to detect changes over time, an N-60 result often is analyzed along with not only other\nverification data but also other N-60 results. This is especially true among establishments who\nthemselves take numerous N-60 samples over time as part of a statistical process control\nprogram. Notably, FSIS has recommended that industry take numerous, regular N-60 samples\nas part of a statistical process control program since 2008.1 Thus, although a single N-60\nsample result may not indicate definitively the success or failure of an establishment\xe2\x80\x99s process\n\n\n1\n Compliance Guideline for Sampling Beef Trimmings for Escherichia coli O157:H7, Draft for Stakeholder Comment, FSIS, August\n12, 2008.\n\n\n\nFSIS Form 2630-9 (6/86) EQUAL OPPORTUNITY IN EMPLOYMENT AND SERVICES\n\x0c 2\n\n\ncontrols for beef trim, it can be an important component of process control and verification\nprograms, especially if the establishment or FSIS has taken multiple N-60 samples over time.\n\nIn regard to the recommendation that FSIS determine a national prevalence, FSIS agrees that\nthis is important information for properly designing a sampling program.2 However, a national\nprevalence estimate is not sufficient information to determine how to collect a sample from a lot,\nowing to the distinction between determining how many lots to test and how to collect a sample\nfrom each lot. In other words, prevalence data could inform how many lots to test nationwide,\nbut not how to collect a sample from each lot. A sampling program, such as FSIS\xe2\x80\x99 trim sampling\nprogram, is a different concept than a sample collection method, such as N-60.\n\nOur responses to the recommendations should be read in light of this discussion. Although the\naudit report acknowledges FSIS\xe2\x80\x99 approach of using sampling and inspection to verify an\nestablishment\xe2\x80\x99s food safety plan and to ensure the safety of the food supply, the report does not\nconsider the combined effect of these activities and instead considers sampling in isolation.\nThe audit report criticizes FSIS\xe2\x80\x99 trim sampling program for having insufficient statistical power to\ndetect contaminated product with high confidence. FSIS\xe2\x80\x99 mission is not to screen the food\nsupply through testing, but to ensure production of safe and wholesome food through\ninspection.\n\nIn response to the recommendations, needed improvements to FSIS verification of process\ncontrols for E. coli O157:H7 in beef trim and other ground beef components will include changes\nto multiple inspection and analytical activities, focusing on the improved collection and\nintegration of various data to better identify establishments with process control problems.\nFurther, FSIS agrees that a plan for conducting additional baselines for beef trim and other\nproducts is necessary, but these baselines also will be used to inform FSIS verification of\nestablishment food safety systems over time, rather than to design an acceptance testing\nmethod for individual lots of product.\n\nRecommendation 1\n\nDevelop a plan with specific timeframes and milestones to prioritize and perform necessary\nbaseline studies of beef trim and ground beef in order to determine the estimated prevalence\nrate of E. coli O157:H7 for redesigning FSIS\xe2\x80\x99 sampling program. The plan should include how\noften the prevalence should be reassessed.\n\nAgency Response\n\nFSIS will develop a plan for prioritizing and performing E. coli O157:H7 baseline studies of beef,\nincluding ground beef, trim and other ground beef components, for the purpose of informing and\nimproving overall FSIS verification programs for control of the pathogen. FSIS will publish the\nplan for comment before it is made final.\n\nCompletion Date: December 2011\n\n\n\n\n2\n FSIS recently published the national prevalence estimate of pathogen contamination of trim based on the 2005-07 beef trim\nbaseline study: http://www.fsis.usda.gov/PDF/Baseline_Data_Domestic_Beef_Trimmings_Rev.pdf.\n\x0c 3\n\n\nRecommendation 2\nRe-evaluate its sample parameters (size and confidence level) and redesign a sampling\nprogram for E. coli O157:H7 in beef trim that provides higher confidence in FSIS accurately\ndetecting contaminated product in order to effectively verify process controls at beef processing\nplants across the nation.\n\nAgency Response\n\nFSIS will evaluate its sampling program in the context of prevalence data and inspection\nverification data. FSIS believes that effective verification of food safety systems must include\ninspection of an establishment\xe2\x80\x99s production process to ensure that controls are adequately\ndesigned and implemented to prevent, eliminate or reduce contamination with E. coli O157:H7.\nTo improve inspection program personnel verification of process controls for the reduction of E.\ncoli O157:H7 at both beef slaughter and processing establishments, FSIS will be taking the\nfollowing actions during 2011. Together, these actions will give inspection program personnel\ncomprehensive, aggregate data on establishment process controls for E. coli O157:H7.\n\n \xc2\xb7 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n \xc2\xb7 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef\n or bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following\n a confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n \xc2\xb7 FSIS will implement a new inspection procedure, the Hazard Analysis Verification\n (HAV), which will examine an establishment\xe2\x80\x99s hazard analysis and decision making\n documents. The HAV will serve as a screening process to identify issues of concern in\n the design of the establishments\xe2\x80\x99 food safety systems. HAVs will be performed at\n specified frequencies and when certain information prompts identify establishments that\n may have modified their HACCP system or may have failed to change their HACCP\n plan in response to a significant change in the establishments\xe2\x80\x99 process. A HAV will\n verify that the establishment\xe2\x80\x99s hazard analysis addresses the applicable food safety\n hazards for the process, product, and intended use. It will also verify that there is at\n least one CCP for the hazards that are reasonably likely to occur and that there is\n support for any decisions that applicable hazards are not reasonably likely to occur. It\n will also verify that any prerequisite programs (such as an SOP, GMP, purchase\n specifications, etc.) which are intended to prevent the occurrence of the food safety\n hazard are adequately justified and properly implemented.\n\nCompletion Date: December 2011\n\x0c 4\n\n\nRecommendation 3\nThoroughly document the scientific support and rationale for FSIS\xe2\x80\x99 revised E.coli O157:H7\nsampling program design, including the contamination level that will be associated with the new\nsample parameters and how the estimated prevalence rate impacts the new design. Publish\nFSIS\xe2\x80\x99 revised E. coli O157:H7 sampling design with its support and rationale for public\ncomment, and consider any recommended changes before implementing the new sampling\nprogram.\n\nAgency Response\n\nFSIS will develop a plan for prioritizing and performing E. coli O157:H7 baseline studies of beef,\nincluding ground beef, trim and other ground beef components, for the purpose of informing and\nimproving FSIS verification programs for control of the pathogen. FSIS will publish the plan for\ncomment before it is made final.\n\nFSIS also will be taking the following actions in 2011 (as stated above in response to\nRecommendation #2) to improve inspection program personnel verification of food safety\nsystems for the prevention, elimination or reduction of E. coli O157:H7 at both raw beef\nslaughter and processing establishments. Together, these actions will give inspection program\npersonnel comprehensive, aggregate data on establishment process controls for E. coli\nO157:H7.\n\n \xc2\xb7 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n \xc2\xb7 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef\n or bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following a\n confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n \xc2\xb7 FSIS will implement a new inspection procedure, the Hazard Analysis Verification (HAV),\n which will examine an establishment\xe2\x80\x99s hazard analysis and decision making documents.\n The HAV will serve as a screening process to identify issues of concern in the design of\n the establishments\xe2\x80\x99 food safety systems. HAVs will be performed at specified\n frequencies and when certain information prompts identify establishments that may have\n modified their HACCP system or may have failed to change their HACCP plan in\n response to a significant change in the establishments\xe2\x80\x99 process. A HAV will verify that\n the establishment\xe2\x80\x99s hazard analysis addresses the applicable food safety hazards for the\n process, product, and intended use. It will also verify that there is at least one CCP for\n the hazards that are reasonably likely to occur and that there is support for any decisions\n that applicable hazards are not reasonably likely to occur. It will also verify that any\n\x0c 5\n\n\n prerequisite programs (such as an SOP, GMP, purchase specifications, etc.) which are\n intended to prevent the occurrence of the food safety hazard are adequately justified and\n properly implemented.\n\nCompletion Date: December 2011\n\nRecommendation 4\nDevelop a detailed operational plan with specific timeframes and milestones to implement an\ninspection system that focuses E. coli O157:H7 sampling and testing resources primarily at\nplants that are likely to be of higher risk and consider the use of specialized sample collection\nteams.\n\nAgency Response\n\nFSIS will evaluate its sampling program in the context of prevalence data and inspection\nverification data. FSIS believes that effective verification of food safety systems must include\ninspection of an establishment\xe2\x80\x99s production process to ensure that controls are adequately\ndesigned and implemented to prevent, eliminate or reduce contamination with E. coli O157:H7.\nTo improve inspection program personnel verification of process controls for the reduction of E.\ncoli O157:H7 at both beef slaughter and processing establishments, FSIS will be taking the\nfollowing actions during 2011. Together, these actions will give inspection program personnel\ncomprehensive, aggregate data on establishment process controls for E. coli O157:H7.\n\n \xc2\xb7 FSIS will revise Directive 6410.1, Verifying Sanitary Dressing and Process Control\n Procedures in Slaughter Operations of Cattle of Any Age, with a special emphasis on\n improving and clarifying verification of sanitary dressing, given that contamination of\n carcasses is the primary cause of ground beef component adulteration by E. coli\n O157:H7.\n \xc2\xb7 FSIS will issue a new Notice regarding traceback methodology for E. coli O157:H7 in\n ground beef components after FSIS or other Federal or State agencies find ground beef\n or bench trim tests presumptive positive for E. coli O157:H7. This Notice will provide\n instructions concerning how to conduct product traceback from the grinder or bench trim\n establishment and what to do at originating supplier slaughter establishments following\n a confirmed positive for E. coli O157:H7 in ground beef, including assessing whether\n evidence of a high event period may have occurred and how to manage information\n gathered during a product traceback.\n \xc2\xb7 FSIS will implement a new inspection procedure, the Hazard Analysis Verification\n (HAV), which will examine an establishment\xe2\x80\x99s hazard analysis and decision making\n documents. The HAV will serve as a screening process to identify issues of concern in\n the design of the establishments\xe2\x80\x99 food safety systems. HAVs will be performed at\n specified frequencies and when certain information prompts identify establishments that\n may have modified their HACCP system or may have failed to change their HACCP\n plan in response to a significant change in the establishments\xe2\x80\x99 process. A HAV will\n verify that the establishment\xe2\x80\x99s hazard analysis addresses the applicable food safety\n hazards for the process, product, and intended use. It will also verify that there is at\n least one CCP for the hazards that are reasonably likely to occur and that there is\n\x0c 6\n\n\n support for any decisions that applicable hazards are not reasonably likely to occur. It\n will also verify that any prerequisite programs (such as an SOP, GMP, purchase\n specifications, etc.) which are intended to prevent the occurrence of the food safety\n hazard are adequately justified and properly implemented.\n\nCompletion Date: December 2011\n\x0c'