jurisdiction: US

text: b' OFFICE OF INSPECTOR GENERAL \n\n\n\n\n Catalyst for Improving the Environment\n\n\nEvaluation Report\n\n\n\n\n Limited Knowledge of the Universe\n of Regulated Entities Impedes\n EPA\xe2\x80\x99s Ability to Demonstrate\n Changes in Regulatory Compliance\n\n Report No. 2005-P-00024\n\n\n September 19, 2005\n\x0cReport Contributors: Erin Barnes-Weaver\n Kim Bryant\n Manju Gupta\n Jeff Hart\n Ben Webster\n\n\n\n\nAbbreviations\n\nCAA Clean Air Act\nCWA Clean Water Act\nECHO Enforcement and Compliance History Online\nEPA Environmental Protection Agency\nFIFRA Federal Insecticide, Fungicide, and Rodenticide Act\nGAO Government Accountability Office\nNPDES National Pollutant Discharge Elimination System\nOECA Office of Enforcement and Compliance Assurance\nOIG Office of Inspector General\nRCRA Resource Conservation and Recovery Act\nSDWA Safe Drinking Water Act\nTSCA Toxic Substances Control Act\n\x0c U.S. Environmental Protection Agency 2005-P-00024\n\n Office of Inspector General September 19, 2005\n\n\n\n\n At a Glance \n\n Catalyst for Improving the Environment\n\n\nWhy We Did This Review Limited Knowledge of the Universe of\n Regulated Entities Impedes EPA\xe2\x80\x99s Ability to\nTo enforce its regulations and\nachieve maximum compliance, Demonstrate Changes in Regulatory Compliance\na regulatory agency must know\nits entire regulated universe. What We Found\nWe sought to determine how\nwell the U.S. Environmental OECA has limited knowledge of the diverse regulated universe for which it\nProtection Agency (EPA) maintains responsibility. OECA has not updated its universe table since\nOffice of Enforcement and generating it in 2001, even though some universe figures for reviewed program\nCompliance Assurance\n areas have changed substantially. EPA has used the 2001 table as a source for\n(OECA) knows the\ncomposition and size of its describing the size of its regulated universe in public documents. Various data\nregulated universe, as well as quality issues impact OECA\'s ability to adequately identify the size of its\nhow OECA determines and regulated universe and associated compliance information. OECA concentrates\nreports compliance levels most of its regulatory activities on large entities and knows little about the\nacross the regulated universe. identities or cumulative impact of small entities. OECA cannot effectively use\n universe figures to assist with its regulatory activities. OECA does not develop\nBackground programmatic compliance information, adequately report on the size of the\n universe for which it maintains responsibility, or rely on universe figures to assist\nOECA compiled its regulated with planning.\nuniverse table to provide\nconsistent numbers when OECA\xe2\x80\x99s limited universe knowledge prevents it from determining overall\npresenting compliance compliance levels in five of the six regulatory program areas we reviewed. This\ninformation to Congress, the hinders OECA\xe2\x80\x99s ability to generate valid programmatic compliance information\npublic, and other stakeholders. and effectively determine program success. In addition, OECA lacks adequate\nThe information also aids EPA transparency in publicly reporting some currently available compliance\nin making management information.\ndecisions about compliance and\nenforcement resource What We Recommend\nallocations. In the universe\ntable issued in September 2001, We recommend that OECA biannually update publicly released universe figures,\nOECA reported an inventory of and produce complete, reasonably accurate, and current universe data. Further,\napproximately 41.1 million OECA should better describe its enforcement and compliance role, develop an\nregulated entities. objective to obtain better reporting from States, and request EPA program offices\n to analyze and report on the cumulative impact of violations from small entities.\nFor further information, contact Also, we recommend that OECA develop and publish information that\nour Office of Congressional and demonstrates changes in compliance levels, and better share existing compliance\nPublic Liaison at (202) 566-2391.\n data and analyses that will provide external stakeholders with an improved\nTo view the full report, understanding of programmatic compliance levels. EPA agreed with some of our\nclick on the following link: recommendations, but not those related to biannually updating universe figures,\n developing an objective to obtain better reporting from States, or for developing\nwww.epa.gov/oig/reports/2005/ programmatic compliance information.\n20050919-2005-P-00024.pdf\n\x0c UNITED STATES ENVIRONMENTAL PROTECTION AGENCY\n WASHINGTON, D.C. 20460\n\n\n\n OFFICE OF \n\n INSPECTOR GENERAL\n\n\n\n\n\n September 19, 2005\n\nMEMORANDUM\n\nSUBJECT: Limited Knowledge of the Universe of Regulated Entities Impedes\n EPA\xe2\x80\x99s Ability to Demonstrate Changes in Regulatory Compliance\n Report No. 2005-P-00024\n\nFROM: Jeffrey K. Harris /s/\n Director for Program Evaluation, Cross-Media Issues\n\nTO: Granta Y. Nakayama\n Assistant Administrator\n Office of Enforcement and Compliance Assurance\n\n\nThis is our final report on the subject evaluation conducted by the Office of Inspector General\n(OIG) of the U.S. Environmental Protection Agency (EPA). This evaluation report contains our\nfindings that describe the problems we have identified and corrective actions we recommend.\nThis evaluation report represents the opinion of the OIG and the findings contained in this report\ndo not necessarily represent the final EPA position. EPA managers will make final\ndeterminations on matters in this report in accordance with established procedures.\n\nWe met with Office of Enforcement and Compliance Assurance managers on June 13, 2005, to\ndiscuss our preliminary findings, and provided our draft report on June 30, 2005. EPA did not\nconcur with all of our recommendations. EPA provided its official written comments in two\ndocuments. We have included EPA\xe2\x80\x99s summary memorandum response in its entirety as\nAppendix D. EPA\xe2\x80\x99s second document \xe2\x80\x93 the Agency\xe2\x80\x99s response, including detailed comments on\nour recommendations \xe2\x80\x93 is available on OIG\xe2\x80\x99s Web site, with the report. We are also including\nlists of publicly available compliance and enforcement measures, provided to us in EPA\xe2\x80\x99s\nresponse, in Appendix F.\n\x0cAction Required\n\nEPA Manual 2750 requires you as the action official to provide this office with a written\nresponse to this report within 90 calendar days of the final report date. Your response should\naddress all recommendations and must include your concurrence or nonconcurrence with all\nrecommendations. For corrective actions planned but not completed by the response date, please\ndescribe the actions that are ongoing and provide a timetable for completion. If you do not\nconcur with a recommendation, please provide alternative actions addressing the findings\nreported. For your convenience, this report will be available at http://www.epa.gov/oig/.\n\n\nAttachment\n\n\ncc: \t Phyllis Harris, Principal Deputy Assistant Administrator, OECA\n Michael M. Stahl, Director, Office of Compliance, OECA\n Walker B. Smith, Director, Office of Civil Enforcement, OECA\n Greg Marion, Audit Followup Coordinator, OECA\n\x0c Table of Contents \n\nAt a Glance\n\n\nChapters\n 1\t Introduction ........................................................................................................... 1\n\n Purpose .......................................................................................................... 1\n Background .................................................................................................... 1\n Scope and Methodology ................................................................................ 3\n\n\n 2\t OECA Has Limited Knowledge of the Regulated Universe ............................... 6\n\n Universe Table Not Updated Since 2001 Despite Significant Changes ......... 6\n\n Data Quality Problems Affect OECA\xe2\x80\x99s 2001 Universe Figures ...................... 7\n\n OECA Could More Clearly Describe Its Role ................................................. 10 \n\n OECA Does Not Focus Its Compliance Monitoring and Enforcement \n\n Activities on a Greater Part of the Regulated Universe ........................ 11 \n\n OECA Does Not Know the Cumulative Impact of Small Entities.................... 14 \n\n Recommendations ......................................................................................... 15\n Agency Comments and OIG Evaluation......................................................... 15 \n\n\n 3\t OECA Cannot Determine Changes in Compliance for Five of Six \n\n Sample Program Areas......................................................................................... 16\n\n OECA Cannot Generate Programmatic Compliance Information\n for Five of Six Program Areas .............................................................. 16 \n\n OECA Is Not Sufficiently Transparent with Available Compliance Data......... 19 \n\n Recommendations ......................................................................................... 20\n Agency Comments and OIG Evaluation......................................................... 20 \n\n\n\nAppendices\n A OECA\xe2\x80\x99s Regulatory Universe Identification Table \xe2\x80\x93 Executive Summary ....... 22 \n\n B Environmental Statute Information and Program-Specific \n\n Facility Size Classifications ................................................................................. 26 \n\n C Details on Scope and Methodology..................................................................... 28 \n\n D Agency Response Memorandum to Draft Report .............................................. 32\n\n E Agency Comments to OIG Recommendations and OIG Evaluation ................ 36\n\n F Publicly Available Compliance and Enforcement Measures ............................ 43 \n\n G Distribution ............................................................................................................ 48\n\n\x0c Chapter 1\n Introduction\nPurpose\n To enforce its regulations and achieve maximum compliance, a regulatory agency\n must know its entire regulated community. This knowledge enables an agency to\n make more informed management decisions about compliance and enforcement\n resource allocations, and provides Congress and the public with data on whether\n compliance is increasing. Further, if entities realize they are unknown to a\n regulatory agency, they may be less likely to comply with environmental\n regulations, and enforcement actions by the U.S. Environmental Protection\n Agency (EPA) may be unlikely to occur.\n\n The EPA is responsible for implementing a large number of environmental\n regulations over a diverse universe of regulated entities of differing size, nature of\n operations, and complexity. To evaluate the effectiveness of EPA\xe2\x80\x99s enforcement\n and compliance efforts, we plan to evaluate several interrelated issues. This\n report addresses the nature and composition of the regulated universe, and\n answers the following questions:\n\n \xe2\x80\xa2\t How well does EPA\xe2\x80\x99s Office of Enforcement and Compliance Assurance\n (OECA) know the composition and size of its regulated universe?\n\n \xe2\x80\xa2\t How does OECA determine and report levels of compliance in the regulated\n universe?\n\n In subsequent reports, we plan to address how to best measure compliance status,\n determine EPA\xe2\x80\x99s tools and strategies for improving compliance, and consider the\n changes that occur as a result of those tools and activities.\n\nBackground\n Importance of Knowing the Regulated Universe\n\n Knowledge of the size and character of the regulated community is fundamental\n to a regulatory agency\xe2\x80\x99s effectiveness. Identifying the regulated universe enables\n regulatory agencies to develop effective enforcement and compliance strategies,\n as well as establish deterrence among the regulated community. Current and\n complete universe information facilitates evaluation, highlights the scope of\n responsibilities, increases transparency, and improves the ability to manage\n enforcement and compliance efforts.\n\n\n\n 1\n\n\x0c We developed the following list of the benefits of a known universe based on\n interviews with OECA staff and reviews of environmental policy literature\n (see Table 1-1):\n\n Table 1-1: Benefits of Knowing the Full Regulated Universe\n\nEvaluation\t Current and complete universe figures can allow for improved internal and external\n analysis and reporting of the impact of a regulatory agency\xe2\x80\x99s activities.\n\nScope\t Universe data discloses the size and nature of a regulatory agency\xe2\x80\x99s\n responsibilities. Knowledge of the universe provides the agency with a definitive\n baseline on the number, size, location and character of entities subject to particular\n regulations and statutes. This is especially important as entities shut down, start up,\n move, or change their operations.\n\nTransparency\t EPA has a longstanding commitment to transparency \xe2\x80\x93 to publicly account for its\n decisions and explain why it took or did not take certain actions. During his\n confirmation hearing, current EPA Administrator Stephen Johnson listed one of his\n guiding principles as pursuing \xe2\x80\x9cas open and transparent a decision making process\n as possible.\xe2\x80\x9d Increased information disclosure allows external stakeholders to\n better understand a regulatory agency\xe2\x80\x99s activities. Since the passage of the\n Emergency Planning and Community Right-to-Know Act in 1986, EPA has used\n information disclosure as a means to improve environmental performance. OECA\n documents note how continued and increased release of information keeps with the\n Administrator\xe2\x80\x99s emphasis on greater transparency. Public disclosure also holds the\n Agency accountable for all lessons learned, including both successes and failures.\n\nManagement\t Current and complete universe information can assist with the following\n management activities:\n \xe2\x80\xa2\t Targeting: Updated universe data provides information on how many entities\n can and will be affected by regulatory agencies\xe2\x80\x99 enforcement and compliance\n activities, and thus enables better targeting strategies.\n \xe2\x80\xa2\t Retrospective Analyses: Comprehensive regulated universe information can\n assist a regulatory agency in conducting retrospective analyses on the success\n of their enforcement and compliance activities in certain program areas.\n \xe2\x80\xa2\t Rule-making: OECA staff said EPA has used universe data in regulatory\n decisions and analyses.\n \xe2\x80\xa2\t Priority-Setting: Knowledge of the regulated universe enables regulatory\n agencies to more easily establish defensible priorities in focusing resources.\n\n\n\n OECA\xe2\x80\x99s 2001 Universe Table\n\n In September 2001, OECA compiled figures detailing the size of the regulated\n universe for which it was responsible. OECA produced its 2001 Regulatory\n Universe Identification Table (see Appendix A) after receiving criticism for not\n having an adequate knowledge of its regulated universe. According to senior\n\n\n\n 2\n\n\x0c OECA managers, OECA also generated the universe figures to provide a single\n definitive source for internal briefings and external presentations. For example:\n\n \xe2\x80\xa2\t Prior to 2001, OECA staff cited a variety of inconsistent numbers when\n discussing the regulated universe. During congressional budget hearings,\n OECA could not provide consistent accounts of its regulated universe.\n\n \xe2\x80\xa2\t In a July 2001 report,1 the Government Accountability Office (GAO) stated\n OECA could not demonstrate \xe2\x80\x9cthe universe of entities subject to regulation\n under federal environmental laws.\xe2\x80\x9d GAO also noted OECA could not\n adequately explain variations in enforcement activity across regions, or how it\n distributed resources and determined priorities.\n\n The 2001 Regulatory Universe Identification Table shows that OECA\xe2\x80\x99s universe\n totaled approximately 41.1 million entities. OECA developed the table by using\n Agency databases and consulting 36 sources for the 58 program areas included in\n the table. In developing the table, OECA produced a methodology that could be\n used to generate future universe data. Staff in the Office of Compliance\n coordinated with the Office of Regulatory Enforcement (now the Office of Civil\n Enforcement) and program offices to obtain numbers for universe program areas.\n OECA staff asked relevant program staff to provide program descriptions, a data\n source to generate the universe number, and any data caveats. OECA staff\n emphasized that a source, database, or document had to support the universe\n number used in the table; institutional knowledge or a \xe2\x80\x9cgut\xe2\x80\x9d feeling were not\n adequate. OECA staff described the methodology as a resource-intensive process.\n Since 2001, EPA has described the number of regulated entities as approximately\n 41 million based on OECA\xe2\x80\x99s universe table. For example, EPA cited this figure\n in public documents, such as the EPA Strategic Plan 2003-2008.\n\nScope and Methodology\n We conducted our evaluation field work on EPA\xe2\x80\x99s knowledge of its regulated\n universe and compliance status between January and April 2005. We generally\n performed our evaluation in accordance with Government Auditing Standards,\n issued by the Comptroller General of the United States.\n\n To answer both the universe and compliance objectives, we judgmentally selected\n six sample program areas, as case studies, from OECA\xe2\x80\x99s 2001 Regulatory\n Universe Identification Table. We based this sample on information gathered\n from our preliminary research. An initial analysis of the Regulatory Universe\n Identification Table showed that OECA\xe2\x80\x99s universe consisted of a total of\n approximately 41 million entities. These included a diverse and complex mix of\n entities of varying sizes and types. Small entities made up a much greater part of\n\n1\n GAO-01-812, Human Capital: Implementing an Effective Workforce Strategy Would Help EPA to Achieve Its\nStrategic Goals, July 2001.\n\n\n 3\n\n\x0cthe universe than major and large entities. A review of OECA\xe2\x80\x99s planning\ndocuments also showed that a handful of programs were selected as national\npriorities, and others were described as core programs. While OECA is ultimately\nresponsible for regulating all 41 million entities, it focuses its regulatory attention\non national priority program areas, while the States are primarily responsible for\ncore programs. To conduct a balanced evaluation of OECA\xe2\x80\x99s knowledge of the\nregulated universe, it was important to select a sample that reflected the mix of\nprogram areas for which OECA is both ultimately responsible and it had\nrepresented as falling under its regulatory authority in its Regulatory Universe\nIdentification Table.\n\nWe selected the sample program areas using the following criteria:\n\n\xe2\x80\xa2\t Program areas from across major environmental statutes;\n\xe2\x80\xa2\t Program areas including a mix of large and small entities because OECA\n included both groups in its overall universe count; and\n\xe2\x80\xa2\t A mix of national priority and core programs.\n\nApplying these criteria, we selected the sample program areas listed in Table 1-2.\nWe included details on the applicable statutes, program areas, and distinctions\nbetween the different sizes of entities in Appendix B.\n\n Table 1-2: Environmental Statutes and Sample Program Areas\n\n Statute Program Area\n Clean Air Act (CAA) Minor Stationary Sources and Synthetic Minors *\n Clean Water Act (CWA) Stormwater Permits\n (National Pollutant Discharge Elimination System)\n Federal Insecticide, Fungicide, Total Number of Farms and Business Sites\n and Rodenticide Act (FIFRA) Regulated under Pesticides Programs\n Toxic Substances Control Act Core TSCA \xe2\x80\x93 Other Manufacturers, Processors,\n (TSCA) Distributors, Users, and Exporters\n Safe Drinking Water Act Public Water Systems: Community Systems\n (SDWA)\n Resource Conservation and Small Quantity Generator Facilities\n Recovery Act (RCRA)\n * Synthetic minors are stationary sources that have the capacity to emit at or above the\n major source threshold but are considered minor sources because they place physical\n or operational limitations on their capacity to emit pollution.\n\n\n\nWe did not quantitatively extrapolate the results from our judgmental sample\nacross OECA\xe2\x80\x99s entire regulated universe. Similarly, we did not use our\njudgmental sample to generalize to OECA\xe2\x80\x99s entire regulated universe. During the\ncourse of our review, however, we did find conditions that were systemic. We\ninterviewed OECA managers to determine whether a condition was limited to the\nprogram area under review, or whether it was an OECA-wide policy or practice.\n\n\n 4\n\x0cWhen OECA managers confirmed that it was an OECA-wide policy or practice,\nwe presented it as a finding with an appropriate recommendation. These findings\nand recommendations may therefore be broader than the program areas under\nreview within the sample. However, they are accompanied by appropriate\nexplanatory caveats. Both Chapters 2 and 3 include such findings and\nrecommendations.\n\nWe did not independently verify the accuracy or reliability of data provided by\nOECA staff for our program area sample. We conducted limited work regarding\nfraud, and did not test internal controls. We followed the guidance and\ndefinitions provided in the GAO guidance document, Assessing the Reliability of\nComputer-Processed Data, to answer our evaluation objectives. We reviewed\nexisting information on OECA\xe2\x80\x99s data problems, limitations, and corrective\nactions, and conducted interviews with OECA staff knowledgeable about\nenforcement and compliance data systems. We designed our scope to examine\nuniverse data at a national level and, as such, did not trace documents or data\nfrom original points of regional, State, or local data entry to final database output.\n\nSee Appendix C for more details on our scope and methodology.\n\n\n\n\n 5\n\n\x0c Chapter 2\n OECA Has Limited Knowledge of the\n Regulated Universe\n OECA has limited knowledge of the regulated universe for which it maintains\n responsibility. OECA has not updated its Regulatory Universe Identification\n Table since generating that table in 2001, even though some universe figures have\n changed substantially. In addition, various data quality issues impact OECA\'s\n ability to adequately determine reliable and current information on the size and\n composition of its regulated universe. OECA concentrates most of its compliance\n monitoring and enforcement activities on large entities, and knows little about the\n identities or cumulative pollution effects of small entities. Therefore, OECA\n cannot effectively use universe figures to assist with its regulatory activities. It\n cannot develop programmatic compliance information, adequately report on the\n size of the universe for which it maintains responsibility, or rely on these numbers\n to assist with management and regulatory planning. Further, in publications,\n OECA could more clearly qualify its role in relation to the roles played by States\n and others.\n\nUniverse Table Not Updated Since 2001 Despite Significant Changes\n While the number of entities in our universe sample has changed, OECA has not\n updated the universe table as a whole. To determine whether any significant\n changes in the size of the sampled universe have occurred, we requested that\n OECA staff provide us with the current number of regulated entities for our\n sampled program areas. Between 2001 and 2005, the universe for these program\n areas increased by 35 percent. Although we had limited our review of the size\n increase to only the six sampled areas, this indicates that the numbers of entities\n changed over time and OECA needs to update the numbers. Table 2-1 illustrates,\n for example, how some areas changed very little, while others changed\n significantly:\n\n\n\n\n 6\n\n\x0c Table 2-1: Change in Sample Universe Figures by Program Area, 2001 and 2005\n Percent\n Program Area 2001 2005 Change\n CAA \xe2\x80\x93 Minor Stationary Sources 96,866 101,370 +5%\n CAA \xe2\x80\x93 Synthetic Minor Sources 17,248 20,795 + 21 %\n CWA \xe2\x80\x93 Stormwater Permits 380,000 >550,000 + 45 %\n FIFRA \xe2\x80\x93 Farms and Businesses, etc. 2,246,512 2,168,241 -4%\n TSCA \xe2\x80\x93 Other Manufacturers, etc. 3,758,176 6,063,948 + 61 %\n SDWA \xe2\x80\x93 Public Water Systems: 54,101 52,838 -2%\n Community Systems\n RCRA \xe2\x80\x93 Small Quantity Generator 202,965 178,771 - 12 %\n Facilities\n Total 6,755,868 9,135,963 + 35 %\n\n Following the release of the Regulatory Universe Identification Table in\n September 2001, OECA planned to begin updating the table in December 2001.\n OECA intended future iterations to provide regional breakouts of the regulated\n universe in addition to national figures. However, OECA has not produced new\n universe figures since it released the original table in 2001, and continues to cite\n the number of entities from the 2001 table. OECA staff attributed this to a lack of\n resources. However, they indicated they are considering updating information on\n certain areas selected as national priority programs. OECA selected a handful of\n programs as national priorities based on patterns of noncompliance or the threat of\n significant environmental risk. In a December 8, 2004, response to our questions,\n senior managers from OECA\xe2\x80\x99s Office of Compliance indicated they did not\n believe the universe numbers had changed dramatically. Some OECA staff,\n however, said it is difficult to provide precise universe figures for some program\n areas because they fluctuate frequently.\n\n Regulatory agencies should attempt to maintain a reasonably accurate count of the\n regulated entities, by program area, under their purview. Knowledge of the\n regulated universe allows an environmental regulatory body like OECA to better\n understand the patterns of noncompliance in a sector or population, assess the\n production process and practices used by entities, and determine the state of\n environmental management sophistication in the sector or population. Without\n reliable universe information, OECA lacks both a definitive baseline on the\n number, size, and character of entities subject to regulation, as well as the\n information necessary to provide a denominator for compliance rates.\n\nData Quality Problems Affect OECA\xe2\x80\x99s 2001 Universe Figures\n The data that constitute OECA\xe2\x80\x99s numbers in the universe table are subject to\n several data quality issues. With the exception of SDWA, we found universe data\n for the sampled program areas was not reliable. The EPA Office of Inspector\n\n\n 7\n\n\x0cGeneral (OIG), GAO, and the National Academy of Public Administration have\npreviously reported data quality problems. We based our conclusions using\ndefinitions from GAO\xe2\x80\x99s Assessing the Reliability of Computer-Processed Data,\nwhich defines reliable data as being both complete and accurate, and provides the\nfollowing definitions:\n\n\xe2\x80\xa2\t Complete: The data contain all of the data elements and records needed for\n the engagement.\n\xe2\x80\xa2\t Accurate: The data reflect the data entered at the source or, if available, in the\n source documents.\n\nTable 2-2 shows where we determined that a number of program areas do not\nhave reliable or current data. Some program areas rely on estimated or\nextrapolated data, and the estimates were not based on supportable information.\nAccording to OECA, its inability to ensure desired data quality occurred due to\ndata gaps. Data gaps occur in cases where regulations do not require States to\nreport data to OECA. OECA managers said that obtaining additional reporting\nfrom States would require approvals by the Office of Management and Budget, or\nadditional regulatory actions.\n\n Table 2-2: Data Quality of Sample Program Areas\n Program Area Reliable Data * Current Data\n CAA \xe2\x80\x93 Minor Stationary Sources No No\n CAA \xe2\x80\x93 Synthetic Minor Sources No No\n CWA \xe2\x80\x93 Stormwater Permits No Unknown\n FIFRA \xe2\x80\x93 Farms and Businesses, etc. No Unknown\n Core TSCA \xe2\x80\x93 Other Manufacturers, etc. No No\n SDWA \xe2\x80\x93 Public Water Systems: Yes Yes\n Community Systems\n RCRA \xe2\x80\x93 Small Quantity Generator No Unknown\n Facilities\n * Reliable data must be both complete and accurate.\n\nResults from Table 2-2 are explained below:\n\n\xe2\x80\xa2\t CAA Minor Stationary Sources: CAA minor source data are incomplete\n because States are not required to report CAA minor stationary source data.\n In addition, State and local agencies do not evaluate all minor source entities\n regularly, so the data are not kept current.\n\n\xe2\x80\xa2\t CAA Synthetic Minor Sources: According to OECA staff, the Agency does\n not have a complete, accurate, or current universe of CAA synthetic minor\n sources even though State and local regulatory agencies are required to report\n the data to EPA. The staff also noted that State and local agencies do not\n evaluate all synthetic minor sources regularly, so the data are not kept current.\n\n\n\n 8\n\n\x0c\xe2\x80\xa2\t CWA Stormwater: EPA\xe2\x80\x99s Office of Water staff said they estimated the\n figure used in the 2001 table, but they could not provide a supportable basis\n for the estimate. They suggested the number might have come from\n extrapolated projections from the seven States for which EPA had not\n delegated the CWA stormwater program. It, therefore, does not necessarily\n represent a reliable estimate for the true national universe of entities subject to\n stormwater regulations.\n\n\xe2\x80\xa2\t FIFRA: Approximately 99 percent of the FIFRA universe of 2,246,512\n entities is estimated. It includes three subgroups: pesticide production\n establishments (12,442), commercial pest control firms (40,000), and farms\n (2,194,070). OECA has limited knowledge of the largest subgroup: farms.\n FIFRA provides primacy to States, and OECA has limited involvement with\n the FIFRA universe. The statute requires commercial establishments to\n register pesticides, and this component of the universe includes only those\n who register. The number of non-registrants is not known.\n\n\xe2\x80\xa2\t TSCA: The numbers for the TSCA universe are not reliable for the sample\n program area. The sample included a large group (3,758,176) described as\n \xe2\x80\x9cother manufacturers, processors, users, distributors, and exporters of\n chemical substances in the U.S....\xe2\x80\x9d OECA estimated this number from the\n Dun and Bradstreet sector data of entities, which is composed of\n manufacturing, wholesale, and retail trades. However, according to OECA\n staff, its compliance and enforcement activities only focus on a small subset\n of the total Core TSCA universe \xe2\x80\x93 the 13,513 chemical manufacturers. This\n number for chemical manufacturers was generated from the 1997 U.S.\n Economic Census. This 1997 figure was used in the 2001 universe table, and\n was also provided as the current universe number. Therefore, both the 2001\n and 2005 figures are outdated.\n\n\xe2\x80\xa2\t SDWA: We take no exception to the published universe figures for SDWA.\n The sample for SDWA included Public Water Systems: Community Systems.\n This universe is well defined. We did not independently test the SDWA data.\n\n\xe2\x80\xa2\t RCRA: The RCRA small quantity generator facility universe number is\n incomplete. The designation of large quantity versus small quantity depends\n on the amount of waste generated and stored. According to OECA staff, these\n facilities are not required to notify EPA if their status changes from a small to\n large facility. The facilities can easily change from small to large generators,\n and the staff explained how it is difficult for OECA to generate a reliable\n universe figure because of this flux.\n\n\n\n\n 9\n\n\x0c The EPA OIG, GAO, and the National Academy of Public Administration have\n previously reported problems with OECA\xe2\x80\x99s data quality.2 High quality data in\n EPA\xe2\x80\x99s national databases are essential for credible measures, reports, and\n analyses. In response to weaknesses identified by external sources, OECA issued\n guidance in March 2002 on improving data quality, titled Final Enforcement and\n Compliance Data Quality Strategy, which states:\n\n EPA managers and members of Congress rely on this data to hold EPA\n programs accountable and to inform their decision-making\xe2\x80\xa6 Accurate\n data, in an accessible and usable form, is essential to support Agency\n planning and targeting\xe2\x80\xa6 Documenting performance [for the Government\n Performance and Results Act] requires timely, high quality data\xe2\x80\xa6 Proof\n of high quality gives rise to confidence among the public, industry, and\n other users of enforcement and compliance data.\n\nOECA Could More Clearly Describe Its Role\n Of the 41.1 million entities listed in the 2001 universe table, EPA maintains only\n 1.6 million facility or entity records in its compliance and enforcement databases.\n This represents less than 4 percent of the total facilities for which OECA is\n responsible. Many of those entities represent those for which EPA has direct\n regulatory authority. For most entities, EPA has authorized States to implement\n environmental programs and conduct enforcement activities in accordance with\n environmental laws. Among the program areas we evaluated, OECA is the primary\n regulatory authority only for TSCA. For the other program areas, the States are the\n primary regulators, with OECA providing an oversight role, although in certain\n instances EPA may become involved in compliance activities and enforcement\n actions.\n\n Some documents in which EPA includes the 41.1 million figure do not clearly\n indicate OECA\xe2\x80\x99s shared role in conducting its regulatory responsibility. These\n include the Agency\xe2\x80\x99s 2005-2007 National Program Guidance, and EPA responses\n to prior EPA OIG and GAO reports. EPA\xe2\x80\x99s 2003-2008 Strategic Plan and\n OECA\xe2\x80\x99s response to the 2006 Office of Management and Budget Program\n Assessment Rating Tool do explain that EPA works cooperatively with States, as\n well as local and tribal agencies, to secure and maintain compliance. OECA staff\n stated they never intended to imply that OECA had direct responsibility for\n 41.1 million entities. However, without appropriate explanatory caveats, external\n stakeholders might misinterpret the figure to mean that OECA actually ensures\n environmental protection by directly regulating all 41.1 million entities.\n\n\n\n2\n These documents include OIG\xe2\x80\x99s EPA\xe2\x80\x99s Key Management Challenges (2003), GAO\xe2\x80\x99s Human Capital:\nImplementing an Effective Workforce Strategy Would Help EPA to Achieve Its Strategic Goals (2001), and the\nNational Academy of Public Administration\xe2\x80\x99s Evaluating Environmental Progress: How EPA and the States Can\nImprove the Quality of Enforcement and Compliance Information (2001).\n\n\n 10\n\n\x0cOECA Does Not Focus Its Compliance Monitoring and Enforcement\nActivities on a Greater Part of the Regulated Universe\n According to OECA staff, EPA\xe2\x80\x99s enforcement and compliance monitoring\n activities focus on major and large entities or sources, which represent only a\n small fraction of the total universe shown in OECA\xe2\x80\x99s universe table. OECA has\n mostly focused on larger and major entities, and has not conducted or obtained\n analyses showing the cumulative impact of the vast number of entities that emit\n pollution below the threshold of major or larger entities. EPA has focused on\n major and larger entities because any one of the larger entities can have a greater\n individual impact than any of the smaller entities by itself. However, given the\n much greater number of small entities in the sample, and the potential cumulative\n impact from this vast part of the regulated universe, we find it is important for\n OECA to know the cumulative environmental impact of entities that fall below\n the major or large threshold. Information on small entities could help OECA\n better prioritize where to focus resources and facilitate effective management.\n\n The following charts illustrate the proportion of large and small entities in the\n CAA, CWA, RCRA, and SDWA program areas (FIFRA and TSCA do not\n formally distinguish between large and small entities). We generated each chart\n using data from OECA\xe2\x80\x99s 2001 Regulatory Universe Identification Table. The\n charts demonstrate that smaller entities comprise a much larger percentage of\n OECA\xe2\x80\x99s total universe than large entities. We include general statutory\n information and program-specific facility size classifications in Appendix B.\n\n CAA: OECA concentrates its CAA resources on major stationary sources and\n large polluters. Using 2001 universe figures, major stationary sources, minor\n sources, and synthetic minor sources total 137,982 entities. Major stationary\n sources represent only 4 percent of OECA\'s CAA regulated universe when\n including other CAA entities such as mobile sources.\n\n Chart 2-1: CAA Regulated Universe - 2001\n\n Large Facilities (Major\n Stationary Sources)\n 4%\n 14%\n Small Facilities (Minor\n Stationary Sources)\n\n 17%\n Small Facilities\n 65% (Synthetic Minor\n Stationary Sources)\n Other Facilities\n (includes mobile\n sources)\n\n\n\n\n 11\n\n\x0cCWA Stormwater Permits: The CWA stormwater program is a subset of the\nNational Pollutant Discharge Elimination System (NPDES) program. The\nstormwater permits program was instituted in two phases. The first phase\nrequired permits for large municipal waste water systems, industrial plants, and\nlarge commercial construction sites. Phase two permit requirements cover a much\ngreater number of smaller municipal waste water systems and smaller\nconstruction sites. In a majority of cases, States issue general permits, and only a\nsmall percent of these permits are in OECA\xe2\x80\x99s databases. Analyses have shown\nthat stormwater overflow is a major source of water pollution. The universe of\nstormwater sources is not well known even though stormwater has been a priority\narea since fiscal year 2001. Stormwater continues as a national enforcement\npriority program in fiscal year 2005, and OECA has developed a national strategy.\nThe strategy puts emphasis on providing compliance assistance and developing an\ninventory of watersheds. OECA plans to devote more resources to focus on these\nsources.\n\n\n Chart 2-2: CWA-NPDES Regulated Universe - 2001\n\n 1%\n 16%\n Large Facilities (Major\n Permitted Facilities)\n\n\n Small Facilities (Minor\n Permitted Facilities)\n\n\n Other Facilities (includes\n wet weather facilities\n 83% (stormwater permits))\n\n\n\n\nRCRA: OECA concentrates most of its regulatory resources on large quantity\ngenerator facilities, although they comprise less than 10 percent of the combined\ntotal of RCRA facilities, according to OECA\xe2\x80\x99s 2001 universe figures. The\npercentage of large quantity generator facilities in the total RCRA universe drops\nto 4 percent when including other facilities such as conditionally-exempt small\nquantity generators.\n\n\n\n\n 12\n\n\x0c Chart 2-3: RCRA Regulated Universe - 2001\n\n\n 4% Large Facilities (Large\n Quantity Generators)\n\n\n\n Small Facilities (Small\n 42% 35% Quantity Generators)\n\n\n\n Small Facilities\n (Conditionally-Exempt\n Small Quantity\n Generators)\n\n 19% Other Facilities\n\n\n\n\nSDWA: While OECA is familiar with the SDWA public water system universe,\nOECA does not expend its regulatory resources on all types of drinking water\nentities. The majority of the U.S. population gets its drinking water from a small\nnumber of public community water systems. EPA\xe2\x80\x99s annual report to Congress on\nnational public water system compliance includes information on large public\ncommunity water systems. The States, not OECA, are responsible for most\ncompliance activities across drinking water systems. SDWA was a national\npriority enforcement area for OECA until 2005.\n\n\n Chart 2-4: SDWA Regulated Universe - 2001\n\n\n 5%\n 11% Large Facilities (Public Water\n Systems - Community Water\n Systems)\n\n Small Facilities (Non-\n Transient and Transient\n Community Water Systems)\n\n Other Facilities (includes\n Class I-V Wells)\n 84%\n\n\n\n\nTSCA: While Core TSCA does not formally distinguish between large and small\nentities, OECA concentrates its compliance and enforcement resources on 13,513\n\n\n\n 13\n\n\x0c chemical manufacturing entities out of 3,771,689 entities in the universe table.\n This is 0.4 percent of the stated Core TSCA universe.\n\n OECA staff mentioned that they use compliance assistance to help smaller entities\n become compliant. They also provided various reasons why they generally focus\n compliance monitoring and enforcement resources on large entities. OECA\n officials said they should concentrate resources on large entities because:\n\n \xe2\x80\xa2\t Large entities are often priority enforcement and compliance monitoring\n areas;\n \xe2\x80\xa2\t States address small entities through core program implementation and OECA\n specifies to States how frequently they should inspect small entities;\n \xe2\x80\xa2\t The large size of program area universes, and an accompanying lack of\n resources; and\n \xe2\x80\xa2\t The difficulty in identifying and analyzing RCRA small quantity generator\n facilities because of their transient nature.\n\nOECA Does Not Know the Cumulative Impact of Small Entities\n In most program areas in our sample, OECA does not know the cumulative\n effects of pollution from small entities. OECA did not provide information on\n any analyses that study the cumulative environmental or health effects of small\n entities. Of our sample program areas, OECA focuses compliance and\n enforcement resources on small and medium entities in only the CWA stormwater\n and Core TSCA program areas. EPA\xe2\x80\x99s concern that stormwater runoff\n significantly impairs water quality contributed to designating the stormwater\n program as a national enforcement priority in prior years, and continuing it as a\n priority in fiscal 2005. While Core TSCA regulations do not formally distinguish\n between large and small entities, OECA TSCA staff recently focused on smaller\n chemical manufacturers because the staff believed they are less likely to\n understand their Core TSCA obligations than large chemical manufacturers.\n\n While no formal impact assessments have been undertaken, some States and EPA\n regions have argued that RCRA small quantity generator facility inspections\n represent some of the most environmentally significant activities that regions and\n States conduct. RCRA enforcement staff said they are collecting small quantity\n generator facility information, and may consider shifting OECA\xe2\x80\x99s RCRA\n resources in coming years.\n\n We acknowledge that analyzing the cumulative environmental effect of small\n entities is not OECA\xe2\x80\x99s direct responsibility. However, given that OECA\n crosscuts all of the Agency\xe2\x80\x99s program offices, we believe OECA should request\n that program offices conduct these studies if OECA does not do the analyses\n themselves. The knowledge generated by these analyses would allow OECA to\n establish more defensible priorities of its own activities, as well as better organize\n and coordinate the efforts of its State partners.\n\n\n 14\n\n\x0cRecommendations\n\n We recommend that the Assistant Administrator for Enforcement and Compliance\n Assurance:\n\n 2-1 \t Biannually update publicly released universe figures by tracking and\n recording the number of entities over which OECA has oversight and\n primary regulatory responsibility.\n\n 2-2 \t When producing its biannual universe update, use reliable data to generate\n complete and current universe numbers that meet national data quality\n standards similar to those outlined in OECA\xe2\x80\x99s Final Enforcement and\n Compliance Data Quality Strategy.\n\n 2-3 \t Describe OECA\xe2\x80\x99s enforcement and compliance role in relation to States\n and other partners when the Agency publicly releases universe figures.\n\n 2-4 \t Develop an objective of having the most up-to-date and reliable data on all\n entities that fall under its regulatory responsibility. OECA should adopt\n the goals of requiring States to track, record, and report data for entities\n over which States have regulatory responsibility. To achieve this goal,\n OECA should develop a multi-State, multi-program pilot program of\n collecting data that States track, record, verify, and report.\n\n 2-5 \t Request that EPA program offices analyze and report to OECA the\n cumulative impact of violations by regulated entities that pollute below the\n thresholds of major or large entities. OECA should use any cumulative\n impact analyses conducted by program offices to support OECA\xe2\x80\x99s\n management decisions.\n\nAgency Comments and OIG Evaluation\n\n Even though OECA agreed with the premise that knowing the size and character\n of the regulated universe is a fundamental activity for a regulatory agency, OECA\n has not agreed to update its knowledge of the universe of entities for which it\n maintains regulatory responsibility. Specifically, OECA agreed with\n Recommendations 2-2 and 2-3, but disagreed with Recommendations 2-1, 2-4,\n and 2-5. Appendix D includes the Agency memorandum that contains summary\n comments. Appendix E includes the Agency\xe2\x80\x99s comments on specific\n recommendations and other general comments, and our evaluation of those\n comments. The Agency\xe2\x80\x99s detailed comments are available on the EPA OIG Web\n site. We have made some appropriate changes to the report in response to the\n Agency\xe2\x80\x99s comments.\n\n\n\n\n 15\n\n\x0c Chapter 3\n OECA Cannot Determine Changes in Compliance for\n Five of Six Sample Program Areas\n OECA could not determine or report on the levels of compliance with\n environmental regulations for five of our six sample regulatory areas. OECA\xe2\x80\x99s\n National Program Guidance for 2005-2007 states that \xe2\x80\x9cOECA\xe2\x80\x99s national\n enforcement and compliance assurance program is responsible for maximizing\n compliance.\xe2\x80\x9d OECA conducts several activities to determine and report\n compliance. However, in five of the six sample program areas, OECA could not\n determine and report overall programmatic compliance levels because it lacks\n current and complete knowledge of the regulated universe. Data quality problems\n further hindered OECA\xe2\x80\x99s ability to generate valid programmatic compliance\n information. The lack of programmatic compliance information impedes OECA\xe2\x80\x99s\n ability to determine which programs are working and how effective they are in\n achieving goals. As a result, OECA cannot demonstrate changes in overall\n compliance in the regulated universe. OECA generates region- and State-specific\n noncompliance statistics and data for internal reports, but lacks sufficient\n transparency by not publicly sharing some of this compliance-related information\n with external stakeholders.\n\nOECA Cannot Generate Programmatic Compliance Information for\nFive of Six Program Areas\n\n Reliable compliance information is essential for a regulatory agency to plan, set\n goals, evaluate the results of its strategy, and demonstrate results to external\n stakeholders. Statutes and regulations provide the basic framework of regulatory\n compliance, along with various policies.\n\n OECA cannot generate programmatic compliance information for five of the\n program areas in our sample due to its limited knowledge of the full universe of\n regulated entities. In these five sample areas, OECA maintained only facility-\n specific noncompliance information for a limited part of that universe: the larger\n or major entities in the program. OECA cannot determine the compliance of most\n minor and synthetic minor stationary air sources, small quantity waste generator\n facilities, and stormwater dischargers because OECA does not currently require\n States to report data on minor and small sources. As discussed in Chapter 2,\n minor and smaller entities constitute a much greater number of OECA\xe2\x80\x99s total\n regulated universe than large entities.\n\n Lack of knowledge of the number, location, and levels of compliance of this\n significant portion of the universe leaves a large gap in OECA\xe2\x80\x99s knowledge of\n overall compliance in each program. Within most of our sample, OECA has not\n\n\n 16\n\n\x0cbeen able to quantitatively demonstrate the success of its strategies, nor show an\nincrease in the percent of entities in compliance, because OECA does not know\nthe full universe of regulated entities.\n\nIn Table 3-1, we summarize whether compliance information for the program area\nwas available, and provide additional information after the table, based on\ndiscussions with OECA staff, about available programmatic compliance\ninformation. During discussions with OECA managers, we found that the lack of\ndata was systemic and not limited to only the program areas under review.\nTable 3-1 shows by program area whether: OECA could provide programmatic\ncompliance information, OECA has defined a significant noncompliance or high\npriority violator category, and the area is or has been a national priority program\nsince 2001.\n\n Table 3-1: Summary of Programmatic Compliance Information for Sample Areas\n National\n Programmatic Priority\n Compliance SNC/HPV * Program\n Sample Program Area Information Applicable 2001 2005\n CAA \xe2\x80\x93 Minor Stationary Sources No Yes No No\n CAA \xe2\x80\x93 Synthetic Minor Sources No Yes No No\n CWA \xe2\x80\x93 Stormwater Permits No No Yes Yes\n FIFRA \xe2\x80\x93 Farms and Businesses, etc. No No No No\n Core TSCA \xe2\x80\x93 Other Manufacturers, No No No No\n etc.\n SDWA \xe2\x80\x93 Public Water Systems: Yes Yes Yes No\n Community Systems\n RCRA \xe2\x80\x93 Small Quantity Generator No Yes No No\n Facilities\n * SNC/HPV: Significant Noncompliance/High Priority Violator\n\nCAA Minor Stationary Sources: OECA did not have information on levels of\ncompliance for minor sources because it did not require States to report them in\nthe national Air Facilities Subsystem database or provide associated compliance\nand enforcement information.\n\nCAA Synthetic Minor Sources: OECA does not have complete data on the\nsynthetic minor sources, even though States and local regulatory agencies are\nrequired to report on these entities. OECA staff said they have focused mainly on\nmajor sources because OECA considers them a greater source of air pollution.\nIn April 2001, EPA issued the Clean Air Act Stationary Source Compliance\nMonitoring Strategy, which recommends that State and local agencies focus\ncompliance monitoring activities on a subset of the synthetic minor universe.\nHowever, OECA staff indicated that they do not have current or complete\nknowledge of compliance status of all synthetic minor sources.\n\n\n\n 17\n\n\x0c CWA: According to OECA staff, there is a high level of noncompliance with\n stormwater regulations. Even though this has been a priority area since 2001,\n OECA cannot provide programmatic or facility-specific compliance information\n because it does not have a good knowledge of this universe. In most cases, States\n issue general permits for stormwater regulations, and EPA does not have detailed\n data on these permits. In fiscal year 2005, OECA again selected the Stormwater\n program area as a national enforcement priority program due to runoff concerns.\n\n FIFRA: OECA cannot provide overall compliance information for the FIFRA\n program area because it does not have data for the universe of 2,246,512\n regulated entities. OECA staff said that, according to statutes, States have\n primacy in implementing the FIFRA enforcement and compliance program.\n OECA provides grants to States, and States then report the number of inspections\n and violations found. However, OECA cannot demonstrate levels of\n programmatic compliance based on the results of States\xe2\x80\x99 targeted inspections.\n\n TSCA: OECA cannot determine overall compliance for the Core TSCA universe\n because OECA can only make compliance determinations by conducting\n inspections. According to OECA staff, OECA has not inspected even a small\n portion of the 3,758,176 Core TSCA regulated entities. Core TSCA enforcement\n staff uses the Enforcement Response Policy to assess the gravity of violations and\n determine appropriate enforcement actions on a case-by-case basis. After\n concentrating compliance monitoring on larger manufacturing entities for many\n years, OECA staff indicated that they would pay more attention to providing\n compliance assistance to smaller entities. However, OECA cannot presently\n provide compliance information on the Core TSCA universe.\n\n SDWA: EPA reports on the status and progress of the SDWA program to\n Congress annually, and SDWA has been a national enforcement and compliance\n priority for many years. OECA generated programmatic compliance rates to\n report changes in compliance in the SDWA Public Water Systems \xe2\x80\x93 Community\n Systems universe. However, OECA generated the rates with incomplete\n violations data, resulting in underreporting. States conduct the majority of\n SDWA inspections and enter information and violations in the Safe Drinking\n Water Information System database. Both EPA OIG and OECA audits3 of that\n database have shown that States do not provide consistently high quality\n violations data and do not enter many violations in the database.\n\n RCRA: OECA does not know the universe or compliance levels for the small\n quantity generator universe and therefore cannot provide information to show\n levels of RCRA compliance. As discussed in Chapter 2, RCRA facilities can\n change generator size status from small to large. However, if they do not notify\n EPA, the status change will not be discovered until the facility is inspected.\n\n3\n EPA OIG Report No. 2004-P-0008, EPA Claims to Meet Drinking Water Goals Despite Persistent Data Quality\nShortcomings, March 5, 2004; and OECA 2002 National Public Water System Compliance Report, Providing Safe\nDrinking Water in America, December 2004.\n\n\n 18\n\n\x0c According to OECA staff, not all RCRA facilities are inspected annually, and a\n significant percentage of small quantity generator facilities have never been\n inspected.\n\nOECA Is Not Sufficiently Transparent with Available Compliance Data\n While OECA generates many types of compliance reports for management use\n from the data provided by States and EPA regions, OECA does not release many\n of these reports to external stakeholders. The internal reports include monthly\n management reports, watch lists, and performance analyses. From the\n information provided by States and regions in Agency databases, OECA develops\n data, statistics, rates, and analyses for internal OECA and regional management.\n OECA organizes analyses and data by region and State. Some of the rates that\n measure noncompliance include significant noncompliance rates, recidivism rates,\n time to attain compliance, and \xe2\x80\x9chit rates\xe2\x80\x9d that show the number of significant\n noncompliance violations found from inspections.\n\n OECA provides performance results annually to external audiences in measures\n such as pounds of pollutants reduced, dollars of fines and penalties, number of\n inspections, and compliance assistance provided. The external reports and\n publicly reported data do not demonstrate results in terms of levels of compliance\n or noncompliance across regulated program universes. OECA provides facility-\n specific compliance information on approximately 800,000 facilities in its public\n Enforcement and Compliance History Online (ECHO) database, but the system\n does not provide programmatic compliance information.\n\n Transparent regulatory information allows Congress and the public the\n opportunity to review and analyze compliance data. Also, State compliance data\n can show regional and geographical differences in environmental problems,\n approaches, and solutions. EPA has a longstanding commitment to transparency,\n which can improve data quality because States might more timely and carefully\n enter enforcement and compliance data that is open to public scrutiny.\n\n Through fiscal 2002, OECA publicly released most of the rates and data in its\n Measures of Success Reports, but OECA currently releases this information only\n to Agency personnel. According to OECA staff, they do not publicly release\n internal rates and data because:\n\n \xe2\x80\xa2\t People may misunderstand and misrepresent the rates and data derived from\n targeted inspections and compliance monitoring, since the data do not\n represent the noncompliance levels of the whole regulated universe.\n \xe2\x80\xa2\t Congress and the public may be unduly alarmed by the high level of\n noncompliance because inspections are targeted based on high risk or\n suspected noncompliant activity.\n\n\n\n\n 19\n\n\x0c \xe2\x80\xa2\t Experts could create programmatic compliance rates from the facility-specific\n information in the ECHO database; therefore, OECA does not need to\n generate programmatic information.\n \xe2\x80\xa2\t States do not want to publish statistics because people may make comparisons\n among States and draw incorrect inferences without the proper context.\n \xe2\x80\xa2\t OECA must keep some information confidential for enforcement actions.\n\n While we agree that some information must be kept confidential for conducting\n enforcement actions, we do not agree with the other reasons. If the data is based\n on targeted inspections, or only represents some States or regions, OECA can\n provide explanatory notes to avoid misrepresentation and prevent Congress and\n the public from misunderstanding what the data represents. Providing significant\n noncompliance rates, with appropriate qualifying explanations, can still\n demonstrate the success of pursuing known violators. The credibility of EPA\xe2\x80\x99s\n decisions can increase when external audiences understand how the Agency came\n to its decisions. Further, even experts cannot generate programmatic compliance\n information from the facility specific compliance information in the ECHO\n database because, according to OECA statements, this database only contains\n compliance data on approximately 800,000 entities, whereas OECA has described\n its regulated universe as approximately 41 million entities.\n\nRecommendations\n We recommend that the Assistant Administrator for Enforcement and Compliance\n Assurance:\n\n 3-1 \t To show the results of its national enforcement and compliance program\n in maximizing compliance with environmental statutes, develop and\n publish information that demonstrates changes in compliance levels within\n the regulated universe, by program areas. Include any appropriate\n explanations of data quality issues or data caveats.\n\n 3-2 \t Share compliance data and analyses with external stakeholders to provide\n a better understanding of programmatic compliance levels; include\n explanatory notes as needed to ensure proper representation and\n understanding.\n\nAgency Comments and OIG Evaluation\n OECA did not concur with Recommendation 3-1, and as an alternative proposed\n that it would share with the public any statistically valid noncompliance rates\n developed in the past year and any that OECA will develop in the future. This\n alternative is not adequate considering the limitations OECA itself acknowledged\n in its September 29, 2004, memo to us requesting assistance in developing\n methodologies for producing statistically valid noncompliance rates. The Agency\n concurred with Recommendation 3-2 to increase transparency. OECA included\n\n\n 20\n\n\x0ctwo tables with lists of reports that we included as Appendix F. However, OECA\ndid not agree to publish other compliance information that shows significant\nnoncompliance rates, \xe2\x80\x98hit\xe2\x80\x99 rates, recidivism, and the time taken to bring violators\nto compliance. Appendix E includes the Agency\xe2\x80\x99s comments on specific\nrecommendations and other general comments, and our evaluation of those\ncomments.\n\n\n\n\n 21\n\n\x0c Appendix A\n\n OECA\xe2\x80\x99s Regulatory Universe Identification Table \xe2\x80\x93 \n\n Executive Summary \n\n\nOur analysis of OECA\xe2\x80\x99s September 2001 Regulatory Universe Identification Table found that\nOECA listed 14 statutes that had one or more programs (or programmatic descriptions). OECA\nincluded \xe2\x80\x9cfederal facilities,\xe2\x80\x9d \xe2\x80\x9ctribes,\xe2\x80\x9d \xe2\x80\x9cwetlands,\xe2\x80\x9d \xe2\x80\x9cUST\xe2\x80\x9d (underground storage tanks), and\n\xe2\x80\x9cmulti-media\xe2\x80\x9d as statutes even though they are not, by themselves, statutes. We identified 58\nseparate program areas under the 14 statutes. Further, OECA consulted 36 different sources to\nobtain the universe numbers for the 58 programs. Data sources came from both within and\noutside of EPA and included enforcement and media-specific databases, final rule documents,\nBureau of Indian Affairs data, and Census information. The universe table also included\nestimates when no database or source existed for a given regulatory area. In this case, according\nto OECA staff, the estimate served as the best source for the universe number.\n\nIn the cover letter that accompanied OECA\xe2\x80\x99s universe table, OECA explained that the total\nuniverse of entities in EPA\xe2\x80\x99s compliance and enforcement databases is approximately\n1.6 million. OECA derived the rest of the universe numbers from various other sources,\nincluding Agency publications and databases. We noted that OECA incorrectly totaled its\nuniverse of regulated entities. While the Executive Summary states that OECA\xe2\x80\x99s universe is\n41.1 million entities, we found, after reviewing the figures, that OECA\xe2\x80\x99s 2001 regulatory\nuniverse should have been listed as 41.8 million entities. We also found other math transcription\nerrors while analyzing OECA\xe2\x80\x99s universe table, including two instances where the number listed\nin OECA\xe2\x80\x99s universe table for a particular program did not match the number OECA listed in the\nexecutive summary for that same program. For example, under the CWA National Pollutant\nDischarge Elimination System statute, OECA lists a number of 29,688 for \xe2\x80\x9cbiosolids/sludge\nactive POTWs\xe2\x80\x9d (Publicly Owned Treatment Works) in the universe table, but lists that figure as\n29,668 in the Executive Summary.\n\nWe transcribed the following table and endnotes directly from OECA\xe2\x80\x99s September 2001\nRegulatory Universe Identification Table. To maintain the integrity of the source document, we\ndid not spell out abbreviations or correct any spelling, punctuation, or spacing errors.\n\n\n\n\n 22\n\n\x0c OECA Regulatory Universe Identification Table - Executive Summary\n\n Statute Programmatic Description Universe Estimate Universe in Database\nCAA Major Stationary Sources (AFS) 23,868\n Minor Stationary Sources (AFS) 96,866\n Synthetic Minor Stationary Sources (AFS) 17,248\n Chlorofluorocarbon Sources (AFS) 4,292\n Asbestos Work Practice Standards Notifications 96,954\n (NARS)\n Section 112(r) Risk Management Plans 15,081\n Mobile Sources 324,830\nCWA Major permitted facilities (PCS) 6,599\nNPDES\n Minor Permitted facilities (PCS) 87,844\n Pretreatment significant industrial users 30,000\n Biosolids (Sludge) (PCS) 29,668 5,143\n Wet Weather facilities - Stormwater permits 380,000 41,613\n CAFOs 12,660\nWetlands Total acreage 105.5 M\nRCRA Treatment, Storage, and Disposal (TSD) 2,393 2,973\n Facilities*\n Large Quantity Generator (LQG) Facilities* 20,876 30,231\n Small Quantity Generator (SQG) Facilities 202,965\n Conditionally Exempt Small Quantity Generator 108,780\n (CESQGs) Facilities\n Transporter Facilities 13,223\n RCRA Non-Notifier Facilities 7,076\n \xe2\x80\x9cOther\xe2\x80\x9d Facilities in RCRAInfo 230,870\nSDWA Public Water Systems: Community Systems 54,101\n Public Water Systems: Non-Transient Non- 20,429\n Community Systems\n PWS: Transient Non-Community Water Systems 93,034\n UIC: Class I Wells 533\n UIC: Class II Wells 156,215\n UIC: Class III Wells 15,452\n UIC: Class IV Wells 4\n UIC: Class V Wells 650,000 207,429\n\n\n\n 23\n\n\x0c Statute Programmatic Description Universe Estimate Universe in Database\nFIFRA Total Registered Pesticides 20,000\n Total Registrants with active pesticide 1,963\n registrations\n Pesticide Production Establishments (Active) 12,442\n Commercial Pest Control Firms 40,000\n Agricultural Sector\n Total Number of Farms 2,194,070\n Total Amount of Farm Acreage 947,340,000\n Total Number of Agricultural Workers 4,135,315\n Total Number of Farms And Business Sites 2,246,512\n Regulated Under Pesticide Programs\nEPCRA Toxic Release Inventory (TRI) Reporting 22,639\n Universe - Total number of facilities\n Total Number of 311/312 Facilities 559,600\nTSCA Core TSCA - Chemical Manufacturers 13,513\n - Other manufacturers, processors, 3,758,176\n distributers, users and exporters\n Polychlorinated Biphenyls (PCBs) - Storage &\n Disposal Sectors 89\n Commercial Storers 48\n Commercially Permitted Disposal Companies 4\n Commercial Decontamination Facilities 7\n Scrap Metal Oven Facilities\n PCBs\xe2\x80\x93 Utility/Non-utility Waste Generators\n Electrical Utility Industry 3,215\n Total Number of Non-Utility Establishments 6,404,244\n with PCB-contaminated Equipment 2,500\n Company-registered Transformer Facilities 20,742\n - Transformers in operation\n Lead-based (Pb) Paint - 402/404\n Total Training Program Providers 177\n Lead-based Paint activities\n \xe2\x80\x93 Firms 4,069\n \xe2\x80\x93 Individual Workers 17,249\n Pb-based paint - 1018\n - Targeted Housing 99,500,000\n - Real Estate Firms 92,000\n - Real Estate Agents 352,000\n - Property Managers 243,000\n\n\n\n\n 24\n\n\x0c Statute Programmatic Description Universe Estimate Universe in Database\n Pb-based Paint - 406\n - Annual Number of Target Units Subject to 18,500,000\n Renovation Requirements 482,000\n - Total Number of affected Renovation Firms\n - Total Number of Contractor Personnel most 2,272,000\n likely to be involved in Pb-based paint renovation\n activities:\n Total Number of Entities Regulated under 5,356,984\n Asbestos AHERA/MAP Programs\n Federal Number of Federal Facilities Subject to 11,670\n Facilities Environmental Regulations\n Tribes Number of Federally Recognized Tribes 556\n Total Population 1,400,000\n Total Acreage of Tribal Lands 54,893,267\n Federal, industrial, commercial, and municipal 1,645\n facilities located on Tribal areas\n CERCLA National Priorities List (NPL) Sites (non-federal 1,349\n facility) \xe2\x80\x93 Proposed, Final & Deleted\n Total number Non-NPL Sites (non-federal 8,884\n facility)\n NPL Federal Facilities \xe2\x80\x93 Total number of sites 176\n (Proposed, Final & Deleted)\n Non-NPL Federal Facilities \xe2\x80\x93 Total number of 781\n sites\n OPA Oil Storage Facilities Subject to SPCC 469,289\n Requirements\n UST Active Underground Storage Tanks 713,666\n Multi Facilities tracked by two or more media programs 15,733\n media (CAA, CWA, or RCRA)\n\nThe total universe estimate of entities under the responsibility of EPA\xe2\x80\x99s enforcement and compliance programs is approximately\n41.1 million (the sum of all bolded numbers in the universe estimate and universe in database columns).\nThe total universe of entities maintained in EPA databases is approximately 1.6 million. This is the sum of the numbers in the\nuniverse in database column, except for the RCRA TSD and LQG numbers, which were summed from the universe estimate\ncolumn (see the asterisked footnote below regarding the RCRA data), and excluding the federal facility, tribal, and multimedia\nnumbers, which are accounted for in the media-specific counts.\nNote: In summing entities across media categories, an unknown amount of multiple-counting (i.e., of the same facility regulated\nby several EPA programs) is inevitable.\nUniverse Estimate provides a documented estimate of all facilities/entities, taken from Agency publications, ICRs, databases, and\nother sources. Universe in Database provides a snapshot of the number of facilities/entities that are tracked by EPA program\ndatabases. Please refer to the full version of this table for detailed information about each universe category and estimate.\n*: Both universe estimate and universe in database numbers are generated from RCRAInfo data. The universe estimate number\nrepresents the subset of handlers with inspection, violation, and/or enforcement activity within the past five years (\xe2\x80\x9cactive\xe2\x80\x9d per\nFY2001RECAP definition).\nSeptember 21, 2001 Version 4.exec\nUSEPA / OECA / OC / EPTDD / IUTB\n\n\n\n\n 25\n\n\x0c Appendix B\n\n Environmental Statute Information and \n\n Program-Specific Facility Size Classifications \n\n\nThe following briefly summarizes the environmental statutes that cover our sample program\nareas. Where applicable, we described how OECA defined large and small, or major and minor\nentities (FIFRA and TSCA programs do not distinguish between large and small entities).\n\n CAA Background\n The Clean Air Act (CAA) created a national program to control the damaging effects of\n air pollution.\n Large/Small Entity Classifications\n Stationary sources of air pollution include factories, processing plants, chemical\n plants, refineries, and utilities.\n Major Stationary Source: Any stationary source or group of sources located in a\n contiguous area, with potential to emit more than 100 tons of pollutant per year, or,\n for hazardous air pollutants, 10 tons per year.\n Minor Stationary Source: Any stationary source that has the potential to emit air\n pollutants at less than the major stationary source thresholds, or is not classified as a\n synthetic minor source, and is subject to Federal regulations.\n Synthetic Minor Source: Any stationary source that has the capability to emit air\n pollutants at or above the major stationary source threshold but is considered a minor\n source because it places physical or operational limitations on its capacity to emit\n pollution.\n\n\n CWA Background\n The Clean Water Act (CWA) is the primary Federal law that protects the Nation\xe2\x80\x99s\n waters, including lakes, rivers, and coastal areas.\n Large/Small Entity Classifications\n The National Pollutant Discharge Elimination System (NPDES) program controls\n water pollution by regulating point sources that discharge into U.S. waters. The\n majority of NPDES facilities are classified as either:\n Major Permitted Facilities: Municipal or industrial facilities that directly discharge\n effluent, based on design flow or a qualifying permit rating score.\n Minor Permitted Facilities: Municipal or industrial facilities that directly discharge\n effluent, but are not designated as major dischargers.\n Stormwater Permits: Permits issued through a two-phased national program for\n regulating non-agricultural sources of stormwater discharges and run-off. Phase I\n required stormwater permits for large municipalities, industrial plants, and commercial\n construction sites. Phase II requires permits for smaller municipalities and smaller\n construction sites.\n\n\n FIFRA Background\n The Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) regulates the sale,\n distribution, and use of pesticides through a registration system.\n\n\n\n 26\n\x0cRCRA Background\n The Resource Conservation and Recovery Act (RCRA) regulates hazardous and\n municipal and industrial solid waste generated nationwide. RCRA Subtitle C\n establishes a system for controlling hazardous waste from the time of generation until\n ultimate disposal and includes regulations for the generation, transportation,\n treatment, storage, or disposal of hazardous wastes.\n Large/Small Entity Classifications\n The statute defines three categories of generator facilities based on volume:\n Large Quantity Generator Facilities:\n \xe2\x80\xa2 Greater than 1,000 kg of hazardous waste per calendar month (approximately\n 2,200 lbs); or\n \xe2\x80\xa2 Greater than 1 kg of acutely hazardous waste per calendar month (approximately\n 2.2 lbs).\n Small Quantity Generator Facilities:\n \xe2\x80\xa2 Between 100 kg (approximately 220 lbs) and 1,000 kg of hazardous waste per\n calendar month; and\n \xe2\x80\xa2 Accumulate less than 6,000 kg (approximately 13,200 lbs) of hazardous waste at\n any time.\n Conditionally-Exempt Small Quantity Generator Facilities:\n \xe2\x80\xa2 Less than 100 kg of hazardous waste per calendar month; or\n \xe2\x80\xa2 Less than 1 kg of acutely hazardous waste per calendar month; and\n \xe2\x80\xa2 Accumulate less than 1,000 kg of hazardous waste, 1 kg of acute hazardous\n waste, or 100 kg of any residue from the cleanup of a spill of acute hazardous\n waste at any time.\n\n\nSDWA Background\n The Safe Drinking Water Act (SDWA) ensures public health protection by public water\n systems complying with all health-based standards, including monitoring and reporting\n requirements. Through the Public Water System Supervision program, EPA\n implements and enforces drinking water standards to protect public health.\n Large/Small Entity Classifications\n EPA defines Public Water Systems, by type and number of people they serve, as\n either community systems or non-transient non-community systems. A community\n system is a public water system that supplies water to at least 15 service connections\n or 25 people year-round in their primary residences. Non-transient non-community\n systems serve at least 25 of the same persons over 6 months per year. Transient\n non-community water systems serve at least 25 persons (not the same 25 persons)\n over 6 months per year. EPA does not regulate drinking water wells that supply water\n to fewer than 25 people.\n\n\nTSCA Background\n The Toxic Substances Control Act (TSCA) authorizes EPA to secure information on\n and regulate all new and existing chemical substances, as well as control any\n substances determined to cause unreasonable risk to public health or the\n environment. \xe2\x80\x9cCore\xe2\x80\x9d TSCA regulates industrial chemicals, exclusive of\n Polychlorinated Biphenyls, lead-based paint, and asbestos.\n\n\n\n\n 27\n\n\x0c Appendix C\n\n Details on Scope and Methodology\nTo gain a general understanding of OECA\xe2\x80\x99s regulated universe and associated enforcement and\ncompliance activities and resources, we reviewed policy and guidance documents, including:\n\n \xc2\x89 Supplemental Information for EPA Fiscal \xc2\x89 Enforcement Response Policies for program\n Year 2003 Annual \xe2\x80\x93 Integrity Act Report and area sample\n Key Management Challenges\n \xc2\x89 Memoranda of Agreement Guidance \xc2\x89 Fiscal Year 2005 Annual Performance Plan\n Priorities \xe2\x80\x93 Fiscal Years 1996-2007 and Congressional Justification\n \xc2\x89 Fiscal Year 2005 Budget \xc2\x89 End-of-Year Accomplishments Reports\n \xc2\x89 Fiscal Year 2005-2007 National Program\n Guidance\n\nWe also interviewed staff in the following OECA offices:\n\n Office of Site Remediation Enforcement\n Office of Civil Enforcement\n \xc2\x89 Air Enforcement Division\n \xc2\x89 Water Enforcement Division\n \xc2\x89 RCRA Enforcement Division\n \xc2\x89 Toxics and Pesticides Enforcement Division\n Office of Compliance\n \xc2\x89 National Planning Measures and Analysis Division\n \xc2\x89 Enforcement Targeting and Data Division\n \xc2\x89 Compliance Assessment and Media Programs Division\n \xc2\x89 Compliance Assistance and Sector Programs Division\n\n\nWe reviewed relevant reports, including:\n\n EPA Office of Inspector General\n \xc2\x89 EPA\xe2\x80\x99s Key Management Challenges (2001, 2002, 2003)\n \xc2\x89 Information Technology: Unreliable Data Affects Usability of DOCKET Information (2002)\n Government Accountability Office\n \xc2\x89 Environmental Information: EPA Needs Better Information to Manage Risks and Measure\n Results (2000)\n \xc2\x89 Human Capital: Implementing an Effective Workforce Strategy Would Help EPA to Achieve Its\n Strategic Goals (2001)\n \xc2\x89 Major Management Challenges and Program Risks \xe2\x80\x93 Environmental Protection Agency (2003)\n National Academy of Public Administration\n \xc2\x89 Evaluating Environmental Progress: How EPA and the States Can Improve the Quality of\n Enforcement and Compliance Information (2001)\n \xc2\x89 Environmental Performance Measures in a Federal System (2000)\n Environmental Council of States\n \xc2\x89 Report to Congress - State Environmental Agency Contributions to Enforcement and\n Compliance (2001)\n\n\n\n 28\n\n\x0cSample of Six Regulatory Program Areas\n\nWe reviewed and summarized environmental statutes for our sample program areas (see\nAppendix B for descriptions). The table below lists the 2001 universe numbers for the six\nenvironmental statutes covering our sample program areas, with specific program areas reviewed\nin bold type. Our six sample areas comprise just over 16 percent of OECA\xe2\x80\x99s 2001 total universe\nof regulated entities. We obtained the numbers from the Executive Summary OECA included\nwith its 2001 Regulatory Universe Identification Table (see Appendix A).\n\n Statute Programmatic Description 2001 Universe Number\nCAA Major Stationary Sources 23,868\n Minor Stationary Sources 96,866\n Synthetic Minor Stationary Sources 17,248\n Others (includes mobile sources) 441,157\n CAA Total 579,139\nCWA NPDES - Major Permitted Facilities 6,599\n NPDES - Minor Permitted Facilities 87,844\n NPDES \xe2\x80\x93 Stormwater Permits 380,000\n NPDES - Others (includes biosolids and Concentrated 72,328\n Animal Feeding Operations)\n CWA Total 546,771\nRCRA Treatment, Storage, and Disposal Facilities 2,393\n Large Quantity Generator Facilities 20,876\n Small Quantity Generator Facilities 202,965\n Conditionally Exempt Small Quantity Generator Facilities 108,780\n Others (includes transporters, non-notifiers, and other 251,169\n facilities included in EPA\xe2\x80\x99s RCRA data system)\n RCRA Total 586,183\nSDWA Public Water Systems: Community Water Systems 54,101\n Public Water Systems: Transient and Non-Transient 113,463\n Non-Community Systems\n Others (includes underground injection control wells) 822,204\n SDWA Total 989,768\nFIFRA Total Number of Farms and Business Sites Regulated 2,246,512\n under Pesticide Programs\n FIFRA Total 2,246,512\nTSCA Core TSCA - Chemical Manufacturers 13,513\n Core TSCA - Other Manufacturers, Processors, 3,758,176\n Distributors, Users and Exporters\n Others (includes facilities related to Polychlorinated 31,341,591\n Biphenyls and lead-based paint)\n TSCA Total 35,113,280\n Sample Program Area Total 6,755,868\n Table Total 40,061,653\n\n\n\n\n 29\n\n\x0cTo answer both the universe and compliance objectives, we judgmentally selected the six sample\nprogram areas, as case studies, from OECA\xe2\x80\x99s 2001 Regulatory Universe Identification Table.\nThis sample was based on information gathered from our preliminary research. A preliminary\nanalysis of the Regulatory Universe Identification Table showed that OECA\xe2\x80\x99s universe of\n41 million entities included a diverse and complex mix of entities of varying sizes and types.\nSmall entities made up a much greater part of the universe than the major and large entities.\nA review of OECA\xe2\x80\x99s planning documents also showed that a handful of programs were selected\nas national priorities, and others were described as core programs. While OECA is ultimately\nresponsible for regulating all 41 million entities, it focuses its regulatory attention on national\npriority program areas, while the States are primarily responsible for core programs. To conduct\na balanced evaluation of OECA\xe2\x80\x99s knowledge of the regulated universe, it was important to select\na sample which reflected the mix of program areas for which OECA is both ultimately\nresponsible and it had represented as falling under its regulatory authority in its Regulatory\nUniverse Identification Table.\n\nOECA\xe2\x80\x99s Knowledge of Its Regulated Universe\n\nTo determine OECA\xe2\x80\x99s knowledge of the composition and size of its regulated universe, we\nanalyzed OECA\xe2\x80\x99s 2001 regulated universe table. We compared 2001 universe figures to 2005\ndata provided by OECA staff for six sample program areas. We met with staff in EPA\xe2\x80\x99s Office\nof Water to discuss OECA\xe2\x80\x99s universe number for the CWA-NPDES stormwater program. We\nalso interviewed OECA staff to understand any challenges associated with obtaining accurate\nuniverse numbers for our sample regulatory programs, as well as to discuss the benefit of\nknowing the composition and size of the regulated universe. We did not independently verify\nthe accuracy of data provided by OECA. We did review EPA\xe2\x80\x99s fiscal 2005 Annual Performance\nPlan for Goal 5, where EPA identified data problems, limitations, and reliability issues in\nOECA\xe2\x80\x99s databases and data. We used that information to identify limitations in the reliability of\nOECA\xe2\x80\x99s universe numbers. In addition, we applied a methodology described in GAO\xe2\x80\x99s October\n2002 report, Assessing the Reliability of Computer Processed Data (GAO-03-273G). We\nreviewed additional data quality materials, including OECA\xe2\x80\x99s:\n\n \xc2\x89 \xe2\x80\x9cEnforcement and Compliance Reporting \xc2\x89 \xe2\x80\x9cEnsuring Integrity of Reporting Enforcement\n Process for FY 2004\xe2\x80\x9d Memorandum and Compliance Data" Memorandum\n (March 12, 2004) (May 6, 2003)\n \xc2\x89 \xe2\x80\x9cFinal Enforcement and Compliance Data \xc2\x89 Information from OECA\xe2\x80\x99s Web site on various\n Quality Strategy\xe2\x80\x9d (March 25, 2002) enforcement and compliance databases\n \xc2\x89 Data quality quarterly reviews\n\n\n\nChanges in Regulatory Compliance\n\nTo determine how OECA determines and reports levels of compliance in its regulated universe,\nwe reviewed general information on compliance monitoring as well as how OECA defines\ncompliance through its enforcement response policies. We interviewed representatives from the\nNational Academy of Public Administration, the Environmental Law Institute, and the\nEnvironmental Compliance Consortium on how OECA measures changes in regulatory\ncompliance. We interviewed OECA managers and staff to understand how they determine and\n\n\n 30\n\n\x0creport compliance in our sample program areas, as well as any challenges associated with\ndemonstrating changes in regulatory compliance. We discussed conditions we identified during\nthe review of the sample with OECA managers to determine whether the problems were limited\nto the program areas, or whether they were systemic and OECA-wide conditions. In instances\nwhere the condition was systemic, we reported it as a general condition. We analyzed the cause\nof the problem, and developed recommendations to address the cause of the condition.\n\n\n\n\n 31\n\n\x0c Appendix D\n\n Agency Response Memorandum to Draft Report\n\n\n UNITED STATES ENVIRONMENTAL PROTECTION AGENCY\n WASHINGTON, D.C. 20460\n\n OFFICE OF\n ENFORCEMENT AND\n COMPLIANCE ASSURANCE\n\n\n\n\n August 1, 2005\n\nMEMORANDUM\n\n\nSUBJECT: \t The Office of Enforcement and Compliance Assurance=s Agency Response to the\n Draft, AEvaluation Report: Limited Knowledge of the Universe of Regulated\n Facilities Impede=s EPA=s Ability to Demonstrate Changes in Regulatory\n Compliance,@ dated June 30, 2005\n\nFROM: T\n \t homas V. Skinner\n Acting Assistant Administrator\n\nTO: \t Jeffrey K. Harris\n Director\n Program Evaluation, Cross-Media Issues\n Office of Inspector General\n\n\nIntroduction\n\n Today, on behalf of the Office of Enforcement and Compliance Assurance (OECA) and\nas the designated AAction Official,@ I am forwarding to you our consolidated Agency Response\n(Response) regarding the Office of Inspector General=s (OIG) draft AEvaluation Report: Limited\nKnowledge of the Universe of Regulated Facilities Impede=s EPA=s Ability to Demonstrate\nChanges in Regulatory Compliance,@ dated June 30, 2005.\n\n In accordance with the instructions provided in your June 30, 2005 memorandum, the\nResponse addresses the factual accuracy of the draft Evaluation Report. Consistent with those\ninstructions, the Response also specifically indicates whether OECA concurs with each of the\nrecommendations proposed by the OIG. Further, to the extent that action has already been\ninitiated or planned to address issues identified in the draft Evaluation Report, the Response\n\n\n 32\n\n\x0cspecifically identifies those actions that have been initiated or planned. Finally, your June 30,\n2005 memorandum expressly states that the Afinal report will include an assessment of [the]\ncomments@ made in the Response. Consequently, I am specifically requesting that this\nmemorandum and the attached Response be attached to, and be made a part of, the final version\nof the draft Evaluation Report.\n\nPrincipal Concerns\n\n While we appreciate the effort made by the OIG to provide a useful review of the data we\nhad developed in 2001 to better understand the regulatory universe, we were troubled by a\nnumber of practices which were used in this evaluation. We would ask that the OIG consider our\nconcerns both for purposes of possible revisions to this evaluation and as suggestions about the\nconduct and content of future evaluations. Given the amount of time that the staff and managers\nof OECA and OIG are investing in these evaluations, it is important that the evaluations focus on\nthe most important issues, provide clear and meaningful findings, and yield recommendations\nthat add value and improve the effectiveness of our Agency programs.\n\n Here are the general comments we offer about the report as a whole. Comments on\nspecific findings and recommendations are included in the attached response.\n\n 1.\t Limitations of Judgmental Sampling. We are very concerned that judgmental\n sampling of six segments of the regulated universe produced results that are biased\n (i.e., not representative of the larger population being\n See OIG Response in\n examined). Judgmental sampling can only characterize the Appendix E, Note 1\n types of problems that might exist in the full set of universe\n data, but it cannot quantify the size of the problem for the full\n set of universe data. While the OIG states that it Acannot use our judgmental sample to\n generalize to OECA=s entire population,@ that caveat did not constrain the OIG from\n making a very sweeping set of recommendations to address the issues it identified\n through the use of a technique with very significant limitations. (See pages 2 and 3 of\n our attached response for a full description of our concerns.)\n\n 2. \t Findings Broader Than the Supporting Evidence. In several instances, the evaluation\n presents findings that are much broader than the issue the OIG is raising or the evidence\n it is providing. The primary example of this problem is the\n finding entitled, AOECA Does Not Focus Activities on See OIG Response in\n Majority of Facilities,@ on page 8 of the draft evaluation. Appendix E, Note 2\n While it may be true that in recent years enforcement actions\n have focused increasingly on larger facilities associated with OECA=s national\n enforcement priorities (though we note that the OIG presents no quantitative evidence of\n this), OECA=s compliance assistance program since its inception has been focused on\n serving smaller facilities which often lack the resources or expertise to achieve and\n maintain compliance. Of the 14 OECA-sponsored online Compliance Assistance Centers\n (visited more than one million times last year), 12 are serving sectors comprised\n primarily of small businesses. In addition, other compliance assistance initiatives in\n sectors dominated by small businesses (e.g., dry cleaners, construction, auto repair) have\n\n\n\n 33\n\n\x0c been undertaken to provide tailored assistance to meet the specialized needs of small\n businesses. (See pages 8 and 9 of our attached response for a full description of our\n concerns.)\n\n A second example of the IG drawing findings which are too broad is its assertion that\n AOECA Lacks Transparency in Sharing Data,@ on page 15 of the draft report. In our\n response we point out the many different types of compliance data we make available to\n the public. We describe the Enforcement and Compliance History Online (ECHO) web\n site which has been used by two million visitors to find facility-specific inspection and\n enforcement data. We also detail in Table 3 (pages 19-21), the various types of\n compliance data we make publicly available for all the major EPA programs. The OIG\n may want more transparency, but to say that OECA lacks transparency in sharing data is\n inaccurate.\n\n3. \t Recommendations Broader than the Findings Being Addressed. Several of the\n recommendations seem disproportionate to the problem they are meant to solve. The\n primary example of this disproportionality is\n Recommendation 2.4, specifically the portion of the See OIG Response in\n Appendix E, Note 3\n recommendation directing OECA to require states to Atrack,\n record, and report data for facilities over which states have\n regulatory responsibility... [and] develop a multi-state, multi-program pilot program for\n tracking, recording, verifying, and reporting of state data.@ Because the OIG has\n identified that OECA has not updated its universe table since 2001 and a determination\n (through a non-representative judgmental sample of six regulated populations) that the\n universe table must be updated, it now recommends a very large and resource-intensive\n goal of increasing state reporting. In our response, we describe the significant regulatory,\n procedural and resource barriers to increasing state reporting. We believe that the\n Asolution@ (i.e., significant new data reporting requirements for states) is a complete\n mismatch and way out of proportion to the Aproblem@ (i.e., the need for more reliable\n universe data). (See page 6 of our response for a full description of our concerns about\n increasing the reporting burden on states.)\n\n4. \t Exclusion of Relevant Information. We believe the evaluation at times ignores\n important information about OECA programs and leads to poorly conceived findings and\n recommendations. For example, in the section about\n OECA=s description of its role, the evaluation does not See OIG Response in\n provide a complete and accurate description of the relative Appendix E, Note 4\n roles of EPA and the states, opting instead for a very\n simplistic view which seems to color the OIG=s judgment about the findings and\n recommendations in the report. Additionally, as described above, the OIG fails to take\n into account that OECA=s compliance assistance program is and always has been focused\n on sectors comprised of small facilities. As a result, the OIG makes an inaccurate finding\n that AOECA Does Not Focus Activities on Majority of Facilities.@\n\n\n\n\n 34\n\n\x0c 5. Disconnect from Procedural /Resource Barriers and Competing Demands. In some\n of the recommendations in this evaluation, the OIG does not take into account the\n regulatory and procedural obstacles that would be\n encountered and the significant resource commitments that See OIG Response in\n would be necessary for implementation. Similarly, the OIG Appendix E, Note 5\n seems never to consider existing demands already placed on\n organizations and the relative importance of those demands versus proposed OIG\n recommendations. The OIG=s recommendation for increased state reporting of data\n would require a large investment of personnel and energy to overcome the regulatory and\n procedural barriers and resource shortfalls described in our response (see page 6).\n Moreover, it could well disrupt OECA=s and the states= progress toward modernizing\n single-media data systems and incorporating them into the Integrated Compliance\n Information System (ICIS).\n\n\n We look forward to working with you to finalize this report and on future evaluations of\nOECA programs.\n\ncc: P\n \t hyllis Harris\n Michael Stahl\n Walker Smith\n\n\n\n\n 35\n\n\x0c Appendix E\n\n Agency Comments to OIG Recommendations\n and OIG Evaluation\nThe Agency comments consisted of two parts \xe2\x80\x93 a transmittal memorandum with summary\ndiscussion, and a document with detailed comments. We included the memorandum as\nAppendix D; the detailed comments are available on the EPA OIG Web site at\nwww.epa.gov/oig. Two lists of reports that were part of OECA\xe2\x80\x99s detailed comments are\nprovided in Appendix F. Appendix E includes each specific OIG recommendation, the Agency\xe2\x80\x99s\ncomments on those recommendations, and our evaluation of comments from both Agency\ncomment documents. In this appendix, we first evaluate the Agency\xe2\x80\x99s comments from the\ndetailed comment document to our recommendations, and follow this in Notes 1 through 5 with\nour responses to the five points discussed in the Agency\xe2\x80\x99s memorandum in Appendix D.\n\nOIG Recommendation 2-1: Biannually update publicly released universe figures by tracking\nand recording the number of entities over which it has oversight and primary regulatory\nresponsibility.\n\n Agency Comments: Non-concur. As an alternative, only universe figures updated\n within the previous two years will be released to the public. OECA will begin the process\n of updating universe figures for populations associated with its national priorities and\n complete that update within six months. Further, OECA will remove the 41 million\n universe figure from its National Program Guidance and not use it in any future public\n documents.\n\n OIG Evaluation: Updating the universe numbers for populations associated with the\n handful of national priority program areas is a beginning. However, it will not indicate\n the complete universe of regulated entities for which OECA is responsible for ensuring\n compliance. Not releasing the number of regulated facilities will subject EPA to the\n same July 2001 GAO criticism noted in Chapter 1 of this report. GAO had stated that\n OECA could not demonstrate \xe2\x80\x9cthe universe of entities subject to regulation under federal\n environmental laws.\xe2\x80\x9d OECA has not provided any reasons to show there is more value to\n updating the universe of the five national priority areas in lieu of all program areas.\n Additionally, OECA has not provided any timetable for updating universe information on\n a regular basis. OECA officials said OECA management has assigned other projects\n higher priority. Nevertheless, as OECA itself has recognized, knowledge of the size and\n character of the regulated universe is fundamental for a regulatory body, and we continue\n to recommend that OECA assign a higher priority to regularly updating that universe.\n\nOIG Recommendation 2-2: When producing its biannual universe update, use reliable data to\ngenerate complete and current universe numbers that meet national data quality standards similar\nto those outlined in OECA\xe2\x80\x99s Final Enforcement and Compliance Data Quality Strategy.\n\n\n\n\n 36\n\n\x0c Agency Comments: Concur. When updating universe figures as described in response\n to recommendation 2.1, OECA will use complete, accurate, and current data in\n accordance with national data standards including those in OECA=s Data Quality\n Strategy.\n\n OIG Evaluation: OECA qualified its concurrence to ensure the quality of the universe\n data by referencing its response to Recommendation 2-1 on updating the numbers for\n national priority program areas only. We consider this a good start, but it will not meet\n the objective of knowing the full universe of regulated entities. OECA also commented\n that we should describe the data problems in universe numbers as data gap problems,\n explaining that data gaps exist when the statutes do not specifically require States to\n report data to EPA. OECA indicated that requiring more data would be burdensome to\n States due to shortfalls in State resources. OECA explained its use of estimates where\n data gaps existed, and objected to our using GAO criteria to evaluate data quality.\n We have modified the text to provide additional sources we used to find weaknesses in\n data quality. We also revised the discussion of the problems we found in the estimated\n numbers in OECA\xe2\x80\x99s universe table. We cannot provide an opinion on the issue of\n resources as we have not analyzed OECA\xe2\x80\x99s resource allocations and the process by\n which management assigns relative weight to tasks as part of this evaluation.\n\nOIG Recommendation 2-3: Describe OECA\xe2\x80\x99s enforcement and compliance role in relation to\nStates and other partners when the Agency publicly releases universe figures.\n\n Agency Comments: Concur. OECA will develop and add a more precise description to\n relevant documents to more clearly explain the respective roles and responsibilities of\n EPA and the states in maximizing compliance in the regulated universe.\n\n OIG Evaluation: We accept the concurrence.\n\nOIG Recommendation 2-4: Develop an objective of having the most up-to-date and reliable\ndata on all entities that fall under its regulatory responsibility. OECA should adopt the goals of\nrequiring States to track, record, and report data for entities over which States have regulatory\nresponsibility. To achieve this goal, OECA should develop a multi-State, multi-program pilot\nprogram of collecting data that States track, record, verify, and report.\n\n Agency Comments: Non-concur. While OECA believes that it currently subscribes to\n an objective of having current and accurate facility data, we believe this recommendation\n fails to take into account the significant procedural and resource barriers which impede\n expanded collection and reporting of data from states. Further, we believe that given the\n resource constraints under which the states are operating, a multi-state, multi-program\n pilot would be resisted vehemently by the states.\n\n As an alternative, OECA and the states will continue on their current path of\n modernizing single-media data systems and integrating them into ICIS (Integrated\n Compliance Information System). The modernization process is being done in full\n\n\n\n\n 37\n\n\x0c consultation with states, on a schedule developed jointly by EPA and the states, and will\n achieve many of the improvements sought by the OIG.\n\n OIG Evaluation: We are aware of resource and procedural barriers in collecting\n additional information from States, and therefore recommended that OECA obtain\n knowledge in incremental steps by conducting pilot data collection programs. The pilot\n might reveal that data are readily available in some of the States. Also,\n Recommendation 2-4 addresses the problem of smaller and minor entities. OECA still\n needs to indicate how the modernization of OECA\xe2\x80\x99s data systems will address the\n condition under discussion.\n\nOIG Recommendation 2-5: Request that EPA program offices analyze and report to OECA the\ncumulative impact of violations by regulated entities that pollute below the thresholds of major\nor large entities. OECA should use any cumulative impact analyses conducted by program\noffices to inform OECA\xe2\x80\x99s management decisions.\n\n Agency Comments: Non-concur. As an alternative, OECA will request from program\n offices any and all currently available data and analyses of cumulative impact of small\n facilities. OECA will make this request within sixty days of the date of the final version of\n this OIG evaluation. OECA will use this information to identify emerging problems in\n sectors comprised of smaller facilities.\n\n OIG Evaluation: Collecting available analyses from program offices, and using the\n information to identify emerging problems in smaller facilities, is a move in the right\n direction. OECA may find that many such analyses exist within program offices.\n However, we recommend that OECA also request analyses for program areas where such\n analyses have not been done. As shown in Chapter 2, small and minor entities comprise\n a large part of the regulated universe; thus, it is important that OECA know their\n cumulative impact. OECA occupies a distinctive position in EPA in that it cuts across\n the Agency\xe2\x80\x99s program offices. At a minimum, OECA can use these data and analyses to\n support its decisions to focus its enforcement and monitoring activities.\n\nOIG Recommendation 3-1: To show the results of its national enforcement and compliance\nprogram in maximizing compliance with environmental statutes, develop and publish\ninformation that demonstrates changes in compliance within the regulated universe, by program\nareas. Include any appropriate explanations of data quality issues or data caveats.\n\n Agency Comments: Non-concur. As an alternative, OECA will share with the public\n any statistically-valid compliance rates it has developed in the past year and any\n statistically valid rates it will develop in the future.\n\n OIG Evaluation: We reiterate that OECA should develop and publish information that\n shows changes in the levels of compliance within various program universes. OECA\xe2\x80\x99s\n proposed alternative to publish only statistically valid rates falls short of the\n recommendation, considering the limitations OECA itself stated in its request for\n assistance from us in developing methodologies for producing statistically valid\n\n\n\n 38\n\n\x0c noncompliance rates. In the detailed response to this report (available on the EPA OIG\n Web site at www.epa.gov/oig), OECA listed a total of 10 projects to develop statistically\n valid noncompliance rates from Fiscal Years 2000 through 2006. Each of the projects\n deals with a very limited sector or population. Given the limited number and scope of\n OECA\xe2\x80\x99s statistically valid noncompliance rates, releasing only those rates would show\n extremely limited results of OECA\xe2\x80\x99s efforts to maximize compliance in the regulated\n universe.\n\n Additionally, we note that OECA\xe2\x80\x99s response to Recommendation 3-1 is inconsistent with\n its response to Recommendation 2-1. In our opinion statistically valid noncompliance\n rates cannot be generated without knowledge of the size of a program area\xe2\x80\x99s universe.\n Therefore, OECA\xe2\x80\x99s response to this recommendation is untenable.\n\nOIG Recommendation 3-2: Share compliance data and analyses with external stakeholders to\nprovide a better understanding of programmatic compliance levels; include explanatory notes as\nneeded to ensure proper representation and understanding.\n\n Agency Comments: Concur. OECA will expand the amount of compliance data it will\n make available to the public on the EPA web site. The data which will be added to the\n web site is listed in Table 4 and includes a wide variety of information about compliance\n monitoring, enforcement cases, citizen complaints, cases resulting from voluntary\n disclosures, and more. OECA will post this data on the web site within 60 days of the\n release of the final version of this evaluation.\n\n OIG Evaluation: We commend OECA for its concurrence with Recommendation 3-2\n and its expansion of the amount of compliance data it plans to make available on the EPA\n Web site. We have provided, in Appendix F, two tables from OECA\xe2\x80\x99s response.\n Releasing this additional information will increase transparency. However, we found that\n OECA did not include some other compliance-related rates that OECA generates. We\n reiterate the recommendation that OECA should publish the other compliance\n information that shows significant noncompliance rates, \xe2\x80\x98hit\xe2\x80\x99 rates, recidivism, and the\n time taken to bring violators to compliance. If OECA presents the information with\n appropriate explanations, readers should understand what the data represents. This\n information more directly links to compliance than most other measures OECA currently\n releases or plans to release. They show the results of OECA\xe2\x80\x99s enforcement and\n compliance monitoring activities on noncompliant entities and facilities.\n\n\nNotes on Principal Concerns\n\nIn this section we provide a summary of OECA\xe2\x80\x99s comments (italicized) from the Agency\nmemorandum (included in full in Appendix D), followed by our evaluation.\n\nNOTE 1 - Limitations of Judgmental Sampling: We are very concerned that judgmental\nsampling of six segments of the regulated universe produced results that are biased (i.e., not\nrepresentative of the larger population being examined).\n\n\n 39\n\n\x0c OIG Response: We do not consider OECA\xe2\x80\x99s critique of our sampling approach and\n subsequent analysis valid. We used a qualitative \xe2\x80\x93 not quantitative methodology in this\n evaluation. Therefore, OECA\xe2\x80\x99s concerns about quantifying from our sample are not\n applicable. To clarify any questions on our methodology, we added more details in the\n report and provide some additional detail and rationale below.\n\n The use of a judgmental sample was appropriate to answer our objectives. We selected a\n small judgmental sample to obtain case studies for a \xe2\x80\x98qualitative\xe2\x80\x99 and not a \xe2\x80\x98quantitative\xe2\x80\x99\n evaluation. We did not quantify, or extrapolate, the results from our sample across the\n entire universe. While we did determine that the size of the sample had changed by\n 35 percent between 2001 and the present, this figure clearly did not apply to the entire\n universe. During the course of our review, when we found a condition that was systemic,\n we interviewed OECA managers to determine whether the condition was limited to the\n program area under review or was an OECA-wide policy or practice. Only when OECA\n managers confirmed that it was an OECA-wide policy or practice did we present it as a\n broader finding with a recommendation. We included appropriate explanatory caveats in\n those cases.\n\n We determined that case study analysis was an appropriate research design for our review\n because our objective sought to determine how well OECA knew the composition and\n size of universes within program areas. Statistical models are best used to measure\n across cases, while case studies are best to investigate within a case since they allow one\n to explore more in depth. While a judgmental sample limits generalization, the\n accompanying in-depth analysis can allow the evaluator to pursue questions that arise and\n determine whether a problem is endemic across the universe or confined to the sample.\n\n We do not agree with OECA\xe2\x80\x99s opinion that the results of a judgmental sample are\n automatically biased. Selection bias usually occurs when the sampling includes some\n type of systematic error. In designing the sampling plan, we took care to capture an array\n of program areas for which OECA bears responsibility: small and large, across media\n areas, national priority and core programs, and those in which OECA shares regulatory\n responsibility with States versus those it does not. This sampling technique explicitly\n acknowledged the heterogeneity of OECA program areas. This sample is more\n representative of the program areas OECA included in the universe totals than if we had\n included only major programs or national priority programs in a sample.\n\nNOTE 2 - Findings Broader than the Supporting Evidence: In several instances, the\nevaluation presents findings that are much broader than the issue the OIG is raising or the\nevidence it is providing.\n\n OIG Response: We do not agree with OECA\xe2\x80\x99s comment. We revised some of the\n report language to ensure statements may not be interpreted to imply more than intended.\n We also expanded the description of our scope and methodology to explain when the\n finding or recommendation is broader than the specific program area in our sample. As\n discussed regarding Note 1, during the course of our review we found certain conditions\n\n\n\n 40\n\n\x0c that could be systemic across the entire universe for which OECA is responsible. After\n confirming with OECA managers, we presented each as a finding that may be broader\n than the specific program area in our sample. We added language to explain this. Also,\n as suggested by OECA, we added discussion on OECA\xe2\x80\x99s use of compliance assistance\n activities to target small businesses and entities. While OECA provides compliance\n assistance to many small entities, it nevertheless does not focus the majority of its\n compliance monitoring resources on small entities. In reviewing our sample areas we\n found that many of OECA\xe2\x80\x99s databases do not include compliance monitoring histories, or\n even facility identification information for many of the small entities over which OECA\n has oversight responsibility. OECA\xe2\x80\x99s response does not address this point.\n\nA second example of the IG drawing findings which are too broad is its assertion that AOECA\nLacks Transparency in Sharing Data\xe2\x80\xa6.@\n\n OIG Response: We added Tables 3 and 4 from OECA\xe2\x80\x99s response to Appendix F of our\n report (the tables are shown in our report as Tables 1 and 2). These tables list the types of\n compliance information that OECA now releases. However, OECA does not publicly\n release compliance information that can inform Congress and the public as to whether\n compliance has increased across programs. While OECA makes facility-specific\n compliance information available on its ECHO Web site, Congress and the public do not\n have the ability to determine whether OECA is achieving its central goal of maximizing\n compliance across the regulatory programs for which it is responsible. Programmatic\n compliance information will allow OECA to know whether compliance is increasing or\n decreasing, whether its targeting strategies are successful, and whether its mix of\n compliance tools and methods is effective. It also provides Congress and the public the\n means to determine whether there are regional and geographic differences in\n environmental problems, approaches, and solutions. OECA already produces some\n compliance rates in Online Tracking Information System management reports. We urge\n OECA to release to the public the information it currently possesses internally, with the\n necessary caveats.\n\nNOTE 3 - Recommendations Broader than the Findings Being Addressed: Several of the\nrecommendations seem disproportionate to the problem they are meant to solve. The primary\nexample of this disproportionality is Recommendation 2-4, specifically the portion of the\nrecommendation directing OECA to require states to Atrack, record, and report data for facilities\nover which states have regulatory responsibility... [and] develop a multi-state, multi-program\npilot program for tracking, recording, verifying, and reporting of state data.@\n\n OIG Response: We do not agree with OECA\xe2\x80\x99s comment that the recommendations are\n disproportionate to the problems they are supposed to remedy. We developed the\n recommendations to match the breadth of the conditions and causes. If a condition is due\n to a systemic cause, the recommendation addresses the cause of the problem. OECA has\n oversight responsibilities over State compliance and enforcement activities of many small\n entities. OECA agrees in its response that the knowledge of the size and character of the\n regulated community is fundamental to a regulatory agency\xe2\x80\x99s effectiveness. Our analyses\n show that some of OECA\xe2\x80\x99s universe has changed. OECA has explained it has data gaps\n\n\n\n 41\n\n\x0c because States do not report on smaller entities. Therefore we concluded that OECA\n does not know the size, nature, or character of its full universe because of these gaps.\n This limits OECA\xe2\x80\x99s ability to exercise its oversight responsibilities, and we developed\n recommendations addressing this problem. Also, OECA needs a broad range of\n knowledge of regulated entities for which it is responsible to fully exercise oversight, and\n again we made recommendations accordingly. This data will provide OECA, Congress,\n and the public with information on entities for which EPA is ultimately responsible.\n\nNOTE 4 - Exclusion of Relevant Information: We believe the evaluation at times ignores\nimportant information about OECA programs and leads to poorly conceived findings and\nrecommendations.\n\n OIG Response: We do not agree with OECA\xe2\x80\x99s comments. We gathered and reviewed\n extensive amounts of documentation and information, and held numerous meetings with\n OECA staff and managers. We also met with external stakeholders and experts and\n reviewed documents and reports from external sources (see Appendix C). We reported\n the significant conditions that we identified as a result of the evaluation, and included the\n information pertinent to presenting the findings and recommendations. We do not\n believe we omitted important information in our draft report. Nonetheless, as suggested\n by OECA, we added information on OECA\xe2\x80\x99s use of compliance assistance for interacting\n with small entities, as well as additional information on various other issues.\n\n OECA also commented that we offered a simplistic view of OECA\xe2\x80\x99s relationship with\n the States that resulted in mischaracterized findings and recommendations. We recognize\n that OECA has broad and varied relationships with States. One of these central\n relationships is OECA\xe2\x80\x99s oversight of States\xe2\x80\x99 program activities, and without obtaining\n knowledge of the full universe, OECA cannot fully exercise this oversight authority.\n\nNOTE 5 - Disconnect from Procedural/Resource Barriers and Competing Demands:\nIn some of the recommendations in this evaluation, the OIG does not take into account the\nregulatory and procedural obstacles that would be encountered and the significant resource\ncommitments that would be necessary for implementation.\n\n OIG Response: We recognize OECA\xe2\x80\x99s regulatory and procedural constraints in\n obtaining data from States that States are not required by statute to provide. However, in\n our opinion, it is fundamental for OECA to have an adequate range of knowledge about\n the entities it regulates, and OECA agrees with this central premise. Since it is critical for\n OECA to have this knowledge, it should pursue the best possible procedures to get this\n data. Due to the procedural constraints faced by OECA, we recommended that they\n accomplish this incrementally. Expanded universe data will ultimately provide OECA,\n and therefore Congress and the public, with increased information on entities for which\n EPA is responsible for regulating compliance.\n\n\n\n\n 42\n\n\x0c Appendix F\n\n Publicly Available Compliance and \n\n Enforcement Measures \n\n\nOECA provided the following two lists of reports that it plans to make publicly available\nduring 2005.\n\n\n Table1. Enforcement and Compliance Measures and Reports\n FY(s)\n Data Available Reports/Website in Which Measure Appears\nAcres of Wetlands Mitigated 2000-2004 Five-Year Trend Charts1, Numbers-at-a-Glance1\nAdministrative Compliance 2000-2004 Results Summary1, Five-Year Trend Charts1,\nOrders Numbers-at-a-Glance,ECHO2\nAdministrative Penalties 2000-2004 Results Summary, Five-Year Trend Charts,\n Numbers-at-a-Glance, ECHO\nAdministrative Penalty Orders 2000-2004 Results Summary, Five-Year Trend Charts,\n Numbers-at-a-Glance, ECHO\nCivil Case Highlights 2004 Case Highlights1, ECHO\nCivil Judicial Referrals 2000-2004 Results Summary, Five-Year Trend Charts,\n Numbers-at-a-Glance\nCivil Judicial Settlements 2000-2004 Five-Year Trend Charts, ECHO\nCompliance Assistance Activity 2004 Case Highlights\nCompliance Incentives Program 2000-2004 Results Summary\nComplying Actions 2003-2004 Results Summary, ECHO\nCriminal Case Highlights 2004 Case Highlights\nCriminal Defendants Charged 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nCriminal Fines and Restitution 2000-2004 Results Summary, Numbers-at-a-Glance\nCriminal Investigations 2000-2004 Five-Year Trend Charts\nCriminal: Judicial Mandated 2000-2004 Numbers-at-a-Glance\nProjects\nCriminal: Pounds of Pollution 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nReduced, Treated or Properly\nManaged\nCubic Yds of Contaminated Soil 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nto be Cleaned Up\nCubic Yds of Contaminated 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nWater to be Cleaned Up\nEntities Reached through 2000-2004 Results Summary\nCompliance Assistance\nEnvironmental Crime Cases 2000-2004 Five-Year Trend Charts\nInitiated\n\n\n\n\n 43\n\n\x0c FY(s)\n Data Available Reports/Website in Which Measure Appears\nEnvironmental Homeland 2000-2004 Numbers-at-a-Glance\nSecurity Cases Initiated\nFacilities Resolved 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nFinal Administrative Penalty 2000-2004 Results Summary, Five-Year Trend Charts,\nOrders Numbers-at-a-Glance, ECHO\nGallons of Wastewater/ 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nGroundwater Treated\nGPRA Goal 5 Civil Investigations 2004 Annual Performance Report3, Results Summary,\n Five-Year Trend Charts, Numbers-at-a-Glance\nGPRA Goal 5 Complying Actions 2004 Annual Performance Report, Results Summary, ECHO\nGPRA Goal 5 Criminal 2004 Annual Performance Report, Results Summary,\nInvestigations Numbers-at-a- Glance\nGPRA Goal 5 Develop and Use 2004 Annual Performance Report, Results Summary\nCompliance Rates\nGPRA Goal 5 Entities Reached 2004 Annual Performance Report, Results Summary,\nthrough Compliance Assistance Five-Year Trend Charts, Numbers-at-a-Glance\nGPRA Goal 5 EPA-Assisted 2004 Annual Performance Report, Results Summary, ECHO\nInspections\nGPRA Goal 5 Facilities with 2004 Annual Performance Report, Results Summary,\nVoluntary Disclosures Five-Year Trend Charts, Numbers-at-a-Glance\nGPRA Goal 5 Federal 2004 Annual Performance Report, Results Summary,\nInspections Five-Year Trend Charts, Numbers-at-a-Glance, ECHO\nGPRA Goal 5 Pollutant 2004 Annual Performance Report, Results Summary, ECHO\nReductions\nIncarceration 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nInjunctive Relief 2000-2004 Results Summary, Five-Year Trend Charts,\n Numbers-at-a- Glance, ECHO\nJudicial Penalties 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance, ECHO\nPounds of Contaminated 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nSoil/Sediment to be Cleaned Up\nPounds of Pollutants Reduced, 2000-2004 Results Summary, Five-Year Trend Charts,\nTreated or Properly Managed Numbers-at-a-Glance\nLinear Feet of Stream Mitigated 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nNotices of Determination (NODs) 2000-2004 Five-Year Trend Charts, Number at a Glance\nPeople Protected by SDWA 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nEnforcement\nReferrals 2000-2004 Five-Year Trend Charts\nStipulated Penalties 2002-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nSuperfund: % of Cost Recovery 2000-2004 Five-Year Trend Charts, Numbers-at-a-Glance\nStatute of Limitation Cases\nAddressed with Total Past\nGreater than or Equal to\n$200,000\n\n\n\n\n 44\n\n\x0c FY(s)\n Data Available Reports/Website in Which Measure Appears\nSuperfund: % of Remedial 2004 Five-Year Trend Charts, Numbers-at-a-Glance\nAction (RA) Starts Where\nSettlement Reached or\nEnforcement Taken by the Time\nof the RA Start (during the FY) at\nNon-Federal Superfund Sites\nthat Have Known Viable, Liable\nParties\nSuperfund: Private Party 1995-2004 Results Summary, Five-Year Trend Charts,\nCommitments ($ in millions for Numbers-at-a-Glance\npast cost)\nSuperfund: PRP-financed RA 2000-2003 Results Summary, Five-Year Trend Charts,\nStarts Numbers-at-a-Glance\nSuperfund: PRP-lead RA Starts 2000-2003 Numbers-at-a-Glance\n(%)\nSuperfund Private Party 1995-2004 Results Summary, Five-Year Trend Charts,\nCommitments ($ in millions for Numbers-at-a-Glance\nfuture response work including\ncashouts)\nSupplemental Environmental 2000-2004 Results Summary, Five-Year Trend Charts,\nProjects (SEPs) Numbers-at-a-Glance, ECHO\nStatistically Valid Noncompliance 2002, 2004 2004 Combined Sewer Overflow Statistically Valid\nRates for Combined Sewer Noncompliance Rate Study4\nOverflows\n\n\n1\n The six reports comprising OECA=s annual press release for FY 2004 are: Results Summary, Numbers-at-a-Glance,\nCriminal Enforcement Highlights, Civil Enforcement Highlights, Compliance Assistance Highlights, and Five-Year\nTrends. The reports are available on EPA=s Web site: http://cfpub.epa.gov/compliance/resources/reports/endofyear\n2\n The Enforcement and Compliance History Online (ECHO) Web site (http://www.epa.gov/echo) provides facility-\nlevel compliance monitoring, compliance status, enforcement action, and penalty data from 2002-2004 for facilities\nregulated as Clean Air Act (CAA) stationary sources, Clean Water Act (CWA) permitted dischargers (under the\nNational Pollutant Discharge Elimination System, and Resource Conservation and Recovery Act (RCRA) hazardous\nwaste sites.\n3\n The Annual Performance Reports for 1999 through 2004 are available at\nhttp://cfpub.epa.gov/compliance/resources/reports/gpra/\n4\n Available at http://www.epa.gov/compliance/resources/publications/data/planning/priorities/cwacsosvnrstudy.pdf\n\n\n\n\n 45\n\n\x0c Table 2. Enforcement and Compliance Measures and Reports\n\n FY(s)\n Data Available Reports in Which Measure Appears\nAcres of wetlands restored 2002-2003 FY 2002 - FY 2003 National Estimates of\n Environmental Benefits\nAdministrative Penalty Order (APO) 1991-2003 FY 1991 - FY 2003 Administrative Penalty Order\ncomplaints by statute/program Complaints, FY 1999 - FY 2003 Case Initiations-\n Administrative Orders\nAdministrative Compliance Orders by 1999-2003 FY 1999 - FY 2003 Case Initiations-Administrative\nstatute Orders, FY 1999 - FY 2003 Case Conclusions\nAdministrative penalties by statute ($) 1974-2003 1974 - FY 2003 Enforcement Penalties, FY 1999 -\n FY 2003 Administrative and Civil Judicial Penalties,\n FY 1999 - FY 2003 Penalties\nCases against facilities initiated as a 1999-2003 FY 1999 - FY 2003 Voluntary Disclosure Policy\nresult of voluntary disclosure under\naudit policy\nCases against companies initiated as 1999-2003 FY 1999 - FY 2003 Voluntary Disclosure Policy\na result of voluntary disclosure under\naudit policy\nCases with SEPs by statute 1999-2003 FY 1999 - FY 2003 Supplemental Environmental\n Projects (SEPs)\nCitizen complaint responses by 2003 FY 2003 Citizen Complaints\nprogram area\nCitizen complaints received by 2003 FY 2003 Citizen Complaints\nprogram area\nCivil investigations by national priority 2002-2003 FY 2002 - FY 2003 Civil Investigations\nand non-priority program area\nCivil judicial settlements by statute 1999-2003 FY 1999 - FY 2003 Case Conclusions\nCivil judicial referrals 1973-2003 FY 1973 - FY 2003 Civil Judicial Referrals and\n Penalties, FY 1999 - FY 2003 Case Initiations-Civil\n Judicial Referrals\nCivil judicial penalties ($) 1973-2003 FY 1973 - FY 2003 Civil Judicial Referrals and\n Penalties, FY 1974 - FY 2003 Enforcement\n Penalties, FY 1999 - FY 2003 Administrative and\n Civil Judicial Penalties, FY 1999 - FY 2003\n Penalties\nCriminal referrals 1983-2003 FY 1983 - FY 2003 Criminal Referrals and\n Penalties, FY 1998 - FY 2003 Criminal Enforcement\n Program Activities\nCriminal defendants charged 1998-2003 FY 1998 - FY 2003 Criminal Enforcement Program\n Activities\nCriminal cases initiated 1998-2003 FY 1998 - FY 2003 Criminal Enforcement Program\n Activities\nCriminal penalties ($) 1974-2003 1974 - FY 2003 Enforcement Penalties, FY 1983 -\n FY 2003 Criminal Referrals and Penalties, FY 1999\n - FY 2003 Penalties\n\n\n\n 46 \n\n\x0c FY(s)\n Data Available Reports in Which Measure Appears\nExpedited penalty order complaints 2003 FY 2003 Expedited Administrative Penalty Orders\nby program area\nExpedited penalty order settlements 2003 FY 2003 Expedited Administrative Penalty Orders\nby program area\nFederal inspections and evaluations 1999-2003 FY 1999 - FY 2003 Federal Inspections and\nby program Evaluations\nFederal inspections and evaluations 1994-2003 FY 1994 - FY 2003 Federal Inspections and\nby statute Evaluations\nFinal Administrative Penalty Orders 1999-2003 FY 1999 - FY 2003 Case Conclusions\nby statute\nGallons of wastewater/ground water 2002-2003 FY 2002 - FY 2003 National Estimates of\ntreated Environmental Benefits\nIncarceration (years) 1998-2003 FY 1998 - FY 2003 Criminal Enforcement Program\n Activities\nInjunctive relief by case type ($) 1999-2003 FY 1999 - FY 2003 Injunctive Relief\nInjunctive relief by statute ($) 1999-2003 FY 1999 - FY 2003 Injunctive Relief\nNotices of Determination as a result 1999-2003 FY 1999 - FY 2003 Voluntary Disclosure Policy\nof voluntary disclosure under audit\npolicy\nPeople protected by Safe Drinking 2002-2003 FY 2002 - FY 2003 National Estimates of\nWater Act (SDWA) enforcement Environmental Benefits\nPounds of pollution reduced, treated, 2002-2003 FY 2002 - FY 2003 National Estimates of\nor properly managed Environmental Benefits\nPounds of contaminated 2002-2003 FY 2002 - FY 2003 National Estimates of\nsoil/sediment Environmental Benefits\nResolved cases against companies 1999-2003 FY 1999 - FY 2003 Voluntary Disclosure Policy\nas a result of voluntary disclosure\nunder audit policy\nResolved cases against facilities as a 1999-2003 FY 1999 - FY 2003 Voluntary Disclosure Policy\nresult of voluntary disclosure under\naudit policy\nSettlements with/without complying 2003 FY 2003 Complying Actions\nactions by region\nStatistically-valid noncompliance 2004-2005 Statistically-Valid Noncompliance Rates for RCRA\nrates for RCRA inspections of Inspections of Foundries\nfoundries\nSupplemental Environmental Projects 1999-2003 FY 1999 - FY 2003 Supplemental Environmental\nby statute ($) Projects (SEPs)\nTotal penalties by case type ($) 1999-2003 FY 1999 - FY 2003 Penalties\n\nThese reports will be posted to EPA=s Web site at http://www.epa.gov/compliance/data/results/index.html\n\n\n\n\n 47\n\n\x0c Appendix G\n\n Distribution\n\nOffice of the Administrator\nAssistant Administrator for Enforcement and Compliance Assurance\nPrincipal Deputy Assistant Administrator for Enforcement and Compliance Assurance\nDirector, Office of Compliance, Office of Enforcement and Compliance Assurance\nDirector, Office of Civil Enforcement, Office of Enforcement and Compliance Assurance\nAgency Followup Official (the CFO)\nAgency Followup Coordinator\nAudit Followup Coordinator, Office of Enforcement and Compliance Assurance\nGeneral Counsel\nAssociate Administrator for Congressional and Intergovernmental Relations\nAssociate Administrator for Public Affairs\nInspector General\n\n\n\n\n 48 \n\n\x0c'