jurisdiction: US

text: b"OFFICE OF INSPECTOR GENERAL\n\n\nAUDIT OF USAID/KOSOVO\xe2\x80\x99S\nACTIVITIES FOR ECONOMIC\nGROWTH\nAUDIT REPORT NO. 9-000-12-004-P\nAUGUST 21, 2012\n\n\n\n\nWASHINGTON, D.C.\n\x0cOffice of Inspector General\n\n\nAugust 21, 2012\n\nMEMORANDUM\n\nTO: USAID/Kosovo, Mission Director, Maureen Shauket\n\nFROM: IG/A/PA, Director, Steven Ramonas /s/\n\nSUBJECT: Audit of USAID/Kosovo\xe2\x80\x99s Activities for Economic Growth\n (Report No. 9-000-12-004-P)\n\nThis memorandum transmits our final report on the subject audit. In finalizing the audit report,\nwe considered your comments on the draft report and have included them in Appendix II of this\nreport.\n\nThe final report contains 13 recommendations to help strengthen the effectiveness of\nUSAID/Kosovo\xe2\x80\x99s economic growth activities. On the basis of your response to the draft report,\nwe determined that final action has been taken on Recommendation 9 and management\ndecisions have been reached on the remaining recommendations. Please provide the Audit\nPerformance and Compliance Division of USAID\xe2\x80\x99s Office of the Chief Financial Officer with\nevidence of final action to close the 12 open recommendations.\n\nThank you for the cooperation and courtesy extended to the audit staff during this audit.\n\n\n\n\nU.S. Agency for International Development\n1300 Pennsylvania Avenue, NW\nWashington, DC 20523\nwww.usaid.gov/oig\n\x0cCONTENTS\nSummary of Results ................................................................................................................. 1\n\nAudit Findings ........................................................................................................................... 6\n\n USAID/Kosovo Did Not Set Mission-Wide Policy on Municipal Revenues ............................. 6\n\n Project Did Not Implement Activities to Introduce the Use of Electronic Tenders for\n Procurement ......................................................................................................................... 7\n\n Agriculture Project Did Not Formalize Decision to Shift Focus............................................... 8\n\n Partners Implemented Limited Activities That Addressed Gender and Social Inclusion ......... 9\n\n Performance Management Plans Were Weak ..................................................................... 10\n\n Reported Results Were Unreliable and Required Additional Validation ............................... 14\n\nEvaluation of Management Comments.................................................................................. 18\n\nAppendix I \xe2\x80\x93 Scope and Methodology ................................................................................... 20\n\nAppendix II \xe2\x80\x93 Management Comments .................................................................................. 22\n\nAppendix III \xe2\x80\x93 Agricultural Project Reported and Verified Results ..................................... 26\n\nAbbreviations\n\nThe following abbreviations appear in this report:\n\nADS Automated Directives System\nBEEP Business Enabling Environment Program\nB2B business-to-business\nCOR contracting officer\xe2\x80\x99s representative\nDEMI Democratic and Effective Municipalities Initiative\nFTE full-time equivalent\nFY fiscal year\nHACCP Hazard Analysis and Critical Control Points\nIIF innovation and incentive fund\nMOU memorandum of understanding\nNOA New Opportunities for Agriculture\nOIG Office of Inspector General\nPMP performance management plan\nSME small and medium enterprises\nUSG U.S. Government\n\x0cSUMMARY OF RESULTS\nThe Republic of Kosovo has made significant progress in transitioning to a market-based\neconomy since declaring its independence in 2008. However, the southeastern European\nnation continues to face serious economic challenges. Foreign investment is limited, due in part\nto perceptions of official and informal corruption and the country\xe2\x80\x99s continued struggle for\nrecognition. Years of positive growth rates in gross domestic product have done little to reduce\nthe country\xe2\x80\x99s 45 percent unemployment rate. Kosovo\xe2\x80\x99s 1.8 million citizens are among the\npoorest in Europe, and they rely heavily on international assistance and remittances from the\nKosovar diaspora.\n\nAgriculture remains important for the country\xe2\x80\x99s largely rural population. This sector, which has\nrecovered slowly since the conflict of the late 1990s, is inefficient because farms are small,\ntechnology is limited, and productivity is low. According to a 2007 World Bank assessment,\ndomestic agriculture met only 25 to 35 percent of local demand for food. Therefore, food\nimports are high, leading to an agricultural trade imbalance of $637 million in 2008.\n\nUSAID/Kosovo strives to encourage Kosovo\xe2\x80\x99s efforts to develop a viable economy and an\ninclusive democracy on the path to European integration. The mission designed four projects\nunder its Economic Growth program and dedicated $54.5 million to achieving its goal of\nincreasing private sector growth and investment. The Office of Inspector General (OIG) audited\ntwo of these projects:\n\n Business Enabling Environment Program (BEEP). In July 2010, USAID awarded a 3-year,\n $16.1 million task order to Chemonics International Inc. to implement BEEP. The project\n has four main objectives: (1) improve rules and regulations, (2) lower trade barriers,\n (3) improve the efficiency of the financial sector, and (4) introduce the use of electronic\n tenders or payments for local procurements. Project implementation is expected to run from\n July 2010 to July 2013.\n\n New Opportunities for Agriculture (NOA). USAID awarded a 4-year, $15.9 million task order\n to Tetra Tech ARD in January 2011 to carry out NOA. The project is designed to increase\n Kosovo\xe2\x80\x99s agricultural outputs, exports, and rural incomes through activities that (1) link\n products and farmers to markets, (2) diversify and increase agricultural products,\n (3) improve food quality and safety, (4) make credit more affordable and accessible, and\n (5) improve coordination within the agricultural sector. Implementation is expected to run to\n February 2015.\n\nOIG\xe2\x80\x99s Performance Audits Division conducted this audit as part of its fiscal year (FY) 2012 audit\nplan to determine whether USAID/Kosovo\xe2\x80\x99s economic growth activities under the two projects\nwere achieving their goal of increasing private sector growth and investment.\n\nBEEP and NOA activities have made some progress. BEEP accomplished the following\nobjectives:\n\n Improved rules and regulations. The project obtained the Kosovar Government\xe2\x80\x99s\n commitment to improve its World Bank\xe2\x80\x99s Doing Business ranking from 117 to top 40 status\n by 2014. BEEP also eliminated hundreds of unnecessary, obsolete, or illegal administrative\n instructions.\n\n\n 1\n\x0c Lowered trade barriers. The project reduced the number of documents needed for trade\n and eliminated redundant customs registration for traders. BEEP also helped establish a\n trade policy working group in accordance with international standards and best practices.\n\n Encouraged the financial sector to be more efficient. The project obtained a memorandum\n of understanding (MOU) between USAID/Kosovo and the Central Bank of Kosovo to\n upgrade the national credit registry.\n\nNOA promoted increased investment in the country\xe2\x80\x99s fledgling food production industry,\nsupporting producers and processors. The project stimulated investment in agricultural\ntechnology and infrastructure designed to help producers improve the health of their plants and\nto increase yields. NOA encouraged food processors to invest in the equipment needed to\nmeet and obtain certification of compliance with international health and safety standards. It\nalso helped establish linkages between producers and processors that should enable higher-\nquality products to be sold for higher prices. For example:\n\n A farmer who used new gherkin seeds and updated farming practices reported seeing better\n quality plants and multiple harvests per season. He said he plans to pursue linkages with\n processors so that he can sell his goods for a higher price.\n\n NOA introduced asparagus and saffron, two crops new to Kosovo. These high-value plants\n have generated much interest among beneficiaries and could create a significant number of\n new jobs and, once established, export opportunities.\n\n\n\n\n Asparagus grows in a greenhouse (left) before it is transplanted in fields. On the right are\n saffron plants that NOA introduced to Kosovo. (Photos by OIG, April 2012)\n\n Apple farmers reported that trees provided by the project should produce yields of 45 to\n 65 tons per hectare when mature. The trees are shorter than the varieties traditionally\n grown in Kosovo, making them easier to harvest. NOA has helped some of the farmers\n invest in antihail systems to protect their orchards from inclement weather and damage from\n animals.\n\n NOA introduced grape farmers to the use of a modified trellis system. The system was\n designed to support the vines to grow horizontally so that the fruit receives additional\n sunlight, improving the color and health of the plant while decreasing the incidence of\n disease.\n\n\n\n\n 2\n\x0c One food processor successfully obtained certification of compliance with the Hazard\n Analysis and Critical Control Points,1 or HACCP, management system. According to NOA\xe2\x80\x99s\n staff, this internationally recognized standard for food safety will be required for food\n processors in Kosovo by September 2013.\n\nDespite these achievements, not all BEEP and NOA activities were on track to meet their\nobjectives. Because BEEP did not implement activities to increase municipalities\xe2\x80\x99 use of\nelectronic tenders for procurements, it has not made any progress toward this goal. In NOA\xe2\x80\x99s\ncase, because the project did not make much progress toward increasing exports,\nUSAID/Kosovo, with input from NOA staff members, decided to change the project\xe2\x80\x99s focus to\nimport substitution2 to help Kosovo strengthen the domestic market for locally produced goods.\nAdditionally, NOA could not determine whether it made any progress toward increasing rural\nincomes, partly because Tetra Tech ARD never designed an indicator to measure an increase.\n\nOutside factors also influenced the progress of the two selected projects. BEEP could not\ncontrol the timing and achievement of some of its desired results, such as the passage of laws,\nthe adoption of regulations, and Kosovo\xe2\x80\x99s changing political environment. For NOA, which\nbegan in January 2011, famers already had planned what they would plant by the time staff\nmembers were ready to begin working on the project\xe2\x80\x99s first growing season.\n\nThis report outlines weaknesses identified in BEEP\xe2\x80\x99s and NOA\xe2\x80\x99s implementation.\n\n USAID/Kosovo did not set a mission-wide policy on whether to encourage municipalities to\n generate more revenue through fees, taxes, and permits or to abolish excessive ones to be\n more appealing to businesses (page 6).\n\n BEEP did not implement activities to introduce the use of electronic tenders for procurement\n (page 7).\n\n NOA did not formalize its decision to shift the project\xe2\x80\x99s focus to import substitution instead of\n developing products for export (page 8).\n\n Partners implemented limited activities that addressed gender and social inclusion (page 9).\n\n Project performance management plans were weak (page 10).\n\n Reported results were unreliable and required additional validation (page 14).\n\nTo address these problems and to strengthen the effectiveness of USAID/Kosovo\xe2\x80\x99s economic\ngrowth activities, this report includes 13 recommendations. We recommend that\nUSAID/Kosovo:\n\n1. Develop a mission policy on the generation of municipal own source revenue and implement\n this policy in its work with municipalities in programs in the Economic Growth and\n Democracy and Governance Offices (page 6).\n1\n HACCP is a management system in which food safety is addressed by analyzing and controlling\nbiological, chemical, and physical hazards from raw material production, procurement, and handling to\nmanufacturing, distribution, and consumption of the finished product.\n2\n Import substitution is an economic strategy in which a country or organization tries to reduce imports by\nproducing the goods or products locally.\n\n\n 3\n\x0c2. Reevaluate the feasibility of implementing activities under the More Transparent Tenders\n component of the Business Enabling Environment Program. If a decision is made to not\n implement these activities, we recommend that USAID/Kosovo modify Chemonics\n International Inc.\xe2\x80\x99s task order by removing this component from the scope of work and\n reducing the ceiling price by $800,000, the amount of funds dedicated to this component\n (page 7).\n\n3. Modify the statement of work for the New Opportunities in Agriculture project to reflect the\n decision to refocus the goal of the project to supporting import substitution (page 9).\n\n4. Work with Tetra Tech ARD to revise, as appropriate, the target for the performance indicator\n for the Total value of exports as a result of USG [U.S. Government] assistance to better\n reflect the decision to focus the New Opportunities for Agriculture project on supporting\n import substitution (page 9).\n\n5. Work with Tetra Tech ARD to develop and implement a plan that encourages the targeting\n of project interventions that focus on increasing gender and minority inclusion under the\n New Opportunities for Agriculture project (page 10).\n\n6. Work with Tetra Tech ARD to develop procedures to strengthen the New Opportunities for\n Agriculture monitoring and evaluation system so that it adequately measures the results of\n gender and minority inclusion in its activities (page 10).\n\n7. Work with Chemonics International Inc. to establish a performance indicator and\n corresponding target to measure the progress of its activities designed to lower trade\n barriers under the Business Enabling Environment Program (page 12).\n\n8. Work with Tetra Tech ARD to establish a performance indicator and corresponding target to\n measure the change in rural incomes as a result of activities under the New Opportunities\n for Agriculture project. If a proxy indicator is deemed appropriate, we recommend that\n USAID/Kosovo work with Tetra Tech ARD to adequately document the assumptions\n supporting the choice of indicator and the methodology for its collection in the New\n Opportunities for Agriculture project performance management plan (page 12).\n\n9. Work with Chemonics International Inc. to determine a realistic, yet achievable, target for the\n performance indicator Number of requests by businesses and citizens for their own credit\n reports under the Business Enabling Environment Program (page 13).\n\n10. Work with Tetra Tech ARD to streamline the performance management plan for the New\n Opportunities for Agriculture project by determining those performance indicators that best\n fit the criteria defined in Automated Directives System (ADS) 203.3.4.2 and eliminating\n those performance indicators that demonstrate little usefulness for project management\n (page 14).\n\n11. Develop and implement procedures to verify that the performance results reported by its\n implementing partners are reasonably accurate, complete, reliable, and adequately\n supported (page 16).\n\n12. Work with Chemonics International Inc. and Tetra Tech ARD to develop and implement\n procedures to verify that the data collection and analysis methodologies used to report\n\n\n 4\n\x0c results of its activities under the Business Enabling Environment Program and New\n Opportunities for Agriculture project are consistent with those outlined in the projects\xe2\x80\x99\n performance management plans (page 17).\n\n13. Work with Tetra Tech ARD to revise, as necessary, the methodologies used for the\n collection and analysis of data so that reported results under the New Opportunities for\n Agriculture project are reliable and comparable across reporting periods (page 17).\n\nDetailed findings follow. The audit\xe2\x80\x99s scope and methodology are described in Appendix I, and\nUSAID/Kosovo\xe2\x80\x99s comments appear in Appendix II of the final report. Our evaluation of\nmanagement comments is on page 18 of the report.\n\n\n\n\n 5\n\x0cAUDIT FINDINGS\nUSAID/Kosovo Did Not Set\nMission-Wide Policy on\nMunicipal Revenues\nADS 201.3.3.3 discusses USAID\xe2\x80\x99s results framework and missions\xe2\x80\x99 development objectives.\nDevelopment objectives may be mutually reinforcing and should not reflect solely functional\nobjectives. In fact, development objectives focused on specific sectors should build synergies\nwith other development objectives. To this end, integrated technical approaches, principles,\nand resources from various sectors and sources can be used to achieve a common objective by\ndeveloping a unified programmatic approach.\n\nUSAID/Kosovo provides assistance to spur economic development in Kosovo\xe2\x80\x99s municipalities\nthrough projects managed by its Economic Growth and Democracy and Governance Offices.\nDespite their shared goal, the offices have not been able to agree on a policy for municipal own\nsource revenues, which are funds a municipality raises by issuing licenses, fees, and permits.\nBetween 2008 and 2010, this category accounted for 16 to 21 percent of total municipal\nrevenues.\n\nAs an Economic Growth project focused on improving the business environment, BEEP\nencouraged municipalities to become more appealing to businesses by abolishing unnecessary\ntaxes, fees, and permits. However, officials from the Democracy and Governance Office were\nconcerned that these reforms could clash with their Democratic and Effective Municipalities\nInitiative (DEMI), which lists increasing municipal revenues among its primary goals.\n\nUSAID/Kosovo\xe2\x80\x99s technical offices worked independent of each other and therefore did not\nanticipate or identify potential programmatic conflicts between BEEP and DEMI in either the\nplanning or initial stages of implementation.\n\nWithout agreeing on a mission-wide policy on municipal own source revenues, USAID/Kosovo\nrisks sending mixed messages to its implementing partners and beneficiaries. Municipal\ncoordinators working for BEEP acknowledged that they sometimes encounter difficulties\nbecause the two projects have different goals. In a larger concern, the discord within\nUSAID/Kosovo may weaken its reputation as a technical expert. The mission must adopt a\npolicy on municipal revenues. Therefore, we make the following recommendation.\n\n Recommendation 1. We recommend that USAID/Kosovo develop a mission policy on\n the generation of municipal own source revenue and implement this policy in its work\n with municipalities in programs in the Economic Growth and Democracy and\n Governance Offices.\n\n\n\n\n 6\n\x0cProject Did Not Implement Activities\nto Introduce the Use of Electronic\nTenders for Procurement\nAccording to ADS 202.3.6.1, the contracting officer\xe2\x80\x99s representative (COR) must confirm that\nimplementing partners perform in accordance with the terms of their contracts or other\nagreements. Responsibilities for monitoring contractor performance may include, among other\nactions, recommending modifications. As stated in 202.3.6.3, \xe2\x80\x9cMaking Necessary Adjustments,\xe2\x80\x9d\nUSAID missions, offices, and development objective teams must adjust their tactics when\nconditions warrant. This may include developing an entirely new project (or activity) and\ninstrument, or simply modifying and changing existing projects or activities.\n\nThe task order for BEEP outlined four main objectives of assistance: (1) improve rules and\nregulations, (2) lower trade barriers, (3) help develop a more efficient financial sector, and\n(4) increase tender transparency by introducing the use of electronic tenders. However, at the\nend of audit fieldwork, BEEP had not yet begun to carry out any activities toward achieving the\nfourth objective. This objective was designed to reduce procurement irregularities, a principal\nsource of corruption. By introducing electronic tenders, the mission hoped to reduce the\namount of corruption related to the procurement of local goods and services. Funding for this\nobjective amounted to $800,000 or 5 percent of BEEP\xe2\x80\x99s total budget of $16 million.\n\nOfficials for USAID/Kosovo and Chemonics International Inc. said they did not implement these\nactivities because they hoped to avoid duplicating the efforts of a World Bank project with a\nsimilar objective. In fact, they postponed the work on the objective until after the World Bank\nhad finalized its plans. USAID/Kosovo currently is considering whether to restructure this\nobjective or replace it with another.\n\nUntil the mission decides whether to proceed with this objective, the delay reduces the amount\nof time Chemonics International Inc. has to implement activities designed to increase tender\ntransparency. If the activity is not performed, procurement irregularities that the project was\nmeant to address may continue. Furthermore, the money budgeted for this goal could have\nbeen put to better use by funding other activities.\n\nBEEP is set to end in July 2013. For the project\xe2\x80\x99s final year of implementation, USAID/Kosovo\nmust decide what action to take for this objective. Therefore, we make the following\nrecommendation.\n\n Recommendation 2. We recommend that USAID/Kosovo reevaluate the feasibility of\n implementing activities under the More Transparent Tenders component of the Business\n Enabling Environment Program. If a decision is made to not implement these activities,\n we recommend that USAID/Kosovo modify Chemonics International Inc.\xe2\x80\x99s task order by\n removing this component from the scope of work and reducing the ceiling price by\n $800,000, the amount of funds dedicated to this component.\n\n\n\n\n 7\n\x0cAgriculture Project Did Not\nFormalize Decision to Shift Focus\nADS 200.3.5.6 encourages USAID missions to \xe2\x80\x9cimprove coordination and collaboration with\ndevelopment partners, test promising new approaches, build on what works, and eliminate what\ndoes not during the implementation of the Program Cycle.\xe2\x80\x9d Additionally, ADS 203.3.4.5 requires\nmissions to set ambitious, but realistic targets for performance indicators. Missions also should\nbe accountable for achieving these targets.\n\nIn October 2009 the consulting firm Booz Allen Hamilton conducted an analysis for\nUSAID/Kosovo of the constraints and opportunities for growth of Kosovo\xe2\x80\x99s agribusiness sector.\nThe resulting report, Kosovo Agricultural Opportunities Strategy, served as the basis for NOA.\nThe report recommended that Kosovo pursue a diversified crop mix with strong export potential.\nNOA opted to support several of the recommended crops and focus on key points in a crop\xe2\x80\x99s\nvalue chain to improve the production and exportation of agricultural products.\n\nIn setting the goal of increasing Kosovo\xe2\x80\x99s agricultural exports, NOA staff created a\ncorresponding performance indicator measuring the Total value of exports as a result of USG\nassistance. By the time NOA ends in February 2015, it expects to attribute at least $9.7 million\nin exports to its activities.\n\nEarly in the project\xe2\x80\x99s implementation, NOA assessed and prepared a report on new market\nopportunities for five crops in eight regional markets. According to the report, there were few\nnear-term opportunities for export and Kosovo\xe2\x80\x99s low production volume, limited storage\ncapacity, and lack of postharvest technology were serious impediments to expanding external\nmarkets for the five selected crops. The report noted that at least 5 more years of investment\nand development would be needed to overcome these impediments.\n\nDuring the audit fieldwork, local food processors said they had limited opportunities to export\ntheir products. Beneficiary producers said their goods were destined for local markets and\nsupermarkets, not for export markets. Furthermore, by the end of FY 2011, NOA reported no\nexport sales in its results.\n\nAfter approximately 8 months of implementation, USAID/Kosovo officials and NOA staff decided\nto shift the project\xe2\x80\x99s primary focus from developing products for export markets to import\nsubstitution. NOA\xe2\x80\x99s FY 2011 annual report and revised performance management plan (PMP)\nboth announced this change. According to the annual report, NOA plans to keep pursuing\ncertain export opportunities; however, import substitution seems a reasonable goal for many of\nthe project\xe2\x80\x99s targeted value chains.\n\nDespite realigning the project\xe2\x80\x99s focus, USAID/Kosovo and NOA staff members said that\nbecause the project\xe2\x80\x99s scope of work was sufficiently broad, this change did not merit a\nmodification to the statement of work.\n\nNOA\xe2\x80\x99s staff revised the target for the total value of exports as a result of USG assistance in\nOctober 2011. Despite reportedly using value chain analyses, regional market assessments,\nand interviews with producers, processors, and collection points, the updated target does not\naccurately reflect the project\xe2\x80\x99s activities.\n\n\n\n\n 8\n\x0cBy shifting NOA\xe2\x80\x99s priorities away from export development to import substitution, USAID/Kosovo\nofficials and project staff effectively changed the purpose and goal of the project. The statement\nof work is no longer in line with the current activities. Also, the shift to import substitution may\nslow the rates of private sector growth and investment that the mission wanted to achieve.\nFurthermore, the target for Total value of exports as a result of USG assistance is overly\nambitious and unlikely to be achieved. Therefore, we make the following recommendations.\n\n Recommendation 3. We recommend that USAID/Kosovo modify the statement of work\n for the New Opportunities in Agriculture project to reflect the decision to refocus the goal\n of the project to supporting import substitution.\n\n Recommendation 4. We recommend that USAID/Kosovo work with Tetra Tech ARD to\n revise, as appropriate, the target for the performance indicator for the Total value of\n exports as a result of USG assistance to better reflect the decision to focus the New\n Opportunities for Agriculture project on supporting import substitution.\n\nPartners Implemented Limited\nActivities That Addressed\nGender and Social Inclusion\nThe task orders for both BEEP and NOA instruct the implementing partners to address gender\nand social inclusion issues through their activities.\n\nAccording to the BEEP task order, the project would \xe2\x80\x9cwork with minority and women\xe2\x80\x99s business\nassociations and national and local governments to strengthen the entrepreneurial opportunities\nfor women and minorities, both at the national and municipal/local level.\xe2\x80\x9d According to NOA\xe2\x80\x99s\ntask order, that project must take gender and social inclusion issues into consideration in the\noperations and outcomes so that it will promote inclusive economic growth. NOA\xe2\x80\x99s monitoring\nsystem must measure the effects of the project\xe2\x80\x99s activities on women and minorities.\nFurthermore, databases must disaggregate data relating to these areas when appropriate. ADS\n203.3.3.5.c supports this requirement by stating that all USAID\xe2\x80\x99s people-level performance\nindicators must be disaggregated by sex.\n\nUSAID/Kosovo officials and Tetra Tech ARD staff members acknowledged that NOA did not\nfocus on implementing activities to increase opportunities for women and minorities. Instead,\nNOA emphasized implementing activities that would return the greatest results.\n\nUSAID/Kosovo officials and NOA staff members admitted that agriculture is a challenging sector\nfor women. Cultural norms in Kosovo assume that women find it to be largely unattractive work.\nAccording to NOA staff members, women are thought to be more interested in postharvest\nactivities, such as the sorting, grading, and packaging of goods to be sold instead of working the\nland.\n\nNOA\xe2\x80\x99s PMP included ten people-level performance indicators whose results were to be\ndisaggregated by sex and/or minority group. However, NOA rarely included this information in\nits reported results. The project\xe2\x80\x99s FY 2011 annual report disaggregated the result for only one\nindicator\xe2\x80\x94Number of individuals trained in agriculture productivity through USG assistance\xe2\x80\x94by\nsex. No results were disaggregated by sex or minority group in the subsequent quarterly\nreports. USAID officials agreed that gender especially was not addressed adequately in NOA\xe2\x80\x99s\nprogress reporting.\n\n\n 9\n\x0cSince the vast majority of NOA\xe2\x80\x99s beneficiaries are men, there are few results that could\neffectively be disaggregated by sex. In the narrative sections of the performance reports, NOA\nincluded limited information about members of the minority groups with which the project is\nworking.\n\nThe audit determined that BEEP staff believed that too much focus on a specific group when\ndefining laws and legal frameworks could be seen as discriminatory. Therefore, they were\nreluctant to implement activities that catered specifically to gender or to a certain minority group.\n\nEncouraging gender equality and minority participation in the agriculture and business sectors\nare priorities for USAID/Kosovo\xe2\x80\x99s management. However, with few activities geared toward\nthem, little progress can be made. With little reliable information available, USAID/Kosovo\ncannot determine its influence on gender and minority inclusion in these sectors. Therefore, we\nmake the following recommendations.\n\n Recommendation 5. We recommend that USAID/Kosovo work with Tetra Tech ARD to\n develop and implement a plan that encourages the targeting of project interventions that\n focus on increasing gender and minority inclusion under the New Opportunities for\n Agriculture project.\n\n Recommendation 6. We recommend that USAID/Kosovo work with Tetra Tech ARD to\n develop procedures to strengthen the New Opportunities for Agriculture monitoring and\n evaluation system so that it adequately measures the results of gender and minority\n inclusion in its activities.\n\nPerformance Management\nPlans Were Weak\nADS 203.3.3 discusses the importance of performance management\xe2\x80\x94the systematic process\nof monitoring program achievements. By encompassing \xe2\x80\x9cthe tools used for assessing, learning,\nand sharing,\xe2\x80\x9d performance management represents USAID\xe2\x80\x99s \xe2\x80\x9ccommitment to managing\ndevelopment objectives for results in order to achieve the best possible development\noutcomes.\xe2\x80\x9d USAID missions are, according to this guidance, responsible for measuring\nprogress toward the milestones identified for achieving foreign assistance objectives. ADS\n203.3.3.4 requires that development objective teams prepare PMPs to assist with the\nperformance management process.\n\nADS 203.3.3.4 further requires that as part of the process, development objective teams should\nestablish performance indicators. As explained in ADS 200.6, these are used to measure\nprogress by comparing actual results with expected results.\n\nGood performance indicators share several characteristics. According to ADS 203.3.4.2, they\nshould be practical, useful for management, direct, attributable to USAID\xe2\x80\x99s efforts, and\nadequate. All performance indicators require resources and effort to collect, analyze, report,\nand use. So PMPs should contain as many indicators as are necessary and cost-effective for\nresults management and reporting purposes.\n\nAccording to ADS 203.3.4.5, performance baselines and targets should be set for each indicator\nin a PMP. Targets should be ambitious, but realistic, and achievable within stated time frames\n\n\n\n 10\n\x0cand given USAID inputs. Furthermore, ADS 203.3.3.2 states that the performance\nmanagement process should be streamlined so that only the most useful information is\ncollected and reported.\n\nAccording to ADS 203.3.4.7, during project implementation, performance indicators may need to\nbe changed or dropped because changes in priorities or other decisions that affect the scope of\na project may require the use of indicators that are different from those originally selected. They\nalso may need to be adjusted if they prove to be unsuitable; for example, the effort and cost\nneeded to collect them become excessive.\n\nDespite these directives, we identified several weaknesses in both BEEP\xe2\x80\x99s and NOA\xe2\x80\x99s PMPs.\n\nPMPs Lacked Indicators for Two Objectives. The task orders for BEEP and NOA outlined\neach project\xe2\x80\x99s main objectives. One of BEEP\xe2\x80\x99s was to strengthen Kosovo\xe2\x80\x99s international trade\ncapabilities by lowering trade barriers and building the capacity of trade officials and private\nsector groups to make use of international trade practices and conventions. One of NOA's\nobjectives was to increase the incomes of people living in rural areas. However, neither\nproject\xe2\x80\x99s PMP contained indicators to measure the progress toward achieving these desired\nresults.\n\nIn the case of BEEP, the project did not include an indicator that directly tracked the progress of\nits trade objective. Instead, the project used trade data embedded in the World Bank Doing\nBusiness survey and BEEP\xe2\x80\x99s Determination of economic impact indicator to gauge the project\xe2\x80\x99s\nprogress. However, the trade data used were not individually tracked, nor did they fully capture\nthe project\xe2\x80\x99s efforts towards lowering trade barriers.\n\nThe NOA task order required the PMP to include a performance indicator measuring the\nincrease in rural incomes as a result of project activities. However, the staff did not attempt to\ntrack this information.\n\nADS 203.3.4.2.d states that if a direct indicator cannot be used for cost or other reasons, a\nproxy indicator may be substituted to measure the desired result. In this case, the assumptions\nsupporting the selection of the proxy indicator should be documented in the PMP and confirmed\non a regular basis. Although NOA did not formally establish a proxy indicator to serve as an\nalternative measure of changing incomes in rural areas, NOA staff suggested that an estimate\nof this change could be made from the data collected for the indicator measuring the level of\nsales by project beneficiaries.\n\nWhen submitting their initial PMPs for approval, Chemonics International Inc. and Tetra Tech\nARD officials did not propose performance indicators that measured the two important\nobjectives described above. BEEP staff members said they believed that the other performance\nindicators would adequately capture the project\xe2\x80\x99s progress in lowering trade barriers. NOA\xe2\x80\x99s\nstaff acknowledged the difficulty in collecting sound data regarding the income levels of rural\ninhabitants; furthermore, NOA did not establish a baseline from which a change in incomes\ncould be measured.\n\nWithout performance indicators designed to measure these desired results, USAID/Kosovo and\nits partners cannot determine whether any progress has been made in reaching these important\nproject objectives. Therefore, we make the following recommendations.\n\n\n\n\n 11\n\x0c Recommendation 7. We recommend that USAID/Kosovo work with Chemonics\n International Inc. to establish a performance indicator and corresponding target to\n measure the progress of its activities designed to lower trade barriers under the\n Business Enabling Environment Program.\n\n Recommendation 8. We recommend that USAID/Kosovo work with Tetra Tech ARD to\n establish a performance indicator and corresponding target to measure the change in\n rural incomes as a result of activities under the New Opportunities for Agriculture project.\n If a proxy indicator is deemed appropriate, we recommend that USAID/Kosovo work with\n Tetra Tech ARD to adequately document the assumptions supporting the choice of\n indicator and the methodology for its collection in the New Opportunities for Agriculture\n project performance management plan.\n\nBEEP\xe2\x80\x99s Credit Performance Indicator Did Not Have a Target. BEEP\xe2\x80\x99s staff did not set a\ntarget for its Number of requests by businesses and citizens for their own credit reports\nperformance indicator. Staff members said that when the project began, the Central Bank of\nKosovo did not maintain historical information on the number of credit reports requested.\nTherefore, BEEP could not set a meaningful target for this indicator.\n\nAfter BEEP started, the Central Bank of Kosovo began to track the number of credit reports\nrequested. Between July 2010 and April 2012, as shown in Figure 1 below, the number of\ncredit reports requested increased, and an average of 36 credit reports were being requested\nper month.\n\n Figure 1. Number of Credit Reports Requested per Month\n Between July 2010 and April 2012\n\n 70\n 60\n # of Credit Requests\n\n\n\n\n 50\n 40\n 30\n 20\n 10\n 0\n\n\n\n Date\n\n Source: Central Bank of Kosovo\n\nWithout a target for this performance indicator, BEEP\xe2\x80\x99s staff will not have a specific, desired\nlevel of performance for the number of credit reports requested for the final year of the project\xe2\x80\x99s\nimplementation. The availability of the data from the Central Bank of Kosovo should allow\nUSAID/Kosovo and BEEP to set a realistic target for this indicator. Therefore, we make the\nfollowing recommendation.\n\n\n\n\n 12\n\x0c Recommendation 9. We recommend that USAID/Kosovo work with Chemonics\n International Inc. to determine a realistic, yet achievable, target for the performance\n indicator Number of requests by businesses and citizens for their own credit reports\n under the Business Enabling Environment Program.\n\nNOA Indicators Overlapped, Were Impractical, and Could Not Be Attributed to the\nProject. NOA\xe2\x80\x99s monitoring and evaluation staff tracked a set of 32 performance indicators to\nmeasure the project\xe2\x80\x99s progress. However, as discussed later in this report, NOA\xe2\x80\x99s reported\nresults have been largely incomplete, inaccurate, and inconsistent.\n\nUSAID/Kosovo officials and NOA\xe2\x80\x99s staff said the PMP is onerous. Moreover, some of the\nperformance indicators are impractical. According to the PMP, it is difficult to measure and\njudge Number of farmers, processors, and others who have adopted new technologies or\nmanagement practices as a result of USG assistance. Proper data collection for this indicator\nrequires significant resources because monitoring and evaluation staff must verify that new\ntechnologies and management practices:\n\n Have been applied consistently by farmers for an entire planting season.\n\n Have been applied consistently by collectors and processors for at least 20 production\n cycles from start to finish.\n\n Have been declared as \xe2\x80\x9cadopted\xe2\x80\x9d by the customer.\n\nNOA tracks similar indicators with Number of new technologies and/or management practices\nintroduced as a result of USG assistance and Number of individuals trained in agriculture\nproductivity as a result of USG assistance. These indicators require fewer resources because\nthere is no need to verify the long-term, consistent use of the new technology or management\npractice, or to obtain an official declaration from the beneficiary that the process has been\nadopted. As a result, tracking the results obtained from these indicators would be as timely and\nuseful for management (if not more) than tracking the number of people who adopted new\ntechnologies and management practices.\n\n\n\n\nThe root system of a lettuce plant, left, develops in a germination tray before it\xe2\x80\x99s transplanted in a\nfield. \xe2\x80\x9cFertigation\xe2\x80\x9d machines, like the one on the right, deliver fertilizer to plants through an\nirrigation system. (Photos by OIG, April 2012)\n\n\n\n 13\n\x0cIn addition, NOA\xe2\x80\x99s staff doubted the project\xe2\x80\x99s ability to influence the result of some of its\nindicators. For instance, the project tracks two indicators that are very similar: Number of policy\nreforms/regulations/administrative procedures drafted and presented for public/stakeholder\nconsultation as a result of USG assistance and Number of policy\nreforms/regulations/administrative procedures drafted and presented for public/stakeholder\nconsultation and submitted for approval as a result of USG assistance. However, NOA did not\nimplement activities designed to encourage these types of reforms. According to one staff\nmember, NOA\xe2\x80\x99s management was considering removing the indicators because they did not\ndirectly address the project\xe2\x80\x99s activities.\n\nNOA field staff members responsible for monitoring and evaluation were not in place when\nTetra Tech ARD developed the initial PMP. Therefore, they could not contribute to the plan\xe2\x80\x99s\ndesign. A revised PMP, approved by USAID/Kosovo in October 2011, did not significantly\nchange the plan or reduce the number of indicators to be tracked. Furthermore, officials from\nUSAID/Kosovo did not question the ability of NOA\xe2\x80\x99s staff to track and report on so many\nindicators.\n\nNOA\xe2\x80\x99s monitoring and evaluation staff spent significant time and resources collecting and\nanalyzing data for indicators that may not be practical or attributable to USAID\xe2\x80\x99s efforts.\nFurthermore, as discussed later in this report, the verification of NOA\xe2\x80\x99s results uncovered\nnumerous errors in reporting. A more streamlined PMP would allow monitoring and evaluation\nresources to be directed at managing fewer performance indicators and strengthening the\ncompleteness, accuracy, and reliability of reported results. Therefore, we make the following\nrecommendation.\n\n Recommendation 10. We recommend that USAID/Kosovo work with Tetra Tech ARD\n to streamline the performance management plan for the New Opportunities for\n Agriculture project by determining those performance indicators that best fit the criteria\n defined in ADS 203.3.4.2 and eliminating those performance indicators that demonstrate\n little usefulness for project management.\n\nReported Results Were Unreliable\nand Required Additional Validation\nAccording to ADS 203.3.5, USAID missions should balance the quality of reported data and the\ncost that it takes to collect them. In doing so, missions should ensure that data are of\nsufficiently high quality to support the appropriate level of management decisions and as\ncomplete and consistent as management needs and resources permit. High-quality data should\nmeet the standards outlined in ADS 203.3.5.1. They include:\n\n Integrity. Data should have established mechanisms in place for collection, analysis, and\n reporting.\n\n Precision. Data should present a fair picture of performance and enable management\n decision-making at the appropriate levels.\n\n Reliability. Data should reflect stable, consistent data collection processes and analysis\n methods over time.\n\n\n\n\n 14\n\x0cDespite these directives, the audit team found weaknesses in BEEP\xe2\x80\x99s and NOA\xe2\x80\x99s reported\nresults.\n\nBEEP Did Not Report Results for One Performance Indicator Accurately. One of BEEP\xe2\x80\x99s\nprimary performance indicators tracks Economic impact of reforms adopted and implemented\nas a result of project assistance. According to the project\xe2\x80\x99s PMP:\n\n Annual economic impact is project generated or supported changes in\n government legal and/or regulatory acts and/or processes that result in\n reductions of time and/or financial cost to businesses. Impact is an annualized\n figure included in total in the report for the year in which the change occurs to the\n extent it can be reasonably ascertained as valid. Only those changes validated\n by 75 percent of a representative sample of key informants as having a positive\n impact are measured in terms of number of procedures reduced, time saved,\n costs reduced, and total savings.\n\nBEEP staff members reported inaccurate, incomplete information for this indicator. One\neconomic impact was counted twice in the reported results. A second one was not recorded in\nthe main database.\n\nIn addition to these errors, the staff could not document the validity of each economic impact, a\nkey requirement outlined in the PMP. In one case, staff in the project\xe2\x80\x99s main office unilaterally\nincreased the savings realized in one municipality for its Economic impact of the new regulation\non transport services with tow trucks without proper validation. The local municipal coordinator\nreported no economic savings for this reform, while BEEP recorded $404,000 in savings, an\namount that the municipal coordinator never validated.\n\nStaff members did not consistently confirm the amount reported for the overall economic impact\nresulting from reforms that BEEP introduced. Furthermore, the project\xe2\x80\x99s municipal coordinators\ndid not receive sufficient training on the requirements for validating savings that stemmed from\nBEEP reforms. As a result, municipal coordinators did not conduct the required validation\nprocedures properly, nor did they maintain adequate supporting documentation of the validation\nprocess.\n\nNOA\xe2\x80\x99s Reported Results Did Not Meet Data Quality Standards. NOA\xe2\x80\x99s staff used largely\nincomplete, inaccurate, and inconsistent data for activity results in performance reports. In\nsome instances, data could not be validated because of a lack of supporting documentation.\nThe extent of the errors in reporting calls into question the integrity, precision, and reliability of\nthe project\xe2\x80\x99s reported results. Appendix III contains all of those results, as well as those the\nteam verified.\n\nIntegrity. Although NOA\xe2\x80\x99s staff established a data collection and analysis methodology for each\nindicator in its PMP, the approved methodology was not always used to calculate reported\nresults. For instance, for Number of delivery contracts issued for targeted crops, staff members\nreported verbal contracts when only written contracts should have been recorded. Also, a\nmultiplier was used to capture indirect sales that would be included in Value of sales resulting\nfrom linkages created between farmers, processors, and traders as a result of USG assistance;\nthe approved methodology counted only direct sales.\n\nPrecision. USAID/Kosovo and NOA staff members who signed attendance sheets for training\ncourses were regularly included in the results for Number of individuals trained in agricultural\n\n\n 15\n\x0cproductivity through USG assistance. Furthermore, beneficiaries attending multisession training\ncourses were counted as unique participants for each session.\n\nBecause of these practices, NOA reported an inflated result for this indicator. Additionally, the\nstaff reported incomplete results for some indicators. For example, Total value of sales as a\nresult of USG assistance did not adequately capture all of the sales attributed to NOA during\nFY 2011.\n\nReliability. NOA\xe2\x80\x99s staff altered the methodologies used to calculate the results for some\nindicators without making any adjustments to the approved methodologies in the PMP or\nupdating the results already reported. When calculating Value of sales resulting from linkages\ncreated between farmers, processors, and traders as a result of USG assistance for the second\nquarter of FY 2012, the staff included additional calculations that were not considered in\nprevious reporting periods. The results for this quarter include the value-added tax and price\nmarkup paid by the final consumer of the goods.\n\nIn another example, they limited the result for Values of sales/purchases from smallholders for\nproducts as a result of USG assistance for the second quarter of FY 2012 to sales from\nproducers. In previous reporting periods, this result included sales from processors to\nsupermarkets.\n\nNOA officials set ambitious targets for the early phases of implementation. Therefore, the staff\nwanted to show results from the project\xe2\x80\x99s activities by the end of FY 2011\xe2\x80\x94after only 8 months\nof implementation. However, in some cases, these results had not yet been achieved. More\nimportantly, no one from either USAID/Kosovo or Tetra Tech ARD verified the results in the\nprogress reports to identify errors.\n\nNOA\xe2\x80\x99s management has already taken positive steps to address the weaknesses identified in its\nmonitoring and evaluation system. Since the project\xe2\x80\x99s inception, the monitoring and evaluation\nstaff has used Excel spreadsheets to record the results for NOA\xe2\x80\x99s numerous indicators. A\nsophisticated database is currently under development, and it should help reduce reporting\nerrors and improve overall reporting on the project\xe2\x80\x99s accomplishments.\n\nUSAID/Kosovo relies on its implementing partners\xe2\x80\x99 reported results to inform decisions about its\neconomic growth program. However, with inaccurate, incomplete, and inconsistent data, the\nregular progress reports for BEEP and NOA do not accurately reflect the accomplishments\nachieved through project interventions. Incorrect and inconsistent data collection and analysis\npose difficulties in comparing results for all reporting periods to measure the change resulting\nfrom project activities. Without procedures to verify the accuracy of reported results,\nUSAID/Kosovo risks making programming decisions based on weak data.\n\nTherefore, we make the following recommendations.\n\n Recommendation 11. We recommend that USAID/Kosovo develop and implement\n procedures to verify that the performance results reported by its implementing partners\n are reasonably accurate, complete, reliable, and adequately supported.\n\n\n\n\n 16\n\x0cRecommendation 12. We recommend that USAID/Kosovo work with Chemonics\nInternational Inc. and Tetra Tech ARD to develop and implement procedures to verify\nthat the data collection and analysis methodologies used to report results of its activities\nunder the Business Enabling Environment Program and New Opportunities for\nAgriculture project are consistent with those outlined in the projects\xe2\x80\x99 performance\nmanagement plans.\n\nRecommendation 13. We recommend that USAID/Kosovo work with Tetra Tech ARD\nto revise, as necessary, the methodologies used for the collection and analysis of data\nso that reported results under the New Opportunities for Agriculture project are reliable\nand comparable across reporting periods.\n\n\n\n\n 17\n\x0cEVALUATION OF MANAGEMENT\nCOMMENTS\nUSAID/Kosovo agreed with all the recommendations in the draft report. On the basis of\ninformation provided by the mission\xe2\x80\x99s response to the draft report, we determined that final\naction has been taken on Recommendation 9 and management decisions have been reached\non the remaining recommendations.\n\nRecommendation 1. USAID/Kosovo has created a working group that will draft a mission\npolicy on the generation of municipal own source revenue and will meet periodically to ensure\ncoordination on municipality issues. The mission expects to finalize the policy by December 31,\n2012. Accordingly, a management decision has been reached.\n\nRecommendation 2. USAID/Kosovo is preparing the required documentation to modify the\ncontract with Chemonics International Inc. to replace the More Transparent Tenders component\nwith a component on land use reform. Once this is done, the mission will revise the contract\naccordingly. USAID/Kosovo expects to finalize the contract modification by the end of October\n2012. Therefore, a management decision has been reached.\n\nRecommendation 3. USAID/Kosovo is currently drafting a modification of the task order. The\nscope of work will reflect import substitution as one of the project\xe2\x80\x99s objectives. This change will\nmore accurately capture the work being done now. The mission expects to finalize the contract\nmodification by the end of October 2012. Accordingly, a management decision has been\nreached.\n\nRecommendation 4. USAID/Kosovo will revise NOA\xe2\x80\x99s PMP to include a performance indicator\nand target(s) for import substitution, and those revisions will be documented in the revised PMP.\nThe mission expects to finalize the revision by the end of October 2012. Accordingly, a\nmanagement decision has been reached.\n\nRecommendation 5. USAID/Kosovo is working with Tetra Tech ARD to address gender and\nminority inclusion. Tetra Tech ARD will hire a gender specialist to consult on planning,\nimplementation, and monitoring of project interventions. Additionally, the mission will continue\nto work with Tetra Tech ARD to address other mechanisms to target gender and minority\ninclusion. The mission expects to finalize the plan by the end of October 2012. Accordingly, a\nmanagement decision has been reached.\n\nRecommendation 6. USAID/Kosovo will develop procedures to strengthen NOA\xe2\x80\x99s monitoring\nand evaluation. These changes will be documented in a revised PMP. The mission expects to\nfinalize the revision by the end of October 2012. Accordingly, a management decision has been\nreached.\n\nRecommendation 7. USAID/Kosovo will work with Chemonics International Inc. to\ndisaggregate trade indicators from the World Bank\xe2\x80\x99s Doing Business Trading Across Borders\nindicator. Additionally, the mission will add two performance indicators: (1) the number of\ncustoms harmonization procedures implemented in accordance with internationally accepted\nstandards as a result of BEEP assistance and (2) the number of legal, regulatory, or institutional\n\n\n\n 18\n\x0cactions taken to improve implementation or compliance with international trade and investment\nagreements because of BEEP. The mission expects to finalize the revision by the end of\nOctober 2012. Accordingly, a management decision has been reached.\n\nRecommendation 8. USAID/Kosovo is working with Tetra Tech ARD to establish a\nperformance indicator and corresponding target to measure the change in rural incomes as a\nresult of NOA activities. If a proxy indicator is identified, the rationale for the decision and the\nassumptions and methodology will be explained in the revised PMP. The mission expects to\nfinalize the revision by the end of October 2012. Accordingly, a management decision has been\nreached.\n\nRecommendation 9. USAID/Kosovo established a target of 960 credit report requests for the\nperformance indicator: Number of requests by businesses and citizens for their own credit\nreports. We determined that final action has been taken on this recommendation.\n\nRecommendation 10. USAID/Kosovo is working with Tetra Tech ARD to start streamlining the\nPMP for NOA. The COR will review the revised PMP prior to its implementation and will also\nreview its progress periodically. The mission expects to finalize the revision by the end of\nOctober 2012. Accordingly, a management decision has been reached.\n\nRecommendation 11. USAID/Kosovo agreed to review the data quality assessments for BEEP\nand NOA, and analyze their data verification procedures. In addition the mission will assess\nhow the Economic Growth Office monitors the partners\xe2\x80\x99 procedures, identifying gaps and best\npractices in the process. CORs for the projects will then complete new DQAs 3 months after\nthese changes have been implemented to verify that they are compliant. The mission expects\nto finalize the changes by the end of February 2013. Accordingly, a management decision has\nbeen reached.\n\nRecommendation 12. USAID/Kosovo will work with Chemonics International Inc. and Tetra\nTech ARD to review their internal data collection and analysis. To assist, the mission may hold\na workshop with both implementing partners to share best practices, encourage mentoring\nbetween partners, and ensure consistency across the economic growth portfolio. The mission\nexpects to finalize the procedures by the end of October 2012. Accordingly, a management\ndecision has been reached.\n\nRecommendation 13. USAID/Kosovo will work with Tetra Tech ARD to revise, as necessary,\nthe methodologies used for the collection and analysis of data. The mission will review past\nDQAs to improve internal data collection and analysis. The mission will also conduct a follow-\nup DQA to assess any changes to the new methodologies. The mission expects to finalize the\nreview by the end of October 2012. Accordingly, a management decision has been reached.\n\n\n\n\n 19\n\x0c Appendix I\n\n\n\nSCOPE AND METHODOLOGY\nScope\nOIG conducted this performance audit in accordance with generally accepted government\nauditing standards. Those standards require that we plan and perform the audit to obtain\nsufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions\nin accordance with our audit objectives. We believe that the evidence obtained provides that\nreasonable basis.\n\nThe objective of the audit was to determine whether selected USAID/Kosovo\xe2\x80\x99s economic growth\nactivities were achieving their goal of increasing private sector growth and investment. In\nplanning the audit, the audit team reviewed USAID/Kosovo\xe2\x80\x99s economic growth portfolio for\nincreasing private sector growth and investment. The current portfolio has four projects, worth a\ntotal of $54.5 million. The audit team reviewed two, BEEP and NOA, which represent almost\n60 percent of the funding.\n\nUSAID awarded a $16.1 million time-and-materials task order to Chemonics International Inc.\nunder the Support for Economic Growth and Institutional Reform Global Business, Trade, and\nInvestment II indefinite quantity contract to implement BEEP. The 3-year project, expected to\nrun from July 2010 to July 2013, is designed to improve Kosovo\xe2\x80\x99s business enabling\nenvironment. As of February 2012, cumulative obligations and disbursements under BEEP\ntotaled $12.9 million and $6.6 million, respectively.\n\nNOA is implemented through a $15.9 million cost-plus-fixed-fee task order issued to Tetra Tech\nARD through the RAISE PLUS indefinite quantity contract. The 4-year project, expected to run\nfrom January 2011 to February 2015, aims to increase Kosovo\xe2\x80\x99s agricultural outputs, exports,\nand rural incomes. As of February 2012, cumulative obligations and disbursements under NOA\ntotaled $7.4 million and $3.3 million, respectively.\n\nOIG conducted audit fieldwork in Kosovo from April 17 to May 18, 2012. We conducted our\nfieldwork at USAID/Kosovo and implementing partner offices in Pristine and at beneficiary sites\nin and around the cities of Fushe Kosove, Gjakove, Gjilan, Klina, Mamushe, Mitrovice, Mramor,\nPeje, Prizren, Rahovec, Suhareke, and Vushtrri. Our fieldwork covered the period from July\n28, 2010, to May 18, 2012.\n\nIn planning and performing the audit, we assessed management activities and controls\nestablished by USAID/Kosovo and its implementing partners. This included reviewing task\norders and modifications, annual work plans, performance management plans, performance\nreports, data quality assessments, monitoring reports, and portfolio reviews. We also reviewed\nUSAID/Kosovo\xe2\x80\x99s FY 2011 annual self-assessment of management controls for its economic\ngrowth office. In addition, for both of the selected projects, we assessed the implementing\npartners\xe2\x80\x99 activities to address any problems related to trafficking in persons and gender\ninclusion.\n\n\n\n\n 20\n\x0c Appendix I\n\n\nMethodology\nTo answer the audit objective, we obtained an understanding of what USAID/Kosovo intended\nto accomplish through its economic growth program, particularly BEEP and NOA. We\ninterviewed officials from USAID/Kosovo, including the technical officers responsible for\noverseeing the management of the selected projects. We interviewed the implementing\npartners\xe2\x80\x99 personnel responsible for carrying out the selected projects and for reporting the\nresults of activities. Also, we met with officials from the Central Bank of Kosovo, Ministry of\nAgriculture, Forestry, and Rural Development, Ministry of Trade and Industry, and the Prime\nMinister\xe2\x80\x99s Office of Legal Support and Services. We conducted site visits with a judgmental\nselection of project beneficiaries. We reviewed key project documents to obtain an\nunderstanding of the design, principal activities, monitoring and evaluation procedures, and\nprogress achieved for BEEP and NOA. On the basis of the collective results from these\nactivities, we determined the progress of USAID/Kosovo\xe2\x80\x99s selected economic growth activities\ntoward the progress of the program\xe2\x80\x99s goals.\n\nWe assessed USAID/Kosovo\xe2\x80\x99s management of its economic growth program by reviewing\napplicable policies and strategic plans, annual work plans, quarterly and annual progress\nreports, and other key project documents produced by USAID/Kosovo and its implementing\npartners. We compared the expected results with the reported results. We performed site visits\nto observe and discuss the progress and impact of project activities. We judgmentally selected\nsites that represented the various components of the projects.\n\nWe determined the accuracy of the results that the implementing partners reported by\ncomparing the results with the supporting documentation maintained in the partners\xe2\x80\x99 main and\nfield offices. Furthermore, the team reviewed the World Bank Doing Business database for\nBEEP\xe2\x80\x99s reported results. In completing our verification fieldwork, we established a materiality\nthreshold of 5 percent. Therefore, if the results of our validation showed a difference of\n5 percent or less between the reported and verified result, we considered the reported results to\nbe reasonably accurate.\n\n\n\n\n 21\n\x0c Appendix II\n\n\n\nMANAGEMENT COMMENTS\n\n\n\nMEMORANDUM\n\n\nTO: IG/A/PA, Director, Steven Ramonas\n\nFROM: USAID/Kosovo Mission Director, Maureen A. Shauket\n\nSUBJECT: Management Comments to Audit of USAID/Kosovo\xe2\x80\x99s Activities for Economic\n Growth (Report No. 9-000-12-00X-P)\n\n\nDATE: August 06, 2012\n\n===============================================================\n\nThank you for the opportunity to provide USAID/Kosovo\xe2\x80\x99s written comments to the subject\nreport. USAID/Kosovo agrees with all 13 recommendations. Below is USAID/Kosovo\xe2\x80\x99s position\non each recommendation, the planned actions and the target dates for addressing the\nrecommendations. Action on the first recommendation will be completed by December 31,\n2012; action on recommendation 11 will be completed by February 28, 2013; action on the\nremaining recommendations will be completed by October 31, 2012.\n\nUSAID/Kosovo Position on Each Recommendation\n\nRecommendation 1: Develop a mission policy on the generation of municipal own\nsource revenue and implement this policy in its work with municipalities in programs in\nthe Economic Growth and Democracy and Governance Offices.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. The Mission has\nalready created a Municipality Working Group, convened by the Economic Growth Office,\nincluding members from DGO, Program Office and the Financial Management Office. The\nMunicipality Working Group will be involved in the drafting of the mission policy and will also\ncontinue to meet periodically to ensure coordination on municipality issues. We expect the\nmission policy to be finalized by December 31, 2012.\n\nRecommendation 2: Reevaluate the feasibility of implementing activities under the\n\xe2\x80\x9cMore Transparent Tenders\xe2\x80\x9d component of the Business Enabling Environment\nProgram. If a decision is made to not implement these activities, we recommend that\nUSAID/Kosovo modify Chemonics International\xe2\x80\x99s task order by removing this component\n\n\n\n 22\n\x0c Appendix II\n\n\nfrom the scope of work and reducing the ceiling price by $800.000, the amount of funds\ndedicated to this component.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO is currently\ndrafting a modification of the task order that will address this recommendation. The component,\nMore Transparent Tenders, remained on hold during the projects\xe2\x80\x99 first two years, pending the\nresolution on a variety of donors\xe2\x80\x99 intentions. A significant amount of buy-in from the Government\nof Kosovo and other donors is required for this activity to be effective.\n\nWhile e-procurement remained on hold, BEEP assisted the Ministry of Environment and Spatial\nPlanning (MESP) develop a transformational Law on Construction to address long-standing\ndefects in the Kosovo construction legal framework. Following the adoption of the law, work\nremains to be done on implementing the Law on Construction, drafting a Law on Spatial\nPlanning and developing the Construction Code. MESP requires additional resources to\nimplement these activities. EGO is preparing the required documentation to modify the contract\nwith Chemonics International to replace the More Transparent Tenders component with a\ncomponent on Land Use Reform. Once that documentation is finalized, we will know the amount\nby which the contract ceiling is to be revised. We expect the contract modification to be finalized\nby the end of October 2012.\n\nRecommendation 3: Modify the statement of work for the New Opportunities in\nAgriculture project to reflect the decision to refocus the goal of the project on supporting\nimport substitution.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO is currently\ndrafting a modification of the task order that will address this recommendation. The scope of\nwork will reflect import substitution as one of the goals of the project. This change will more\naccurately capture work that is currently on-going; there is no change in activities of the project\nin this request. We expect the contract modification to be finalized by the end of October 2012.\n\nRecommendation 4: Work with TETRA TECH ARD, to revise, as appropriate, the target for\nthe performance indicator for the \xe2\x80\x9ctotal value of exports as a result of USG assistance\xe2\x80\x9d\nto better reflect the decision to focus the New Opportunities for Agriculture project on\nsupporting import substitutions.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. A performance\nindicator and target(s) for import substitution will be documented in a revised Performance\nManagement Plan for the project. We expect this to be finalized by the end of October 2012.\n\nRecommendation 5: Work with TETRA TECH ARD, to develop and implement a plan that\nencourages the targeting of project interventions that focus on increasing gender and\nminority inclusion under the New Opportunities for Agriculture project.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO is currently\nworking with TETRA TECH ARD to address this important issue. TETRA TECH ARD is hiring a\ngender specialist, to advise on planning, implementation and monitoring project interventions\nthat will focus on gender and minority inclusion starting with the project\xe2\x80\x99s support to the Ministry\nof Agriculture Forestry and Rural Development (MAFRD), specifically the Agricultural Extension\nService. Additionally, the mission will continue to work with TETRA TECH ARD to address other\n\n\n\n\n 23\n\x0c Appendix II\n\n\nmechanisms to target gender and minority inclusion. [We expect this to be finalized by the end\nof October 2012.]3\n\nRecommendation 6: Work with TETRA TECH ARD, to develop procedures to strengthen\nthe New Opportunities for Agriculture monitoring and evaluation system so that it\nadequately measures the result of gender and minority inclusion in its activities.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees and supports this recommendation. EGO\nhas already engaged TETRA TECH ARD on this issue. These changes will be documented in a\nrevised Performance Management Plan for the project. We expect this to be finalized by the end\nof October 2012.\n\nRecommendation 7: Work with Chemonics, International to establish a performance\nindicator and corresponding target to measure the progress of its activities designed to\nlower trade barriers under the Business Enabling Environment Program.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. The project has\nbeen using a performance indicator that captures this information along with other information.\nThe World Bank\xe2\x80\x99s Doing Business Trade Across Borders indicator is actually comprised of a\nnumber of sub-indicators that are then combined. Some of the sub-indicators measure the\nprogress of USAID\xe2\x80\x99s activities designed to lower trade barriers. We will work with Chemonics to\ndisaggregate this indicator. Additionally, the following indicators will be added: (1) the number\nof customs harmonization procedures implemented in accordance with internationally accepted\nstandards as a result of BEEP assistance, and (2) the number of legal, regulatory, or\ninstitutional actions taken to improve implementation or compliance with international trade and\ninvestment agreements due to support from BEEP. All reforms that lower barriers to trade will\nbe captured in the disaggregated entries as this measure is directly attributable to project\nactivities and demonstrably contribute to an improved business environment. We expect these\nto be finalized by the end of October 2012.\n\nRecommendation 8: Work with TETRA TECH ARD, to establish a performance indicator\nand corresponding target to measure the change in rural incomes as a result of activities\nunder the New Opportunities for Agriculture Project. If a proxy indicator is deemed\nappropriate, we recommend that USAID/Kosovo work with TETRA TECH ARD, to\nadequately document the assumptions supporting the choice of indicator and the\nmethodology for its collection in the New Opportunities for Agriculture project\nperformance management plan.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO has already\nengaged TETRA TECH ARD on this issue. If a proxy indicator is identified, the rationale for the\ndecision and the assumptions and methodology will be well explained in the revised\nPerformance Management Plan. We expect these to be finalized by the end of October 2012.\n\nRecommendation 9: Work with Chemonics, International to determine a realistic, yet\nachievable, target for the performance indicator \xe2\x80\x9cnumber of requests by businesses and\ncitizens for their own credit reports\xe2\x80\x9d under the business Enabling Environment Program.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. As an ambitious\nbut achievable goal for the \xe2\x80\x9cnumber of requests by businesses and citizens for their own credit\n\n3\n Date of completion was omitted in USAID/Kosovo\xe2\x80\x99s initial response but later added as an amendment.\n\n\n 24\n\x0c Appendix II\n\n\nreports\xe2\x80\x9d, USAID and Chemonics, International have established the target of 960 credit report\nrequests. This is 80 new requests every month for the next 12 months. We consider this\nrecommendation to be closed.\n\nRecommendation 10: Work with TETRA TECH ARD, to streamline the performance\nmanagement plan for the New Opportunities for Agriculture project by determining those\nperformance indicators that best fit criteria defined in ADS 203.3.4.2 and eliminating\nthose performance indicators that demonstrate little usefulness for project management.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO has already\nengaged TETRA TECH ARD on this issue. The Contracting Officers Representative will review\nthe revised Performance Management Plan prior to its implementation and will also review\nprogress on the plan on a periodic basis. We expect this to be finalized by the end of October\n2012.\n\nRecommendation 11: Develop and implement procedures to verify that the performance\nresults reported by its implementing partners are reasonably accurate, complete,\nreliable, and adequately supported.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO will carefully\nreview the Data Quality Assessments (DQA) for both BEEP and NOA, analyze the data\nverification procedures of both partners and also assess how the EGO office monitors the\npartners\xe2\x80\x99 procedures, identifying gaps and best practices. Contracting Officer Representatives\nfor the two projects will then complete new DQAs three months after the changes recommended\nby this audit have been implemented to ensure compliance. In line with the recommendations\nabove, we expect the partners to have new indicators and data collection methods for those\nindicators by the end of October 2012. Therefore our review of the new systems and new DQAs\nwill take place by the end of February 2013.\n\nRecommendation 12: Work with Chemonics International and TETRA TECH ARD, to\ndevelop and implement procedures to verify that the data collection and analysis\nmethodologies used to report results of its activities under the Business Enabling\nEnvironment Program and New Opportunities for Agriculture project are consistent with\nthose outlined in the projects\xe2\x80\x99 performance management plans.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. EGO will work with\nboth partners to review their internal data collection and analysis. A joint workshop between all\nprivate sector EGO partners may be utilized to enhance the sharing of best practices,\nencourage mentoring between partners and ensure consistency across the portfolio. We expect\nthis work to be finalized by the end of October 2012.\n\nRecommendation 13: Work with TETRA TECH ARD, to revise, as necessary, the\nmethodologies used for the collection and analysis of data so the reported results under\nthe New Opportunities for Agriculture project are reliable and comparable across\nreporting periods.\n\nUSAID/Kosovo Response: USAID/Kosovo agrees with this recommendation. This particular\nfinding will be addressed by the actions proposed in Recommendations 11 and 12; a review of\npast DQAs, assistance for improving internal data collection and analysis, and a follow-up DQA\nassessment. We expect this work to be finalized by the end of October 2012.\n\n\n\n 25\n\x0c Appendix III\n\n\n AGRICULTURE PROJECT REPORTED AND VERIFIED RESULTS\n\n Table III-1. NOA Reported and Verified Results FY 2011 (Audited)\n Indicator Reported Verified Difference (%)\nAssistance Objective Indicators\nTotal Value of Sales as a Result of USG\n $522,304 $595,978 14.11\nAssistance\nTotal Value of Exports as a Result of\n $0 $0 0.00\nUSG Assistance\nTotal Value of Domestic Sales as a\n $522,304 $595,978 14.11\nResult of USG Assistance\nNumber of Person-Days/Full Time\nEquivalents (FTE) Generated Through\n 259 Unable to verify -\nTarget Value Chains as a Result of USG\nAssistance\nProducts and Farmers Linked to Markets\nNumber of Delivery Contracts Issued for\n 13 0 -100.00\nTargeted Crops\nValue of Sales Resulting from Linkages\nCreated Between Farmers, Processors,\n $138,594 $75,324 -45.65\nand Traders as a Result of USG\nAssistance\nNumber of Farmers Engaged in Target\nValue Chains as a Result of USG 331 Unable to verify -\nAssistance\nNumber of Participants in Study Tours,\nBusiness to Business Meetings (B2B), 34 31 -8.82\nMarket Investigation, and Trade Shows\nAgricultural Products Diversified and Increased\nNumber of New Markets Entered for\n 5 5 0.00\nTarget Value Chain Products\nValue of Sales/Purchases from\nSmallholders for Products as a Result of $522,304 $276,496 -47.06\nUSG Assistance\nNumber of New Technologies and/or\nManagement Practices Introduced as a 6 6 0.00\nResult of USG Assistance\nNumber of Individuals Trained in\nAgriculture Productivity Through USG 306 284 -7.19\nAssistance\nNumber of Farmers, Processors, and\nOthers Who Have Adopted New\n 70 0 -100.00\nTechnologies or Management Practices\nas a Result of USG Assistance\nNumber of Farmers Receiving USG\nAssistance to Invest in Improved 11 11 0.00\nTechnologies\nNumber of Hectares Under Improved\nTechnologies and/or Management 8.4 8.4 0.00\nPractices as a Result of USG Assistance\nNumber of New Varieties (Including New\nCrops) Introduced as a Result of USG 20 20 0.00\nAssistance\n\n\n 26\n\x0c Appendix III\n\n\n Indicator Reported Verified Difference (%)\nNumber of Farmers Receiving New\nVarieties (Including New Crops) as a 21 23 9.52\nResult of USG Assistance\nFood Quality and Safety Improved\nNumber of Firms Receiving USG\nAssistance that Obtain Certification with\nInternational Quality Control, 1 0 -100.00\nEnvironmental, and Other Process,\nVoluntary Standards, or Regulations\nNumber of NOA-Supported Products\nCertified and Meeting Established 0 0 0.00\nInternational Standards\nValue of Sales of NOA-Supported\n 0 0 0.00\nCertified Products\nNumber of Local Food Inspectors\n 0 0 0.00\nTrained\nIncreased Affordable and Accessible Credit\nValue of Lending of Project Customers\nfor Targeted Crops and Products as a 0 0 0.00\nResult of USG Assistance\nNumber of Institutions Offering New\nProducts Targeted at Agriculture and\n 1 0 -100.00\nAgri-business as a Result of USG\nAssistance\nNumber of Small and Medium\nEnterprises (SME) Receiving USG\n 41 41 0.00\nAssistance to Access Bank Loans or\nPrivate Equity\nNumber of Producers in Target Value\nChains Receiving Assistance to Access 46 46 0.00\nCredit\nValue of Grants and Subcontracts Issued\nfor Value Chain Operations of Target\nCrops and Products as a Result of USG 0 0 0.00\nAssistance (excluding Innovation and\nIncentive Fund [IIF])\nValue of Grants and Subcontracts Issued\nfor Value Chain Operations of Target\n $387,840 $326,856 -15.72\nCrops and Products as a Result of USG\nAssistance (IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 0 0 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (excluding IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 28 28 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (IIF)\n\n\n\n\n 27\n\x0c Appendix III\n\n\n Indicator Reported Verified Difference (%)\nImproved Coordination Within Agricultural Sector\nNumber of Policy\nReforms/Regulations/Administrative\nProcedures Drafted and Presented for 0 0 0.00\nPublic/Stakeholder Consultation as a\nResult of USG Assistance\nNumber of Policy\nReforms/Regulations/Administrative\nProcedures Drafted and Presented for\n 0 0 0.00\nPublic/Stakeholder Consultation and\nSubmitted for Approval as a Result of\nUSG Assistance\nNumber of Donor, Government Of\nKosovo, and Agriculture Sector\n 11 11 0.00\nRoundtables or Other Events Facilitated\nby NOA\n\n\n Table III-2. NOA Reported Results FY 2012, Quarter 2 (Audited)\n Quarterly Indicators Only\n Indicator Reported Verified Difference (%)\nProducts and Farmers Linked to Markets\nNumber of Delivery Contracts Issued for\n 1 1 0.00\nTargeted Crops\nValue of Sales Resulting from Linkages\nCreated Between Farmers, Processors,\n $82,580 $59,040 -28.51\nand Traders as a Result of USG\nAssistance\nNumber of Farmers Engaged in Target\nValue Chains as a Result of USG 389 Unable to Verify -\nAssistance\nNumber of Participants in Study Tours,\nB2B, Market Investigation, and Trade 15 13 -13.33\nShows\nAgricultural Products Diversified and Increased\nValue of Sales/Purchases from\nSmallholders for Products as a Result of $141,246 $97,874 -30.71\nUSG Assistance\nNumber of New Technologies and/or\nManagement Practices Introduced as a 4 4 0.00\nResult of USG Assistance\nNumber of Individuals Trained in\nAgriculture Productivity Through USG 207 199 -3.86\nAssistance\nNumber of Farmers Receiving USG\nAssistance to Invest in Improved 22 22 0.00\nTechnologies\nNumber of Farmers Receiving New\nVarieties (Including New Crops) as a 33 30 -9.09\nResult of USG Assistance\nFood Quality and Safety Improved\nNumber of Local Food Inspectors Trained 0 0 0.00\n\n\n\n 28\n\x0c Appendix III\n\n\n Indicator Reported Verified Difference (%)\nIncreased Affordable and Accessible Credit\nValue of Lending of Project Customers\nfor Targeted Crops and Products as a 0 0 0.00\nResult of USG Assistance\nNumber of SMEs Receiving USG\nAssistance to Access Bank Loans or 0 0 0.00\nPrivate Equity\nNumber of Producers in Target Value\nChains Receiving Assistance to Access 0 0 0.00\nCredit\nValue of Grants and Subcontracts Issued\nfor Value Chain Operations of Target\n $828,145 $995,157 20.17\nCrops and Products as a Result of USG\nAssistance (IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 0 0 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (excluding IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 40 33 -17.50\nGrants and/or Subcontracts as a Result\nof USG Assistance (IIF)\nImproved Coordination Within Agricultural Sector\nNumber of Donor, GOK, and Agriculture\nSector Roundtables or Other Events 12 0 -100.00\nFacilitated by NOA\n\n\n Table III-3. NOA Reported Results FY 2012, Quarter 1\n Quarterly Indicators Only (Audited)\n Indicator Reported Verified Difference (%)\nProducts and Farmers Linked to Markets\nNumber of Delivery Contracts Issued for\n 68 69 1.47\nTargeted Crops\nValue of Sales Resulting from Linkages\nCreated Between Farmers, Processors,\n $138,621 $90,500 -34.71\nand Traders as a Result of USG\nAssistance\nNumber of Farmers Engaged in Target\nValue Chains as a Result of USG 559 574 2.68\nAssistance\nNumber of Participants in Study Tours,\nB2B, Market Investigation, and Trade 37 34 -8.11\nShows\nAgricultural Products Diversified and Increased\nValue of Sales/Purchases from\nSmallholders for Products as a Result of $164,253 $179,554 9.32\nUSG Assistance\nNumber of New Technologies and/or\nManagement Practices Introduced as a 8 8 0.00\nResult of USG Assistance\n\n\n\n\n 29\n\x0c Appendix III\n\n\n Indicator Reported Verified Difference (%)\nNumber of Farmers Receiving USG\nAssistance to Invest in Improved\n 4 0 -100.00\nTechnologies\n\nNumber of Farmers Receiving New\nVarieties (Including New Crops) as a 76 74 -2.63\nResult of USG Assistance\nFood Quality and Safety Improved\nNumber of Local Food Inspectors\n 13 13 0.00\nTrained\nIncreased Affordable and Accessible Credit\nValue of Lending of Project Customers\nfor Targeted Crops and Products as a 0 0 0.00\nResult of USG Assistance\nNumber of SMEs Receiving USG\nAssistance to Access Bank Loans or 0 0 0.00\nPrivate Equity\nNumber of Producers in Target Value\nChains Receiving Assistance to Access 0 0 0.00\nCredit\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 0 0 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (excluding IIF)\nValue of Grants and Subcontracts Issued\nfor Value Chain Operations of Target\n $27,737 $27,737 0.00\nCrops and Products as a Result of USG\nAssistance (IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 1 1 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (IIF)\nNumber of Value Chain Operators of\nTarget Crops and Products Receiving\n 0 0 0.00\nGrants and/or Subcontracts as a Result\nof USG Assistance (excluding IIF)\nImproved Coordination Within Agricultural Sector\nNumber of Donor, GOK, and Agriculture\nSector Roundtables or Other Events 32 32 0.00\nFacilitated by NOA\n\n\n\n\n 30\n\x0cU.S. Agency for International Development\n Office of Inspector General\n 1300 Pennsylvania Avenue, NW\n Washington, DC 20523\n Tel: 202-712-1150\n Fax: 202-216-3047\n www.usaid.gov/oig\n\x0c"


type: caselaw

jurisdiction: EU

text:
 18.4.2009
 ET
 Euroopa Liidu Teataja
 C 90/17
 Eelotsusetaotlus, mille esitas Lietuvos Aukščiausiasis Teismas (Leedu Vabariik) 16. veebruaril 2009 – Kirin Amgen, Inc. versus Lietuvos Respublikos valstybinis patentų biuras
 (Kohtuasi C-66/09)
 2009/C 90/27
 Kohtumenetluse keel: leedu
 Eelotsusetaotluse esitanud kohus
 Lietuvos Aukščiausiasis Teismas
 Põhikohtuasja pooled
 Kaebuse esitaja: Kirin Amgen, Inc.
 Vastustaja: Lietuvos Respublikos valstybinis patentų biuras
 Eelotsuse küsimused
 1.
 Kas määruse nr 1768/92 (1) artikli 19 lõikes 2 osutatud jõustumise kuupäevaks tuleb Leedu puhul pidada Euroopa Liiduga ühinemise kuupäeva?
 2.
 Kui vastus esimesele küsimusele on jaatav, siis milline on määruse nr 1768/92 artikli 19 ja artikli 7 vaheline suhe kuuekuulise tähtaja arvutamisel ning kumba artiklit tuleb kohtuasjas kohaldada?
 3.
 Kas toote müügiluba Euroopa Ühenduses hakkas Leedu Vabariigis kehtima tingimusteta alates kuupäevast, mil ta ühines Euroopa Liiduga?
 4.
 Kui vastus kolmandale küsimusele on jaatav, siis kas toote müügiloa kehtima hakkamine on võrdsustatav selle loa andmisega määruse nr 1768/92 artikli 3 punkti b tähenduses?
 (1) Nõukogu 18. juuni 1992. aasta määrus (EMÜ) nr 1768/92 ravimite täiendava kaitse tunnistuse kasutuselevõtu kohta (EÜT L 182, lk 1; ELT eriväljaanne 13/11, lk 200).


type: wikipedia

text:
Max y Ruby es una serie animada estadounidense-canadiense producida por Silver Lining Productions, 9 Story Media Group, Atomic Cartoons, Treehouse Original, Nickelodeon Animation Studios y Nelvana Limited para Treehouse TV y Nickelodeon. La serie es basada en los libros infantiles creados por Rosemary Wells, los episodios consisten en las historias sobre los conejos Max y su hermana mayor Ruby a las 6:30 a.m. en 21 de octubre de 2002 en Nick Jr. en el Estados Unidos.
 Personajes
 Ruby: Una coneja de 7 años de edad, por lo general usa un vestido de color violeta y delantal amarillo. Y se la ve mayor parte del tiempo cuidando a su hermano menor Max. Los padres de Max y Ruby nunca son vistos en la serie, hasta la temporada 7.
 Max: Un conejo de aproximadamente 3 años de edad, viste una jardinera y oberol. Durante las primeras temporadas de la serie, él suele repetir las palabras o frases cortas que otros personajes, dicen, y conforme avanza la serie, él aprende a hablar con más fluidez y a decir frases más largas
 Louise: Es la mejor amiga de Ruby, es una coneja de pelaje marrón que visita a Ruby cuando tienen que hacer trabajos de Scout juntas.
 La Abuela: Es la abuela de Max y de Ruby. Le gustan tanto las cosas de Max como las de Ruby.
 Señora Hoffinton: Es la capitana de las conejitas exploradoras.
 Señor Hoffinton: Es el papá del Bebe Hoffinton y esposo de las Señora Hoffinton.
 Juguetes de Max
 Hombre Espacial: Juguete favorito de Max. Es un robot rojo con cabeza cónica y cuatro botones en su pecho. Tiene una frase: "Alerta roja, alerta roja. Peligro, peligro".
 Araña Escupebolas de gelatina: Segundo juguete favorito de Max. Es una araña verde y morada con una cuerda para colgarla. Puede escupir bolas de gelatina.
 Langosta de Cuerda: Tercer juguete favorito de Max. Es una langosta roja de cuerda capaz de estar varios minutos sin que se le agote la cuerda. También puede caminar por las paredes. Fue un regalo de los tíos de los protagonistas. Al principio, a Max no le gustó, pero luego le cogió cariño.
 Babosa que no se posa: Peluche con forma de babosa amarilla.
 Loro, saluda al mundo: Un loro rojo capaz de decir "Hola" en diversos idiomas como el inglés o francés, además del español mientras camina.
 Anguila que Brilla en la Oscuridad: Una anguila eléctrica verde con la capacidad de brillar en la oscuridad. Max la usa como linterna.
 Bichos de cuerda: Son unos bichos de diferentes colores que están en una caja.
 Ambulancia: Es el octavo juguete de Max que dice "Emergencia, emergencia" cuando suele estar encendido o al rodar.
 Lugares en donde es emitida la serie
 RETV
 Tooniverse, EBS y Nickelodeon
 Treehouse TV, Nick Jr., TFO
 Kanal 5
 MTV3, SubTV y Yle
 Nickelodeon Junior, France 5,
 ABC
 Rede Record
 ATV
 Disney Channel, Nick, Nick Jr.
 NPO
 LTV1, LTV7
 LRT televizija, LRT Plius, LRT Lituanica
 TVM
 RaiSat Ragazzi, RaiSat Yoyo, Nick Jr.
 Nick Jr. España
 Hop!
 Nick Jr, TVI, RTP1. RTP2, Disney Channel y Canal Panda
 Boomerang (2006-2008), CBeebies (12 de enero de 2009), Nick Jr. (2011-presente)
 Moon TV
 ZTV, Norsk TV1
 ZBC , ExpressAfricaTV , Kidz.Net, Disney Junior, Star Kidz y Nickelodeon
 Nickelodeon y Nick Jr.
 Nick Jr. y Nick Jr. 2
 Nickelodeon
 Señal Colombia
 Cartoon Network, Boomerang, Nickelodeon y Nick Jr.
 Nickelodeon y Nick Jr.
 Referencias
 Enlaces externos
 Sitio oficial de Max and Ruby en TreehouseTV.com
 Max y Ruby en Noggin
 Series de televisión animadas
 Series de televisión infantiles
 Series de televisión de Nickelodeon
 Series de televisión de Nick Jr.
 Conejos de ficción
 Series de televisión infantiles basadas en libros
 Series dobladas al español en Venezuela
 Programas de televisión educativos


text:
6.9.2003 DE Amtsblatt der Europäischen Union C 213/11
 Rechtsmittel der Maja Srl gegen das Urteil der Zweiten Ersuchen um Vorabentscheidung, vorgelegt mit Urteil
 Kammer des Gerichts erster Instanz der Europäischen des Conseil d’État (Belgien), section d’administration
 Gemeinschaften vom 12. März 2003 in der Rechtssache (Verwaltungsabteilung), vom 9. Mai 2003 in dem Rechts-
 T-254/99, Maja Srl gegen Kommission der Europäischen streit Gesellschaft niederländischen Rechts MERCK,
 Gemeinschaften, eingelegt am 27. Mai 2003 SHARP und DOHME B. V. gegen Belgischer Staat
 (Rechtssache C-238/03 P) (Rechtssache C-245/03)
 (2003/C 213/17) (2003/C 213/18)
 Die Maja Srl hat am 27. Mai 2003 beim Gerichtshof der Der Conseil d’État (Belgien), section d’administration (Verwal-
 Europäischen Gemeinschaften ein Rechtsmittel gegen das tungsabteilung), ersucht den Gerichtshof der Europäischen
 Urteil der Zweiten Kammer des Gerichts erster Instanz der Gemeinschaften mit Urteil vom 9. Mai 2003, bei der Kanzlei
 Europäischen Gemeinschaften vom 12. März 2003 in der des Gerichtshofes eingegangen am 10. Juni 2003, in dem
 Rechtssache T-254/99, Maja Srl gegen Kommission der Euro- Rechtsstreit Société de droit néerlandais Merck, Sharp und
 päischen Gemeinschaften, eingelegt. Prozessbevollmächtigte Dohme B. V. gegen Belgischer Staat um Vorabentscheidung
 der Klägerin sind die Rechtsanwälte Paolo Piva, Venedig, über folgende Frage:
 Roberto Mastroianni, Cosenza, und Guy Arendt, Luxemburg.
 Ist die Frist von neunzig Tagen, die gemäß Artikel 6 Absatz 1
 Die Rechtsmittelführerin beantragt, Unterabsatz 1 der Richtlinie 89/105/EWG des Rates vom
 21. Dezember 1988 betreffend die Transparenz von Maßnah-
 — das Urteil des Gerichts erster Instanz (Zweite Kammer) men zur Regelung der Preisfestsetzung bei Arzneimitteln für
 vom 12. März 2003 als rechtswidrig aufzuheben und den menschlichen Gebrauch und ihre Einbeziehung in die
 dementsprechend die angefochtene Entscheidung für staatlichen Krankenversicherungssysteme ( 1) um weitere neun-
 nichtig zu erklären; zig Tage verlängert werden kann, eine Ordnungs- oder eine
 Ausschlussfrist? Welche Folgen hat im letztgenannten Fall eine
 — der Kommission nach den allgemeinen Vorschriften die Fristüberschreitung für die Beantwortung des Antrags auf
 Kosten in beiden Rechtszügen aufzuerlegen. Aufnahme eines Arzneimittels in die Liste der unter das
 Krankenversicherungssystem fallenden Erzeugnisse?
 Ist diese Fristüberschreitung dahin auszulegen, dass sie als
 Rechtsmittelgründe und wesentliche Argumente
 Aufnahme in diese Liste gilt?
 Die Rechtsmittelführerin macht geltend, dass das Urteil des (1 ) ABl. L 40 vom 11.2.1989, S. 8.
 Gerichts erster Instanz vom 5. März 2003 aus folgenden
 Gründen Rechtsfehler aufweise:
 Verstoß gegen die Verordnung Nr. 4028/86 ( 1) sowie gegen
 die Durchführungsverordnung Nr. 1116/88 (2), wie sie auch
 im Licht der „Komitologieentscheidung“ auszulegen seien;
 Begründungsmangel und Folgewidrigkeit der Begründung;
 Verstoß gegen den Grundsatz des kontradiktorischen Verfah- Klage der Kommission der Europäischen Gemeinschaften
 rens und des berechtigten Vertrauens; offensichtliche Folge- gegen die Französische Republik, eingereicht am 17. Juni
 widrigkeit und Widersprüchlichkeit; fehlende Entsprechung 2003
 von Antrag und Entscheidung; Verstoß gegen die Begrün-
 dungspflicht gemäß Artikel 33 der Satzung des Gerichtshofes
 (Rechtssache C-258/03)
 der EG, auf den in Artikel 46 dieser Satzung verwiesen werde;
 Verstoß gegen den Grundsatz der Verteidigungsrechte; Verstoß
 gegen Rechts- und wesentliche Formvorschriften; Verstoß (2003/C 213/19)
 gegen die allgemeinen Grundsätze auf dem Gebiet der Übertra-
 gung von Befugnissen.
 Die Kommission der Europäischen Gemeinschaften hat am
 ( 1) ABl. L 376 vom 31.12.1986, S. 7.
 17. Juni 2003 eine Klage gegen die Französische Republik beim
 ( 2) ABl. L 112 vom 30.4.1988, S. 1.
 Gerichtshof der Europäischen Gemeinschaften eingereicht.
 Bevollmächtigte der Klägerin ist M. Condou Durande, Zustel-
 lungsanschrift in Luxemburg.