WPS5635 Policy Research Working Paper 5635 A Second Look at the Pesticides Initiative Program Evidence from Senegal Melise Jaud Olivier Cadot The World Bank Poverty Reduction and Economic Management Network International Trade Department April 2011 Policy Research Working Paper 5635 Abstract This paper investigates whether the Pesticides Initiative firm-level dataset containing data on sales, employment, Program has significantly affected the export performance and exports by product and destination markets, as well of Senegal’ shorticulture industry. The authors apply as firm enrolment year, over 2000–2008. The results two main microeconometric techniques, difference-in- suggest that wile the program had no significant effect differences and matching difference-in-differences, to on exports pooled over all products and destinations, it identify the effect of the Pesticides Initiative Program on had a positive effect when considering fresh fruits and exports of fresh fruits and vegetables. They use a unique vegetables exports to the European Union. This paper is a product of the International Trade Department, Poverty Reduction and Economic Management Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank. org. The author may be contacted at ocadot@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team A Second Look at the Pesticides Initiative Program: Evidence from Senegal ∗ † ‡ Melise Jaud Olivier Cadot May 2011 Keywords: Senegal, EU, Agriculture, Export Promotion, Technical Assistance, Impact Evaluation. JEL classi�cation codes: F13, F14, L15, L25, O17, O24, C23. ∗ This paper is part of a joint program of Switzerland’s NCCR and the World Bank on the application of impact-evaluation methods to trade-related interventions. Without implicating e e them, we would like to express our thanks to C´line Carr`re, Ana Fernandes, Julien Gourdon, Daniel Lederman, Aaditya Mattoo, Jaime de Melo, Marcelo Olarreaga, Alberto Portugal and Akiko Suwa for useful comments. We are also grateful to the European Commission and to e e Senegal’s Direction G´n´rale des Douanes and Direction Nationale des Statistiques for access to their data. † The World Bank and Paris School of Economics; mjaud@worldbank.org ‡ The World Bank, University of Lausanne, and CEPR; ocadot@worldbank.org Abstract This paper investigates whether the Pesticides Iniative Program has sig- ni�cantly affected the export performance of Senegal’s horticulture industry. We apply two main microeconometric techniques, difference-in-differences and matching difference-in-differences, to identify the effect of the Pesti- cides Initiative Program on exports of fresh fruits and vegetables. We use a unique �rm-level dataset containing data on sales, employment, and exports by product and destination markets, as well as �rm enrolment year, over 2000-2008. The results suggest that while the program had no signi�cant effect on exports pooled over all products and destinations, it had a positive effect when considering fresh fruits and vegetables exports to the European Union. 2 1 Introduction A poll featured in the Financial Times of July 12, 2010 highlighted two striking facts. First, between 60 and 80 percent of the respondents in Western countries felt that budget cuts would help rather than hurt the economy; second, aid to developing countries was ranked �rst as a candidate for cuts in the U.S. and U.K., and second in all other countries. Against this background, in the words of a recent OECD document, “[t]here is a growing political demand among academics, NGOs, international agencies and voters for results in development.� (OECD 2010a). The demand for evaluation applies to aid for trade (AFT) as well as to other forms of aid.1 After declining as a share of Overseas Development Assistance (ODA) in the second half of the 1990s, AFT commitments have increased by 62% since the launch of the AFT initiative at the 2005 WTO Ministerial in Hong Kong, reaching $41.7 billion or 37% of “sector-allocable� ODA in 2008. For the EU, trade- related assistance (TRA) is an important part of the trade-and-development policy nexus, responding to a widely-perceived need to provide adjustment assistance to Southern partners affected by increasingly stringent product standards and by recent changes in the EU’s system of preferences (see te Velde et al. 2006). What do we know about the impact of aid for trade (AFT)? The literature is limited. As for its allocation, Gamberoni and Newfarmer (2009) found that, after controlling for absorption capacity (governance etc.), it correlated with an indicator of ‘demand’ based on indices of under-trading. As for its impact, Wagner (2003) explored whether aid generated trade in the form of exports from the donor country to the recipient (up to the early 1990s, over half of all bilateral aid was at least partially tied to donor exports); he found, using a gravity equation, that this particular form of trade was indeed boosted. By contrast, Osei, Morrissey and Lloyd (2004) ran a gravity equation in �rst differences on a panel of four European donors and 26 African recipients after testing for the direction of causalitybetween bilateral aid and trade flows. They found an unstable and, on the whole, insignif- icant impact of aid on trade (also in the form of exportsfrom donor to recipient). Nelson and Juhasz Silva (2008) used a moreconventional gravity equation includ- ing bilateral aid flows as a regressor (instrumented by their one-year lagged value) and found a signi�cant although small elasticity, again, for flows from donor to recipient. 1 Aid for Trade was de�ned by the 2006 WTO AFT Task Force through six categories: (i) Trade policy & regulations; (ii) Trade development; (iii) Trade-related infrastructure; (iv) Build- ing productive capacity; (v) Trade-related adjustment; (vi) Other trade-related needs. The �rst three are referred to as Trade-related assistance (TRA); the last three are referred to as the ‘wider aid-for-trade agenda’ (EC 2008). 3 Only some of the most recent studies have looked at whether aid raised the export capacity of recipient countries. Cali and te Velde (2009) regressed trade costs and the value of exports on control variables and lagged AFT disburse- ments, using data from the OECD’s Credit Reporting System. Their sample was a panel of countries covering 1995-2007, and they dealt with endogeneity and measurement errors in AFT flows by instrumenting them with the World Bank’s index of civil liberties. The message that seems to emerge across their various speci�cations is that aid to trade facilitation and infrastructure seems (by and large) to have a signi�cant effect (the former on trade costs, the latter on ex- port values), while aid to productive capacity is insigni�cant. When looking at sectorally-targeted aid, controlling for country × sector �xed effects (comparative advantage) they found, again, that aid to infrastructure had a signi�cant impact but aid to productive capacity didn’t. Brenton and von Uexkull (2009) combined mirrored product-level (HS4) export data with export-development (ED) aid data from the GTZ (covering 1975-2005) and from the OECD/WTO Trade Capacity Building Database (covering 2001-2005) for 48 countries. They useda difference- in-differences (DID)2 approach where the performance variable was exports and regressors included lagged exports, country and year × product �xed effects, and contemporaneous and lagged aid coded in binary form (ED program in force = 1). By and large, once outliers were dropped, matching yielded insigni�cant coef- �cients in spite of the relatively large sample size, suggesting that once selection effects were taken into account, export-development programs provided little sig- ni�cant boost to exports. All in all, it is fair to say that, as the literature stands, the effect of AFT on the export performance of bene�ciary countries has not been clearly established on the basis of aggregate numbers. Among TRA programs, technical-assistance ones have scarcely been evalu- ated.3 A brochure published by the EU Commission (EC 2006) tells the interesting story of a Kenyan fruit and vegetables exporter who got assistance from the EU’s Pesticides Initiative program (PIP); the case study presents the program as “[...] to provide support to companies like Myner [the Kenyan ex- porter], to help them get up to speed with European food safety and traceability requirements. [...] Since it began in July 2001, the PIP has had a positive effect on more than 26’000 ACP producers, many of whom are small-scale farmers. Nearly 6.25 million Euros has been 2 See section 3 below for a discussion of the DID and matching approaches. 3 Marcano and Ruprah (2009) is one of the few papers looking speci�cally at technical- assistance programs. Using a multiple-treatment approach, they found a positive impact to Chile’s Neighborhood Improvement Program and Guatemala’s Social Investment Fund (both water-access program) over and above public-works spending. 4 committed to the program, with each applicant allocated around 86’000 Euros. In line with the principles of the ACP-EU partnership agree- ment singed in Cotonou in 2000, the PIP aims to contribute to the development of the ACP’s private sector and to promote regional inte- gration.� As for the case study itself, the brochure explains that “[w]hen Myner Exports began working with the Pesticides Initiative Program, or PIP in 2002, it was exporting about 300 tonnes of French beans, snow peas, passion fruit, and sugar snaps a year to the European Union. Today, the company exports some 900 tonnes a year. � Quoting this particular case study in their assessment of the EU’s TRA, te Velde et al. (2006) noted that “in an ideal world, one would compare this supported company with a similar one that was not supported� (p. 21). This is precisely what we set up to do in the present paper, although using Senegal instead of Kenya as our sample of study. We chose Senegal because the data we had access to provided a unique combi- nation. Through a World Bank project run by Denisse Pierola and Paul Brenton at the World Bank, we obtained raw �les from Senegal’s customs with export data at the transaction level for every year between 2000 and 2008. The data include the exporter’s ID, the product code, the country of destination, and the export value for over 500 HS8 fresh fruit and vegetable (FFV) products. We were able to merge the customs’ dataset with industrial-survey data and a list of PIP bene�cia- ries provided by the EU Commission.4 This unique combination made it possible to construct a treatment group of �rms that got assistance and a control group of �rms that did not, and this for a sample period that ran from before the program to its end.5 Using this rich data set, we used a wide array of approaches to estimate the effect of the PIP on �rm-level exports of treated products (FFV). We ran DID regressions of the value of exports, by product × �rm × destination, on control variables as well as a dummy variable marking ‘treatment’ by the PIP. In order to deal with selection issues, we combined the DID approach with propensity-score matching, although the small size of the sample and the systematic differences in size between treatment- and control-group �rms considerably reduced the power of 4 Thanks to Morag Webb for providing us with the data. 5 Although we had similar customs data from a number of other countries, none covered years before the start of the PIP. After merging, �rm IDs have been deleted from the dataset for con�dentiality reasons. 5 matching. We also used a control-function approach similar to Heckman’s selection model (Heckman 1979). In most speci�cations we tried, we failed to �nd a signi�cant impact. Only when considering the �rm aggregate exports of FFV to the EU, did we �nd a positive and signi�cant effect of the program. Given the small size and peculiarities of the sample, our results should be treated very cautiously and we would certainly stop short of concluding that the PIP was useless on the basis of this single impact- evaluation exercise. Additionally, bene�ciary �rms self-selecting in the program would introduce a bias in the estimated treatment effect. However the direction of the bias is not clear. If self-selected �rms are larger and potentially more efficient than others the estimated treatment effect should be biased upwards. Inversely less productive �rms may be more likely to rely on �nancial aid or rent-seeking, biasing the effect downwards. If so, clearly more research is needed on this issue, possibly on other, larger samples to assess whether the PIP had any impact or not. The paper is organized as follows. Section 2 presents some stylized facts on the exporting environment of Senegalese FFV producers and on how the PIP addresses its objective of alleviating some of the constraints that these producers face. Sec- tion 3 presents the impact evaluation: data, estimation issues. Section 4 presents the baseline results and some robustness tests. Finally, section 5 concludes. 2 Stylized Facts 2.1 Senegal’s Exports of Fresh Fruits & Vegetables According to Senegal’s National Horticultural Direction, national exports of fresh fruits and vegetables (FFV) have been rising at a rate of about 15% a year since 2001. French beans alone – the main export crop – account for most of this increase, with volumes up from 652 tons to almost 9’000 between 2001 and 2007 (Figure 1). 6 Figure 1: Horticulture Exports (in tons) from Senegal, 1997-2007. Geographical and climatic conditions in the Niayes, Senegal River Valley, Casamance and Dakar regions make it possible to export out-of-season vegetables and trop- ical fruits crops in response to a rising demand foryear-round availability in the European market.6 Apart from minor volumes shipped to neighboring countries, the European Union (EU) remains the main destination market. In 2007, France accounted for 40% of Senegal’s FFV export volumes, followed by the Netherlands (35%) and Belgium (16%). Exports are mostly fresh produce and include French beans (42% of the exported volume), cherry tomatoes (23%), mangoes (16%) and minor crops including melons, peppers and hibiscus. In 2008, Senegal ranked fourth among African suppliers of French beans to the EU, after Morocco, Egypt and Kenya. Exporting companies are organized in two federations, ONAPES and SEPAS.7 Most belong to SEPAS, the oldest of the two, which coordinates transport, provides market information and assists members in dealing with overseas buyers. ONAPES was created by the seven largest exporters in 2000 to coordinate compliance with traceability standards and to seek GlobalGAP certi�cation. To penetrate the EU fruit and vegetable market, Senegal’s exporters must com- ply with strict and rising standards. The EU’s legislation imposes (1) common 6 EU imports of fresh fruit and vegetables have experienced a cumulative growth of 39% between 2002 and 2008, with an average 7% increase in value and 6% in volume per year. 7 e e Organisation Nationale des Producteur- Exportateurs Senegalais. S´n´galaise d’Exportation de Produits Agricoles et de Services. 7 marketing standards for FFV; (2) sanitary and phytosanitary (SPS) measures; (3) general hygiene rules based on HACCP control mechanisms; and (4) traceabil- ity standards. The EU’s SPS measures have become notably more stringent in the 1990s. Particularly relevant to FFV are reduced tolerance levels for chemical residues.8 First, about 350 active substances initially approved for use in the EU have been gradually withdrawn (out of the 823 initially allowed). Second, Maxi- mum Residue Levels (MRL)9 and Import Tolerances (IT)10 are imposed at levels speci�c to particular protection chemical-crop combinations. When an IT has not been established, a default value of 0.01 mg/kg corresponding to Level of Detec- tion (LOD) in inspection labs are used. The registration of an IT is a complicated process involving the submission of a complete residue dossier including �eld trials and lab analysis results. For exporters of minor crops – most tropical crops except bananas– from developing countries, the challenge is compounded by the fact that agrochemical companies have little incentive to provide registration residue data for those crops because the bene�t would not cover the cost. HACCP and tracability requirements came into force with the General Food Law of 2002 (EC R 178/2002). Traceability means that EU food companies must document from (to) whom they are buying (selling) so that products can be traced back to their origin if they prove defective or dangerous. Although traceability is legally limited to a ‘one step forward, one step back’ principle within the EU (with no obligation to keep records in third countries), in practice EU buyers tend to go beyond the strict legal requirement. Complete traceability throughout the chain all the way up to the overseas producers is part of many private standards like the GlobalGAP. Legislative changes in EU standards and their potential detrimental effects on small growers/exporters were one of the primary drivers behind the establishment of the COLEACP’s Pesticides Initiative Program (PIP).11 Maertens and Swinnen (2009) show that the E.U.’s rising standards have pro- foundly altered the structure of the supply chain in Senegal’s horticulture sector. 8 In the late 1990s, an updated harmonised legislation package on pesticide Maximum Residue Limits (MRL) – EC Directive 91/414 and subsequent Regulation 396/2005– created concern for ACP horticultural exporters because of its stringency. 9 The MRL is the level of residue legally permitted to remain in/on a food or animal feedstuff following the use of a Crop-Protection-Chemical (CPC) under Good Agriculture Practice (GAP), i.e. under the speci�c label instructions of the approved product. 10 An import tolerance is an MRL set for imported products containing active plant-protection substances not authorised in the EU for reasons other than public health, or when a different level is appropriate because the existing Community MRL was set for reasons other than public health. 11 The purpose of the Europe-Africa-Caribbean-Paci�c Liaison Committee (COLEACP) is to facilitate the flow of trade in fresh fruit and vegetables between Africa-Caribbean-Paci�c (ACP) countries and the EU. 8 First, the �nancial constraints generated by the need to comply with increas- ingly stringent standards have induced consolidation at the intermediation stage, with only the larger �rms able to cope. Second, the relationship between inter- mediaries and producers has changed, with more control by intermediaries over farming methods. Tighter control has been obtained through increasingly precise contracts, technical assistance, and the provision of credit and farm inputs. Third, the induced changes have also affected the structure of upstream farm produc- tion, with a sharp decrease in the incidence of contract farming and a rise in that of large-scale estate production. Interestingly, Maertens and Swinnen’s analysis suggests that these changes have been accompanied by a rise in the incomes of affected households. 2.2 The PIP in Senegal The Pesticide Initiative Program (PIP) is �nanced by the European Development Fund (EDF) and implemented by the COLEACP. With an overall budget of 34.1 million euros, the PIP’s �rst phase started in 2001, initially for a �ve-year period; it was extended by two additional years. The program has two main objectives. The �rst is to enable ACP exporters of FFV to comply with European traceability and food-safety requirements (in particular as regards pesticide residues). The second is to consolidate the position of small-scale producers in the ACP horticultural value chain. The PIP’s support activities are organised around �ve components: (i) good company practises, (ii) training, (iii) capacity building, (iv) regulation & standards, and (v) information & communication. The core of the support (almost 30% of program budget) goes to component (i), which consists of helping produc- ers and exporters to set up internal food-safety management systems in production and marketing operations. The regulation & standards component ensures that all substances recommended in crop protocols (‘technical itineraries’) are authorised in both the EU and origin country. Additionally, when needed, the program in- troduces registration of active substances as well as import-tolerance applications. This is especially important for minor crops of interest to smallholders. Finally, the capacity-building component aims at developing national capacity to provide the services needed by the industry. Bene�ciaries of capacity-building activities include private consultants (training courses on food safety, pesticide use, and IPM); accredited laboratories (pesticide residue analysis); public services (includ- ing extension services and pesticide registration bodies); and strong professional organizations. New traceability requirements and recent changes in the EU’s pesticides regu- lation have been of particular concern to Senegal’s horticultural industry—mainly the green bean industry. Senegal ranks fourth, after Kenya, Ghana and Uganda, 9 for the number of PIP protocols signed. PIP bene�ciaries produce and export essentially green beans, cherry tomatoes and mangoes to the European market. Their needs relate essentially to traceability systems, staff training, access to in- formation, advice on pesticide MRLs, and infrastructure improvement. Over the project’s life, several missions went to Senegal and met with bene�ciary companies to review and adjust the activities carried out as part of the program. PIP support activities in Senegal have consisted of in-company training on hygiene and food- safety procedures, development of traceability systems, safe use of pesticides, and recognition of mango pests for mango exporters. Four companies also requested support to obtain GlobalGAP certi�cation; for those, pre-audits were conducted to help identify and correct problem areas. SEPAM obtained its certi�cation in 2004; Soleil Vert, Baniang and AgriConcept obtained theirs in 2007. Remaining exporters, mainly smaller ones, are neither certi�ed nor undertaking particular investments to get it. Except for SEPAM, SAFINA and GDS, Senegal’s FFV exporters have outgrower contracts with smallholders (involving training and sup- port) rather than own production sites. In 2006, in cooperation with Senegal’s AN- CAR (Agence Nationale de Conseil Agricole et Rural ) the PIP launched ‘Golden Bean’, an awareness campaign directly targeting 1’000 small FFV producers. Finally, under the regulations component, �eld trials have been conducted on PIP priority crops including green beans, cherry tomatoes, okra, avocado, passion fruit, mango, papaya and pineapple.12 Trials were initiated in November 2003 on green beans and cherry tomatoes, and in early 2004 on the remaining crops, with crop samples collected and shipped to European GLP-certi�ed laboratories for residual analysis. A particular technical itinerary was developped for mangoes with the objective of bringing mango production in line with European regulations. 2.3 Selection into the Program Eligibility starts with the completion and submission of a request for PIP interven- tion addressing the applicant’s particular needs and objectives. The request iden- ti�es by self-assessment the problem to be resolved —e.g. MRL, non-accredited plant-protection products, or traceability— and puts forward possible �xes such as training in ICM/IPM systems or safe use of pesticides, implementation of food- safety and traceability systems, or ‘technical itineraries’. It also speci�es antici- pated results, like getting into conformity or maintaining current export volumes. Finally, the request assesses a time line and budget, specifying what is requested 12 Field trials are an important part of the establishment or amendment of crop protocols, enabling experts to analyze pesticide residual levels in fruit and vegetables. Following �eld trials, crop protocols are revised as required to achieve compliance with EU MRLs, and the information gathered in the process feeds back into farming practices. 10 from the PIP and what is provided by the applicant. Applications are considered on a �rst-come �rst-served basis, with no prioritization or selection criteria. To be accepted, a requested intervention must help to achieve product compliance with EU traceability and food-safety (pesticide residues) regulations. Upon acceptance, a protocol stating the actions to be implemented by each party on a cost-sharing basis (50% for each, except for smallholders who are expected to contribute only 20%) is signed. Wages and investment costs are de facto excluded. The actions listed in the protocol are chosen among a menu offered by the PIP under its �ve components; however, the combination in each protocol is speci�c to a �rm and varies across bene�ciaries. So far, most of the �nancing has gone to training costs, technical support and the development of a food-safety toolbox containing crop protocols, GAP guidelines, and Hortitrace, a traceability software developed by the PIP. The program’s �rst phase has covered 21 countries13 and 320 export companies. Out of those, 219 �rms bene�ted from the ‘good company practices’ component (advice and assistance for setting up sanitary quality and traceabil- ity systems, and certi�cation pre-audits). 153 bene�ted from training under the capacity-building component. 2.4 The PIP’s Evaluation An evaluation of the PIP’s �rst phase was undertaken in June 2008. The ap- pendix’s valuation matrix summarizes the report’s �ndings and details objectives, expected results, performance indicators, and outcomes. Performance indicators are both quantitative and qualitative. The evaluation relies on trade data reported by the �rms themselves and from Eurostat. Additionally, a survey was conducted among PIP bene�ciaries and EU importers. As highligthed in the appendix’s eval- uation matrix, program impacts are evaluated at the aggregate level. Outcomes for bene�ciary �rms are generally reported without controlling for location, size or experience in exporting FFV to the EU market, with the exception of spe- ci�c objective (S1), for which outcomes are reported by ACP country or type of intervention (O1, O2, O3). Overall, the evaluation report drew up a very positive image of the program’s impact, contributing to the launch of a second �ve-year phase in 2009. While fairly comprehensive, the PIP’s evaluation suffers from a typical drawback of this type of exercise—namely, the lack of a counterfactual to benchmark the performance of treated �rms and products. 13 The countries covered are Benin, Burkina Faso, Cameroon, the Ivory Coast, Gambia, Ghana, Guinea, Mauritius, Jamaica, Kenya, Mali, Mozambique, Namibia, Uganda, the Dominican Re- public, Senegal, Surinam, Tanzania, Togo, Zambia, and Zimbabwe. 11 3 Impact Evaluation 3.1 The Data Our dataset is constructed using three primary databases which together form a rich and unique combination. First, we have export data at the transaction level (aggregated to the �rm level) from raw customs �les over 2000–2008. Each record includes the �rm’s tax ID, the product code, the country of destination, and the export value (in US dollars) and quantity (in tons) for over 500 HS8 products to 90 countries. Exports flows are reported annually. Second, the PIP’s administration in Brussels provided us with a list of the Senegalese �rms that got assistance from the program in each year of the sample period.14 Finally, we obtained data on employment and sales from the CNI, Senegal’s National Statistics Direction.15 As the CNI also identi�es �rms by their tax ID, we could merge the three datasets. Among the reporting �rms, almost 3% appear only once in the dataset. That is, they export only one product to one destination one year. As these observations are likely to be mis-reports or individuals, we drop them from our sample. We also drop international organizations and embassies, as well as trading and transport companies (the latter group represents about a quarter of all observations). This leaves us with a sample of almost 2’000 observations. Let i an exporting �rm, k be a product, j a destination country, and t a year. As the PIP targets products (FFV), some of a �rm’s products may be covered and some others not. In addition, technical assitance provided under the PIP helps make FFV marketable on EU markets, but does not necessarily help on other markets with different (or no) food-safety requirements. In view of this, we take the (i, k, j, t) vector as our Primary Sample Unit (PSU). Our dependent variable is the annual flow of exports of product k to destination j by �rm i in year t, yikjt . That is, we take the intensive margin as our baseline measure of performance. Table 1 gives descriptive statistics of �rm-level covariates and per- formance indicators for treated and non-treated flows. Covariates include �rm i’s annual (overall) turnover, salesit , and employment, N employeeit , the number of products it exports. We also include N prodit , the number of products it exports to destination j N prodijt , the number of destinations to which it exports product k, N destikt and a binary variable equal to one if �rm i has more than one year of experience in exporting product k to destination j, expikj are included. Perfor- mance indicators at the intensive margin include �rm i’s exports of product k to 14 Variation in the intensity of the treatment across �rms would induce a bias in the estimation of the effect. Unfortunately, we do not have data on the intensity of the treatment. 15 The data is collected by the Centre National d’Identi�cation (CNI), part of the National Statistics Direction. 12 Table 1: Descriptive Statistics, non-Treated and Treated Flows. Non-Treated Flows Treated Flows Mean Mean t-Stat p t Sample 1370 492 Firm characteristics salesit 21.7 21.1 4.9 *** Nemployeeit 4.4 5.3 -6.9 *** Export performace: intensive margin exportikjt 8 10.1 -17.6 *** total exportF F V ijt 8.2 11.9 -22.8 *** total exportikt 8.5 11.1 -20.9 *** total exportF F V it 8.6 13.21 -34.9 *** total exportit 12.4 13.1 -7.7 *** Export performace: extensive margin Ndestikt 1.5 2.2 -11.5 *** NdestF F V it 1.3 8 -19.5 *** Ndestit 5.08 3.4 12.3 *** Nprodijt 14.9 3.2 20 *** NprodF F V ijt 0.6 2.7 -20.1 *** NprodF F V it 1.3 8 -19.5 *** Nprodit 38.9 9.3 231 *** Variables are in logs. *, **, and *** denote statistical signi�cance of the t-statistics at the 10%, 5%, and 1% levels, respectively. destination j in year t, exportijkt ; its total exports of FFV to country j in year t, total exportF F V ; its total exports of product k worldwide, total exportikt ; its total ijt exports of FFV worldwide, total exportsitF F V ; and its total exports worldwide, total exportit . Export performance indicators at the extensive margin include �rm i’s number of destinations with product k in year t, N destikt ; its number of desti- nations with FFV, N destF F V ; its total number of destinations, N destit ; its number it of products to destination j, N prodijt ; its number of FFV products to destination j, N prodF F V ; its number of FFV products wordwide, N prodF F V , and its total ict it number of products wordwide, N prodit . Table 3.1 shows descriptive statistics for participating and non-participating �rms. Two observations are in point. First, participating �rms are larger than non-participating. Thus, size must be controlled for in order for non-participating �rms to provide a credible counterfactual. Second, these are all small �rms, as 13 Table 2: Descriptive Statistics, non-Treated and Treated Firms. Non-treated �rms Treated �rms Mean Mean t-Stat p t Sample 131 100 Firm characteristics salesit 19.5 20.3 -2.0 ** Nemployeeit 2.9 4.7 -4.7 *** Export performace: intensive margin total exporteu it 10.1 11.9 -6.7 *** ffv total exportit 10.0 12.0 -6.8 *** total exportit 10.7 11.9 -4.7 *** Export performace: extensive margin ffv Ndestit 1.6 4.2 -5.9 *** Ndestit 2.4 2.5 -0.3 ffv Nprodit 1.6 4.2 -5.9 *** Nprodit 5.1 4.8 0.4 Variables are detailed in Table 1. *, **, and *** denote statistical signi�cance of the t statistics at the 10%, 5%, and 1% levels, respectively. average numbers of products and destinations are very small. 3.2 Estimation Issues Estimating the effect of the PIP poses a standard missing-data problem—estimating how much smaller would have been the export flows that got assistance, had they not gotten assistance.16 Formally, let 1 if (i, k, j, t) is treated at t dikjt = 0 otherwise and 1 if ∃ : t such that dikjt = 1 dikj = 0 otherwise. That is, dikj marks the treatment group. The basic estimator for the problem at hand is the difference-in-differences (DID): 16 Exports to non-EU destinations may be reduced because of a reallocation effect. If this is the case, the effect of the treatment on the treated would be further increased. 14 yikjt = xikjt β + γdikjt + δikj + δt + uikjt (1) where δikj and δt are respectively �rm × destination × product and time �xed effects, and uikjt is an error term. Fixed effects δikj control for time-invariant �rm characteristics potentially affecting both performance and selection into the program, like managerial ability (see Angrist and Krueger 1999, Smith 2000, or Jaffee 2002). Next, we combine the DID estimator (1) with matching, following Heckman, Ichimura and Todd (1997). From now on, for simplicity of exposition, let us denote by a ‘flow’ a (�rm × product × destination) triplet. Using results by Rosenbaum and Rubin (1983), matching is done on the basis of the estimated propensity score (PS), using a probit or logit regression of the participation status on a vector z of observable �rm characteristics. Letting vikjt be an error term orthogonal to uikjt , the �rst-stage selection equation can be written as Pr(dikj = 1) = f (zikj0 α + vikj ). (2) In (2), the vector z, which contains x and may be identical to it if no outside determinant of participation is available, must be evaluated at the time the par- ticipation decision is made–typically its initial value. The estimated propensity score is then retrieved from (2), and the control group is constructed by select- ing untreated �rms whose propensity scores are “close enough� to those of treated ones. How close is close is the analyst’s choice. Under nearest-neighbour matching, each treated flow is matched with the untreated flow having the closest PS, or with a combination of the n closest untreated flows. Under kernel matching, flow g is matched with a weighted average of untreated flows within a chosen radius, using either uniform weights or weights that decrease with distance in the PS space.17 Practically, DID-with-matching estimation is done in two steps: In the �rst, the participation equation is estimated, yielding an estimated PS and a common sup- port; in the second, the DID equation is estimated on the common support. The latter is formed by disregarding unmatched individuals as well as those with esti- mated PS of zero or one. Note that, if the �rst-stage regression predicts treatment- group status too well, it will reduce the common support and thus the sample on which the DID regression is run; if it predicts it too badly, the matching will be poor and the conditional-independence assumption will be unlikely to hold. The choice of RHS variables to include in z is thus a matter of judgement. The quality of the matching can be assessed by so-called ‘balancing tests’. One, proposed by Rosenbaum and Rubin (1985), compares the mean value of each covariate between 17 On this, see Leuven and Sianesi (2003) or Smith and Todd (2005). 15 the treatment and control group. When the difference is too large, the null hy- pothesis (that the samples are balanced with respect to the covariates when they are balanced with respect to the propensity score) is rejected. We also control for selection using a control-function approach that closely resembles the Heckit procedure (Heckman 1979). The approach proceeds, again, in two steps. The �rst-step regression is as before, i.e. (2). In the second step, inverse Mills ratios retrieved from the �rst step are added to (1) as additional regressors. Before we turn to the results, note that, besides selection bias, other issues may complicate the estimation of γ in (1). One is serial correlation. Persistence in the process driving the error term may be aggravated by the extreme form of serial correlation in the treatment variable. Bertrand, Duflo and Mulainathan (2004) show that ignoring this source of serial correlation can lead to an inflated probability of type-I errors (wrongly rejecting the null hypothesis of no effect, i.e. being over-optimistic in the evaluation of the treatment’s impact). This calls for a correction that they suggest. The correction consists of a two-step procedure in which, in Step 1, individual performance for both treated and untreated individuals is regressed on all observables except the treatment. In Step two, residuals from Step 1 for the treated individuals only are retrieved and averaged for (i) the pre- treatment period, (ii) the treatment period. The procedure requires a common treatment period, since otherwise the pre- and post-treatment periods would be unde�ned for the control group. Those average residuals form a two-period panel. They are then regressed on 1 if τ = 1 and (i, j, k) ∈ T dikjτ = 0 otherwise, where τ = 1 denotes the treatment period. The estimated ATT is the coefficient on dikjτ in the second step. 4 Results 4.1 Baseline Results Balancing properties are addressed by testing for equality of means between treated and matched controls for nearest-neighbour matching. Table 4.1 reports results from balancing tests. The table reports, for each covariate included in the probit model determining selection into treatment, the percentage bias after matching, the reduction in the bias, and the t-test statistics for the difference in means between treated and control groups after matching. Variables included in the 16 propensity score speci�cation are the natural logarithm of �rm i’s initial turnover, ln(salesit0 , the natural logarithm of its initial number of employees, ln(N Employeesit0 ), the initial number of products exported by �rm i to destination j, N prodij0 , the initial number of countries served with product k, N destikt , the initial natural logarithm of total export value of FFV products from �rm i to destination j. Table 3: Balancing Properties of Covariates in Treated and Control Groups Sample Mean Mean % % t-test treated control bias between reduction Mean(treated) �flows� �flows� treated bias =Mean(control) and controls t p¿t ln(salesit0 ) Unmatched 21.39 21.25 11.5 1.3 0.19 Matched 21.39 21.56 -13.6 -18.6 -1.72 0.08 ln(Nemployeesit0 ) Unmatched 5.02 4.18 55.2 7.05 0.00 Matched 5.02 4.94 4.7 91.6 0.44 0.66 Nprodij0 Unmatched 2.38 32.03 -144.2 -14.77 0.00 Matched 2.38 2.37 0.1 100 0.16 0.87 Ndestikt Unmatched 2.41 1.37 114.5 15.1 0.00 Matched 2.41 2.3 9.7 91.6 0.84 0.40 ln(total exportF F V ) ijt0 Unmatched 11.93 7.51 154 16.63 0.00 Matched 11.93 12.13 -7 95.4 -1.21 0.23 Matching is by nearest neighbour. *, **, and *** denote statistical signi�cance of the t statistics at the 10%, 5%, and 1% levels, respectively. Results show that, for many covariates, there is a strong bias before matching but matching eliminates it. The null hypothesis of balanced sub-samples is not rejected except for turnover. Table 4 reports difference-in-difference (DID) estimates on the treated, for our baseline speci�cation. That is, the average effect of the PIP on assisted �rms where the export performance indicator is the export value in US dollars, of prod- uct k, from �rm i, to destination country j, in year t. Column (1-3) reports DID estimates without matching, and with or without covariates. Column (4) reports 17 DID estimates with matching, i.e. restricting the sample to the common sup- port de�ned in the NN-PSM procedure. Matching is done at the �rm, product destination level. Column (5) reports treatment-effect estimates using Heckman’s two-step procedure, i.e. estimates from the second-step regression run with the inverse Mills ratio. Finally, column (6) reports results from the second stage of the procedure suggested by Bertrand, Duflo and Mulainathan (BDM) (2004). Results for the �rst step of the Heckman and BDM procedures are reported in Appendix 9 and 10. All regressions are run at the i, k, j, t level and standard errors are clustered at the �rm level. Here the variable of interest is the treatment indicator variable treatmentikjt taking value one if �rm i exporting product k to destination country j in year t, bene�ted from the PIP program. Coefficients in all speci�cations are not signi�cant, suggesting no effect of the program on �rms export performance. The dependent variable is the natural logarithm of the export value of product k from �rm i to country j in time t. All regressions control for (�rm times product times country) and time �xed effects. Our main variable of interest is treatmentikjt , a dummy variable taking value 1 if �rm i exporting product k to country j bene�ted from the PIP in year t. Columns 1-3 show difference-in-difference estimations, where the control variables include the natural logarithm of annual turnover of �rm i in year t ln(salesit , the natural logarithm of the number of employees for �rm i in year t, ln(N employeesit ), �rm i’s experience in servicing product k to country j, and a dummy variable taking value 1 if the �rm exported at least two years product k to country j before time t, experienceikjt . Column 4 shows matching difference-in-difference estimation where results are reported using the Nearest Neighbourg (NN) estimator with caliper (0.04). Column 5 shows two-stage Heckman estimation where λ is the inverse Mill’s ratio retrieved from the �rst step. The �rst-step is a regression of the participation status on a vector of observable �rm characteristics (see Appendix 9). Column 6 shows two stage BDM estimation where residuals from the �rst step are retrieved and averaged for (i) the pre-treatment period, (ii) the treatment period. Step 1, individual performance for both treated and untreated individuals is regressed on all observables except the treatment (see Appendix 10). 18 Table 4: Baseline Results, Average Effect of the PIP on Assisted Firms. (1) (2) (3) (4) (5) (6) Diff Diff Diff Diff in Diff Two stage BDM in in in with Heckman Correction Diff Diff Diff Matching treatmentikjt 0.354 0.216 0.112 -0.013 0.238 0.111 (0.235) (0.288) (0.322) (0.418) (0.362) (0.293) ln salesit 1.062*** 1.125*** 1.246*** 1.263*** (0.212) (0.262) (0.243) (0.246) ln Nemployeeit -0.099 0.017 0.009 (0.155) (0.152) (0.148) experienceikjt 0.183* -0.047 0.018 -0.143 -0.145 (0.104) (0.168) (0.172) (0.169) (0.169) λ -0.153 (0.153) constant 7.94*** -14.59*** -15.94*** -18.98*** -19.39*** -0.073 (0.225) (4.552) (5.422) (5.128) (5.246) (0.166) Observations 1,862 1,193 1,071 698 698 176 R-squared 0.132 0.179 0.188 0.207 0.207 0.007 Number of id 1,134 657 577 369 369 155 Robust standard errors clustered at �rm level are in parentheses. *, **, and *** denote statistical signi�cance at the 10%, 5%, and 1% levels, respectively. 4.2 Robustness In this section we present estimation results of the effect of the PIP on assited �rms, for two alternative export performance indicators. Table 5 reports DID estimates when considering �rm i export of product k to the EU-15 in year t, as the export performance indicator. Balancing tests results are provided in Appendix 7. There is no problem of unbalanced covariates in our model. All regressions are run at the i, k, t level. The coefficients on the treatment variable are not signi�cant for any of the speci�cations in Table 5. Results for the �rst step of the Heckman and BDM procedures are reported in Appendix 9 and 19 10. Finally, Table 6 reports results from regressions run at the level of the �rm instead of the �rm × product × destination combination. Estimation at the �rm level may drastically reduces sample size and means mixing up exports that are covered by the program with exports that are not (namely, products other than FFV). However, it is advisable, since the decision to participate and some of the covariates are at the �rm level. The export performance indicator is the export value of FFV to the EU-15 exported by �rm i in year t. All regressions are run at the i, t level. The coefficient on the treatment variable is signi�cant at the 5% level only in column (1). Results for the �rst step of the Heckman and BDM procedures are reported in Appendix 9 and 10. All in all, results suggest that while there seem to be an effect of the program when considering total FFV exports to the EU, the effect disappears when looking at a more disaggregated level (Table 4 and 5). These results are in line with �ndings in the program evaluation report. In Table 5, the dependent variable is the natural logarithm of the export value of product k from �rm i to the EU-15 at time t. All regressions control for �rm x product and time �xed effects. 20 Table 5: Robustness I, Average Effect of the PIP on Assisted Firms. (1) (2) (3) (4) (5) (6) Diff Diff Diff Diff in Diff Two stage BDM in in in with Heckman Correction Diff Diff Diff Matching treatmentikt 0.100 -0.023 -0.115 -0.114 -0.669 -0.410 (0.215) (0.264) (0.305) (0.327) (0.813) (0.564) ln salesit 0.909*** 0.923*** 0.948*** 0.890*** (0.217) (0.287) (0.293) (0.290) ln Nemployeeit 0.063 0.074 0.078 (0.105) (0.107) (0.103) experienceikjt 0.299 -0.252 -0.228 -0.253 -0.213 (0.248) (0.373) (0.473) (0.504) (0.485) λ 0.333 (0.357) constant 8.95*** -9.36 -10.01 -18.98*** -8.54 0.17 (1.008) (5.834) (5.975) (5.128) (6.054) (0.311) Observations 681 368 288 281 286 93 R-squared 0.153 0.211 0.227 0.218 0.230 0.036 Number of id 373 189 142 139 140 78 Robust standard errors clustered at �rm level are in parentheses. *, **, and *** denote statistical signi�cance at the 10%, 5%, and 1% levels, respectively. 21 Table 6: Robustness II, Average Effect of the PIP on Assisted Firms. (1) (2) (3) (4) (5) (6) Diff Diff Diff Diff in Diff Two stage BDM in in in with Heckman Correction Diff Diff Diff Matching treatmentit 0.533** 0.381 0.666 0.666 0.711 0.477 (0.239) (0.128) (0.421) (0.426) (2.898) (0.292) ln salesit 0.529 0.496 0.520 0.518 (0.372) (0.532) (0.538) (0.626) ln Nemployeeit -0.181 -0.188 -0.187 (0.262) (0.266) (0.265) experienceit -0.007 -0.007 -0.218 -0.237 -0.236 (0.233) (0.233) (0.460) (0.468) (0.530) λ -0.026 (1.570) constant 10.07*** 0.855 2.193 1.992 2.017 -0.278 (0.704) (7.943) (11.724) (11.795) (12.850) (0.214) Observations 199 119 85 79 79 16 R-squared 0.301 0.380 0.354 0.362 0.362 0.308 Number of id 69 38 26 21 21 9 Robust standard errors clustered at �rm level are in parentheses. *, **, and *** denote statistical signi�cance at the 10%, 5%, and 1% levels, respectively. 5 Concluding Remarks By and large, we �nd no signi�cant impact of the PIP on Senegal’s FFV export flows when taking similar, untreated export flows as the counterfactual. There are two ways of interpreting such a no-impact result. The naive interpretation is that the PIP simply fails to achieve its objective. That may well be true, but our failure to reject the null of no impact is not suffi- cient to reach that conclusion. First, as we briefly discussed in the introduction, the choice of Senegal as a testing ground was driven by data availability (Senegal 22 was the only country for which we had data for the pre-treatment period). It has no claim to be a representative or random sample. Different conclusions may be reached from other samples, and clearly a full, cross-country impact evalua- tion should be undertaken. Second and more importantly, it is possible that the PIP affected not only the treated export flows, but also untreated ones, through spillovers. Participating �rms are the largest and more efficient of Senegal’s FFV sector. On one hand. this means that they are more susceptible than others to bene�t from the program. Thus, what we obtain is an estimate of the average effect of the treatment on the treated (ATT), which may over-state the program’s potential effect on the whole population of producers. On the other hand, it may also mean that smaller �rms, although left out of the program, can bene�t from it through imitation of best practices and even unobserved assistance from larger �rms. The argument is even more potent for untreated products: when a �rm gets PIP assistance for its FFV activities, it is quite possible (indeed, likely) that its other activities bene�t as well from improved managerial practices; or destination countries: export flows to non-EU destinations may bene�t from the program for the same reason. In the presence of such unmeasured spillovers, the PIP’s im- pact would be underestimated by impact-evaluation methods. This is important to keep in mind, as public assistance (whether from local governments or donors) should be justi�ed by market failures, like spillovers, rather than a positive rate of return to bene�ciaries (which would simply create a market demand for assis- tance services without justifying use of public funds). Thus, impact evaluation of technical-assistance programs like the PIP is a double-edged sword and must be interpreted with caution. The second conclusion that should be avoided is that, either because the data are not sufficiently reliable or comprehensive or because of the caveats just dis- cussed, rigorous impact evaluation should not be undertaken. The lack of rigorous impact evaluation undermines the credibility of claims about the program’s ben- e�ts made on the basis of case studies, because it is impossible to know whether they are representative or not. Indeed, this is the message conveyed by the 2006 assessment of the impact of EU TRA (te Velde et al. 2006). Worse, in a context where taxpayers are asking for acccountability and results in development aid, Paul Milgrom’s ‘unraveling principle’ applies: Rational taxpayers are likely to take all the news that is not told to be detrimental. In other words, the bad news that impact evaluation can possibly generate (as in the present case) is probably fully anticipated. The more the development community can provide rigorous evidence that at least some programs do make a difference; or that some components do; or that, when not, failure is part of useful experimentation and action is being taken to remedy the observed ineffectiveness, the more support there will be for development aid. 23 However, as the present study highlights, it is difficult to ‘improvise’ impact evaluation ex-post when a program was not designed to be evaluated. Far bet- ter would be to think seriously about evaluation ex-ante, so that TRA programs generate experimental settings out of which useful lessons could be drawn. We hope that this study will help convince the European Commission and other de- velopment agencies of the need to plan for impact evaluation at program-design time. By this, we mean to (i) clarify what measurable performance indicator the program seeks to improve; (ii) collect, before, during and after the program pe- riod, the data needed to track this indicator for treated and non-treated �rms and products, as well as its non-program determinants; (iii) amend the design of the program (in particular the assignment rule) to ensure the existence of a proper control group against which to benchmark its impact. 24 References [1] Abadie, A. (2005), Semiparametric Difference-in-Difference Estimators; Re- view of Economic Studies Wiley Blackwell, vol. 72(1), pages 1-19, 01. [2] Angrist, Joshua, and A. Krueger (1999), Empirical Strategies inLabor Eco- nomics; in O. Ashenfelter and D. Card, eds., Handbook of Labor Economics; Elsevier. [3] Bertrand, Marianne; E. Duflo and S. Mulainathan (2004), How Much Should We Trust Difference-in-Difference Estimates? The Quarterly Journal of Eco- nomics, MIT Press, vol. 119(1), pages 249-275. [4] Blundell, Robert, and M. Costa Dias (2002), Alternative Approaches to Eval- uation in Empirical Macroeconomics; CEMMAP Working Paper CWP10/02. c [5] Bourguignon, Fran¸ois, and M. Sundberg (2007), Aid Effectiveness—Opening the Black Box; American Economic Review Papers & Proceedings 97,316-321. [6] Brenton, Paul, and E. von Uexkull (2009), Product-Speci�c Technical Assis- tance for Exports—Has it Been Effective?; Journal of International Trade & Economic Development 18, 235-254. [7] Cali, Massimiliano, and D.W. te Velde (2009), Does Aid for Trade Really Improve Trade Performance?, mimeo. [8] EC (2006), Making trade work for development. Trade-related assistance: An update, DG Trade. [9] EC (2008), Making trade work for development. Aid for Trade: A selection of case studies from around the world, DG Trade. [10] Gamberoni, Elisa, and R. Newfarmer (2009), Aid for Trade: Matching Po- tential Demand and Supply; Background paper for the WTO Expert Group Meeting, September 2008; mimeo, The World Bank. [11] Heckman, James (1979), Sample Selection Bias as a Speci�cation Error; Econometrica 47, 153-211. [12] —, H. Ichimura, and P. Todd (1997), Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program; Review of Economic Studies 64, 605-654. [13] —, H. Ichimura, J. Smith and P. Todd (1998), Characterising Selection Bias using Eperimental Data; Econometrica 66, 1017-1098. 25 [14] Jaffee, Adam (2002), Building Program Evaluation into the Design of public Research Suppport Programs; Oxford Review of Economic Policy, Oxford University Press, vol. 18(1), pages 22-34, Spring. [15] Leuven, Edwin, and B. Sianesi (2003), PSMATCH2: Stata mudole to perform full Mahalanobis and propensity score matching, common-support graphing, and covariate imbalance testing; Department of Economics, Boston College. [16] Marcano, Luis, and I. Ruprah (2009), Does Technical Assistance Matter? An Impact Evaluation Approach to Estimate its Value Added; Office of Evalua- tion and Oversight, Inter-American Development Bank. [17] Maertens, Miet, and J. Swinnen (2009), Trade, Standards, and Poverty: Ev- idence from Senegal; World Development 37(1), 161-178. [18] Nelson, Douglas, and S. Juhasz Silva (2008), Does Aid Cause Trade? Evidence from an Asymmetric Gravity Model; University of Nottingham Research Pa- per 2008/21. [19] OECD (2010a), How to Evaluate Aid for Trade: Approaches, Methodologies, and Processes; OECD, COM/DCD/TAD(2010)2. [20] OECD (2010b), Aid for Trade flows: 2008; OECD, COM/DCD/TAD/RD(2010)4/RD1. [21] Osei, Robert, O. Morrissey and T. Lloyd (2004), The Nature of Aid and Trade Relationships; European Journal of Development Research 16, 354-374. [22] Rajan, Raghuram, and A. Subramanian (2008), Aid and Growth: The Cur- rent Debate and Some new Evidence; in P. Isard et al., eds., The macroe- conomic management of foreign aid: Opportunities and Pitfalls; IMF, pp. 43-60. [23] Rosenbaum, Peter, and D. Rubin (1983), The Central Role of the Propensity Score in Observational Studies for Causal Effects; Biometrika 70, 41-55. [24] Smith, Jeffrey (2000), A Critical Survey of Empirical Methods for Evaluat- ing Active Labor Market Policies; Swiss Journal of Economics and Statistics (SJES), vol. 136(III), pages 247-268. [25] Smith Jeffrey, and P. Todd (2005), Does Matching Overcome Lalonde’s Cri- tique of Nonexperimental Estimators?, Journal of Econometrics 125(1-2), 305- 353. 26 [26] Te Velde, Dirk Willem (2006), A Critical Assessment of the EU’s Trade- related Assistance to Third Countries; Lessons from the Past, Policy Options for the Future. Overseas Development Institute, Study for the International Trade Committee of the European Parliament. [27] Volpe, Christian, and J. Carballo (2008), Is Export Promotion Effective in Developing Countries? Firm-level Evidence on the Intensive and Extensive Margins of Exports; Journal of International Economics 76, 89-106. [28] Wagner, Don (2003), Aid and Trade—An Empirical Study; Journal of the Japanese and International Economies 17, 153-173. [29] Wooldridge, Jeffrey (2005), Instrumental Variables Estimation of the Aver- age Treatment Effect in the Correlated Random Coefficient Model; mimeo, Michigan State. 27 6 Appendix Table 7: Balancing Test Results for Table 5 The table reports, for each covariate included in the probit model determining selection into treatment, the percentage bias after matching, the reduction in the bias, and the t-test statistics for the difference in means between treated and control groups after matching. Variables included in the propensity score speci�cation are: initial natural logarithm of �rm i turnover (salesit0 ), initial natural logarithm of number of employees in �rm i (N Employeesit0 ), initial number of products exported by �rm i to destination to the EU (N Prodeu ), initial number of countries j served with product k (N Destikt0 ), it0 initial natural logarithm of total export value of product k from �rm i to the EU(total exporteu 0 ). *, **, ikt and *** denote statistical signi�cance of the t statistics at the 10%, 5%, and 1% levels, respectively. Sample Mean Mean % % t-test treated control bias between reduction Mean(treated) �flows� �flows� treated bias =Mean(control) and controls t p¿t ln(salesit0 ) Unmatched 21.44 20.34 74.3 5.83 0 Matched 21.46 21.53 -5 93.2 -0.5 0.618 ln(Nemployeesit0 ) Unmatched 4.87 3.68 69.2 5.73 0 Matched 4.83 4.93 -6 91.3 -0.43 0.666 Nprodeu it0 Unmatched 6.58 4.26 103.8 8.21 0 Matched 6.56 6.51 2.1 98 0.2 0.845 Ndestikt0 Unmatched 2.02 1.35 78 6.79 0 Matched 1.97 1.95 2.2 97.2 0.14 0.892 ln(total exporteu 0 ) ikt Unmatched 11.12 9.74 58.2 4.81 0 Matched 11 10.80 8.7 85.1 0.6 0.548 28 Table 8: Balancing Tests Results for Table 6 The table reports, for each covariate included in the probit model determining selection into treatment, the percentage bias after matching, the reduction in the bias, and the t-test statistics for the difference in means between treated and control groups after matching. Variables included in the propensity score speci�cation are: initial natural logarithm of �rm i turnover (salesit0 ), initial natural logarithm of number of employees in �rm i (N Employeesit0 ), initial number of FFV products exported by �rm ffv ffv i to the EU (N Prodieut0 ), initial number of countries served with FFV product (N Destikt0 ), initial ffv natural logarithm of total export value of FFV product from �rm i to the EU (total exportieut0 ). *, **, and *** denote statistical signi�cance of the t statistics at the 10%, 5%, and 1% levels, respectively. Sample Mean Mean % % t-test treated control bias between reduction Mean(treated) �flows� �flows� treated bias =Mean(control) and controls t p¿t ln(salesit0 ) Unmatched 21.49 20.02 91.9 3.96 0 Matched 21.49 21.55 -3.6 96.1 -0.17 0.869 ln(Nemployeesit0 ) Unmatched 4.75 3.28 86.6 3.82 0 Matched 4.75 4.45 17.9 79.4 0.68 0.5 ffv Nprodit0 Unmatched 5 3.1957 83.9 3.7 0 Matched 5 5 0 100 0 1 ffv Ndestit0 Unmatched 5 3.1957 83.9 3.7 0 Matched 5 5 0 100 0 1 ffv ln(total exportieut0 ) Unmatched 12.88 11.67 69.6 2.98 0.004 Matched 12.88 12.73 8.9 87.2 0.37 0.711 29 Table 9: First Stage Estimation: Probability of Selection into the Treatement In column (1) the dependent variable is the probability of treatement: for a �rm i product k and destina- tion j (treatementikjt ). In column (2) the dependent variable is the probability of treatement for a �rm i product k (treatementikt ). And in column (3) the dependent variable is the probability of treatement for and for a �rm i (treatementit ). Control variables are detailled in Appendix 7 and 8. *, **, and *** denote statistical signi�cance at the 10%, 5%, and 1% levels, respectively. Two stage BDM estimation (column 6): residuals from the �rst step are retrieved and averaged for (i) the pre-treatment period, (ii) the treatment period. Step 1, individual performance for both treated and untreated individuals is regressed on all observables except the treatment (see Appendix 10). (1) (2) (3) treatment ikc treatment ik treatment i 1st Step Probit 1st Step Probit 1st Step Probit ln(salesit0 ) 0.122** ln(salesit0 ) 0.157** ln(salesit0 ) 0.165 (0.059) (0.080) (0.142) ln(Nemployeesit0 ) 0.086* ln(Nemployeesit0 ) 0.107 ln(Nemployeesit0 ) 0.109 (0.048) (0.066) (0.137) ffv Ndestikt0 0.432*** Ndestikt0 0.356*** Ndestit0 0.332 (0.069) (0.122) (0.214) ffv Nprodijt0 -0.039*** Nprodeu it0 0.178*** Nprodit0 -0.029 (0.014) (0.036) (0.080) ffv ffv ln(total exportijt0 ) 0.113*** ln(total exporteu 0 ) ikt 0.029 ln(total exportieut0 ) 0.035 (0.034) (0.041) (0.113) Constant -5.076*** Constant -5.899*** Constant -5.339** (1.164) (1.487) (2.613) Observations 698 286 79 30 Table 10: First Stage BDM Procedure Step 1 of the two stage BDM estimation: individual performance for both treated and untreated in- dividuals is regressed on all observables except the treatment. In column (1) the dependent variable is the natural logarithm of the export value of product k from �rm i to country j in time t. In column (2) The dependent variable is the natural logarithm of the export value of product k from �rm i to the EU in time t. In column (3) the dependent variable is the natural logarithm of the export value of FFV product from �rm i in time t. Control variables are detailled in Appendix 7 and 8. *, **, and *** denote statistical signi�cance at the 10%, 5%, and 1% levels, respectively. (1) (2) (3) Yijkt Yikt Yit 1st Step BDM 1st Step BDM 1st Step BDM ln(salesit0 ) 1.122*** ln(salesit0 ) 0.934** ln(salesit0 ) 0.449 (0.389) (0.412) (0.676) ln(Nemployeesit0 ) -0.098 ln(Nemployeesit0 ) 0.047 ln(Nemployeesit0 ) -0.126 (0.233) (0.181) (0.329) experienceikjt 0.030 experienceikt -0.259 experienceit -0.110 (0.237) (0.584) (0.613) Constant -15.86* Constant -9.495 2.888 (8.052) (8.281) (15.049) Observations 1,071 288 85 R-squared 0.921 0.920 0.913 31 Figure 2: The PIP’s evaluation matrix Program Conclusion of the component Objective or Expected results Performance indicator Outcome report Global ACP countries maintain their share (in value terms) in EU imports of Indicator G1: The share in EU Extra-EU imports of FFV O1: Share of ACP exporters in EU imports value of FFV increased from 6.1% (371'116 Achieved fresh fruits and vegetables. remains stable at 7% in value. thousands euros) in 2001 to 6.2% (534'681 thousands euros) in 2006.: Share with regards to volumes decreased from 7% (335'647 tons) to 6% (384'437 tons). Specifique ACP FFV suppliers meet european Maximum Residues Limit (MRL) Indicator S1: Exporters with established internal systems of O1: The 219 firms that signed a protocle with PIP account for 74% of ACP FFV exports to the Achieved and traceability regulations requirements. food safety management in the production and marketing EU in 2006 (276'000 tons*). process account for 80% of exports of FFV to the EU by the end of the program. O2: The 145 firms that set up a traceability system account for 60% of ACP FFV exports to the EU in 2006 (229'084 tons*). O3: The 191 firms that benefited from staff training in food safety procedures account for 70% of ACP FFV exports to the EU in 2006 (265'395 tons*). Out of these, 56 obtained certification for their food safety management system, and account for 37% of exports in 2006 (142'233 32 Indicator S2: Thoses exporters suffer less shipments tons*). NA. Data on rejections and removals (once the product entered in the EU market) was not NA rejections due to food safety issues on the EU market. available. Indicator S3: Improvement in the degree of satisfaction of O1: 47% of importers (34 respondents) believe the impact of PIP was determinant in bringing Achieved european importers regarding the level of conformity of ACP ACP exports in conformity with regulations requirements. exports with public and private standards requirements. O2: 15% believe its was not sufficient. O3: With regards to private standards requirements 35% believe PIP intervention was determinant. While 21% estimate it as not sufficient. O4: 55% fear that the end of the program will negatively impact the quality of products. Information and Producers and exporters in ACP countries are informed in time of Indicator R1: Is the information system** effective and O1: Among the 176 beneficiaries that answered, 80% are satisfied with the Achieved communication destination markets requirements with a focus on MRL and Import efficient in providing the needed information to both importers information/communication toolkit developped by the program. Tolerances (IT). Therfore, they can adapt their production and and exporters. O2: Among inporters (34 respondents), 59% know about the PIP, and 50% are aware that their production pratices to changes in importers requirements. suppliers benefited from the PIP. Importers are informed of exporters compliance efforts. Regulation and Crop protocols (technical itineraries) are drawn up by the PIP for the Indicator R2: Products targeted by the component account O1: Products for which technical itineraries were elaborated account for 91.4% of ACP FFV Achieved standards main crops exported to the EU market. Applications for Tolerances for 90% of export flows by the end of the project exports to the EU in 2006 (384'437 thousands euros). Import are submited for combinations of product and active substance for which the pesticide use led to residue levels above the relevant EC MRL . Good company Producers adopt Good Agricultural and Production Practices (GAP Indicator R4: Indicator S1 See above Achieved practices and GPP). Exporters set up contol and traceability systems to insure food safety. Export flows from small scale producers are maintained (in value Indicator R5: The decrease in small scale producers export NA NA terms) as must as possible. values of FFV to the EU is bounded to 20%. Capacity building ACP actors of the FFV industry participate in at least ten national task Indicator R6: At least ten task forces are created and are NA NA forces. These task forces act as coordination platforms between the financially viable by the end of the program. producers/exporters, the private professional organisations, the relevant public services and the authorities in charge of pesticide controls. ACP exporters and professional organisations in the FFV industry Indicator R7: An european network is established NA NA develop an european network of technical structures which serve as Notes relays on the ground. These relay structures organise training sessions EU is the EU-15 EU imports correspond to extra -EU imports. FFV = Fresh fruits and vegetables excluding bananas and citrus fruits that are not covered by PIP * Based on firms' own declarations. ** Include website, technical documents, newsletter INFOPIP, technical itineraries.