WPS5437 Policy Research Working Paper 5437 Research for Development A World Bank Perspective on Future Directions for Research The World Bank Development Economics Senior Vice Presidency September 2010 Policy Research Working Paper 5437 Abstract This paper provides an overview of the history of priority for our future research: (i) securing economic development research at the World Bank and points transformation; (ii) broadening opportunities to to new future directions in both what we research and participate in the benefits of, and contribute to, such how we research. Six main messages emerge. First, transformation; (iii) dealing with emerging risks at all research and data have long been essential elements levels; and (iv) assessing the results of development of the Bank's country programs and its contributions efforts, including external assistance. Fifth, a new multi- to global public goods, and this will remain the case. polar world requires a new multi-polar approach to Second, development thinking is in a state of flux and knowledge; the Bank must learn from, and collaborate uncertainty; it is time to reconsider both the Bank's with, developing-country researchers and institutes. research priorities and how it does research. Third, Sixth, greater emphasis must be given to producing a more open and strategic approach to research is the data and analytic tools for others to do the research needed--an approach that is firmly grounded in the themselves and providing open access to those tools. key knowledge gaps for development policy emerging And open data initiative needs to be extended to open from the experiences of developing countries, including knowledge. This will better inform development policy the questions that policy makers in those countries debates and allow for deeper engagement with the direct ask. Fourth, four major sets of problems merit high stakeholders in the outcomes of those debates. This paper--a product of the Development Economics Senior Vice Presidency (DEC)--is part of a larger effort in DEC to reflect on future priorities for development research--both what is research and how it is researched. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at research@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Research for Development: A World Bank Perspective on Future Directions for Research This paper presents views of the World Bank's Development Economics Senior Vice-Presidency (DEC)--under the leadership of the Bank's Chief Economist, Justin Lin--on where research on development economics should be heading. Primary responsibility for drafting the paper went to DEC's Directors for Research (Martin Ravallion) and Development Policy (Alan Gelb, followed by Ann Harrison in January 2010), who drew on inputs and comments from colleagues in DEC and consultations with many others within and outside the World Bank. Special thanks go to the managers and staff of the Research Group. The paper benefited greatly from extensive comments (on multiple versions) from the World Bank Group's President, Robert B. Zoellick. Knowledge is at the heart of the World Bank's comparative advantage as a development institution. Research and data collection are an essential component of the Bank's core country- based programs and the leading edge of the Bank's contribution to the global public goods agenda. To maintain its usefulness, however, knowledge must grow as new challenges arise and as new findings confirm or call into questions previous ideas. And knowledge must be used well to be useful for development. Development economics, both within and outside the Bank, faces many challenges in its quest to inform development policy making. Much has been learned about development from research and operational experience. Development data are also more available and accessible than in the past. However, some observers are asking whether development economics in its present form is well positioned to tackle the most urgent issues of the day. Even before the current global crisis, we had a sense that the course of development economics needed rethinking. The crisis has only made this more compelling. It is now clear that the global economic and financial architecture is itself a development issue. How this evolves after the crisis will influence the progress of developing countries as well as growth in the emerging multi-polar world economy. Looking forward, developing countries will be more active participants in the new financial system. There has been enormous overall progress against extreme poverty in the world, helped by faster economic growth in developing countries. The number of people living below $1.25 a day (at 2005 purchasing power parity) is estimated to have fallen from 1.9 billion in 1981 to 1.4 billion in 2005. There has also been great progress overall in improving health and education. However, progress has been uneven across countries and over time. For example, excluding China, the number of people living below $1.25 a day has stayed around one billion or more since the early 1980s (though of course the proportion of people who are poor has fallen). The wave of trade reform that swept the developing world--with the move away from exchange rate overvaluation, export taxation of agriculture, use of nontariff barriers, and with a two-thirds reduction in average tariffs--has been associated with a dramatic 2 increase in the importance of exports of manufactures and services from developing countries. However, here too progress has been uneven, with the benefits of reform and globalization not reaching many of the world's poor. There is a perception that orthodox prescriptions paid too little attention to managing the risks posed by globalization. The development community has engaged in spirited debate over the adequacy of prevailing development frameworks. Some feel that developing countries are recovering rapidly in part because they have followed orthodox policies. Others point to the high economic and social costs of the crisis and the difficulties encountered by countries that have followed such policies but nevertheless have not succeeded in moving into middle or upper income status. Different periods of history have been associated with the rise and fall of different schools of thought in economics. Keynesianism was popular in the 1930s and 1940s, the neoclassical and more laissez-faire tenets of Milton Friedman and others reached ascendance in the 1980s. In development circles, the Washington Consensus was popular into the 1990s though in the 2000s we have been asking ourselves how we can go beyond the Washington Consensus. Some countries appear to have done well with policies that involve a more active role for the state than envisaged by the Washington Consensus. China's active effort to solicit and encourage foreign investors through tax holidays is a case in point. We are also seeing that pro-poor social policies, such as better education and health services for poor people and certain transfer schemes, can help assure more equal access to the opportunities unleashed by market-driven development. However, here too progress has been uneven, with governance problems plaguing efforts to deliver better services to poor people. Many of the most important issues facing developing countries are at the sector level. How can local institutions and public services be made more responsive to the needs of the poor? How can the private sector play a more significant role, and how can this be encouraged? How can farm productivity be raised? How can energy and water use be made more efficient and part of a larger effort toward more rapid and sustainable 3 development? Addressing such issues requires a broad range of skills, including insights from disciplines other than economics. Development faces new challenges, including: (i) an important global public goods agenda; (ii) the emergence of a larger middle-income group in the developing world with new aspirations, yet with many living just above poverty levels; (iii) feeding the extra three billion people expected by 2050; and (iv) new challenges for developing countries in managing risks, capital and goods flows, and technological development in a globalized world. What should be the role of World Bank research within the global knowledge-creation effort, and what research is needed to address today's major development challenges? After reviewing the history of research at the Bank, this paper outlines what we see as the priorities for research looking forward--our vision of where we want to go. The paper then turns to implementation--how we plan to get there; here we recognize that the world has changed in ways that influence the best ways to generate knowledge in the future. History of Development Research at the World Bank Throughout its history, the World Bank has been at the forefront of research on development. For instance, Paul Rosenstein-Rodan, one of the founding fathers of development economics and originator of the Big Push theory, was Assistant Director of the World Bank's Economics Department in the 1950s. While consulting for that department, Albert Hirschman carried out the first major mission to a developing country (Colombia) in the early 1950s.1 An organized research program took shape in the early 1970s, with the focus on producing quantitative and data-heavy research, when Hollis Chenery became the first Chief Economist of the Bank. Empirical development economics can be said to have begun in earnest at that time. (Simon Kuznets received the Nobel Prize in Economics in 1971 "for his empirically founded interpretation of economic growth, which has led to new and deepened insight into the economic and social structure and process of development.") 1 See Michele Alacevich, The Political Economy of the World Bank: The Early Years. Stanford, CA: Stanford University Press, 2009. 4 During the 1950s, the Economics Department was mainly responsible for conducting financial feasibility studies of proposed projects. During the 1960s, under the presidency of George Woods, the Economics Department recruited some 200 PhDs in economics and also sought to have economic analysis carried out in area departments. When the first research program was formalized in 1971, senior management gave it four basic objectives: (i) to support all aspects of the Bank's operations, including assessment of the development progress of member countries; (ii) to broaden understanding of the development process; (iii) to improve the Bank's capacity to provide advice to members; and (iv) to develop research capacity in member countries. Considerable emphasis in the 1970s was given to reducing the gap between rich and poor countries. Following Robert MacNamara's 1973 Nairobi speech, the Economics/Research Department promoted research projects aimed at improving understanding of the causes of poverty and the search for policy options for poverty reduction, focusing especially on rural areas. In the 1980s, the research focus shifted toward market incentives, getting prices right, and macroeconomic adjustment. Energy was also high on the agenda. Gender and the environment appeared on the Bank's research agenda in the late 1980s. The reform of socialist economies and the emergence of AIDS became special areas of after 1989. Poverty, inequality and corruption reemerged as research topics during the 1990s. A large effort, based in the Development Economics Vice-Presidency (DEC), developed new operational tools for poverty analysis that were taken up by the Bank's country Poverty Assessments. "Beyond the Washington Consensus" emerged as a topic in need of special attention. In the 2000s, emerging economies, in particular China and India and their impact on the world economy, as well as the role played by infrastructure and agriculture--after years of neglect in lending--have been high on the agenda. So too have the problems of fragile states, whose weak institutions make them prone to conflict and pose special development challenges. The Bank is now well known for the quality and relevance of its research. By purely academic criteria--based on research publications and citations by other researchers--the Bank 5 is the leading institution working on development economics globally (in the RePEc rankings).2 An independent evaluation of Bank research was done in 2006 by a panel of about 20 distinguished academics, led by Professor Angus Deaton of Princeton University. Based on the evidence it assembled, the interviews it conducted, and its own consideration, the panel concluded that: "Research is a central part of quality control in the Bank, and is crucial to its claim to be a `Knowledge Bank.' Without a research-based ability to learn from its projects and policies, the Bank could not maintain its role as the world's leading development agency." The panel found that the Bank's research economists have led the world in the measurement of poverty and inequality, including inequality in health, and have done pioneering research on the delivery of educational and health services that has changed the way we think about these issues and the way the Bank lends money for such projects. The Bank's ground- breaking efforts in these areas stimulated further academic research. The Deaton report also pointed to important work on managing the environment and "extremely visible work on globalization, on aid effectiveness, and on growth and poverty." However, the panel also had criticisms of how research was sometimes used to proselytize on behalf of Bank policy, without always taking a balanced view of the evidence and without expressing appropriate skepticism. The management of the Bank's research department has taken these criticisms seriously. Since the mid-1990s, the Bank as a whole has been publishing an average of around 100 books per year on economic development. The best known of these books--which surely everyone working on development knows--is the Bank's annual World Development Report (WDR). The first of 31 WDRs was issued in 1978, entitled Prospects for Growth and Alleviation of Poverty. The latest is on Climate Change and Development. The research department has also produced 20 Policy Research Reports on specific topics, starting with the East Asian Miracle in 1992; the latest was on Conditional Cash Transfers and the next is on Localizing Development (spring 2011). Bank authors have published some 2,000 other books. But this output of books is dwarfed by the Bank's output in scholarly journals--9,000 articles to date. A great many of these publications have influenced development thinking; this is evident, for example, in the citations found using objective sources such as Google Scholar and bibliographic 2 If we restrict the analysis to the output of the Bank's research department, only the University of Chicago ranks higher. 6 data bases.3 It is hard to imagine that the Bank could have emerged as the leading development agency without this portfolio of publications on development. The Bank's research is naturally skewed toward topics relevant to low and middle- income countries. If one focuses on publications addressing countries except the United States in the top 200 economics journals, the Bank has a similar output--about 1,000 papers--to the top five universities combined (Harvard, MIT, Chicago, Stanford and Princeton). The difference is that half of the Bank's publications reported research findings on the poorest 40 percent of countries (ranked by GDP per capita), while those countries only accounted for about one- quarter of the publications of the top five universities.4 Data have long been an important research product. The Bank has been at the lead in expanding the volume, quality and accessibility of development data, as the foundation for development knowledge and well-informed policy making. These data are not simply economic; they include a range of sociological and political information. Early development thinking was often weak on facts about what was happening on the ground. Some of the major advances in development data emerged out of a growing awareness of that early weakness: When Robert McNamara, as President of the World Bank, saw the table of distributional data from household surveys in the first World Development Report, he was shocked to see so many holes. For all except 17 developing countries there were no data in the 1979 WDR, yet macroeconomic aggregates (from national accounts) were available for virtually all countries. McNamara told his research staff to go collect the data--and that was the beginning of the Bank's still thriving Living Standards Measurement Study, based in the research department Since the early 1990s, the Bank's researchers have put considerable emphasis on collecting new micro data sets, on individuals, households, facilities and enterprises based on field work. 3 For an overview of the Bank's publications and their scholarly influence, as revealed by citations, see Martin Ravallion and Adam Wagstaff, "The World Bank's Publication Record." Policy Research Working Paper 5374, World Bank, Washington, DC, 2010. 4 These calculations are based on the data set compiled by Jishnu Das, Quy-Toan Do, Karen Shaines and Sowmya Srinivasan, "U.S. and Them: The Geography of Academic Research." Policy Research Working Paper 5152, World Bank, Washington, DC, 2009. 7 We now know much more about the developing world, including how poor it is. For example, the World Bank's first estimate of the "$1 a day" poverty counts in 1990 used data from only 22 national household surveys for 22 countries; today the estimates use 700 surveys for 115 countries. An interactive software platform, PovcalNet, provides free access to the data derived from these surveys, allowing users to replicate the Bank's global poverty counts and make their own estimates under different assumptions (such as alternative poverty lines). The World Development Indicators has emerged as a central data resource. Key development data are now available through Google. One important change in April 2010 has been to provide these data free of charge to all users through the Open Data website. As a result of the Bank's research and data collection, and the efforts of Bank researchers and their colleagues to promulgate the findings and lessons of that research, the global community knows more than in the past, and is better placed to confront theory with evidence. Past development debates involved wide swings in philosophy and theories and, as a result, the Bank also experienced major shifts in terms of research priorities.5 Pronouncements on policy are now more firmly grounded in market, governmental and institutional realities. A consensus has evolved among development economists that there is no simple recipe for development, applicable to all countries and situations. Markets and states need to complement each other, in a more pragmatic approach that recognizes the diversity of situations and confronts policies and programs with close empirical scrutiny. Similarly, economy-wide development policies and sector-based policies to enhance efficiency and inclusiveness need to be complementary. This shift toward more empirically-grounded development research, while still reflecting the need to formulate coherent and testable theory, is the hallmark of current development research. To achieve these goals, World Bank researchers must challenge themselves as well as reach out to other disciplines and practitioners in the field. This includes working with experts in 5 Jean-Jacques Dethier, "World Bank Policy Research: A Historical Overview." Policy Research Working Paper 5000, World Bank, Washington, DC, 2009. 8 history, political science, sociology, anthropology, psychology, political economy, and sometimes the physical and biological sciences. Priorities for Future Development Research In thinking about future priorities, one naturally builds on lessons from past research, to identify significant and important knowledge gaps in the light of current and anticipated future needs. The need to look forward in identifying research priorities is crucial. Given the inherent time lags in the research process, a static perspective on research priorities means always running to catch up, rather than being out in front. This hinders not only the direct relevance of research, but also its role in quality assurance. The Bank invests large sums each year in shorter-term analytical work, broadly defined, to address various more immediate needs. Without an anticipatory approach to research priorities, research findings cannot be ready to provide a testing ground for the conclusions of other analytical work. Some continuing themes from past research include the greater emphasis now given to the quality of the growth process, notably whether it can deliver inclusive and sustainable development. The impacts of policies and economic changes on the poorest, once routinely neglected, are now given high priority in research. Inequality--notably inequality of opportunity--is now seen as an important problem constraining growth and poverty reduction. Another important ongoing theme concerns the role of the state and international development institutions themselves, vis-à-vis individual decisions in markets. This topic is unavoidable given the fact that private investment globally and in a large number of individual countries (except some low-income states) dominates public-sector investment and international assistance. This has striking implications for how the Bank and other institutions both can and should approach their development missions, both investments and technical assistance. The topic cuts across the landscape, including infrastructure, finance, many traditionally public services like education and health, rural development, and natural resource management. A more pragmatic and empirical approach to policy making has also emerged and this is to be encouraged moving forward. Some of the accepted policy orthodoxies of the past will need 9 to be revisited in the light of new ideas and information, including knowledge from the broad range of experiences of developing countries. There are concerns about the relevance of some aspects of current development research. Is it consistently addressing the most important problems facing developing countries, and is it doing so effectively? Relevance is defined not just by the questions asked and the internal validity of the answers given--establishing that the conclusions about the setting follow from the data. Relevance is also about the external validity of research findings--their validity for the purposes of drawing lessons for other settings, including scaling up. Relevance is defined as well by the usefulness of the research for addressing the questions that policy makers and clients are asking ­ or highlighting other questions that need to be addressed. There are many challenges ahead in improving development data, and engagement with both the research and policy communities is crucial for meeting those challenges. There are concerns about incomplete coverage across countries and over time of some core areas of development data, including on health, education, infrastructure and gender indicators. There are also concerns about the internal consistency of development data, such as the discrepancies found between consumption aggregates and growth rates as estimated from national accounts versus those obtained from household surveys. There are concerns about the quality and coverage of public finance data, including at sub-national levels. Data on physical infrastructure stocks and government spending on infrastructure are weak in most countries. Another generic theme in our thinking about future directions is our belief that not enough attention has been given to the need for differentiated policy recommendations for countries at different stages of development. Differences between countries in (inter alia) physical and human capital endowments and their distribution, and associated differences in the evolution of production activities at different points in the development process, lead to the need for different implications regarding the appropriate institutions and policy frameworks. This theme cuts across all areas of development policy making. There is a dearth of research on development strategy, including on the policies and mechanisms that have been used by some fast-growing countries to accelerate structural transformation. The Growth Commission suggests that "Growth entails a structural 10 transformation of the economy, from agriculture to manufacturing, from a rural workforce to an urban one". At the Washington Summit Declaration, G20 leaders listed "inconsistent and insufficiently coordinated macroeconomic policies" and "inadequate structural reforms" as the two main reasons for the global crisis. Greater emphasis on development strategy must be balanced against the now well-understood fact that there is no "one size fits all" solution to the challenge of promoting growth and overcoming poverty (both from one country to the next, and from one time period to the next). It is important to understand the specific opportunities, circumstances, and constraints facing lagging countries in an increasingly globalized world. There is concern that much present-day research--both micro and macro--is too narrowly focused, and too weak on external validity, to provide the kinds of insights needed to help bring a large and lasting reduction in global poverty. All too often, research looks for topics "under the light." The positive outcomes for policy making are occasional by-products of research rather than its objective from the outset. Many research economists do not start with the key knowledge gaps facing development practitioners but rather search for questions they can answer with the field's currently favored tools. Considerable emphasis is given to internal validity, but external validity is not addressed with the same rigor. The core problem today is that the impressive set of data and analytic tools now available is not sufficiently anchored to the most pressing questions facing developing country leaders, advisors, and investors. It is like a map of the world being filled in by careful study of non-randomly chosen villages, one at a time. A more strategic approach to research is needed--an approach that is firmly grounded in the key knowledge gaps for development policy. The subject of development economics needs to return to tackling the big questions of how to sustain economic growth and how to eliminate poverty, but to do so with all the power that the subject's data and analytic tools can now provide. A key premise for such a research agenda is to recognize that the analytical framework for countries at different stages of development needs to be different and that the policies appropriate for different countries may not be the same.6 6 See Justin Yifu Lin, Economic Development and Transition: Thought, Strategy, and Viability. Cambridge: Cambridge University Press, 2009. 11 Table 1: Research Agenda Looking Forward Priorities for: Analysis Data 1. Transformations Understanding the relationship between Macro data, especially on sectoral composition change in an economy's structure and broader of output. development goals, including poverty Better data on infrastructure investments. reduction. Better data on policy interventions over time. Roles of states, markets and the private sector Firm-level data, especially to study productivity in promoting economic transformation. growth and firm-industry dynamics. Appropriate policies at each stage of Informal sector data, to analyze the factors development. behind entry into and exit from informality. Governance issues for industrial upgrading Household-level data (archival + new) to link to and structural change. welfare outcomes and to integrate with Role of agriculture versus other sectors; macroeconomic/sectoral/public finance data sectoral priorities and trade-offs. including sub-national. Relevant country types: Low income predominantly agrarian or resource-rich economies seeking to move into middle-income status; middle-income economies seeking to upgrade industries and diversify from manufacturing into services. 2. Opportunities Better understanding of why some countries Household, facility-level, public expenditure and places attain faster poverty reduction and tracking, service delivery data. more inclusive development than others at a Data disaggregated by gender given rate of economic growth. Financial services data. Governance challenges in assuring better New measures of inequality of opportunity. education, health and social protection. Information to strengthen provider incentives. Policies to enhance access to finance. More inclusive global integration and/or regional integration, linked to global markets. Relevant country types: High levels of inequality and/or slow rates of poverty reduction; poor performance in moving towards MDGs; regions with low levels of educational attainment and high infant mortality rates. 3. Risks More effective and cost-efficient social Panel data sets tracking same households or protection, including more automatic firms over time. stabilizers in poor countries. Addressing core data weaknesses in fragile Fragile and conflict-ridden states. states. Managing new environmental risks. Better data on environmental and natural Addressing financial-sector vulnerabilities. resource risk factors, their consequences, and Global governance reform and policies to the costs of amelioration. support new multipolar growth and Data on trade flows and trade policies; made interconnected risk. available at no cost. Managing macro-financial risks posed by globalization. Relevant country types: Fragile states, resource-rich economies, regions exposed to extreme weather conditions such as floods or drought; regions highly integrated with global trade, financial markets or people's movement. 4. Results Measures of country performance, including New types of data (mixed qualitative- benchmarking and identifying comparators. quantitative methods). Broader approach to "evaluation," New modeling tools suitable for "non-assigned" emphasizing external validity, drawing on interventions. richer economic modeling, more diverse types Different types of results measures (rates of of data, multiple disciplines, and tailored to return, poverty and other human development strategic knowledge gaps in the above areas. indicators) Relevant country types: All developing countries; countries with a high share of aid to GDP; regions where aid results are weak. 12 What then are the key researchable questions that the World Bank should give highest priority to in the future? We can distinguish four major sets of problems that merit high priority: (i) achieving efficient economic transformation, which also means the ability to tailor policy advice to different stages of development; (ii) broadening opportunities to ensure that all individuals can participate in the benefits of, and contribute to, such transformation; (iii) dealing with emerging new global challenges related to risk and vulnerability; and (iv) assessing the effectiveness and results of development efforts, including external assistance. Each of these themes has implications for both analysis and data, as summarized in Table 1. Each of these themes is also linked with certain country types--the World Bank's "clients" --as indicated in Table 1. The discussion below elaborates. First, we need to better understand the roles of states, markets, and the private sector in promoting the transformation of economies. Why have some countries been able to achieve sustained growth and address environmental degradation, while others appear to remain trapped in dire poverty? The Growth Commission identified 13 countries that were able to maintain high sustained growth over 25 years. Why so few? How do countries transform their economies in an environmentally friendly way and in the process reduce poverty and provide employment opportunities for all? To address these fundamental questions requires a deeper understanding of the process of how an economy's structure evolves. This is not just about the shift from agriculture to industry and services. Average income can rise if an economy moves from producing some types of goods to producing others, or if it changes how those goods are produced, such that higher output is attained with the same inputs. In China, for example, internal economic reforms starting in the late 1970s achieved a large increase in agricultural productivity and dramatic reduction in poverty incidence.7 This step was probably essential for the next stage of productivity gains through a shift of labor and production to manufacturing and services, which in turn required population urbanization, with 7 On China's rural economic reforms see Justin Lin, "Rural Reforms and Agricultural Growth in China." American Economic Review, 1992, 82: 34-51. On China's subsequent success against poverty since 1980 see Martin Ravallion and Shaohua Chen, "China's (Uneven) progress Against Poverty." Journal of Development Economics, 2007, 82(1): 1-42. 13 further gains to the rural poor. The Green Revolution in India helped greatly in reducing extreme poverty through a transformation within agriculture, facilitated by new technologies and supportive changes in input markets and policies. Yet elsewhere we see continuing stagnation in farm economies and weak labor absorption by the non-farm sector. In a country like Uganda, over 80 percent of the population makes its living in rural areas, where productivity is very low relative to the non-farm sector. How does the movement into higher productivity activities happen in such an economy, both within the rural economy and through urbanization? This question is about the micro-foundations of economic growth. Within agriculture, services or industry, how does the process of moving into higher quality goods and services happen? Within firms, why are some able to move into export markets and start producing new goods, while others languish? How do economies move into information technology and services? What determines a country's economic structure and what contributes to the diversification in an economy? How do firm and industry dynamics--entry, exit and the reallocation of resources across firms within sectors and across sectors--contribute to efficiency, productivity and growth? One implication of this research agenda is the need for differentiated policy recommendations for countries at different stages of development. We must move from the realization that "no one size fits all" to "which policies for which stage of transformation?" Policies are likely to differ across income levels as to the scope of effective fiscal stimulus, the role of the government, and innovation policy. Policy advice for countries moving from a low- income agrarian economy to a middle-income industrial economy is likely to differ from the advice for those stuck in a "middle-income trap", if such traps indeed exist. Countries that are well-endowed with a rich natural resource base also face different policy problems, some related to Dutch disease and others to the need to develop a sustainable strategy, diversify their production base and ensure against commodity shocks. Another area where appropriate policies at different stages of development are likely to vary is finance. How can we assure that financial systems, which may be structured differently at different stages of development, serve the real economy--rather than the reverse? The ongoing crisis has taught us that financial systems need better oversight in all countries. What 14 are the optimal regulatory structures across different stages of development that will prevent these crises from reoccurring? Are capital controls more appropriate at higher or lower stages of financial development? An important question related to this agenda is what should be the respective responsibilities of the market, the state and the private sector in sustaining the dynamics of structural change. What roles (both positive and negative) do markets and states play in the transformation? How do political economy and governance factors constrain the scope for effective public action in promoting structural transformation? What should be the role of infrastructure, and who should provide it? The answers may vary, depending on political leadership and the ability of governance structures to withstand capture by powerful elites. A competitive market should be the economy's fundamental mechanism for resource allocation. Only such a market can generate prices that reflect the relative scarcity of goods and factors and so guide firms to enter (and exit) industries and to choose appropriate technologies according to the economy's comparative advantage. In the process of industrial upgrading and structural change, "hard" infrastructure, as well as social and economic institutions or "soft" infrastructure, needs improvement for the economy to achieve x-efficiency. The dynamic transformation requires information and coordinated investments, and many externalities may arise. Market failures abound and the state has a role to play in the process. State failures (including an inability to correct market failures) are also abundant. The role of the state in promoting certain patterns of industrial development has been a highly contentious area. Most governments attempt to promote industry, yet evaluations have been few and rarely rigorous. There is a pressing need for new research efforts to evaluate these policies. One important research question is whether to directly encourage transformation by focusing on "discovery" of new sectors of comparative advantage, or by simply working with existing industries and clusters to deal directly with the coordination failures that limit their productivity and expansion. Specific mechanisms to encourage new activities will vary across client country categories. It would be misleading to generalize from the success of a few countries with 15 particular policies against a particular national backdrop, to general prescriptions for successful economic transformation policies. In many cases, strengthened roles for the government will be in the areas of infrastructure provision, skills development, and creating an enabling regulatory framework. China, for example, instead of using blanket subsidies for exports and FDI, has given tax incentives and provided infrastructure for special economic zones in key sectors like electronics and telecommunications to attract multinationals. Many public-private partnerships have been successful in Latin America. Several attempts to export flowers in the 1960s and 1970s in Ecuador failed in part because of the lack of reliable air transport to the main destinations. A key difference in the 1980s was an effort by the association of owner-exporters, EXPOFLORES, to convince the government and the national airline to set up the required number of cargo flights for this activity. Flower exports boomed from less than half a million dollars in 1984 to more than $400 million in 2006. To develop new exports of broccoli and mangoes, Ecuador faced a challenge in finding the best seeds and meeting international phytosanitary standards, which presented producers with significant coordination problems. Collective action fostered and implemented by several private, public and mixed agencies was important in dealing with such problems and in facilitating the development of these new sectors.8 Another challenge is how to ensure that the government's policies for structural change are effective and not captured by rent seekers. Guidelines on how states can be built that are less prone to rent-seeking and finding ways that governments can assist economic transformation would be useful. Is there a way to structure policies to ensure that emerging and not declining sectors are encouraged? Research has shown that good governance often is associated with more growth-promoting policies, and it is believed that better governance can foster private entrepreneurship and greater economic and political freedom generally. The government plays a critical role in ensuring the maintenance of property rights, which in turn ensure more efficient transactions, better investments, increasing environmental sustainability, and productivity. But how can governance improvements be achieved and sustained? And to what extent are these 8 For references on this and further discussion, see: Ann Harrison and Andres Rodriguez-Clare. 2009. "Trade, Foreign Investment, and Industrial Policy for Developing Countries." Handbook of Development Economics 5, Amsterdam: North Holland. 16 relatively conventional prescriptions sufficient? Answers to this question require a great deal more work on political economy than is often encountered in development research studies. Agriculture deserves special emphasis in growth-related development studies. Higher agricultural productivity has been a pre-condition for industrial transformation in most successful developing countries. Many of the world's poorest depend on agriculture and so agricultural growth is often more poverty-reducing than growth in other sectors.9 Private firms, especially small, family farms, seem to be remarkably efficient in adopting and using available production technologies, but they have no incentive to develop new technologies. How might agricultural productivity be raised, especially in Africa where there has been remarkably little progress on this front? How much might improvements in technology, in land tenure, in irrigation, in marketing, and in access to finance contribute? By how much might improvements in rural infrastructure promote higher farm productivity and a more diversified rural economy? How can formal and informal mechanisms for effective management of natural resources "commons" ­ shared fodder, forested areas, and water catchments ­ be strengthened to increase agricultural productivity, and thereby also increase household surpluses available for other investments in human as well as built capital? While agricultural development remains important to poverty reduction, in some economies--notably those with high inequality in command over agricultural land--the service sector emerges as no less important to the poor.10 The importance of industrial growth tends to be highly contingent on prior inequalities, notably in access to human and financial capital-- linked in turn to delivery mechanisms for education, health and financial services. This is an instance of a more general point that both the aggregate level and distribution of initial endowments influence both the growth path and how the benefits of that growth are distributed. In time, and with the right policies, the distribution of endowments can also be improved in a virtuous cycle of more inclusive development. 9 See World Bank, World Development Report 2008: Agriculture for Development. World Bank, Washington, DC, 2007. 10 See the discussion of the relative importance of agriculture versus services in the economic history of poverty reduction in Brazil, China and India in Martin Ravallion, "A Comparative Perspective on Poverty Reduction in Brazil, China and India." Policy Research Working Paper 5080, World Bank, Washington, DC, 2009. 17 The sectoral pattern of growth matters to both the level of growth and its distributional impacts. The fact that a rising share of GDP in non-farm sectors is associated with higher GDP per capita, yet agricultural growth is especially pro-poor in many countries, raises the possibility of a trade off in poverty reduction between the growth impacts of structural transformation and its distributional impacts. Agricultural growth often has better distributional impacts, but non- agricultural growth rates tend to be higher. The extent of this trade-off is unclear, however, and will depend crucially on the sources of growth in each sector, and on country initial conditions and how they evolve in the course of economic development. More research is needed on the poverty impacts of structural change. This agenda also has important implications for data collection, so as to properly test hypotheses and lay a firmer foundation for policy making. Here there is a rich and diverse cross- country experience that needs to be properly documented, drawing on both national accounts data and the wealth of household and enterprise surveys that have been done in developing countries, as well as implementing new surveys that are tailored to the most important knowledge gaps. Structural transformation is not, of course, an end in itself. At the heart of this empirical research agenda is the task of properly documenting the successes and failures in attaining long- term progress against poverty and the role played by structural transformation. Newly available data on trends in poverty and inequality and improved methodologies offer an opportunity to re- examine how countries have fared over time. They have shown, for example, that China and India have experienced rising inequality, which has attenuated the gains to the poor from more rapid economic growth while in Brazil, inequality has fallen in recent years (due in large measure to more effective social policies), which has facilitated a high proportionate rate of progress against poverty despite lower growth rates.11 A systematic effort to compile up-to-date "stylized facts" on such trends is a prerequisite to an analysis of the underlying drivers of improvements in well-being. In thinking about data and methods it is acknowledged that examining growth and distributional change on the basis of cross-country comparisons is useful, but also that it runs the 11 See Martin Ravallion ibid. 18 risk of masking the enormous variation experienced by different countries over time. This kind of analysis should be complemented by studies with more detailed country and location-specific analysis. In a growing number of countries, studies based on long series of cross-sectional household surveys have uncovered important roles for agricultural intensification, urbanization, rural non-farm diversification, land reforms, human capital, migration, social protection and redistributive policies, and political and governance reforms. With increased availability of appropriate data the number of countries in which such studies are undertaken has expanded over time. Further research of this sort is needed to better understand structural transformation and its growth and distributional impacts. Second, we need to better understand how access to economic opportunities can be broadened to ensure rapid poverty reduction and promote human development. We have learned that inequalities of opportunity lie at the heart of inequitable growth processes in developing countries. Lack of access to opportunities, particularly for investing in education, health, and other forms of human capital, stifles growth and makes the growth that does occur less effective in reducing poverty. The large differences in the rate of progress against poverty at a given rate of growth that one sees across countries can be linked both theoretically and empirically to inequality. On the data side, advances in the measurement of inequality of opportunity can help us see how societies have expanded opportunities to their populations.12 We have learned that ensuring better services for poor people is one key element. However, across countries, social outcomes are only weakly related to development spending in the social sectors and, in many countries, additional spending on social services is subordinated to other priorities. Understanding why similar programs work better in some countries than others and identifying the binding constraints to better service delivery are research priorities. We know that governance plays a role in explaining the inadequacy of services to the poor. Past research has documented large scale absenteeism and poor on-the-job performance by front-line service providers. Reforms to improve provider incentives frequently face substantial political opposition, because improving the quality of service delivery is not as politically salient as the provision of protected jobs or price subsidies. Too often front-line service providers and political 12 World Development Report 2006: Equity and Development (World Bank. Washington, DC) broke important new ground on this topic. 19 decision makers have weak incentives to improve services. Documenting publically and understanding these deficiencies in service provision is key to assuring better services. One important direction for future research is experimenting with information dissemination interventions and transparency to strengthen the incentives of providers and governments. Quite diverse results are obtained from different types of information provision and across different political contexts. One type of intervention focused on using information to train and mobilize local community agencies in India, with legal powers to improve the village public school. This did not have any impact on the public school system but did succeed in stimulating large scale private volunteer teaching which improved literacy levels dramatically. Building on these findings, new work by local civic society organizations in Africa and India is experimenting with making information more politically salient, such as by disseminating performance indicators at the level of electoral constituencies. Information availability has large implications for accountability and is a likely candidate for reform. Research needs to focus on ways in which information can be strengthened to improve governance. This brings to the fore again the need for research on political economy to better understand not just what might work better, but also how to implement effective changes. One promising direction is impact evaluations that alter institutional arrangements in the market for education and health care. A long-term study in Pakistan documents a large increase in the number of private schools and shows that more than half the population in the largest province, Punjab, now lives in villages with eight different public and private schools from which to choose. In this environment, policies aimed at improving only public schools could prove counter-productive if they prompted many children to switch from private to public schools, such that the costs of overcrowding outweighed the benefits of reform. In an experimental study, providing information on school and child-level test scores led to declines in private school fees and improvements in test scores in both public and private schools. This intervention targeted a governance problem (the lack of information about provider performance) and the broader educational market rather than specific schools. Two obstacles to scaling up this reform point to other areas where research is vital. First, service delivery is often difficult to measure. Research is needed to design meaningful, 20 comprehensive measures of service quality that allow providers to be given incentives to deliver higher quality services. Second, public options are often the only choice that citizens have. Do information interventions work in settings where government provision is the only option? Researchers have launched numerous initiatives to answer this question, for example, asking whether the presence of local, community radios in Benin increases the ability of citizens to demand and gain access to policies ranging from reduced school fees to the free distribution of anti-malaria bednets. Inadequate information is not the only reason for governance failure. As noted, deeper political economy phenomena can limit scope for reform even when information is perfect. In an environment in which political competition revolves around vote-buying, clientelist promises or violent confrontation--characteristics that are endemic in fragile states but are not absent in most others--even well-informed citizens are unlikely to be able to hold political decision makers accountable for poor service delivery. A goal of research is to understand better the conditions under which politicians have an incentive to promise and deliver better services to the poor. This research will inform and improve all those interventions that fall under the rubric of "demand- side interventions," aimed at improving the ability of the poor to hold governments accountable for the services they provide. Another important area for research is on gender equity, the focus of the 2012 World Development Report. Gender disparities in rights, access to resources and economic opportunity, and in voice and political participation vary across and within countries but remain pervasive worldwide. On the one hand, more rapid development can help attenuate gender disparities. Higher incomes, and better development outcomes more generally, are likely to benefit even those who are disadvantaged in terms of opportunities (generally girls and women). In the other direction, reducing gender inequality can help improve development outcomes. There is increasing awareness of the costs that gender disparities impose on individuals and societies, and of the resulting implications of these inequities for development prospects. Development practitioners still face large gaps in knowledge both in how and why gender equity matters for development and in the understanding of how best to take account of and address gender differences in policy design. The World Development Report for 2012 aims to fill these gaps by 21 providing concrete evidence on the extent of gender gaps, why these inequities matter, which ones are of greatest concern and what policies can help resolve them. Broad access to financial services is believed to be crucial for inclusive growth.13 There is a need to assess and evaluate the channels through which access to finance can contribute to more pro-poor growth processes, such as promoting entrepreneurship, innovation and the process of technology adoption. The role of financial literacy and gender differences in this process should also receive greater attention. Ultimately, the aim should be to evaluate the impact of firms' financing constraints and households' inability to access financial services on economic growth and poverty alleviation, and to better identify different ways to improve this access, ranging from microfinance innovations to making improvements in the functioning of mainstream financial institutions and systems. For example, one frequently used policy for improving micro, small, and medium-size enterprises' access to bank loans is creating a credit guarantee scheme that insures banks against borrower default.14 Credit guarantee schemes can benefit small and medium enterprises (SMEs), since these firms often lack the necessary collateral and accounting information to obtain a loan without a credit guarantee. Several World Bank units are currently providing technical assistance to governments related to the implementation of such credit guarantee schemes. However, there is very little rigorous evidence on the impact of these schemes that can guide policy discussion. Open questions include the following. Do credit guarantee schemes really generate additional loans or is participation largely confined to SMEs that would have access to loans in the absence of the scheme? How strict should the selection criteria be to ensure that the credit guarantee scheme reaches the firms that need it most, while balancing this with sustainability of the scheme? What is the impact of additional loans on firm investment and growth? DEC researchers are currently collaborating with experts from the IFC on designing impact evaluations of credit guarantee schemes. 13 The arguments and evidence on this point are reviewed in Finance for All? Policies and Pitfalls in Expanding Access, Policy Research Report, Development Research Group, World Bank, 2007. 14 For an overview of credit guarantee schemes, see: Thorsten Beck, Leora Klapper, and Juan Carlos Mendoza, "The Typology of Partial Credit Guarantee Funds around the World." Policy Research Working Paper 4771, World Bank, Washington, DC, 2008. 22 Broadening opportunities also has an important global dimension. How can we ensure a more inclusive process of global integration? The economic crisis led to the biggest annual decline in world trade in the last sixty years. Many developing countries--in part due to World Bank advice--have made global integration a key component of their growth strategy. Will the export-led growth model--famously adopted by a number of Asian countries--continue to succeed, especially in light of China's continued dominance in manufacturing? Or will the current reality point more to domestic growth paths, associated with a different set of obstacles such as the need to improve agricultural productivity and boost local demand?15 The importance of maintaining open markets in light of the financial crisis points to the critical lessons that can be learned from economic history. One reason why countries have not actively pursued protectionist policies following the sharp decline in trade after the crisis is the knowledge that such policies played a pivotal role in accelerating and lengthening the Great Depression. Important insights have been generated by economic historians such as Doug Irwin, who discovered that much of the rise in protection in the United States was inadvertent, generated from specific tariffs which increased protection levels as average prices fell. Knowledge about the enormous social and global costs of those increases in protection has left a golden legacy of caution in this current crisis. In a post-crisis world, what is the best way to ensure that the gains from global integration also help the poor? A number of measures--some in partnership with the private sector--have recently been initiated by the World Bank to increase trade credit, facilitate trade, and provide aid for trade. Yet we still lack knowledge about how trade credit and trade facilitation efforts affect trade flows. We would also like to identify the proper role for industrial and export promotion policies to increase exports and encourage diversification as a means of broadening the opportunities for benefiting from globalization. The policy challenges in this area are particularly stark given the very strong patterns of specialization in trade that we observe at the country-product level. For example, 42 percent of US imports of women's cotton suits come from just one country (Fiji), and 94 percent of Italy's imports of bathroom ceramics come 15 These questions about the continued validity of the export-led growth model are the topic of the first two series of Regional Development Debates, jointly sponsored by DEC, the WBI, and PREM. 23 from just one country (Egypt).16 We as yet know very little about the role of policymakers in countries like Fiji or Egypt in identifying such "hits" in export markets. We need to pay closer attention to the role of the private sector in ensuring global gains from international integration. In the 2000s, FDI inflows were the single biggest source of capital for developing countries and a critical input for technology transfer in developing country firms. The IFC has played an important role as a catalyst for these activities. Indeed, the IFC and DEC's research department are both exploring the important questions of what are the right policies to attract and retain FDI, without disadvantaging domestic firms. The issue is also critical for Africa, which needs to encourage foreign investment inflows and yet protect its valuable resources for posterity. Foreign investment, as well as mobilizing private savings, can also play a potentially important role in promoting adaption to and mitigation of climate change. Unequal opportunities for development are also generated by weak incentives for political actors to make decisions in the interests of citizens generally. We need to advance our understanding of the sources of such adverse political incentives, of how to shape development interventions that moderate these incentives and, when this is not possible, of how to design interventions that equalize opportunities in an unsupportive political environment. Third, we need improved means to meet existing and new risks facing economies and people at local, regional, and global levels. Our world is riskier than many of us thought. Recent history has confirmed that a large fraction of the developing world's population remains vulnerable to shocks. These can be myriad in form, ranging from natural disasters (earthquakes, tsunamis), to health pandemics, to wars and civil strife, to economic crises such as the East Asian financial crisis of the 1990s, oil and food price shocks of 2007-08, and the global economic crisis of 2008-09. Developing countries are asking themselves how the most recent crisis, which generates continuing uncertainty, will affect their growth prospects. They are also asking what they can do to better protect themselves going forward against current and prospective risks. New research has been launched to assess what factors helped countries achieve resiliency to the ongoing 16 William Easterly, Ariell Reshef, and Julia Schwenkenberg, "The Power of Exports." Policy Research Working Paper 5081, World Bank, Washington, DC, 2009. 24 economic and financial crisis. Preliminary results from DEC research suggests that many developing countries were much better prepared to respond to this crisis than more developed countries, in part because of higher fiscal surpluses, a better build-up of reserves, lower inflation, and avoidance of currency mismatches which could have led to exchange rate speculation and insolvency. Assessing the impact on poverty and well-being of such events is difficult and, when data are collected, they are often difficult to interpret. Yet such evidence is essential to improve our understanding of how long, and in what manner, shocks reverberate through affected populations. We need to better understand how shocks affect populations, seeking to highlight the way that societies can re-emerge from such setbacks and resume long-term paths of poverty and inequality decline. We also need new paradigms that allow us to incorporate risk into policy recommendations, allowing policies "wiggle room" to face unanticipated shocks. These conclusions apply with equal force to risks of natural disasters as to risks from disruptions in the global financial system. For example, measuring farmers' exposure to drought and other weather-related risks has been a longstanding concern. Some Bank research based on pilot programs that tie crop insurance to objective measures of general climate conditions--like total precipitation at a regional level--indicates that such mechanisms could be promising for providing affordable and effective insurance without "moral hazard" problems resulting from difficulties in observing individual farmer behavior. Yet, more work is needed to establish if and how such schemes can be deployed to provide financially sustainable commercial insurance products at a larger scale. In the area of natural disasters from extreme weather events such as cyclones, floods, and earthquakes, research in Bangladesh has highlighted the importance of relatively simple and cost-effective protection measures such as providing emergency shelters large enough to accommodate family members and their livestock, to counteract tendencies to remain in the path of a disaster to protect the bulk of many families' wealth.17 Other work on disaster preparedness has emphasized the need for regulatory reform as well as new regulations to lower hazard exposure. For example, land use regulations to limit settlements in flood-prone 17 See Susmita Dasgupta, Mainul Huq, Zahirul Huq Khan, Manjur Murshed Zahid Ahmed, Nandan Mukherjee, Malik Fida Khan and Kiran Pandey, "Vulnerability of Bangladesh to Cyclones in a Changing Climate: Potential Damages and Adaptation Cost." Policy Research Working Paper 5280, World Bank, 2010. 25 areas are often difficult to enforce; reform of commercial building codes and regulations, on the other hand, may help to reduce housing shortages and provide stronger price signals to housing suppliers for improving quality and safety. One important area of research is the identification of safety nets at reasonable costs. That does not necessarily mean "better targeting," which can come at (often hidden) costs, including to poor people. Past work has focused on the problem of avoiding "leakage" to the non-poor and given too little attention to other issues, including assuring better coverage, maintaining incentives for escaping poverty by other means, and devising social protection policies that can act as more automatic stabilizers. Such stabilizers do not require long lead times to provide help and do not inhibit access for those in need. The scope for non- governmental partnerships in social protection has also been under-emphasized. Safety nets for the poor are sometimes seen as harmful, or at best neutral, to economic growth. And bailouts in times of need are seen to encourage excessively risky behavior in the future. Recent research has suggested that some of these traditional concerns need to be re- considered. Counter-arguments can be made, suggesting that efficient safety nets may actually foster longer-term growth by addressing market failures, and that lack of opportunities for managing risk sis an often bigger efficiency concern than moral hazard. However, our empirical knowledge on these issues is still weak, which constrains proper assessments of the costs and benefits of this class of policies. Most countries, even including those that have experienced rapid development success underpinned by financial deepening, have suffered from financial crises interrupting the growth process, and sometimes setting back progress for a decade or more. Heightened risks at the level of the individual firm and households have reinforced the role of financial instruments and markets for hedging and managing risk also at this micro level. Risk management, including crisis prevention, needs to become a central part of the research agenda on financial development. Building on past work, further effort is needed to investigate the impact of supervision strategies as well as the impact of compliance with Basel Core Principles on bank stability, the interaction of bank insolvency resolution and deposit insurance policies, and the impact of financial globalization on bank efficiency and access to financial services. 26 Within this category of research questions, greater attention also needs to be given to better understanding the development possibilities in fragile and conflict-affected states. Depending on the definition used, between 500 million and 1 billion people live in states seriously affected by conflict or fragile governance. In contrast to most of the remaining developing world, these have seen little progress in terms of development outcomes and are becoming an increasingly serious part of the global development problem. The 2011 World Development Report addresses this question by examining the relationships among peace- building, state-building and development. The Report also explores how to build confidence and state legitimacy, through inter-related actions on security, economic development, governance, and the provision of justice. However, sustained research will be needed to improve our understanding of alternative approaches to this problem, as well as the associated challenge of reconciling local identity and governance concerns with an increasingly globalized development process. Another important sub-theme concerns how to better utilize and protect various aspects of the global commons in a growing and rapidly urbanizing world. While globalization has been a potent force for development, the crisis has also exposed the fragility of growth to policy failures and spillover effects among nations in relation to natural resources and the environment. Economic growth also places increasing pressure on common resources. Given demographic imbalances between countries, and the threat of larger-scale environmental disruptions in the future, migration also looms as a growing issue. How should global arrangements be reformed to ensure that these challenges are widely seen as important and legitimate, and that international responses provide a robust and supportive framework for sustainable development? Ensuring markets and financial flows is critical but we must also look beyond these. How can developing countries better mitigate and adapt to the longer-term effects of climate change, and how can the prospects for rapid economic progress without unmanageable climate change be enhanced? Widespread instances of water scarcity, depletion of fisheries, deforestation, and loss of biodiversity pose similarly serious challenges, even in the nearer-term. With rapid urbanization expected to continue, moreover, what urban development initiatives are most important in terms of infrastructure expansion, increased energy efficiency, and improved resilience? 27 New challenges can also be expected for food security. Demand for food is expected to grow substantially over the next half-century in response to population growth and rising incomes. Agricultural productivity growth is needed, given constraints on the use of additional land or water, but productivity growth appears to have slowed, in part because of past cutbacks in support for public agricultural research and development. Further sharp spikes in food prices, as in 2007-8, are expected when adverse shocks coincide with low stock levels. Climate change also seems likely to increase the frequency of adverse shocks. Fourth, a broader approach is needed to the task of assessing development effectiveness. The development community is increasingly focused on results. How should evaluations be conducted and how can delivery mechanisms be improved to generate better results? What innovative roles can public-private partnerships provide? How can development loans and aid avoid undermining accountability mechanisms in recipient countries, and promote rather than discourage policy reform? Aid and loans, whether stemming from public or private sources, are not the main drivers of development success. The dominant role has to be played by the populations and governments in the countries concerned. This controversial topic is of particular interest for the development community, including of course the World Bank. At the macro level, research has long had an important role in creating the tools, especially the data, needed for monitoring overall development effectiveness at country and sub- national levels. (For example, the World Bank's global poverty monitoring effort has been housed in research units since it began in 1990.) Our ability to learn from country experiences rests heavily on the quality of that data. There is still much work to be done in improving these data, such as in reducing comparability problems, understanding data inconsistencies, reducing biases in data collection and better linking across types of data. At the more micro level, credible evaluations of development policies and projects (both ex ante and ex post) are accompanied by continuing methodological and data-related challenges. Methods will vary depending on the nature of the interventions that are being studied. There have been important advances in the use of both experimental and non-experimental evaluation methods for assigned programs (meaning that some observational units get the programs and some do not). However, these tools can be misleading for informing development policy 28 (including scaling up) and these tools are not appropriate for many of the most important questions we face in development policy making, where the intervention is not assigned or its assignment entails large spillover effects. A wide range of careful ex-post impact evaluations have been launched by DEC, examining a multiplicity of interventions. The Bank's special role is in evaluations with governments, which poses special opportunities and constraints (such as concerning what methods are politically feasible). A distinctive feature of the Bank's evaluative research is that linkages between the specific intervention and the rest of the (public and private) service system get extra attention. There has been less progress on ex-ante assessment of options, including particularly the application of cost-benefit analysis (CBA) and related approaches to comparing the impacts of different policies or investments. According to a recent IEG report, the percentage of projects at the World Bank where ex-ante cost-benefit analysis was performed dropped from 70 percent to 25 percent between 1970 and 2008.18 While half of that decline is due to a change in the composition of projects, which have moved towards sectors where the World Bank has not traditionally done CBA, the IEG report highlights the important challenges in moving towards a more comprehensive framework in assessing aid both ex ante and ex post. The retreat from CBA amongst development agencies has stemmed in part from a continuing shortage of information about key inputs to CBA, notably for quantifying and valuing benefits. For example, we still know rather little about the implicit prices ("shadow prices and wages") that should guide project choice in poor countries. But the IEG report also indicates a deeper set of institutional questions.19 There has been an expansion in ex-post evaluations that can help fill knowledge gaps, though in practice it remains unclear how well this evaluative effort is matching up with the knowledge gaps or how much the effort informs subsequent 18 See Cost-Benefit Analysis in World Bank Projects, Independent Evaluation Group, World Bank, Washington DC, 2010. 19 Internal incentives and resource constraints may increase the emphasis placed on speed of project preparation, and increase the perceived risk of subjecting judgments of project developers and preferences of borrowers to some independent quantitative review, which is the essence of what CBA can do. The fact that project appraisal occurs only after a project is developed limits the usefulness of CBA as a more strategic (albeit not perfect) tool for economic analysis of alternatives. 29 actions.20 Moreover, while ex post evaluations can inform the degree to which an intervention was effective in achieving its desired aims, only a comparison across options can provide information on the relative cost-effectiveness of options conditional on their effectiveness. A menu of evaluation approaches, which include ex-ante CBA, impact evaluation, and other ex- post methods, can contribute to assessing both aid effectiveness and cost-effectiveness of various interventions.21 In the longer term, there are important challenges associated with assessing aid in a more comprehensive and systematic way, both at the World Bank and elsewhere. Looking forward, the task of addressing the most pressing knowledge gaps facing us today will require that researchers remain open on the most appropriate theoretical starting points and empirical methodologies; it is the policy question that should drive the research agenda, and not the researcher's disciplinary background or favorite method. To make greater progress in filling the persistent knowledge gaps about development effectiveness, we will need to draw on a broader set of evaluative tools and on insights from a wider range of disciplines. Implementation in a Changing World Rethinking our research agenda and setting development priorities along the above lines are only the first stage in transforming the role of research at the World Bank. The next stage is to ensure that new research is implemented and that the right lessons are drawn for knowledge and policy making. Ultimately the most important measure of the relevance of development research is whether it leads to better policies. The 40 or more years of the Bank's research efforts hold enumerable examples and results of how such efforts can help inform policy making. Of course, not every single piece of research needs to pass the test of immediate policy relevance. By its very nature, "blue sky" research may have little immediate policy impact, though well chosen topics for such research can still mean that it plays an important role in ongoing policy debates. Given the combination of global operational engagement cutting across many sectors, it 20 For further discussion of these issues, see: Martin Ravallion, "Evaluation in the Practice of Development." World Bank Research Observer 24 (1): 29-54, 2009. 21 The complementarity of approaches needs to be emphasized. The Bank is both a development institution and a fiduciary institution. As such it needs to assess both the effectiveness of its actions and the relative cost- effectiveness of different approaches for obtaining desired results. 30 is essential to sustain the role of the Bank as an intellectual leader in the development field. This means being in a position where it can help identify major emerging issues, both of a global and country-level nature, and play a catalyzing role in the intellectual debate. Toward that end, the Bank must and will continue to maintain a high-quality in-house research capability, aiming to attract the most talented researchers to the task of tackling important development problems. Analytical work is found throughout the Bank, as appropriate to the needs of specific units. The Bank's research department is, however, unusual in that its raison d'être is research, although its staff spend a significant part of their time providing advice and support to other Bank units. The Bank provides its researchers with a unique vantage point on development. The experience of working with colleagues in operations and networks, and partners in government and civil society, helps the Bank's researchers identify key gaps in knowledge, generates awareness of emerging issues, and provides the testing grounds for new ideas and tools. It also provides powerful opportunities for disseminating research findings and insights and supporting their translation into concrete actions. It is no accident that many topics in Bank research are inspired by the 30 percent of their time that researchers provide to operational activities. The Bank's in-house research capability also helps it play a role in helping to draw lessons of more general validity from scattered individual micro-assessments of the effectiveness of development projects. These need not be Bank projects; neither need the studies be done solely by Bank researchers. Yet with proper engagement and planning, the Bank's researchers can help to facilitate rigorous comparative analysis across a set of comparable interventions, to better understand the institutional and other conditions under which a particular approach will yield results. The Bank's research agenda needs to shift even further toward a focus on the results of development policies and projects. As mentioned earlier, this will require gathering more evidence and data to assess the effectiveness of development efforts, including aid. It will also require a broader range of analytic tools. The assessment of results in development also needs much greater attention from the economics profession as a whole. 31 This research agenda and the knowledge more broadly generated can benefit developed as well as developing countries. Research on the more effective and efficient provision of public services, public-private models, and risk management for natural disasters is likely to yield important policy implications for all countries. The World Bank Group is a cooperative and the benefits of this broader knowledge agenda could accrue to all its members. For example, there are likely to be valuable lessons for higher income countries from experience with conditional cash transfers in Mexico, investment in infrastructure and green technologies in China, and pension reform in Chile. While the Bank needs to maintain a strong in-house capability for relevant research, it also needs to recognize that there are some important ways in which the world has changed in the 40 years it has been doing development research. There are changes in the set of partners-- the connected people and institutions also doing research on development--and changes in the technology and portability of the tools of research, coming alongside the enhanced capabilities for research of the Bank's partners. To stay relevant the Bank's model of how research is done needs to adapt to these changes. Global Research Partnerships: The World Bank is only one institution in the global context, and the vast majority of researchers working on development economics are not affiliated with the Bank.22 A great deal of new development thinking and experience in the wider academic and development community is now found in the developing countries themselves. Outside experts and centers of excellence are prominent in many areas, and this has increased the diversity of sources or advice to which developing countries can turn. We see this as a plus for our clients and for development in general. The World Bank must expand its scope to lead and leverage within the global research community, but also to learn and collaborate. Establishing and strengthening partnerships with developing country researchers and institutes will help accomplish this goal. The World Bank can play an important role in connecting the different nodes of knowledge through its various platforms, including DEC, the World Bank Institute (WBI), and the World Bank networks. 22 For example, there are almost 1900 authors of research papers classified as "development economics" in RePEc, while there are 240 authors of economics papers affiliated with the World Bank. 32 These networks include the Poverty Reduction and Economic Management Network, the Sustainable Development Network, the Human Development Network, and the Financial and Private Sector Development Network. These networks play a critical role in absorbing and synthesizing research. WBI and the networks play a particularly critical role as a connector and disseminator of practitioner knowledge and experience. The work of talented in-house researchers, knowledgeable about their fields, also facilitates the Bank's convening role as a connector and disseminator of research produced in developing countries and outside the institution. There is, in fact, a fair degree of engagement between the Bank and outside researchers and research institutions to build on: The most financially significant institutional partnership and one of the oldest is that with the Consultative Group on International Agricultural Research (CGIAR). The Bank has been a founding member of several prominent research institutions and networks in developing countries, including the African Economic Research Consortium (AERC), the Economic Research Forum (ERF) for the Arab Countries, Iran and Turkey, and a number of institutions in Central and Eastern Europe and the CIS, including the New Economic School (NES) in Moscow, and others. More recently, the Bank has encouraged the building of research capacity in developing countries through the Global Development Network (GDN). Bank researchers work with consultants from some 90 different countries, providing many of these with opportunities for networking and engagement with peers worldwide. Bank research is supported also by a number of development partners, and this provides further opportunities for intellectual exchange. Yet more can and should be done. In countries and regions where adequate (or better) research capacity exists the Bank should be able to leverage its strength and that of developing country research institutions where researchers are closer to the ground. They will understand more clearly the reality of their development processes, the development priorities of their governments and the underlying political economy that shapes policies. The Bank should also 33 revive, in collaboration with other partners, its support for building research capacity in developing countries and sub-regions where such capacity is still weak or non-existent. Working with development partners, the Bank needs to give greater weight to their views on priority areas for research, and then shape its own programs to enable good proposals to benefit from a global comparative perspective and methodology that facilitates cross-country learning. This is the pattern, for example, in the ongoing study on Peace and Development, a multi-disciplinary study that brings together case material from some 20 countries. A recent research initiative with the Economic Research Forum develops several thematic studies of mutual interest to countries in the Middle East and North Africa, such as regional economic integration, equity and inequality, and the management of natural resources and economic diversification, which can also benefit from a more global perspective. The Bank needs to find ways of engaging more actively in researcher exchanges with developing regions. We need to use programs such as the Visiting Research Fellows Program to increase the diversity of our research engagements. Also important are new opportunities for research staff to be able to spend some longer periods in the field, whether as part of exchanges with other research institutions or through well-designed development assignments to address particular issues of importance to both operations and the research program. The assets of researchers in developing countries make them vital partners. Their knowledge of development challenges at local levels complements the that of the Bank's researchers. This is especially important in addressing political economy issues. Researchers in developing countries also can play a vital role in the task of identifying, collecting, and using data, such as the research department's LSMS household datasets and enterprise survey datasets, which appear to be under-utilized by developing country researchers and policy makers. Importantly, researchers in developing countries are on the "front line" in identifying key development challenges and are uniquely placed to translate new research findings into effective policies in a local context. Following our comparative advantage in access to experiences across countries in the world, the production of high quality, rigorous empirical research (including development data and analytical tools) and dissemination of the findings in an accessible and impartial form is 34 what will position the Bank as a leader in development research and thinking--in a world where we act as a catalyst and global connector for new ideas and can influence development debates because of the quality, diversity and relevance of what we produce. Dissemination and Knowledge Management: Just as there are opportunities to improve the Bank's research functions, we must find more effective ways to disseminate the high quality research we produce jointly with our partners, and make it accessible to policymakers and the development community. This need is in no way unique to research at the Bank, but it is nonetheless of singular importance for this "product line." While part of the challenge is that internal and external non-research "customers" can be uninterested in (or even suspicious of) research, or are simply too pressed by their other responsibilities to accord it greater attention, there is more that researchers and Bank managers can do as well. The management of the research department is putting increased emphasis on effective non-academic dissemination and application of research in addition to the research product itself and its validation by peers. Managers in other parts of the Bank also can contribute through the quality standards they set for knowledge products, sharing of information on priority knowledge gaps, and efforts to promote greater unit-level as well as project-level engagement with DEC on filling those needs. The Bank has recognized that greater access to knowledge and greater effectiveness in the use of knowledge is necessary, if not sufficient, to improvement its own results-effectiveness in the face of constraints on financial and human resources. This has led to the consideration of numerous initiatives for improving internal knowledge management--making analytical and experience-based knowledge more readily accessible across the Bank--and for stimulating new inflows of such knowledge from outside experts. Maintaining and increasing the Bank's research capacity, and further increasing its effectiveness and linkages to various partners, is a critical complement to those efforts. It is through such measures that the Bank can gain additional insights on what interventions have worked better than others, and what additional issues need to be studied to effectively address lingering conundrums. Open Data, Open Knowledge and Open Solutions: So far this discussion has been premised on the long-standing "retailing model" for research. This entails that researchers investigate a specific issue over a period and produce a research product--a paper or volume-- 35 on their findings. This is then disseminated to the public, including to other researchers and policy makers. As has been noted, the Bank has long been a world leader in producing such retail products on development. However, many things have changed since this retailing model first emerged. As was also noted, while the Bank remains a leading institution in research on development economics, it is only one institution in the global context, and the vast majority of researchers working on development economics are not affiliated with the Bank. The public has also become more demanding of openness and transparency from institutions such as the World Bank. And how we produce and manage knowledge has changed. Dramatic changes in our information technology have expanded the types of data that can be brought to bear on any question, and made openness in answering that question more feasible than ever before. However, in keeping with much academic research, the Bank's analytic work (in both research and operations) often lacks sufficient transparency--not least among those who would be affected most by the policies derived from those analyses. Civil society groups have been suspicious that advocacy was being dressed up as analytics, given that there can be large barriers and costs to understanding and evaluating analysis. It is time then to develop a supplementary "wholesaling model," in which the emphasis is on producing the tools for others to do the research themselves and providing open access to those tools. There are three objectives for such an initiative. This first is to empower researchers in developing-countries to do better research for informing development policy and development practice. This switches the focus of the traditional "capacity-building" model from the task of "teaching the lessons from past research" to facilitating new learning in specific contexts. Success in this endeavor will also help in a second, longer-term, goal of greatly expanding opportunities for a "collaborative retailing model" in which Bank staff can work more closely with colleagues in developing countries as full peers, to the benefit of both. The third goal is to help assure a more open and transparent policy analysis. The discipline of policy analysis (particularly analysis rooted in economics) is increasingly sophisticated in its theories and methods. There are still long lags between the introduction of 36 new theories and methods on the one hand, and their application to real-world problems on the other. There is a further lag between application of new research approaches to gain additional understanding, and the translation of such understanding into operational terms. The Bank can play an important role in reducing the costs of increased understanding and its application, even with the most sophisticated policy analysis, if technical capabilities also increase among key stakeholders. Through innovative uses of technology, DEC is particularly well positioned to help broaden access to data, information, and experience, so as to better inform development policy debates and to better engage the direct stakeholders in those debates. Building on the Open Data Initiative, the Bank will need to become more deeply involved in producing tools to facilitate better development analysis and policy making by non- Bank researchers. These tools include development data, in particular survey data on households, enterprises and facilities using questionnaires that are geared towards addressing the most important development issues. They also embrace software tools for accessing and using existing data. The aforementioned data tool PovcalNet is an example. Another can be found in a new product of the Bank's research department, ADePT. This is an innovative software program designed to simplify and speed-up the production of standardized tables and graphs in many areas of economic analysis, focusing particularly on the Bank's analytic work at country and regional levels. ADePT is a free, stand-alone program, available for download to anyone in the world. Another example is the Apps for Development Competition, which aims to bring together the developer and the development communities to create innovative tools, applications, and mashups. Such tools allow practitioners in developing countries to easily access the most powerful analytic methods available. Looking ahead, the challenge is in combining more effective "retail research"-- addressing the key knowledge gaps emerging from development practice--with well-chosen applications of this new wholesale model for how we do research. New research products outside the Bank will also start to emerge from open access to the Bank's data and analytic tools. Our vision is that the data, knowledge and tools that provide the solutions to development problems will ultimately be generated collaboratively through active peer-to-peer participation by those who have the most to gain from the success of those solutions. 37