Learning What Works

Children eating in a kindergarten in Brazil.

A new push for evidence on the impacts of development programs is prompting careful assessment of what works and under which conditions, potentially resulting in more effective interventions for poor people.

In recent years, many development program coordinators have been asked to provide something they have rarely needed to offer in the past: empirical evidence that their programs work.

Although some fields—especially medicine—have long embraced the habit of evaluating effectiveness, rigorous impact evaluation is relatively new in international development. When the International Initiative for Impact Evaluation (3ie) got underway in 2009, Executive Director Howard White said, “Each year billions of dollars are spent on development programs with scant evidence on whether those programs are having any impact at all on poverty, child mortality, getting girls to school, and so on. Policymakers simply don’t know the best use of their resources.”

If evaluating impact—that is, attributing a particular result to a particular program or policy—were easy, it probably would have been commonplace long ago. In fact, conducting high-quality evaluations faces a host of political, financial, and technical obstacles. Nonetheless, aid donors and governments have shown new interest in acquiring solid evidence of the effectiveness of development programs. “Impact evaluation has received more attention in the past decade since the Millennium Development Goals were set,” says Mywish Maredia, an associate professor at Michigan State University. “There has been a push for evidence-based policy.” This new focus has resulted in a number of careful evaluations that have shown definitively whether programs were working and how they could be improved.

Outputs, outcomes, and impact It is helpful to be able to distinguish between different kinds of results when discussing impact evaluation. The terms “outputs,” “outcomes,” and “impact” are not always used consistently in the literature, and the distinctions between them can be fuzzy in some cases. Typically, economists use “outputs” to mean the goods and services generated by a program (for example, bags of seeds delivered to farmers); “outcomes” to mean stakeholders’ use of and satisfaction with these goods and services (for example, the rates at which farmers use the seeds); and “impact” to mean the overall effect on people’s living standards (for example, higher incomes or greater food security).

The “How To” of Agricultural Transformation

Phase Two of the Ghana Strategy Support Program

Shashi L. Kolavalli

These are particularly challenging times for African countries like Ghana that are seeking to develop their agriculture sectors. After decades of neglect, there is increasing interest in investing in agriculture, fueled initially by the food crisis and sustained by the enthusiasm surrounding the Comprehensive Africa Agriculture Development Program (CAADP). How to achieve agricultural growth, however, continues to be a puzzle; what to invest in is the key question. The attention of policymakers and donors appears to have shifted from developing pro-agriculture policies and strategies to developing investment plans that can deliver the desired growth.

What should the Ghanaian Ministry of Food and Agriculture be doing to facilitate private investments? How should it work with the private sector? Should it be subsidizing fertilizers, pursuing block farming, or setting up grain reserves? If so, how? In addition to these important issues there are concerns about the capacity of ministries and in-the country administrative systems to make effective use of available resources. To some extent, divergent thought on these issues is what holds back the alignment of country and donor policies and, thereby, greater aid effectiveness.

While the focus continues to be on private-sector led growth in agriculture, two noteworthy approaches have emerged in Ghana and elsewhere. The first is a gradual shift away from making general improvements in the way input and output markets function and, instead, improving specific value chains through the delivery of marked development interventions (that is, anchoring interventions around improving the competitiveness of certain commodities). The second notable approach is a subtle emphasis on commercial agriculture: in addition to linking smallholders to markets, this type of agricultural development seeks to combine aspects of large-scale farming and foreign direct investments with expectations of benefiting from the transfer of technologies and capabilities from elsewhere. All of this entails strategic coordination with the private sector, which has begun to happen.

In practice, value chain development, which embodies market and private-sector development, goes beyond merely seeking to overcome market failures and instead aims to transform relationships between economic agents. The assumption behind this approach is that, in the absence of trust, agents interacting with each other at arms length relationships, in a competitive environment are driven by opportunistic behavior. Value chain development also recognizes the existence of coordination failures between agents. The objective in further developing value chains therefore is to foster cooperation between agents that will ultimately create mutually beneficial commercial relationships, which are inevitably superior to what they are under competitive behavior. Value chain development often entails identifying and supporting an agent in the chain who is in a position to influence other agents to upgrade their operations and likewise down the line through mutually beneficial relationships. A key expectation is that such relationships would make it feasible to develop smallholders in a private-sector led approach to develop agriculture.

Value chain development entails risks that are usually associated with industrial policy—picking winners and risking rent-seeking behavior—but it can be an effective tool for strategic targeting of regions and households. Countries like Ghana—where IFPRI has had a strategic support program since 2005—are also targeting commodities and products for import substitution. The approach is also a clearer and an aggressive way to develop private sector as it goes beyond merely providing a facilitating environment. Value chain analyses can offer simple diagnostics of binding constraints and potential investment opportunities. The interventions that would emerge from such analyses would include the usual public investments and infrastructure development in addition to direct support to private entities.

The Ghana Strategic Support Program, a research and capacity-strengthening initiative, generates knowledge with the aim of improving agricultural and rural development research, decision making, policies, and strategies. After having supported policymaking with strategic analysis during the first phase of the Ghana Strategy Support Program—focusing on understanding the structure and linkages of Ghana’s economy, the nature of growth required to meet the developmental objectives, and the returns to public expenditures, in addition to governance of delivery of key services—the ongoing second phase is designed to answer the “how to” of agricultural transformation focusing on the emerging approaches. Stakeholders felt that previous analysis did not go far enough to identify binding constraints, investment opportunities, or make ex ante assessments of potential options. A lack of availability of data, particularly with adequate levels of disaggregation, limited the capability of strategic research from being adequately supportive of decisions to develop investment plans.