Putting Poverty and Hunger Solutions to the Test

Angelino Viceisza, a researcher at the International Food Policy Research Institute, explains the trust game to Solomon, who translates it to the participants.

Development researchers are applying innovative experimental methods and new technologies to the study of poverty and hunger. Can these new approaches help point the way to more effective policies and programs?

Livestock are central to the livelihoods of the Maasai people of East Africa, and if they could market the livestock products they produce, they could greatly improve their quality of life. But getting their products to market can be difficult. One promising innovation has been the introduction of contract farming. In Tanzania, Maasai milk producers have signed contracts with milk processors, committing themselves to deliver a certain quantity of milk on a certain date. Such contracts offer milk producers a number of advantages: contracts give farmers an assured market for their milk at a prearranged price; farmers benefit from reduced marketing costs; and the milk processors often provide advice and technologies for producing more and better-quality milk. Milk processors also benefit from the contractually assured supply and price of milk. It’s a textbook win-win solution.

And it would probably work—if real people behaved the way people do in textbooks.

Instead, however, the Maasai producers often fail to deliver the milk as promised. If the price of milk in the local market is higher than the price specified in the contract, many producers sell the milk there to benefit from the highest price. It is hard for them to believe that the price offered by the milk processor is fair when they see the market price surpassing the contract price. This situation frustrates the milk processors who are expecting the milk deliveries. It also threatens the whole contracting arrangement and could ultimately result in a riskier situation for the producers—no market at all. The milk processors face a dilemma: Should they try new arrangements to encourage consistent milk delivery? Should they reward loyal milk producers with money, household goods, or goods and services to help improve livestock production? Should they direct incentives to the Maasai men who own the livestock or to the Maasai women who manage the milk production?

To help answer these questions, IFPRI researchers are drawing on the tenets of experimental economics—a growing branch of economics that uses experiments to test economic theories and better understand the working of markets. By conducting experiments, researchers can isolate the effects of alternative institutions and incentives on individual behavior. “We want to understand and measure how people behave under certain conditions,” says Maximo Torero, director of IFPRI’s Markets, Trade, and Institutions Division. “Experimental economics allows us to clearly identify a causal link.”

In Tanzania, researchers are now using the IFPRI Mobile Experimental Economics Laboratory (IMEEL) to test different versions of contracts, and the final results should point to the type that has the best chance of succeeding. The IMEEL is not only a set of equipment like computers and handheld devices, but also a set of experimental techniques. These techniques are becoming increasingly common in academic settings, in international institutions, and in developing-country policy circles. The hope is that the knowledge gleaned will lead to better strategies, policies, and programs for fighting poverty and hunger.

“Often we have sharply conflicting theories about the same behavior,” says Glenn Harrison, C.V. Starr Chair of Risk Management and Insurance at Georgia State University. “Experiments make it easier to tease apart one theory versus another theory. Depending on which theory is at work, you’ll design different policies.”

Taking Experiments from the Lab to the Field

It was once thought that economics could be studied only through observation of social systems, not through experiments. But in the mid-20th century, economists drew on practices used in psychology to come up with ways to apply experimental methods to the study of people’s economic behavior. They set up “games” in which participants (usually university students, who were handy subjects for university economists) made choices based on a treatment and a control. “Experimental economics started as theory testing,” says Ragan Petrie, associate professor at George Mason University. “Say we have a theory, but we haven’t seen it in action. So in an experiment we ask ‘Are people behaving as my theory would predict? If not, what accounts for that?’” This controlled variation allowed economists to test theories in ways that complemented the more traditional economic research tools, like models and surveys. Laboratory experiments gave researchers more control over the participants’ environment than they would have in real life. By, for example, changing one variable to see how people’s behavior changed, they could more accurately identify cause and effect.

Although these lab results generated important information, they were not necessarily transferable to environments outside the lab. A 20-year-old American student may behave quite differently from a 40-year-old Chinese farmer or a 60-year-old Kenyan grandmother. Economists thus began bringing experiments into the field. If the research question concerns Ethiopian farmers, for example, then the subjects of the experiment are actual Ethiopian farmers engaging in activities that reflect their real-life concerns. Field experiments allow researchers to capture the salient aspects of the decisionmaking environment in a somewhat simplified way and to quickly and easily find out people’s decisions.

Understanding How People Behave in the Face of Risk

Angelino Viceisza, a postdoctoral fellow at IFPRI, and Ruth Vargas Hill, an IFPRI research fellow, have applied these techniques to study Ethiopian farmers’ use of fertilizer. It is clear that farmers in Ethiopia could boost their crop yields by applying more fertilizer—but doing so is risky. If the weather is good, then the money farmers spend on fertilizer will turn out to be a profitable investment. But if the rains fail—not an uncommon occurrence in Ethiopia—farmers will lose not only their crops, but also the money they spent on fertilizer. One way to help farmers produce more while reducing their risk would be to offer weather insurance—if the weather is bad, insurance will compensate farmers for their losses. But will Ethiopian farmers go for such a scheme? Would weather insurance encourage them to buy more fertilizer? It’s hard to know. Weather insurance is not currently widely offered in Ethiopia, and many farmers don’t even know what it is.

To study the issue, the IFPRI team carried out an experiment with more than 250 farmers in southern Ethiopia. The experiment consisted of two games. In both games, farmers first had to decide whether to buy fertilizer. Farmers then faced a weather scenario—good rains or poor rains—that determined whether the harvest was abundant or meager. At the end of each round of these games, farmers collected their income—real money that they could keep—from this “harvest.” Then, in one of the games, researchers changed one variable: farmers were given crop insurance. Would this change affect their willingness to fork out money for fertilizer? In other words, would the presence of insurance lead farmers to take greater risks by purchasing fertilizer? The experiment showed that farmers with insurance were indeed likely to buy more fertilizer than those without—especially if they had experienced good weather and a large harvest in preceding rounds of the game or had showed a strong understanding of the insurance contract. The results suggest that crop insurance could be a useful way to help reduce risk and push up crop yields through fertilizer use in Ethiopia. This experiment was followed by a pilot program in which the farmers were given money that they could choose to keep or exchange for insurance. Just over half of them chose to purchase the insurance. Now an Ethiopian insurance company is preparing to provide insurance on a market basis for these farmers and others in nearby villages.

IFPRI’s Ethiopian research shows how experiments can be used to measure people’s willingness to take risks—an important characteristic that can mean the difference between policy success and failure. “People in the developing world live with a great deal of risk, and it’s not just financial risk. If you perturb their income a bit, they collapse. A change in income can dramatically change their health and the health of their family,” says Glenn Harrison from Georgia State University. Policies need to take this reality into account. For example, people living in poverty may prefer a policy that results in a slightly lower but more stable income over one that increases their average income but makes it more variable.

Experiments are a useful way to measure these kinds of preferences and to help answer other important questions confronting policymakers. If an agricultural program makes improved seeds available, will farmers be willing to try growing new crops that could bring higher profits? Will people take advantage of credit programs to borrow money and start small businesses? If a government starts a new program to promote girls’ education, will parents send their daughters?

Studying Firms and Farmers

The Tanzanian milk production study forms part of a recent trend in which private companies are using experimental techniques, with the help of researchers from IFPRI and other institutions, to test systems for dealing with farmers in developing countries. The potential for the private sector to help improve people’s lives and livelihoods in developing countries has often been touted, but private schemes for strengthening markets can go awry if they fail to account for people’s real-life attitudes and behaviors. Experimental testing of different options can help lay the groundwork for successful partnerships that will benefit both farmers and firms.

Like the contract farming scheme in Tanzania, a similar arrangement in Vietnam has also been veering off the tracks, but for different reasons. The milk-processing company tests the milk supplied by contract farmers to make sure it meets certain quality standards. The farmers, however, distrust the results reported by the milk company and believe they are being cheated. IFPRI researchers have set up an experiment that involves giving some farmers a voucher to get the milk tested by an external laboratory when they believe the milk company has cheated them. The researchers also decided to delve deeper into the issue, conducting experiments to test whether the success or failure of the voucher system varies with farmers’ trust attitudes. The results of these experiments should show whether the voucher option will work and under what conditions, or whether the company needs to go back to the drawing board.

Experimenting with Social Programs

Experimental techniques have also revolutionized how policymakers and researchers answer the question, how do you know if a program is working? For decades, it was common to measure the impact of a government program by reporting on the inputs poured into the program: how much money was spent, how many bed nets were distributed, how many school lunches were served, and so on. But in 1997 a landmark experiment in social policy began when the government of Mexico launched the Programa de Educación, Salud y Alimentación, or PROGRESA. The program itself was innovative in the way it linked social services together. PROGRESA gives families cash if their children regularly attend school and use the services of health clinics. It also gives health benefits and nutritional supplements to preschool children and pregnant and lactating mothers. The idea is to improve the educational, health, and nutritional status of poor families, particularly children and their mothers.

Mexico’s tight budget situation prevented it from rolling out the program nationwide all at once, so it took advantage of this constraint to see how introducing the program to a relatively small number of people affected their educational, health, and nutritional status compared with that of similar people who were not in the program but who would be enrolled later. This phased approach, with those starting earlier and later both selected randomly, meant that as it rolled out the program could be evaluated using experimental methods. “Mexico had a political commitment to a rigorous evaluation of the program,” says IFPRI senior research fellow John Hoddinott.

From 1998 to 2000, IFPRI evaluated this experiment to see how well it was working, examining measurable changes in participants’ education and health. Its findings were instructive. PROGRESA helped increase enrollment in high school girls by 20 percent and high school boys by 10 percent. Preschool children enrolled in PROGRESA had a 12 percent lower incidence of illness than non-participating children. PROGRESA also increased families’ visits to clinics and their use of prenatal health care, and raised their spending on food and their calorie consumption.

But this experiment also revealed areas for improvement in PROGRESA (now renamed Oportunidades). For example, IFPRI’s evaluation led Oportunidades to link benefits to high school attendance, rather than stopping at the junior high level, and to tie benefit to school performance, not just attendance.

PROGRESA/Oportunidades has served as a model for dozens of other conditional cash transfer programs in developing countries, including in Cambodia, Honduras, and Turkey. Similar programs have adopted not only the careful linking of benefits to education, health, and nutrition behaviors, but also the experimental design that allows for continual evaluation and improvement.

Smaller-scale experiments have also identified ways making social programs more effective. In many developing countries, attempts to improve children’s school attendance and performance have run up against a common problem—the teachers don’t show up for work. In 2006, Michael Kremer, an affiliate at the Poverty Action Lab at the Massachusetts Institute of Technology (MIT), and his coauthors conducted a survey of six countries—Bangladesh, Ecuador, India, Indonesia, Peru, and Uganda—showing that teachers were absent an average of 19 percent of the time, with serious implications for children’s learning. In the mid-2000s, the Poverty Action Lab worked with Seva Mandir, an Indian nongovernmental organization (NGO), to determine the best method of reducing teacher absenteeism at the organization’s schools. Researchers divided 120 Seva Mandir schools in Udaipur, India, into a control group and an incentive group. Teachers in the control group, who were offered no special incentive for better attendance, were paid the usual $23 a month. Teachers in the incentive group were paid $11.50 a month plus $1.15 for each day they were present. Every day the teachers in this group had to take two photos of themselves with students, five hours apart, using a camera with a tamper-proof date and time stamp. Absenteeism in the incentive group fell by half, and after one year students in that group posted much better test scores. “A big hurdle is just getting teachers to show up,” says Iqbal Dhaliwal, director of policy at the Poverty Action Lab. “Once they show up, the question is, what do they do? They are as likely to teach as a teacher in the control group.”

Embracing New Technologies Research-based policies are only as good as the data underlying the research. Untimely data, incompletely or carelessly filled-out surveys, and hard-to-read handwriting can all obscure researchers’ view of how people are behaving and why. To improve research results, researchers at IFPRI and elsewhere are using new technologies to conduct experiments and other kinds of research. For example, the IMEEL uses portable computers and handheld devices that can be carried into the field. These devices can help raise the accuracy of participants’ responses to questions or situations. IFPRI is also using global positioning systems (GPS) to help identify the exact location of rural households that participate in traditional surveys. This information can tell researchers, for example, how far a farmer is from a market town and help explain why he or she behaves in a certain way. In several Peruvian schools, IFPRI researchers have undertaken an experiment, incorporating new technologies, to see if children can help change their parents’ behavior in ways that improve children’s health. Researchers installed Internet connections in the schools and sent messages to children instructing them to go to the doctor and get iron pills. To test whether these messages, transmitted to parents from their schoolchildren, actually raise children’s blood iron, researchers have set up a series of cognitive aptitude tests for the kids using the Nintendo Wii video game. “The Wii is easy and interesting for kids,” says Torero, “which solves the problem of their being bored when responding to traditional questionnaires and therefore improves the quality of the information collected.” These high-tech tools can give faster and more accurate results than other more traditional research tools, like pencil and paper, says Eduardo Maruyama, an IFPRI postdoctoral fellow. But it is not always appropriate to use computers and other technologies in the field. The decision depends on the research question being asked, the nature of the experiment, and the technical facility of the participants in the experiment.

Perfecting Experimentation

Of course, not all policies related to development are amenable to testing through experiments. As Glenn Harrison points out, it is not possible to conduct an experiment, with a treatment and a control, for a macroeconomic policy or a trade policy, yet policymakers need to move forward with such policies, finding other ways to design and evaluate them.

Critics note that the results of experiments may not be generalizable. In other words, the outcomes of a field experiment based on a hypothetical scenario, like the weather insurance experiment in Ethiopia, may not carry over into real-world settings, or the results of an experiment in Mexico may not be applicable in India. Of course, this problem with generalizability applies to other research methods as well. To avoid the risk of generating overly broad or misleading conclusions, it is important to present results carefully.

There is still much to be gained from “experimenting with experiments,” says Viceisza. For instance, questions remain about how best to design experiments. How should research questions be framed to get the most accurate results? What if a treatment cannot be randomly assigned? What if the effects of a treatment spill over into a control group? More broadly, how will more alliances between researchers and public or private enterprises influence development policies and programs?

Yet at a time when many governments and aid donors are confronting a situation of limited funds, experiments can identify which programs and policies are worth spending money on by showing what works—and what doesn’t work. “Often policymakers can’t make huge changes, but they can tweak policies to make them more effective,” says Dhaliwal. “Experimental economics can help identify programs that give you a bigger bang for your buck.”

—Reported by Heidi Fritschel