Zap It to Me: First Randomized Trial of Micro-transfers

In 2009, I blogged the first two randomized trials of microcredit and the first of microsavings. Now comes what is in a sense the first randomized test of the impact of a phone-based micro-transfer service. The study is reported in a new CGD working paper by Jenny Aker, who once resided here as a post-doctoral fellow and is now a non-resident fellow based at Tufts University. Her coauthors are Rachid Boumnijel, Amanda McClelland, and Niall Tierney, who all work for Concern Worldwide, the charity that conducted the experiment.

Don't think of this is a broad test of what happens to poor people when a mobile money service arrives in their lives. The intervention tested is quite specific. Facing a severe drought in Niger last year, Concern decided to distribute cash to women in particularly vulnerable families. With all the excitement about M-PESA in Kenya, and the arrival in Niger of an M-PESA rival called Zap, people at Concern wondered whether they could use phones to get money to people more cheaply or effectively. To find out, they teamed up with Jenny, whose CV includes both an economics Ph.D. and years of experience managing programs for Catholic Relief Services in western and central Africa. Together, they decided to compare two ways of delivering money: the old way, in envelopes doled out from armored trucks as designated stops, and the new way.

Since most intended recipients lacked phones, Concern had to distribute those too before they could "Zap" money to them. For the research, that raised the challenge of distinguishing the impacts of getting a phone from the impacts of getting cash over a phone. After all, a new way to communicate over distance could change one's life in many ways. So Concern created a third experimental group, a "placebo" group, whose women got free phones but still had to meet the armored truck to get their money. Assignment of subjects among the three groups was random, by village. Notice that, as a matter of humanitarian necessity, women in all the groups received money. Since all the study's results come from comparisons between families receiving money, the study does not measure the impact of cash transfer per se, only the delivery method.

In all, the experiment delivered 110,000 CFA ($215) each to 1,2000 women in 96 villages during the five-month hungry season.

Among the findings:

• Zapping money is cheaper. Direct cash delivery cost $12.76/person to administer. Phone-based delivery cost $13.65---but that includes $8.80 for the phone, which could be reused. Keep in mind that most people receiving e-money probably turned it into cash. Apparently, Zap's larger-scale, behind-the-scenes cash transfer system is more economical that Concern's purpose-built one.

  Perhaps more importantly, zapping money saved recipients time. Cash delivered the old-fashioned way only got within 1 kilometers (1.25 miles) of recipients' homes, on average. The women had to walk it the rest of the way. In contrast, the nearest Zap kiosk was 0.45 kilometers (a quarter mile) away. The authors translate this savings into compelling terms:

  This is equivalent to an opportunity cost savings of 30 minutes for each cash transfer, or 2.5 hours over the entire program. Based upon an average daily agricultural wage of USD $3.60, this time savings would translate into USD $.92 over the cash transfer period. This is equivalent to 2.5--3 kilograms of millet, enough to feed a family of five for one day.

• "There was no impact on consumption of staple foods, namely grain and cowpeas."
• Zapped households were less likely to sell off non-durable assets such as flashlights, presumably as a way to get money for food.
• There is evidence that people receiving money via phone spent it on more kinds things (mostly more kinds of staple foods) and had more diverse diets. "Households in zap villages were 4--5 percentage points more likely to consume fruits---and 6--12 percentage points more likely to consume fats than households in cash and placebo villages, a 28-percent increase." Overall, because this pattern emerges both in what people said they spent the money on and, separately, in their descriptions of their diets, I think it is unlikely to be a fluke.

  But it is not quite clear how much to believe this finding. Back in 2009, in reviewing the Karlan & Zinman microcredit impact study, I explained the problem of "multiple hypothesis testing": if you check for impacts on enough outcomes, a few of the treatment-control differences will be improbably large by pure, meaningless chance, just as a fair coin will sometimes come up heads five times in a row. Indeed, perhaps the majority of the cases where Karlan & Zinman find significant impacts can be explained away in this manner. (Of course, mostly they don't find significant impacts.) When Karlan previewed that study at CGD, commentator David McKenzie suggested that Karlan & Zinman perform a formal adjustment for this multiple hypothesis testing. They did not. But David soon did in a paper of his own; and, imitating him, so did Aker and coauthors.

  According to the adjustment procedure, the finding that zapping money diversified diets might just be one of those quirks in the data you get by chance if you check enough outcomes. This disclaimer does not apply to the results I cited earlier.

Why would the way money is received affect how it is spent? The authors hypothesize that it was easier for women to conceal their receipt of electronic money from others, including their husbands. Receiving manually disbursed money is a more public act since all recipients in a village must trek to the distribution point at the appointed time. Receiving electronic money meant discreetly noting a beep or text message on the phone, then cashing in at a time of one's choosing, not so far from home. Perhaps privacy---secrecy---gave women more control. That would be "empowerment."

I think the biggest caveat for this study is that the sample was restricted to recipients. We don't know whether the gains of those who got zapped money came at the expense of neighbors or relatives who did not. Greater privacy may have helped women evade the notice of relatives who otherwise have pressed for a share of the funds. (A similar caution applies to the microsavings study.)

Relatedly, it would be nice to have some confirmation that the program's eligibility criteria were adhered to, or at least that those determining who was eligible were unaware of which villages would receive money electronically. (Non-enforcement of a formal eligibility rule is a big issue in the Pitt & Khandker study in Bangladesh.) Without such confirmation, it is conceivable that there were systematic differences among the study groups in the violations of the eligibility criteria. Maybe in villages where Zap accounts were offered as part of the package, better-off households were more likely wheedle their way into the program, creating the illusion of impact. I don't find the story particularly compelling, but more might be done to rule it out.

Overall, I think this paper offers a tantalizing glimpse of the way a mobile payment system can rebalance power dynamics within families. I hope further research, perhaps of a qualitative nature in the same villages, will show us more.