Randomized Test of Microcredit in Mongolia

A few years ago, Alaka Holla and Michael Kremer, the latter a leader in the randomization revolution, opened a CGD working paper with this interesting observation:

Over the past 10 to 15 years, randomized evaluations have gone from being a rarity to a standard part of the toolkit of academic development economics. We are now at a point where, at least for some issues, we can stand back and look beyond the results of a single evaluation to see whether certain common lessons emerge.

Their paper reviewed the evidence on how pricing affects uptake of health-related products such as bednets (is demand sensitive to price? is there a big difference in uptake between really cheap and free?). I think it was one of the first instances where enough randomized trials had been done on a question that one could begin to generalize with confidence.I think we are approaching that point in the study of the short-term (1--2 year) impacts of microcredit. First there was Karlan & Zinman's study of a "cash lender" in South Africa---a for-profit company that lent for four months at a time, generally to people with jobs or job prospects. This was not what is usually considered microcredit; but the study showed that it is possible for poor people to benefit from loans while paying interest that compounds to 586%/year.Then in 2009 came two studies of microcredit as that term is usually conceived---another by Karlan & Zinman, in Manila, and one by Abhijit Banerjee, Esther Duflo, Rachel Glennerster, and Cynthia Kinnan of J-PAL in Hyderabad, India. Neither study found an impact on key indicators of poverty, though in Hyderabad microcredit did stimulate microenterprise and shift spending away from "temptation goods" toward durable goods such as sewing machines.Last March came another by Duflo and company in rural Morocco, which I failed to report. (But Duflo's October 2010 preview was blogged by Laura Starita at Philanthropy Action and Lee Crawfurd of FAI.) Despite the radically different context, the short-term impacts in rural Morocco resembled those in urban India: no change in poverty; but people who already had "businesses" (raising sheep, goats, and such) expanded and diversified their businesses, worked less outside the home, and cut spending on non-durables.Aside from the diversity of the locales, which are a boon for generalization, the biggest distinction among those studies is the method of randomization. The J-PAL studies randomize across slums or villages, offering some geographic areas microcredit, but not others (yet). They work with lenders who are expanding into new territory and are willing to randomize the order in which they do so. In contrast, the Karlan & Zinman studies randomly "unreject" some applicants for individual credit (not group-based loans) who would otherwise not make the cut based on such factors as income and credit history.As you would expect, each approach has advantages. For one, each answers a different but practical question. The J-PAL area-based approach measures the impact of expansion into new areas. The K&Z unrejection method looks at the impact of pushing the margins of creditworthiness where operations are already established. The area-based approach has the advantage that in "treated" areas, once where microcredit is offered, surveyors can visit non-borrowers as well as borrowers. This lets researchers estimate the average effect on a community: if a borrower expands her tomato-selling business at the expense of non-borrowing sellers (a major concern for Milford Bateman, called displacement), only the net effect is counted since all types of people enter the averages.Meanwhile, as I've noted before, the K&Z approach is premised on the lender collecting a fair amount of information about borrowers and plugging it into a computer-based credit-scoring program. This expense tends to be justified only for less-poor clients taking larger (individual) loans, so that, where J-PAL has studied group credit for fairly poor people, the K&Z studies focus on a somewhat better-off clientele. The unejection studies also haven't factored in effects on non-borrowers. On the other hand, those studies were administered on established, experienced lending operations, whereas the area-based approach assesses expansion offices that may have new and inexperienced personnel, making them less representative of going microfinance.Some fresh playwrights just wrote the next act of the microcredit evaluation drama. It is set in Mongolia. The writers are Orazio Attanasio, Britta Augsburg, Ralph De Haas, Emla Fitzsimons, and Heike Harmgart, and are all affiliated with London-based institutions: the European Bank for Reconstruction and Development (EBRD), the Institute for Fiscal Studies, and University College London. And some of the same authors are finalizing a study in Bosnia and Herzegovina (sneak peek at the abstract here). I guess the EBRD funded both these new studies.The Mongolia paper  most resembles the Morocco one. Randomization is over villages, in the countryside. The researchers worked with XacBank as it expanded lending in five provinces. In each province, three villages were randomly chosen for offers of group-based credit, three for individual loans, and two for no credit at all---at least for the 18 months of the study. In all of these 40 villages, in the spring of 2008, XacBank ran information sessions to explain microcredit and to announce that there was a good chance that it would soon begin lending to relatively poor women there, not revealing whether a given village was in a treatment or control group. (XacBank already did business with a wealthier, generally male, clientele in the villages.) Interested women below certain poverty thresholds were to assemble themselves into groups of 7--15 to await credit offers. Altogether 1,148 women did so, becoming the subjects of the study.XacBank proceeded to set up its microcredit operations in the treatment villages. For mundane (and unknown to us) administrative reasons, it did not begin lending in all at the same time. And in the group lending villages at least, further delay occurred because women first had to save a certain amount with the bank before they could borrow. In the fall of 2009, follow-up surveyors came to ask the women lots of questions and collect lots of data. They managed to track down 86% of the women. (Where did the other 14% go? It is a nomadic society...) The time between first access to credit and follow-up visits ranged between 8 and 16.7 months for group lending villages, averaging 13.7. So in the average group lending village, one woman had been borrowing for 13.7 months and the rest for less. (This according to e-mail from Ralph De Haas.)Only about half the women in treatment villages who prepared to borrow followed through (57% in group-lending villages, 50% in individual-lending ones). They were the better-off ones, as measured by ownership of such things as fences, wells, and tools. Most of the non-borrowers either decided not to apply for the loan when the time came, or turned down the offer once they got it because "the amount was too small, the interest rate too high, or the repayment schedule unsuitable." A minority were rejected by XacBank, apparently for insufficient collateral or already having too much debt.The Mongolia study differs from the Morocco one in two ways:

1. While women who initially signed up to participate in the study but never borrowed remained in the sample, so that their microcredit-free life trajectories were averaged into the reported impacts, women who never joined such groups were outside the study. So the study population may not be representative of the entire villages, and any displacement effects on women (or men) outside the sample---the non-borrowing vegetable sellers out-competed by the borrowing ones---were not captured. On the other hand, neither were positive side-effects, as might arise from the credit's stimulus to the village economy.
2. Importantly, this study is the first to rigorously compare individual and group lending. This allows sharper answers to questions like: Do people prefer to borrow in groups or solo? Does being monitored by peers change what one does with a loan? Does it affect outcomes such as poverty? For example, above I noted a hint that women preferred group loans---or felt more compelled by peer pressure to take them: uptake was 57% in group lending villages and 50% in individual lending villages.

I was struck in reading the study by how microcredit---a set of innovations that arose in the dense villages of Bangladesh and barrios of Latin America---has been adapted for such a sparsely populated land. How sparse is it?

The average distance from a village to the nearest province centre – small towns where XacBank's branches and loan officers are based—is 116 kilometres. Because the distance between a village and the nearest paved road is on average 170 kilometres, travel between villages, and between villages and province centres, is time consuming and costly.

Mongolia has half the area of India and less than half the population of Hyderabad. As a result, weekly meetings with loan officers just aren't practical. They meet monthly. Distance and the infrequency of meetings must weaken the ability to monitor and pressure borrowers into using credit productively. It appears to me that as a result, XacBank relies especially heavily on dynamic incentives to coax repayment---rewards tomorrow for good behavior today. Good customers can expect larger and longer loans next time, at lower interest rates.Also interesting is that this study, like those in India and Morocco and the forthcoming Bosnia one, took place amid worries about overheated lending and overindebtedness (as NPR reported). Yet it finds no gross harm to clients---food for thought.The authors summarize the results this way:

Although the loans provided under this experiment were originally intended to finance business creation, we find that in both the group---and in the individual-lending villages, about one half of all credit is used for household rather than business goals. Women who obtained access to microcredit often used the loans to purchase household assets, in particular large domestic appliances. Only among women that were offered group loans do we find an impact on business creation: the likelihood of owning an enterprise increases for these women by 10 per cent more than in control villages. We also document an increase in enterprise profits but only for villages that had access to microcredit for longer periods of time. In terms of poverty impact, we find a substantial positive effect of access to group loans on food consumption, particularly of fruit, vegetables, dairy products and non-alcoholic beverages.In terms of individual lending, overall we document no increase in enterprise ownership, although there is some evidence that as time passes women in these villages are more likely to set up an enterprise jointly with their spouse. Among women in individual-lending villages we also detect no significant increase in (non-durable) consumption, although we find that women with low levels of education are significantly more likely to consume more.

The increase in spending on food in group credit villages is novel. But there is also a lot of consonance with the earlier studies. Overall spending does not increase, but spending patterns shift. As in Hyderabad and Morocco, group loans led to more enterprise. As in Manila, individual loans didn't. That suggests that peer monitoring really does affect loan use. Takers of individual credit may have been more apt to pass the money to extended family ("may have" because of caveat #2 below.)For those wanting deeper insight into the methodology, here are two caveats:

1. The degree of randomization---dividing 40 villages into three groups---is a tad low. As usual, the best way to understand this kind of conceptual point is to take it to the extreme. Suppose XacBank had drawn a line down the center of Mongolia and flipped a coin to decide whether to offer microcredit in the east or the west. That would have been a randomized control trial. But the capital, Ulaanbataar, with 40% of the population, is in the east, so you can imagine that poverty levels and such would differ systematically between the twohalves even before the microcredit was introduced---yet which would be attributed to microcredit under standard methods. Randomization relies on the law of large numbers, which says that if you randomly split enough people or villages or playing cards into groups, the groups will converge statistically.The researchers took one step to reduce the chance that a small number of randomization units would make the treatment and control groups statistically different before microcreditarrived: they stratified. As described earlier, they picked five provinces, then within each randomly picked 3 group villages, 3 individual villages, and 2 "neither" villages. By guaranteeing the same number of treatment and control villages in each province, this two-stage approach makes the treatment and control groups more comparable, to the extent that villages are more similarwithin provinces than across them.Here's an analogy. Imagine if five countries of Europe or states of India had been chosen, then within each 3 towns were randomly picked for group lending, 3 for individual, and 2 for neither. You could imagine some differences might crop by chance between the 15 group villages, the 15 individual ones, and the 10 "neithers" in such things as average income and number of microbusinesses---less so than if the numbers were 150, 150, and 100.The tests for good randomization (Table 1 of the paper) can't be expected to pick up such deviations. What they tell you is the equivalent of the observation that getting heads 100% of the time is unremarkable if you flip a coin once.No doubt the researchers would have preferred to randomize across more villages. But budgets or other practicalities prevented them. I view this as a modest caveat, one to keep in the back of your mind to guard against taking the quirkiest results too literally. As I wrote of Karlan & Zinman, the key is to look for patterns in the data that tell a consistent story. Some apparent differences might be due to imperfect randomization. Take the number of words I devote to it as a measure of the seriousness not of my doubts, but of my desire to explain this complex issue to non-statisticians.
2. Following standard practice, the paper reports differences in average outcomes for all studied women in the treatment and control villages at the time of the follow-up surveys. Distinctively, however, it also breaks out results according to how many months women could borrow, or how many loans they took in sequence. The idea is to go beyond the coarseness ofdifferences in averages and check whether the tendency of group loans to lead to more enterprise, say, gradually emerges. But this granularity comes at a price, in the form an additional and debatable assumption about the "intensity" of treatment: that the number of months women could borrow, or the number of loan cycles they could pass through, was effectively random.Recall that XacBank didn't start lending in all villages at the same time. If they rolled dice to decide where they would lend first, then this would have been an excellent embellishment of the experiment, randomizing not just who got the treatment but how long they got it. What they actually did is not clear. Maybe they set up first in the villages that areless poor, closer to roads; or maybe the managers who got a quick start also ran their programs better, producing better results. These scenarios would generate differences in outcomes that would be falsely attributed to longer exposure to microcredit. The authors express appropriate caution about their estimates of the effects over time but do cite such findings when summarizing.I think it's best to moderately discount the results about the time-evolution of impacts if they are not backed by the more purely randomized and rigorous simple averages. So, for example, the authors "document an increase in enterprise profits but only for villages that had access to microcredit for longer periods of time." That is, profits rose more (or fell less) for women who could borrow longer. But I don't completely buy this finding because the most rigorous check---average profits in treatment vs. control villages---shows no effect.

Overall, I enjoyed reading this study and I think you will too. It is an important addition to the continuing investigation into the impacts of microcredit. What I felt most keenly at the end was that, as we get a fix on the short-term impacts, the longer-term impacts loom as a research priority. What happens after 18 months? Perhaps J-PAL will answer that question soon, for Hyderabad.