Reply to Microcredit Summit Campaign

Larry Reed, the new director of the Microcredit Summit Campaign, along with Jesse Marsden, its research and operations manager, recently posted a gentle review of my book, with which I disagree.I think the central argument is:

...he makes no effort to recognize the differences among the many microfinance programs employed globally. His analysis relies most heavily on just two studies from 2009 which examine the data from a few programs studied within a very short timeline of 12--18 months. While his book references many studies that have widely ranging scopes and methods, Roodman chooses to focus on a very limited set of data to draw generalizations about the wide variety of microfinance programs in use globally.Yet, if not all microfinance providers are the same, it would follow that some might place more emphasis on helping their clients climb out of poverty than others who might emphasize greater financial performance of the institution itself. A study that focuses on institutions aiming to maximize client benefit and movement out of poverty might show different results than a study that fails to make a distinction among its test subjects....Our belief is that the way in which microfinance is implemented matters greatly to the very poor. We suspect that those MFIs which judge their success according to the measureable improvements in the lives of their clients end up designing very different programs and, we also suspect, have very different results from those that focus on market share, share price, and investor interests.

• I do rely on a small number of studies. They are the only ones I believe it is safe to rely on. I devote many pages of chapter 6 to explaining this view (whether I do a good job, I don't know). I have spent a good deal of time over the last ten years scrutinizing the sorts of non-randomized quantitative studies that I discount, even writing computer programs to replicate their math. So I speak from knowledge and experience. And my view is mainstream among academic researchers who are trained to measure impacts. The post does not address these arguments. In fact, it seems to greatly underestimate how hard it is to measure impacts, when it refers to "MFIs which judge their success according to the measureable improvements in the lives of their clients." The fact is that even MFIs that seek to judge their success according to measurable improvements basically can't unless they perform the sorts of rigorous studies on which I rely so heavily. Almost none has. As a result, they don't have a firm statistical grasp on what their clients' lives would have been like without microfinance, the counterfactual that is needed in order to measure impacts.I'd rather not generalize from such a small number of studies. But one must make decisions today based on the information one has. Fortunately, more studies are appearing at a steady clip, albeit of short-term impacts. I hope that studies of longer-term impacts will eventually come.
• It is entirely possible programs that are intended to help the poor do so more. But that is a hypothesis (something they "suspect") and needs to be tested. After all, programs run mainly for profit often help people too (think of mobile phones) and programs intended to help the poor often fail (see Pakistan). So I hope the Microcredit Summit Campaign will encourage the microfinance institutions that it favors to collaborate on rigorous research to see if they make more of a difference. In the meantime, I believe it is most prudent to generalize from the evidence we have and to bias toward the assumption that interventions do not reduce poverty among clients until proven otherwise.