I happened to hit the home page of FINCA, a signer of last month's statement on the impacts of microfinance. Among the fly-down menus at the top is a phrase that caught my eye: client research. 2009 was a banner year for microfinance impact studies. I blogged five and, it turns out, missed one, which FINCA published in October under the name, Bottom-Up, Free Enterprise Solutions for Ending Poverty. Actually, it is a project report to the Templeton Foundation, which funded the work with a cool $1.337 million.The report reads like a caricature of bad research. As far as studying impacts goes, it's $1.337 million wasted. Here's some running commentary:
Report text | Comment |
This project proposed that free enterprise from the bottom-up---represented by the global microfinance movement---has already created the largest and most ubiquitous enterprise-based solution to poverty in human history. Since the late 1980’s, hundreds of microcredit programs, collectively channeling several billions of dollars in loan capital, have already doubled the incomes of over 100 million of the world’s poorest families....This is precisely the...story that FINCA wants to help the Templeton Foundationtell in order to convincingly make its case for the transformative potential of free enterprise to end poverty. | "Double the incomes"?? According to whom? In other words: We already know microcredit reduces poverty. Now we just need to prove it. |
FINCA has worked to overcome the many barriers to measuring the effects of "bottom-up free enterprise" by creating a "clearing house for microfinance research"---a specialized entity designed to catalyze a strategic breakthrough in public awareness of, and funding for, the global microfinancemovement. | The party doing the research has a strong vested interest in the results. |
Over the course of this project, over 16,000 client interviews were conducted by more than 100 Client Assessment Research Fellows in 33 countries throughout the world, consisting of over four million data points collected that depict the social and financial status of our clients and their families. [emphasis in original] | FINCA interviewed thousands of clients, asking hundreds of questions. In a few countries, the report explains elsewhere, they resurveyed the people so that they could measure changes in clients' lives after starting with microcredit. There was no attempt to construct a control group by interviewing non-clients. The assumption is that all measured changes in the lives of FINCA borrowers in Mexico, Uganda, and other countries since 2007 are caused by FINCA microfinance. No doubt FINCA asked people how old they are; and presumably they aged between survey rounds; so by the logic of the study, microfinance causes aging. More to the point, the study cannot rule out aging or general economic developments, such as changes in crop prices, as sources of whatever "impacts" are found. The fundamental problem is the confusion of observed outcomes with impacts, the latter being differences between observed outcomes and (unobserved) outcomes in the absence of FINCA. |
The Structured Query Language (SQL) data warehouse (SQL Server 2005) currently holds millions of data points from thousands of FINCA Client Assessment Tool client interviews realized from 2003 to 2009, along with social performance data from additional research projects. The architecture includes hundreds of tables that allow for the easy creation of static reports, in addition to key documentation such as a business requirements guide and a data dictionary. | Sounds impressive. I use SQL Server 2005 too. I bet the motorcycles or SUVs driven by the FINCA staff are complicated too. |
FINCA is reaching the poor and very poor...Clients spend most of their income on food | No serious quibble here, though I was surprised that only a third of subjects live below $4/day. |
FINCA is reaching the poor and very poor...For example, with data from one year only, there are a number of potential reasons that clients’ daily per capita expenditures (DPCE) may have increased after receiving a loan from FINCA....This increase may be attributed to the loan (Relationship 2), or to a good harvest (Relationship 1) or an increase in demand (Relationship 3). However, by tracking the same clients over time, we are better able to identify (pinpoint) the exact relationship between the receipt of a FINCA loan and change in DPCE....This is a major step forward for both FINCA and the microfinance industry because for the first time we can narrow or hone the possible outcomes of the data collected. We can progress from three likely outcomes to one definitive outcome. | We can do nothing of the sort: see above. And actually it's not the first time. |
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
