With rigorous economic research and practical policy solutions, we focus on the issues and institutions that are critical to global development. Explore our core themes and topics to learn more about our work.
In timely and incisive analysis, our experts parse the latest development news and devise practical solutions to new and emerging challenges. Our events convene the top thinkers and doers in global development.
Microfinance, foreign aid, Commitment to Development Index, debt and debt relief
David Roodman, a former CGD senior fellow, worked at the Center from March 2002 to July 2013. His work at the Center focused on microfinance, debt relief, and aid effectiveness. His widely praised book Due Diligence confronts questions about the impacts of microfinance and how it should be supported. He wrote the book through a pathbreaking Microfinance Open Book Blog, where he shared questions, discoveries, and draft chapters.
Roodman was an architect and manager of the Commitment to Development Index since the project's inception in 2002. The Index ranks the world's richest countries based on their dedication to policies that benefit the 5 billion people living in poorer nations; it is widely recognized as the most comprehensive measure of rich-country policies towards the developing world.
Roodman wrote several papers questioning the capacity of common cross-country statistical techniques to shed light on what causes economic development. He co-authored a 2004 American Economic Review paper that challenged findings of World Bank research that aid works in a good policy environment. His non-technical Guide for the Perplexed builds on analysis of methodological problems and fragility in other studies. Among econometricians Roodman is best known for his computer programs that run in the statistical software package Stata; articles about them won him the inaugural Stata Journal editors' prize in 2012. Also in 2012, Roodman aged off the RePEc list of top young economists in the world, at number 6.
Europe provides the majority of the world’s foreign aid. But economic convulsions at home and a shifting global landscape pose challenges to the status quo. In a tight series of mini-presentations, five European experts will assess the performance and prospects for European development policy.
Over the decade, donors have publicly declared that they would improve how they operate in order to make aid work better. They would coordinate better, let recipient countries take more ownership of project design, and so on. Ten years and ten days ago, there was the Rome Declaration. Then came the Paris Declaration, the Accra Agenda for Action, and the Busan Partnership. Probably you doubt as much as I do whether these statements are worth their weight in paper. But their impact should be judged not by our biases, but by evidence---evidence of how donors actually behave. And the ultimate test for donors, suggests a finely written new report, will come in Myanmar. After the remarkable turnabout there---symbolized for foreigners by longtime dissident Aung San Suu Kyi becoming a member of the legislature last April---dozens of public and private donors are flocking to the country.
Write Lex Rieffel of Brookings and James Fox, a former USAID senior economist:
Every respectable aid agency and international NGO in the world is planning to initiate or expand operations in Myanmar. The best and the brightest in these organizations are pushing to be posted in Yangon or to manage the Myanmar account. We were told that in a recent survey of World Bank employees, 80 percent listed Myanmar as their first choice for an overseas posting. Administrators of the Princeton-in-Asia program have described a fellowship in Myanmar as “the hot ticket” for its current applicants.
Rieffel and Fox are admirably forthright about their limitations as analysts of Myanmar---they've worked on development for decades but only spent a few months in the country, and don't speak the language---but Too Much, Too Soon? The Dilemma of Foreign Aid to Myanmar/Burma seems a fair portrait of donor behavior in the early days of an aid rush. Already, Myanmar's thin government is overwhelmed by requests for meeting from "ministers from donor countries, business leaders, [and] movie stars." Already, donors that vowed to cooperate are withholding information from each other. In this light, it may be a strange blessing that the government suddenly moved its capital in 2005 from Yangon to the terribly planned new city Naypyitaw. "For the government of Myanmar, encouraging embassies and donor offices to remain in Yangon could have the advantage of slowing the flow of visitors and making it easier to avoid them altogether."
The news is not all bad though. Rieffel and Fox single out the UK's Department for International Development for praise: "arguably the most innovative and principled aid agency in the world today." One thing DFID did right was, along with the EU and Australia, to help establish four multidonor trust funds. For example, the "3 Diseases Fund," fills the void left by the departure of the Global Fund to Fight AIDS, TB, and Malaria in 2005. By pooling contributions from many countries, it prevents them from each setting up their own projections pell-mell.
My one disappointment is that the report doesn't quite live up to its title. It doesn't answer the question, "Too Much, Too Soon?," nor even confront it with the directness I'd like. There is much (good) discussion of whether aid quality is too low---whether donors are living up to their Paris Declaration---but less of whether the sheer amount of aid in prospect could do more harm than good, no matter the quality. Even when aid is perfectly harmonized, coordinated, untied, country-owned, aligned, defragmented, managed for results, channeled through country systems, and done with mutual accountability and civil society consultation, it can still distort the political economy of the receiving country. The more that revenue comes from abroad rather than from the population, the less this fragile democracy may be compelled to respond to the needs of its citizens. Or the more, as the report worries, that dealing with donors will distract officials from existential challenges such as making peace with ethnic minorities.
Although Rieffel and Fox don't say this directly, their descriptions imply that the best hope for coordinating the donors lies with the recipient. "Aid professionals generally do not get promoted in their organizations for being good cooperators. They advance by responding to the headquarters
agenda, by showing that their organization is doing something that makes a difference, and by speeding up disbursement of resources under their control." Perhaps only the government of Myanmar, by leading a planning process, can get the donors to fall into a line. Here, early signs are promising. It began work on a Framework for Economic and Social Reforms last May, presented it in draft to donors in December, and brought them together to discuss it in January. The donors offered "high praise."
Encouraging too is the deep debt reduction the government won from the Paris Club, the association of creditor nations, earlier this year. So far I've found no one who was involved in the 19-hour negotiation who is willing to tell me what happened. I surmise, however, that the government of Myanmar knew it had a good hand going in and played it well---owing to what blend of native savvy and foreign advisers, I know not. Norway says the talks almost collapsed. Japan, Myanmar's largest creditor, had already promised deep debt relief, and seems set to become the country's largest donor. With that set, Myanmar was apparently prepared to let its arrears to Norway, France, and other creditors languish, which could have frozen them out of the aid action. I've questioned whether Myanmar needed such deep debt relief. But if they were smart enough to squeeze it out of the donors, like a clever gambler who beats the house, I figure they deserve it.
At any rate, if you need to understand the aid situation in Myanmar, this is an essential read. If your interest is more general, I still highly recommend the report as a humble, historically informed, and insightful snapshot of a country on the eve of major change.
Don't get me wrong: sarcastic headline aside, I'm not in favor of the exploitation of children. However, I feel moved to speak against a recent push, I guess led by Hugh Sinclair, to insert a ban on child labor into the lending policies of microfinance institutions (MFIs), microfinance investors, and such accrediting programs as the Smart Campaign and the Seal of Excellence. The concern behind this movement is serious: that microcredit is financing, thus increasing, the exploitation of children. So the cause it leads to is understandable: a push for policies to break any such link.
Legality. Hugh argues that child labor is wrong because it is illegal in many countries with microfinance. Excellent point! In fact, most microfinance clients are engaged in illegality one way or another: squatting on city land to build houses the width of a queen-sized bed, failing to pay taxes on their meager earnings, failing to register their tiny businesses with the authorities... So to expunge microfinance of scofflawery, we need to shut it all down. Seriously, Hernando de Soto showed how, at least in Latin America, elites have purposely complicated the law in order to make formality---legality---a privilege rather than a right. Being poor means you are almost automatically illegal. Thus legality is a wobbly compass microfinance.
Ethics. We are all descendants of children who survived to adulthood only by laboring, whether as farmers or herders or gatherers. Only with their labor could the family subsist. I look forward to the day when there is no child on earth for whom this is the best choice. But we are not there yet. And we are not as close as you might think. Going by the numbers, the world has made great progress getting kids into school. However, a huge number of these children aren't learning very much. So how quick should we be to tell parents struggling under circumstances far different from our own what the right choice is? Many of them agree with you on the value of education. Whether it is best to put their children in the schools they can afford today is another matter.
Evidence. The effect of microfinance on child labor is an empirical question, whose answer will probably vary by context. On the hand, microfinance sometimes stimulates at-home businesses, leading parents to pull kids out of school and employ them at home. On the other, it gives parents a new way to finance school fees, providing them the discipline to set aside money each week for this purpose.
The evidence, like the ethics and the legal argument, is ambiguous. A good non-randomized study in Thailand found credit to increase child labor. One in Guatemala found the opposite. (Hat tip to Hugh for both.) The randomized studies, which I trust more, have mostly found little impact. In Hyderabad microcredit availability did not lift or lower the number of kids in school. In Manila, loans made no difference for the average response to "Any Household Member Helping in Family Business?" Ditto, essentially, in Mongolia. In Morocco, children worked 5.05 hours/week in areas with more microcredit versus 4.88 in areas without, a difference that is not statistically significant; meanwhile the number of children per family in school was slightly but statistically higher in the microcredit treatment areas, at 0.76 instead of 0.73.
An exception appeared in Bosnia & Herzegovina. Among less-educated, and presumably poorer families, microcredit caused more 16--19-year-olds to work at home, where "home" often meant "farm." As I blogged before, it is not easy to second-guess poor families in the midst of a major economic crisis if they use credit to invest in their farms, perhaps in more livestock, and put these near-adults to work.
Principals and agents. Now, one could retort that even if microfinance does not press children into labor on average, it still must do so sometimes. After all, how comforted would you be if I told you that microcredit does not increase slavery on average? And microfinance increases child labor in some cases, then one can argue that MFIs should vow never to finance such exploitation, and that microfinance investors must demand such vows in return for funds.
I do hope that microfinance officers don't leave their ethics at the home, that if they find a child laboring in great duress, and they know that a loan would make things worse, then they will not lend. But in general, microfinance, especially group microfinance for the lower-income clientele, has succeeded by not taking much interest in clients' business. Monitoring clients takes time, time costs money, and higher costs lead to higher interest rates. Anyway, trying to determine what people do with their loans or savings withdrawals is often a fool's errand because of fungibility. Moreover, as Hugh's book dramatizes, reality tends to diverge from rhetoric as one moves along the microfinance investment chain---from individual investor, to investment fund, to MFI headquarters, to field practice. MFIs may say they have banned loans for child labor, and MFI investors may buy that reassurance rather easily---but should we believe them? It will be a great achievement if a program like the Smart Campaign can reliably monitor and certify microfinance field practice as being transparent and non-coercive. I think it is a goal too far to certify what is being done with microfinance in each household. Microfinance investment funds promising to rid their portfolios of child labor will be setting the stage for hypocrisy.
This issue exemplifies a larger problem in international aid and philanthropy. A donor that enters a country with plans to make loans or drill wells or build roads cannot understand, much less control, many of the consequences of those interventions. Often local political structures undermine intentions: bed nets meant to be donated to the poor are pilfered and sold to the highest bidder. Pouring microloans into an Indian slum will perturb the paths of thousands of families. In some, more children will walk out the door each day, headed for the local schoolhouse. In others, more will be enslaved at looms (though I do wonder how much of the most vile child labor occurs in household-level enterprises). In the face of diversity and uncertainty of the outcomes, donors can either proceed or not. I think they'll do best to base their choices on the evidence---which looks pretty good in this case---and a general theory about how the intervention contributes to development. Roads, for example, do harm as well as good, but in many cases clearly more good. Similarly microfinance (if the credit is administered in moderation) is a generally useful service that can give people more control over their lives. Some will use that control for ill, but that doesn't make giving the poor more options a bad idea.
The nine client-members on the board of the Grameen Bank, all women, have made a sassy public response to the interim report of the commission investigating the Grameen Bank. (Hat tip to the Grameen Foundation.)
I have to admire the pointed prose:
The Commission even records in their report that we do not participate in board discussions. They complain that even the learned representatives of the Governments did not participate in the discussions either. What a great board! None of the members participate in the discussion, but the bank still wins the Nobel Peace Prize! A non- speaking board has created the most admired bank in the world! Why does the Commission underestimate us? Could it be because we are poor and because we are women!
I was a bit disappointed that they prefer to play the gender card, pointing out that the commission is all male, to addressing the substance of the interim report. Truly the two sides are talking past each other.
But Asif Dowla's recent comment helped me appreciate one thing the board members wrote:
Most of us never went to school, but that did not mean that we did not know how to run our business and run Grameen Bank. We managed the bank better than anyone else in the banking world. We did not let our bank become a corruption-ridden bank, like the government banks are.
Maybe they are not finance wizards, but they are sterling board members by historical Bangladeshi standards in one sense, as Asif explained:
One has to compare Grameen’s governance with governance in state owned and private banks in Bangladesh. The state owned banks board membership are handed out to people not because they are “experts in law or finance”, but because they are supporters of ruling political party and the membership is a form of patronage and payoff. Until recently, the board members of private banks were full of people who were defaulting on loans from state owned banks! In many cases, the directors of private banks were taking loans from their own banks!
Business training programs are a popular policy option to try to improve the performance of enterprises around the world. The last few years have seen rapid growth in the number of evaluations of these programs in developing countries. We undertake a critical review of these studies with the goal of synthesizing the emerging lessons and understanding the limitations of the existing research and the areas in which more work is needed. We find that there is substantial heterogeneity in the length, content, and types of firms participating in the training programs evaluated. Many evaluations suffer from low statistical power, measure impacts only within a year of training, and experience problems with survey attrition and measurement of firm profits and revenues. Over these short time horizons, there are relatively modest impacts of training on survivorship of existing firms, but stronger evidence that training programs help prospective owners launch new businesses more quickly. Most studies find that existing firm owners implement some of the practices taught in training, but the magnitudes of these improvements in practices are often relatively modest. Few studies find significant impacts on profits or sales, although a couple of the studies with more statistical power have done so. Some studies have also found benefits to microfinance organizations of offering training. To date there is little evidence to help guide policymakers as to whether any impacts found come from trained firms competing away sales from other businesses versus through productivity improvements, and little evidence to guide the development of the provision of training at market prices. We conclude by summarizing some directions and key questions for future studies.
The machinations around the Grameen Bank over the last two years have a had a paradoxical, dreamlike quality. Harsh words have been spoken by mighty leaders. Eminent dignitaries have rushed to the defense. Court battles have been fought. Crimes have been alleged. The mighty Muhammad Yunus has fallen. The government has wrested away the power to replace him. Someone was even tortured. Yet through it all, as in a nightmare in which you are running fast but not moving, the Grameen Bank has hardly changed. Yes, Yunus is no longer the Managing Director, but no permanent successor has been picked, so his longtime deputy Mohammad Shahjahan is filling the seat for now. Yunus continues to be Yunus, speaking around the world about social business. Atop the Grameen Bank home page, Yunus and a borrower are still seen accepting the Nobel Prize. There has been no run on the Bank. The products appear not to have changed. And the fraction of the loan portfolio considered to be at risk of nonrepayment continues to crawl upward, reaching 10.9% at the end of last month.
In short, it is still unclear whether catastrophe---corrupt political meddling, sweeping loan forgiveness, or a bank run---will befall the Grameen Bank or whether it will thrive beyond its founder, as all involved profess to hope.
On February 9, the second commission appointed in recent years by the government to investigate the Grameen Bank released an interim report. Where the fist commission's report focused on themes in the Tom Heinemann documentary---Did Grameen charge high interest? What happened to that Norwegian aid money?---this one sticks mostly fundamental to legal questions. Is the Grameen Bank a public institution or do its 5 million-plus shareholders, mostly poor women, own it? Was it legal for the Grameen Bank---or at least its top people---to have created a web of related companies? Did the governing board exercise appropriate oversight?
After a close reading of the ordinance that created the Bank, board minutes back to 1983, and other documents, the Commission arrives at controversial conclusions. The nine female member-borrowers on the Grameen Bank's board should resign immediately. The Bank should un-invite 90% of its members by buying back their shares. The remaining 10% should elect new board members, with only those who have completed 7th grade eligible to run. The license to operate a digital mobile network held by Grameenphone, Bangladesh's largest phone company, should immediately be suspended---but as a matter of practicality, the firm should keep operating. All these conclusions rest on what are, if not technicalities, then purely legal issues. For example, as the Grameen Bank grew, it needed authorization to sell shares to more members; the report says that share expansions in 1991 and 1994, unlike in 1986, were merely approved by the Ministry of Finance and not effected by amendment of the Ordinance. Therefore they were illegal and all shares sold in excess of of the 1986 limit---90%---should be bought back. Similarly, the Bank's support for some other Grameen organizations, given through loan guarantees, was illegal because those organizations strayed from the Bank's official mission to serve landless poor in rural areas. Are you indignant yet?
I learned a lot from the careful analysis in this interim report, and I think it contains valid criticisms and questions about how the Grameen Bank was run. In particular, and unsurprisingly for a successful founder-led institution, it appears that the Board was ineffectual and that Muhammad Yunus ran the bank with a free hand. The highly touted female borrowers who constitute the board's majority could not be expected to understand the octopus-like complexity of the Grameen family of companies, assuming they were apprised of it. And they certainly could not be expected to perform appropriate oversight. In the event, they hardly ever said anything at the meetings beyond pleasantries.
But I think the best way to understand the report is to start with what it does not say and does not change:
After this report, as before it, there is no suggestion that Yunus was corrupt, or that any of the alleged illegalities were perpetrated for purposes other than the social good. This in a country known for corruption petty and grand. (A possible exception has to do with the alleged failure of the Norwegian phone company Telenor to comply with an agreement to sell Grameen Telecom 16% of Grameenphone. But nothing in the report implicates people at the Grameen Bank in connection to this.)
After this report, as before it, it was clear that the Grameen Bank suffered from ills typical of successful founder-led organizations, notably the inability of the board to impose accountability. And it was clear that the founder's departure creates both challenges and opportunities.
After this report, as before it, the government can do whatever it wants to the Grameen Bank, because Prime Minister Sheikh Hasina commands an overwhelming majority in Parliament. The report's rationales for government intervention were hardly required. Hasina awaited no such investigative findings before changing the rules last summer to take control of the process for picking Yunus's successor.
And, on that point, after this report, as before it, the Grameen Bank is less independent of a historically corrupt and unstable government than it was for its first 29 years.
Here are some key points from the report, with commentary:
The Grameen Bank is a public institution, created by a special law. Even if Bangladeshi women own almost all the shares, the Bank is not private property. In reasserting control, the government has not seized private assets. The shares entitle holders only to participate in elections of board members, and perhaps to dividends---and probably not even that. I find these arguments largely convincing. I do wonder whether they go too far in interpreting the Ordinance's dispensation that the "annual profit...shall be utilised in such manner as the Board may determine" as barring dividend issuance. ("Utilise," the report's argument goes, doesn't encompass giving the money to others.) I do buy that the Grameen Bank is a public institution whose governance happens to give clients a say in the selection of the majority of the board. That doesn't change the fact the the government just, um, emasculated the board by taking away its most important function, picking the director.
The Grameen Bank has illegally strayed beyond the ends and means set forth in the Ordinance. The Ordinance opens this way:
Whereas it is expedient to establish a Grameen Bank to provide credit facilities and other services to landless persons in the rural areas and to provide for matters connected therewith or incidental thereto...
Later, the text defines "landless" as owning less than half a hectare of land, or total assets worth less than a hectare. The commission's lawyer-members read this to say that Grameen should not be engaged in, say, microcredit in the cities or solar energy in the countryside. Yet you see how the text fuzzes the boundaries of Grameen's aims ("matters connected therewith or incidental thereto"). And even if those fuzzy words are ignored, it must be recognized that strict respect for the law's limits is impractical and inefficient. If you want to make loans to poor people in Bangladesh, you should want to keep interest rates down, which means limiting costs, which means lending on a mass scale, to all people poor enough that they are willing to accept the inconvenience of group microcredit. It means not devoting staff labor to tracking clients' land holdings in real time. This is why the Grameen Bank has long lent to people over half an acre, why that has never bothered me, and why, if the law literally means what the commission says, it is an ass.
Similarly, the commission questions the legality of the support that the Grameen Bank has given to other organizations with "Grameen" in their names. I think the main objection is to loan guarantees. In these arrangements, a separate organization---imagine Grameen Healthcare Services---borrows money from some other bank. To reassure the other bank as to creditworthiness, the Grameen Bank provides a guarantee: if Grameen Healthcare Services defaults, the Grameen Bank will repay, and then can try to recover its money from Grameen Healthcare Services. Now, providing guarantees is something banks do. But can Grameen do them? Giving guarantees is not in section 19's list of permitted functions, so maybe not. However, the list ends with this clause, which the Commission does not mention:
generally the doing of all such acts and things as may be necessary, incidental or conducive to the attainment of the object of the Bank.
Again, the ordinance seems to give the Bank a lot of discretion. Now, the commission may well be right that some of Grameen's activities have strayed beyond the spirit of the law. But given the flexibility of the law and social purposes of the activities---Grameen Shakti has brought solar power to a million families----I don't see cause for indignant finger-wagging.
The nine elected ladies are a sham. The commission argues that they are selected more than elected, with heavy involvement of Bank staff in each step. Qazi Nazrul Huque describes it nicely in this comment. In a hierarchical process, all the Center leaders, Grameen clients who each represent about 40 peers, who are served by a given branch convene to select one from among them to represent the branch. Branch representatives do the same at the area level, then the zone, then the region level. Each of the nine regions chooses one board member. In the Commission's telling, Bank staff play major, and perhaps heavy-handed, role in the nominations. There is no direct, secret ballot of the 5.6 million shareholders, nor even of the the 540,000 whom the Commission says hold shares legally. Meanwhile, the board members clearly are unqualified to assess the prudence of the Grameen Bank's complex operations. The report says the "election" of board members
....was a powerful marketing tool, which resonates throughout the world. It has been used to cloak the hollowness of an elective procedure, which has placed as many as four electoral stages between the borrower-shareholder and the post of a Director of the Board. Worse, the description of this process by an ‘elected’ Director as recorded in the Bank’s Board meeting exposes the exploitation of an emotive slogan.195 There is need to make the participation of the borrower-shareholder more direct and meaningful so that the needs of the landless and poor women is more honestly addressed.
I imagine that the tension between democracy and competence is common in cooperative financial institutions. Probably the solution is to balance the board between elected representatives and selected, highly qualified ones, and to arrange for candidates to be more qualified than most who vote for them.
Certain dealings between Grameen Telecom (the nonprofit Grameen company), Grameenphone (the commercial mobile operator controlled by Telenor), and the government look shady. You figure that any process in which the government of Bangladesh sells a private company a potentially valuable asset---in this case, a license---involves some kickbacks. Beyond that, I have not tried to understand the details, which are complex and incomplete. Maybe someone else can explain.
So overall, I find the report interesting and serious, if with occasional flares of bias, archness, and naivete about social science (there is no credible evidence that "but for this mismanagement within Grameen Bank...more people could have been helped"). Strengthening rule of law is essential to Bangladesh's political and economic development. So I'm glad these lawyers advocate for it. But one must consider context before precisely equating any illegality with immorality. In 2011, the report says, a Bangladeshi court found that all laws passed under the martial law regime lasting from March 24, 1982, to November 11, 1986---including the Grameen Bank Ordinance and its first round of amendments---were null and void! All these years, the Grameen Bank was illegal, you see. When then argue about whether it was exactly within the law's limits? And under martial law just a few years ago, the current Prime Minister languished in jail for corruption.
The poverty of most Bangladeshi's being urgent, the eagerness of outsiders to work with the Grameen Bank being great, and the dangers of dependence on the government being obvious, it is understandable that Muhammad Yunus and others at the Grameen Bank pushed the limits of their independence in searching for ways to make a difference. I think that is the right way to view the situation even if, in retrospect, we judge that they sometimes strayed too far from the law.
In November 2009, some guy nobody had heard of, Daniel Rozas, wrote an article asking whether there was a microcredit bubble in south India:
The spark that sets off a large-scale delinquency crisis can be anything and could come at any time – a rapid drop in economic growth, a populist political movement, a religious decree, or a collections effort gone bad. One can’t control the spark, but one can control how much fuel that spark can ignite.
Eleven months later the government of Andhra Pradesh ambushed the microcredit industry within its borders. Rozas became the Roubini of microfinance.
Now Daniel is worried again, this time about Chiapas, in Mexico, or maybe Mongolia or Cambodia:
So is Chiapas the next Andhra Pradesh? At the surface it seems that the comparison doesn’t hold. Chiapas is not the crucible of Mexico's microfinance market in the way that Andhra Pradesh was. A crisis in Chiapas would hurt the local lenders badly, but direct damage to the large lenders operating across the country would likely be limited.
However, Chiapas also differs from Andhra Pradesh in a way that is more concerning. India’s huge states (Andhra Pradesh alone is nearly the size of Mexico) are separated by language, caste, religion, as well as local media – differences that are especially strong among the poorer, less educated social segments that comprise microcredit customers. This regionalism played an important role in containing the Andhra Pradesh crisis within the state – a pattern also seen during an earlier repayment boycott in the Indian state of Karnataka.
But in a country like Mexico, with a strong national media, a common language, and fewer social barriers, it’s not hard to imagine a payment boycott in Chiapas easily spreading to other parts of the country, as was the case with the crisis in Nicaragua.
Of course there are no certainties in this business. But, as he did last time around, Daniel brings data to the table. So you can see what you think.
CGD does not have data on how much public and private aid is responding to the Haiti earthquake. (But see this.) As background, here are a charts on recent patterns in Haiti's aid, debt, and remittance receipts. Post comments to request others. The graphs and data shown here are in this spreadsheet (2006--08 version). Aid figures come from the Paris-based Development Assistance Committee (DAC), which collects its data from donor governments. The latest aid figures---just released---are for 2008.
The graph above shows "Net Official Development Assistance" (Net ODA), the standard DAC measure of aid quantity. Source: DAC Table 2a. 2006--08 version
The graph above compares total foreign aid from governments to "remittances," mainly money sent home by Haitians living abroad. Sources: DAC Table 2a, World Bank. Figures are adjusted for inflation, into dollars of 2008.
Most of the remittances must come from the United States:
For more on the significance of migration for Haitians, watch CGD fellow Michael Clemens.
The graph above shows "gross disbursements" of aid. It is from a different data set, which is less comprehensive but more detailed. So aid totals do not match those in the previous graph. This one shows government aid channeled through private charities, but not true private charity, for which data are unavailable. Data are also missing for two large donors, the Inter-American Development Bank and the IMF. Source: CRS. 2006--08 version
The graph above returns to the "Net ODA" measure. DAC defines humanitarian aid as "assistance designed to save lives, alleviate suffering and maintain and protect human dignity during and in the aftermath of emergencies." Source: DAC Table 1. 2006-08 version
Also in the spreadsheet, but hard to embed here, is a table on aid giving to Haiti by sector (health, education, etc.).
An institution with a very long name (Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) at the Social Science Research Unit at the Institute of Education, University of London) just published a review of the academic literature on the impacts of microfinance in Africa. The study was funded by the U.K. aid agency, the Department for International Development (DFID).
I peer-reviewed the study protocol barely six months ago and a draft in early November, just as the reality of the Andhra Pradesh crisis was sinking in. A lot has happened since. Looking over the report today, I realized how it could become a football in the ongoing arguments about the efficacy of microfinance. Proponents can point to the preponderance of + signs in the tables that summarize findings about effects on outcomes such as wealth and health (pages 29--37) as well as to the citations of "high quality studies." Doubters can invoke the negativity of the summary:
We conclude that some people are made poorer, and not richer, by microfinance, particularly micro-credit clients. This seems to be because: they consume more instead of investing in their futures; their businesses fail to produce enough profit to pay high interest rates; their investment in other longer-term aspects of their futures is not sufficient to give a return on their investment; and because the context in which microfinance clients live is by definition fragile.
There is some evidence that microfinance enables poor people to be better placed to deal with shocks, but this is not universal.
The emphasis on reaching the ‘poorest of the poor’ may be flawed. There may be a need to focus more specifically on providing loans to entrepreneurs, rather than treating everyone as a potential entrepreneur.
Micro-savings may be a better model than microcredit, both theoretically (because it does not require an increase in income to pay high interest rates and so implications of failure are not so high) and based on the currently available evidence. However, the evidence on micro-savings is small and further rigorous evaluation is needed.
The rhetoric around microfinance is problematic and damaging. ‘Clients’ could also be called ‘borrowers’ or ‘savers’, and ‘micro-credit’ might just as well be called ‘micro-loans’ or even ‘micro-debt’. There is an obligation amongst donors and policy-makers not to falsely raise expectations with development aid in this way. The apparent failure of microfinance institutions and donors to engage with evidence of effectiveness perpetuates the problems by building expectations and obscuring the potential for harm. A growing microfinance industry may as easily be a cause for concern as one of hope.
Bear two points in mind in interpreting this report.
First, I would mentally italicize "some" in item 1 above. I think the authors mean to convey that microcredit, like all credit, appears to have a spectrum of impacts: it helps some, hurts others, and leaves many about the same. If I am right (as the lead author confirmed below), then the authors are emphasizing the negative in contradistinction with the prevailing, positive perception---or at least what was the prevailing perception when they started writing. They are not asserting that microcredit always or even on average does harm. I'm pretty sure none of the "high quality" studies suggests that.
Second, "high quality" is relative. The authors' assignment was to glean what they could from the literature on Africa, and in this I think they did an impressive job. But as I have argued (blog post, book chapter), the pickings of truly credible studies are thin. That goes double when you restrict to Africa. For example, the Barnes et al. study of microcredit in Zimbabwe, one of the four "high quality" studies cited, is not randomized and not, in my view, credible. (See Beatriz Armendáriz de Aghion and Jonathan Morduch's Economics of Microfinance.) [Correction from Ruth Stewart below: I had the wrong Barnes et al. study. The "high quality" one took place in Uganda---but this does not change my argument.] Roughly speaking, it is "high quality" because it has a control group. I trust the authors' judgement that this puts the study a cut above. But since the treatment and control groups were not randomly constructed, it still doesn't cut it with me.
So if you're not exclusively interested in the impacts of microfinance in Africa, I would rely more on Kathleen Odell's excellent, recent review for the Grameen Foundation. Even if you are only interested in Africa, Odell's review of lessons from other continents is still relevant.
Overall, the texture and tenor of this fuller distillation of the findings feel about right to me:
The available evidence suggests that micro-credit has mixed impacts on the incomes of poor people. Microsavings alone appears to have no impact. Both microcredit and micro-savings have positive impacts on the levels of poor people’s savings, whilst they also both increase clients’ expenditure and their accumulation of assets.
The available evidence suggests that both micro-credit and micro-savings have a generally positive impact on the health of poor people, and on their food security and nutrition, although the effect on the latter is not observed across the board. In contrast, the evidence on the impact of micro-credit and micro-savings on education is varied, with limited evidence for positive effects and considerable evidence that micro-credit may be doing harm, negatively impacting on the education of clients’ children. Having said this, micro-credit does not appear to increase child labour.
There is some evidence that micro-credit is empowering women, but this is not consistent across the reviewed studies. Both micro-credit and micro-savings have a positive impact on clients’ housing. However, there is little evidence that micro-credit has any impact on job creation, and no studies measured social cohesion.
This Thursday, the World bank will host the unveiling of the latest edition of the best-known ranking of think tanks, which is produced by the University of Pennsylvania. The public event will reveal whether the Brookings Institution has lost its hold on "Think Tank of the Year," which tanks made the top 50 worldwide, which are best in Latin America, and so on.
As with the Oscars, the verdicts of the Global Go To Think Tank (GGTTT) Index are rendered on the basis not of performance measurement, but on the perceptions of those in the business, in this case hundreds of journalists, policymakers, and think tank employees. That approach may be one reason expert perceptions of the GGTTT index itself have tended to be highly critical (here, here, here, here, here). Among the concerns: the opacity of the ranking process, the inclusion of institutions that are not think tanks in any usual sense of that term, and fundamental doubts about what it means for a member of such a diverse class to be "the best." Yet there's no doubt the results turn heads each year. If the criticisms from think tank experts are right, then the GGTTT may be distorting the behavior of think tanks, as they strive to raise their standing on dubious metrics, as well as misleading think tank funders.
The combination of continuing criticism and continuing interest made us wonder: could we do better, or at least illustrate the possibility of doing better? Last November we posted indicators of think tank "profile", looking at how often a tank's work is reported, cited, downloaded, or followed. Here, after tweaking and updating the indicators, we blend them into a single index for ranking, which we easily do by drawing upon methods we honed over 10 years for the Commitment to Development Index.
Let us be clear: CGD does not intend to enter the tank-ranking business long-term. As a think tank, we have a dog in this fight and lack the necessary objectivity. Indeed, we acknowledge that our focus on public profile may bias the results in CGD's favor, since public outreach is central to our strategy. Moreover, as we wrote last time, web page hits, media mentions, and scholarly citations are just a subset of the characteristics that can make a think tank effective. Thus the "profile" in our title. Some tanks succeed precisely by flying below the radar. Our purpose is to stimulate and improve the discourse around think tank performance.
These caveats notwithstanding, like many in the think tank community, we are data geeks, committed to the use of objective evidence in our research. We believe that any effort to rank the tanks should begin by gathering the best available data. In the spirit of offering some much-needed competition to the GGTTT, let’s take a closer look at the data.
We start with an update of the last post's first table, covering noted US institutions. Major changes in our data collection include a switch from Google News to Nexis for media mentions counts, the latter having proved more stable from week to week; the use of academic citations of papers published in 2010 instead of 2012, since 2012 is too recent for much mention to have accrued; and the exclusion of Human Rights Watch and NBER as not being think tanks despite being listed as such in the GGTTT:
Indicators of aggregate profile for major US think tanks
Next we turn those numbers into scores by multiplying each performance indicator column by a scaling factor chosen so that an average performer scores exactly 5 (and a twice-average one gets a 10). This puts most scores in the intuitive 0--10 scale. Finally, we average the five categories to get the overall scores in the last column of the table below.
The conservative stalwarts Cato and Heritage perform best in social media and web presence, while Brookings leads in media mentions and scholarly citations---but as we'll explain in a moment, we don't view these rankings as our most useful results, preferring to order another way:
Scores of aggregate profile for major US think tanks (5 = average)
(In the spreadsheet, you can change the weights on the five indicators.)
The next two tables are like the first two except that they divide all performance indicators by annual spending: not total media mentions, for example, but media mentions per dollar of budget. These results seem more practically relevant because a donor of $100,000 is probably less interested in an institution's aggregate profile than its ability to build following per dollar spent. By that measure, the Cato Institute, the Pew Research Center, and Peterson Institute have been most effective:
Indicators of profile per dollar spent for major US think tanks
Scores of profile per dollar spent for major US think tanks (5 = average)
Ranking by profile per dollar spent of major US think tanks
These standings differ from those of the Global Go To Think Tank index. Cato is third in aggregate profile (second table above) and tops in profile/dollar (fourth table and graph above) but only 14th on the 2011 GGTTT. Last year's 'Think Tank of the Year," Brookings, score just 5.5 (5 being average), placing it 7th out of 18.
To defend the GGTTT against this seeming contradiction by real data, one could argue that the GGTTT index covers more than public profile: it encompasses any performance attribute the raters think of, such as timeliness or creativity of policy proposals. Recognizing this argument, for a more meaningful comparison, we plot our results against corresponding rankings in the GGTTT report.
This next chart compares the average of our first three indicators (per-dollar social media fans, web traffic, and incoming links) with an institution's rank on the GGTTT list of "Think Tanks with the Best Use of the Internet or Social Media to Engage the Public." (It drops tanks that don't make the GGTTT list.) If the two approaches agreed, we would see the dots cluster roughly along a line from bottom left (low score from us, high rank number from GGTTT) to bottom right (high score, low rank number such as #1). Some do that, but others are far from the diagonal line of good agreement:
Internet/Social media: GGTTT rank vs. CGD index
This is a similar plot for print and electronic media; it compares our media-mention scores per dollar to the GGTTT "Think Tanks with the Best Use of the Media (Print or Electronic) to Communicate Programs and Research." Here the correlation is even weaker:
Print/electronic media: GGTTT rank vs. CGD index
Our small exercise falls short of an objective ranking of think tanks. Still, we can’t help but believe that expert and popular understanding of think tanks would improve substantially if some of the energy currently put into annually cajoling hundreds of experts to rate thousands of institutions each year were redirected into collecting and analyzing objective, empirical measurements of think tank performance.
A spreadsheet with all results shown above, plus the same for the GGTTT's "international development" think tanks, is here. Perhaps some think tank funders will want to pursue this work. If so, your comments below could contribute to their own thinking about think tanks.
Milford Bateman's new book, Why Doesn't Microfinance Work? The Destructive Rise of Local Neoliberalism, annoys me in much the way that Dambisa Moyo's did last year. I see myself as open-minded, but I guess I am allergic to (as I perceive it) sloppy thinking. The book makes dramatic conspiracy claims, yet is loose in its reasoning; careless in its use of evidence; and heavy in its use of passive voice and weighty abstractions such as "neoliberalism" that obscure for the reader (and perhaps the writer) who is accused of doing what. I found it hard to get through. (For a kinder (re)view, read Malcolm Harper.)
Bateman is clear about his thesis though:
The central argument...is that microfinance is largely antagonistic to sustainable economic and social development, and so also to sustainable poverty reduction. Put simply, microfinance does not work. I fully accept that there are some minor benefits to the be derived from the widespread provision of microfinance to the poor....But I argue that these benefits are very minimal indeed, and anyway wholly insignificant when set alongside the huge longer-term downsides and opportunity costs inherent in the operation of the microfinance model. To focus upon these few minor shorter-term benefits is to deliberately focus on the few trees left standing after having helped the entire forest to burn down. (pp.1--2, emphasis in original)
Bateman is right to challenge the mythology that was constructed around microfinance in order to sell it to donors. But he commits the error of his opponents in advocating extremism too, just of the opposite tenor. It seems as if his passion about the problems of microfinance, fired by personal experience in Eastern Europe, along with his verbal fluency and a proclivity to view public affairs as polarized ideological battles, override his propensity for critical thinking. The result is a story in which the monolithic "international donor community" chooses to "neoliberalize" microfinance worldwide out of blind ideology, or, worse, a "political" drive to make the world safe for corporations. Bateman is not saying that microcredit directly hurts clients on average but rather that those who push microfinance are powerful enough that they could instead push developing nations onto quite different paths, in particular deploying state-led credit models that he says can be relied upon to deliver development. If it were weren't a captive of neoliberalism, the international donor community could make India and Pakistan, finance-wise, more like China and Vietnam.
Seems a strong claim to me.
On page after page, I found factual misinterpretations and vague, dubious generalizations. Rather than enter a full-on debate with the argument, let me show you examples of how Bateman's passion seems to lead him to select and distort evidence. I find it hard to fully engage with a piece of analysis in which the conclusions so seem to drive the evidence. Think of the right column of this table as my marginalia:
Muhammad Yunus stumbled on the idea of microcredit on learning that 42 people in the village near his university were locked into debts worth only $42 total. "He decided to cancel this debt out of his own pocket, and was embarrassed to be almost royally feted for such a small sum." (p. 9)
Yunus did not cancel the debt, but substituted his lenient, no-interest loans for the moneylenders'. And the cited pages of Yunus's autobiography mention no such hero worship.
"By the late 1980s, microcredit and microenterprise development had become the international development community's anti-poverty intervention of choice." (p. 11)
According to a quick tally from the CRS database, the top category for foreign aid commitments worldwide in 1985--89 was agriculture, at $25 billion. Banking and finance was 14th, at $3 billion.
"As leading microfinance advocate Jonathan Morduch admitted..." (p. 13)
Jonathan is an academic who broke into the microfinance field by showing that the Grameen Bank took more subsidies than most people realized, then challenging the Pitt and Khandker study claiming that the Bank was reducing poverty. Seemingly, it takes a black-and-white world view to shoehorn him into the "advocate" category.
Bateman argues that while Yunus and other microfinance pioneers initially embraced outside subsidies if necessary to keep interest rates down for the poor, "the international development community soon began to disabuse the microfinance industry of this notion....The rejection of subsidies was essentially rooted in changing politics: specifically, the rapid ascendance of the neoliberal political project....Great efforts were made to commission as much supporting evidence as possible in favour of the chosen trajectory. Famously stepping up to the plate here was a group of mainly agricultural economists based at Ohio State University in the USA, who very conveniently provided a stream of arguments discrediting the notion of subsidies in rural finance and state involvement." (pp. 13--14).
Bateman very conveniently skips over the arguments and evidence assembled by the Ohio School, above all about how billions of dollars of government-directed, subsidized credit meant for poor farmers in Brazil, Indonesia, and other nations had been captured by rural elites. Meanwhile, the Grameen Bank was not forced into a neoliberal, subsidy-free form. As Bateman notes, Yunus's views on the proper role of government are similar to those of Milton and Rose Friedman, and presumably long have been. Grameen weaned itself off outside aid less because of ideology than because it prized its independence from donors it sometimes saw as meddlesome. And most of those donors---Norway, Sweden, the Netherlands, Japan, the United Nations---were never that enamored of neoliberalism.
On my work with Jonathan Morduch, Bateman writes, "Regarding the headline-grabbing results in the earlier Pitt and Khandker study...Roodman and Morduch obtained the opposite signs--that is, their results suggested negative impact." (p. 63, emphasis in original)
Actually, we wrote: "But we do not conclude that microcredit harms; rather...reverse or omitted-variable causation is driving the results."
"Supporters of the [commercial] microfinance model, centrally including both CGAP and ACCION, initially cheered the Compartamos IPO to the rooftops." (p. 22)
Actually, CGAP said that the IPO "provoked discussion and concern within the microfinance community and elsewhere. We think some of this concern is justified and some is not."
Bateman twists the term "impact evaluation" when he accuses almost all evaluators of measuring the financial state and client rolls of microcreditors rather than evaluating the impact on clients. (p. 63)
He compares microcredit to the abandoned Morgenthau Plan to deindustrialize Germany after World War II (pp. 93--101). He says that historically, profit-seeking commercial banks began moving into microcredit "and so out of traditional SME [small and medium-sized enterprise] lending," (p. 17) which lending he views, with some reason, as important for poverty-reducing economic growth. In fact, he accuses the leading Cambodian microfinance institution ACLEDA of drawing conventional Cambodian banks into microfinance and away from lending to bigger businesses and thus "playing an important role in undermining the country's desperate attempts to escape extreme industrial and agricultural backwardness." (p. 102, emphasis in original) Such claims beg the question of why profit-seeking banks would abandon a profitable line of business---or whether SME lending is in fact unprofitable, and why. In fact, the microcredit movement arose in part because of repeated failures to lend effectively to SMEs. Some combination of lack of credit-worthy customers and adverse business conditions makes it hard in many places. Since this was true before microcredit, it should not be blamed on microcredit.
I could go on.
There has been a lot of great research, writing, and blogging on microfinance in the last 18 months. I have in mind not my own work, but all that I have praised from this blog. If you want a realistic sense of what microfinance can achieve, I don't think you need to read this book.
The pendulum of public perception has swung against microfinance. That leaves the thoughtful observer, wary of extreme claims in any direction, with a puzzle. Is microfinance a bane or a boon or in between?
I suppose it is a measure of the power of randomized trials (RCTs) that arguments about their pros and cons continue to ricochet in the blogosphere. A fortnight ago, Philip Auerswald at The Coming Prosperity posted under the title, "Why Randomized Controlled Trials Work in Public Health...and Not Much Else." He elicited a high-quality comment stream. Yesterday, Tim Ogden developed his comments into the first of two stand-alone posts.
Most of the arguments are familiar ground for me, so I am not inspired to rehash them. For example, Tim lays out the validity and limitations of the external validity critique, so if you're not familiar with that, I recommend that you read his post.
I think there are worthy ideas all around in these debates. But I also espy a pattern of criticizing caricatures. Sometimes people attack extreme versions of others' views. Sometimes, in their enthusiasm, they exaggerate their own ideas, committing self-caricature. Then people abrade each other, and the conversation becomes heated. It makes life interesting because the appearance of disagreement surpasses the reality. The drama can produce confusion because people are talking past each other. But it can also force people to articulate their ideas more fully and carefully.
Thus, Dean Karlan points out that microcredit is not the panacea. Some microfinance leaders retort, "We never said it was." (Good to know.) Likewise, one wonders whether Phil had Dean in mind when he wrote of "those who most vociferously and naïvely advocate that we apply techniques from public health to economics." Whether he did or not, such rhetoric is a sure sign of jousting with caricatures.
Fortunately, I never fall into that trap.
It reminds me a little of Mad Men, which would be a bore if none of the characters lied. Not that I am accusing anyone of lying; far from it.
Anyway, reading Tim's post put me in mind of a paper by the philosopher of science Nancy Cartwright (not the scientologist Nancy Cartwright), which constitutes just about the sharpest critique of RCTs I have seen. (Angus Deaton, who led me to her work, is a smart and useful critic too, though he's jousted a lot with caricatures, seemingly bent on showing all the ways that RCTs are not, after all, perfect.)
Cartwright's paper is entitled "Are RCTs the Gold Standard?" The first sentence ends the suspense: "The answer to the title question, I shall argue, is 'no'." What I take to be Cartwright's main point is worth understanding, though I don't enitrely buy it. Phil alludes to it:
Econometric studies that seek to draw conclusions about effectiveness from data that span large geographical areas or highly varied populations thus typically have lower levels of internal validity, but higher levels of external validity.
So, once again, the fundamental issue is not the purity of the methodology employed (as exciting as such methodological purity is to the technically inclined) but rather the inherent complexity of the world being studied.
I'll explain. The main challenge for non-randomized studies is that to infer the impact of, say, microcredit, they need to assume that the treatment and control groups are statistically the same---or would be but for microcredit being available to or used by those in the treatment group. This assumption allows us to attribute any differences between the two groups to the impact of microcredit. But it generally isn't that credible. As a general matter, do you really think that people who take microloans are the same, statistically, as those who don't?
Creative methods are often used to tweak the treatment and control groups into greater comparability (more generally, to isolate exogenous variation in treatment). But most of the methods are much less convincing than randomization. For example, matching methods, as I explained in 2009, try to line up treated individuals with untreated ones according to observed traits such as income and number of children. But if the father of one family of four that lives on $5/day goes for a loan and the father of another family of four that lives on $5/day does not, wouldn't you wonder whether, despite the outward statistical similarity, the two men have different personalities and abilities; and whether the borrowing one is more likely to be entrepreneurial and to succeed even without the loan? And wouldn't that make it look as if microcredit was helping even when it wasn't?
Control groups in theory correct for the attribution problem by comparing people exposed to the same set of conditions and possible choices. However, control-group design is tricky, and skeptics hover like vultures to pounce on any weakness.— Elisabeth Rhyne, 2001
But suppose a researcher formed 1,000 such pairs and for each pair flipped a coin to decide who got offered a loan. That assignment of treatment and control should make the two groups match, statistically. For example, the groups should have about the same number of entrepreneurial fathers. That would be an RCT. No longer would we need fret about treating groups of people who are probably different as if they weren't. Cartwright:
One thing we certainly can do is to try to take into account all possible sources of difference between the test and target populations that we can identify. This is just what we do in matched observational studies. When it comes to internal validity, however, advocates of the exclusive use of RCTs do not take this to be good enough—matching studies are not allowed just because our judgements about possible sources of difference are fallible.
Then comes Cartwright's zinger: Turns out that RCTs don't get you around the problem of treating different people as if they are the same. After a study is run, you need to generalize from it. And what does an immaculately conceived study of the impacts of microcredit in Manila or Hyderabad tell us about the impacts in Accra or Lima? Nothing, unless you are prepared to make assumptions about the ways in which people in different places are the same:
Economists make a huge investment to achieve rigor inside their models, that is to achieve internal validity. But how do they decide what lessons to draw about target situations outside from conclusions rigorously derived inside the model? That is, how do they establish external validity? We find: thought, discussion, debate; relatively secure knowledge; past practice; good bets. But not rules, check lists, detailed practicable procedures; nothing with the rigor demanded inside the models.
So while RCTs may be superior within their confines---"internally valid"---the process of generalizing from them remains fraught with precisely the difficulties RCTs are supposed to solve. We cannot avoid these difficulties because the derivation of general conclusions from specific results is essential if the RCTs are to be part of science. It's also essential if they are to be useful.
In sum, while non-randomized methods have problems of comparability within, randomized methods have them beyond. RCTs avoid messy questions about who to equate to whom during implementation only to slam into those questions upon interpretation.
I was struck by this symmetry when I first grasped it. As Phil writes, the "inherent complexity of the world being studied" cannot be dodged. But on reflection, I don't think researchers and those who depend on them are as elegantly trapped as this picture suggests. For that to be the case there would need to be an unavoidable and binding trade-off between randomization and breadth of sample. In Cartwright's favor, I imagine that it is less practical to run a pan-Indian RCT than do a non-randomized study of data from nationally representative surveys, as Angus Deaton has. To the extent that work like Deaton's is more practical, it offers an approach to knowledge that may not be as internally valid as RCTs but is easier to extrapolate to all Indians. Cartwright gives a compelling U.S.-based example:
Consider the widespread correlation between low economic status and poor health, and look at two opposing accounts of how it arises… Epidemiologist Michael Marmot from University College London argues that the causal story looks like this:
Low status--> ‘stress’ --> too much ‘fight or flight’ response --> poor health.
In contrast, Princeton University economist Angus Deaton suggests this:
Poor health --> loss of work --> low income --> low status
Deaton confirms his hypothesis in the National Longitudinal Mortality Study (NLMS) data. He reasons: if the income-mortality correlation is due primarily to loss of income from poor health, then it should weaken dramatically in the retired population where health will not affect income. It should also be weaker among women than men, because the former have weaker attachment to the labour force over this period. In both cases these predictions are borne out by the data. Even more, split the data between diseases that something can be done about and those that nothing can be done about. Then income is correlated with mortality from both—just as it would be if causality runs from health to income. Also, education is weaker or uncorrelated for the ones that nothing can be done about. Deaton argues that it is hard to see how this would follow if income and education are both markers for a single concept of socio-economic status that is causal for health.
But logically, there is no inherent trade-off between randomization and breadth of the sample. Economist Roland Fryer just won a MacArthur "genius" award for work that includes "a randomized experiment with well over 20,000 students from more than 200 schools in three cities" in the United States. Why couldn't a national microcreditor work with researchers to conduct experiments on a national scale? In India, the spread of high-tech systems for insurance and identification and voucher distribution may make cheap, large-scale experiments feasible.
Moreover, I think Cartwright does not acknowledge that there is something fundamentally different about experiments, something for which there is no counterpart in large-scale, non-experimental studies. Experiments introduce novel variation. Randomized ones introduce variation that is effectively uncorrelated with everything else in the universe. That is special.
I think that is why, as I reflect on the available studies of the impact of microcredit, Cartwright's dualism doesn't jive. True, the best non-randomized studies are more geographically diverse and representative than the best randomized ones. The Pitt and Khandker studies use data from 87 villages sprinkled among 29 of Bangladesh's 391 subdistricts (as enumerated in 1991). Yet for reasons elaborated elsewhere, I don't believe these studies convincingly measure the impacts of microcredit. They aren't as persuasive as the Deaton example above. (Perhaps this has something to do with the particular difficulties of evaluating an intervention, which Deaton does not do in the example.) Thus the fact that the Bangladesh studies draw data from across the country, that they more externally valid, is paltry compensation. It tells me that microcredit has unknown impacts not just in those 87 villages, but in all of Bangladesh.
When it comes to generalizing, if it's garbage in, it's garbage out. If the choice is between a study done in one place that I can believe and one done everywhere that I can't, then the choice is easy. Based on my experience, a more promising path than large-scale, non-randomized impact studies is randomized ones done in diverse locales.
David Roodman, creator of the Commitment to Development Index (CDI), has devised a measure of foreign aid flows that takes into account the interest payments that developing countries make to rich country creditors. The Net Aid Transfers data set, which is a component of the CDI, is available for download.
Julia Clark and David Roodman investigate whether better ranking of think tanks is possible by exploiting modern tools for measuring citations in both traditional and new media, as well as in academe. They find that with modest effort the status quo of ranking the tanks can be improved.
The Burnside and Dollar (2000) finding that aid raises growth in a good policy environment has had an important influence on policy and academic debates. We conduct a data gathering exercise that updates their data from 1970-93 to 1970-97, as well as filling in missing data for the original period 1970-93. We find that the BD finding is not robust to the use of this additional data. (JEL F350, O230, O400)
The Commitment to Development Index of the Center for Global Development rates 21 rich countries on the “development-friendliness” of their policies. It is revised and updated annually. In the 2004 edition, the component on foreign assistance combines quantitative and qualitative measures of official aid, and of fiscal policies that support private charitable giving.
Recent literature contains many stories of how foreign aid affects economic growth: aid raises growth in countries with good policies, or in countries with difficult economic environments, or mainly outside the tropics, or on average with diminishing returns. The diversity of these results suggests that many are fragile. I test 7 important aid-growth papers for robustness. The 14 tests are minimally arbi-trary, deriving mainly from differences among the studies themselves. This approach investigates the importance of potentially arbitrary specification choices while minimizing arbitrariness in testing choices. All of the results appear fragile, especially to sample expansion.