Yesterday, the UK’s Independent Commission for Aid Impact (ICAI) published a hard-hitting report on the Department for International Development’s private sector development work. Based on an evaluation of DfID’s private sector work in Tanzania, Uganda and Bangladesh, the report highlights two problems—(1) the pressure to demonstrate results against measurable indicators and (2) the lack of assessment of the cumulative impact of DfID’s work. The report gives DfID a grade of “amber-red” which means that “the programme performs relatively poorly against ICAI’s criteria” and that “significant changes should be made.”
In particular, the report questions DfID’s “target culture” which “arguably provides incentives to report large numbers for impact wherever possible.” DfID’s claims that the number of people whose incomes will go up in Tanzania as a result of its Agricultural Growth Programme is nearly 40,000. Given that the programme is relatively new, the ICAI report argues that the logic through which this would occur is “necessarily speculative.” On DfID’s Rural Support programme, the ICAI report found that the margin for error in reporting the number of beneficiaries was “very large—yet unacknowledged.”
This problem applies to other agencies as well. The World Bank Group sometimes makes absurd claims about the number of jobs generated by its projects and programs. The Bank’s Ghana Gateway project claims to have generated over 300,0000 jobs for Ghana’s citizens. The International Finance Corporation (IFC) claims that’s its various investments have created millions of jobs. There is no discussion of counterfactuals—of what would have happened if the Bank or IFC had not been there. Neither is there any mention of whether these projects create new employees or simply move them over from the informal sector, or of the estimation procedures behind the numbers.
DfID’s management would be well-served to read a 2011 blog post from David McKenzie, who is widely regarded as one of the foremost researchers on private sector development. David cites a 2010 IFC report which argues: “We know that it takes more than volume to meet the needs of the poor. That is why we carefully target our resources, selecting where our financing and advice can be deployed most effectively. And we set measurable goals to gauge our impact, and improve our performance. In 2009, our clients provided 2.2 million jobs, including nearly 514,000 in the manufacturing and services sectors.” David says “clearly this is in no way a measure of impact, and presenting it as such is both disappointing to both researchers working on credible measures of the impacts of different projects, as well as to the readers who are being presented this information as if it is in any way informative about the effects of the World Bank Group’s work.”
The development profession is under pressure to report outcomes and to link programmatic investments to measurable indicators. Broadly speaking, this is a good thing. But the measurement of the number of jobs created by a particular development intervention is very difficult and in many cases, impossible. My own recent research, coauthored with Leonardo Iacovone, suggests that there is much that is yet to be explained when it comes to understanding job creation in poor countries.
DfID is a very good development agency with a capable and dedicated staff. There is no doubt that its work is of great value in many poor countries. But it must not rush headlong into inserting jobs or other outcome measures into its projects, and it must not pressure other donors into doing so.
I look forward to DfID’s response to the ICAI report (which I will blog), and to a substantive conversation around the issue of job creation.
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise.
CGD is a nonpartisan, independent organization and does not take institutional positions.
