Why African Stats Are Often Wrong

As African leaders meet in Washington this week, one issue is not on the agenda: the poor quality of basic economic and social data in the region.  Maybe this year’s GDP re-base in Nigeria, which resulted in an 89 percent increase, was a tip-off? While inconvenient to the #AfricaAscending narrative around town, our recent work suggests that many basic data are in fact systematically distorted.

In our paper, we find that misrepresentation of national statistics in education and health does not occur merely by accident or because of a lack of analytical capacity — at least not always — but rather that systematic bias in administrative data systems stems from incentives of data producers to overstate development progress.

Administrative and Survey Data Don’t Match

Comparing administrative and survey data across 46 surveys in 21 African countries, we find a bias toward overreporting school enrollment growth in administrative data. The average change in enrollment is roughly one-third higher (3.1 percentage points) in administrative than survey data (an optimistic bias that is completely absent in data outside Africa. Delving into the data from two of the worst offenders, Kenya and Rwanda, shows that the divergence of administrative and survey data series was concomitant with the shift from bottom-up finance of education via user fees to top-down finance through per-pupil central government grants. This highlights the interdependence of public finance systems and the integrity of administrative data systems. Difference-in-differences regressions on the full sample confirm that the gap between administrative and survey of just 2.4 percentage points before countries abolished user fees grew significantly by roughly 10 percentage points afterward.

Donors also play a role. In 2000, GAVI Alliance offered eligible African countries a fixed payment per additional child immunized against diphtheria-tetanus-pertussis (DTP3), based on reports from national administrative data systems. Building on earlier analysis by Lim et al. (2008), we show evidence that this policy induced upward bias in the reported level of DTP3 coverage amounting to a 5 percent overestimate of coverage rates across 41 African countries.

It’s Not Just Education and Health

Other work by Justin suggests that official estimates of consumer price indices have been inaccurate, and — once correcting for these accuracies — rates of growth and poverty reduction in Africa are modestly slower on average than published estimates based on official data.   

Inaccuracies in basic data are due in part to perverse incentives created by connecting data to financial or reputational rewards without checks and balances. But the problem of inaccuracy is also related to political interference and statistical agencies that have been inadequately and inconsistently funded over the years. Together, these factors make up a political economy of bad data.

To get to a political economy of good data, our joint working group report with the African Population and Health Research Centre lays out some ideas: (i) fund more and differently; (ii) build institutions that can produce accurate, unbiased data; and (iii) prioritize the accuracy, timeliness and availability of the basic data on births and deaths; growth and poverty; sickness, safety and schooling; and land and environment, that policymakers and citizens can use to generate real progress in development.