With rigorous economic research and practical policy solutions, we focus on the issues and institutions that are critical to global development. Explore our core themes and topics to learn more about our work.
In timely and incisive analysis, our experts parse the latest development news and devise practical solutions to new and emerging challenges. Our events convene the top thinkers and doers in global development.
In development economics, statistical analysis often begins with data from many observational units--households, companies, or countries--over just a few time periods. Two analysis techniques, the "difference" and "system" generalized method of moments (GMM) estimators, are becoming popular for studying causal relationships among variables in this "short panel" setting. This working paper by CGD research fellow David Roodman dramatizes a common but underappreciated problem in how the estimators often are applied. It explains the risks, offers some simple techniques for reducing them, and illustrates, with reference to two early and widely noted applications to studying what causes economic growth: Forbes (2000) on income inequality and Levine, Loayza, and Beck (2000) on financial sector development. Endogenous causation proves hard to rule out in both papers, meaning that we cannot be as confident, after all, that inequality or financial development causes growth.
Judging by current practice, many researchers do not fully appreciate that popular implementations of these estimators can by default generate results that simultaneously are invalid yet appear valid. The potential for "Type I errors"--false positives--is therefore substantial, especially after amplification by publication bias.
In particular, the estimators tend to generate many instruments compared to the number of observational units or data observations, which can cause a number of problems already documented in the literature. The instruments can overfit endogenous variables, failing to expunge their endogenous components and biasing coefficient estimates. They can vitiate the Hansen J test for joint validity of those instruments, as well as the difference-in-Sargan/Hansen test for subsets of instruments. The weakness of these specification tests is a particular concern for system GMM, whose distinctive instruments are only valid under a non-trivial assumption.
This paper is forthcoming in the Oxford Bulletin of Economics and Statistics. David Roodman is also the author of the most popular program implementing these estimators (see his CGD working paper 103 How to Do xtabond2).