When Is X-centric Eccentric? The Curious State of Development Economics

“[A]s a general rule, the effects of causes are far more accessible to our study
than the causes of effects.”

John Stuart Mill, A System of Logic

Most research starts from wanting to explain the causes of effects. Why is Denmark productive and prosperous and Haiti unproductive and poor and India in-between? Why did South Korea’s economy grow so much faster than Ghana’s over the last 50 years (such that Korean GDP per capita was 10 times as high in 2011)? Why did China’s economy grow so rapidly after 1978 following decades (centuries?) of economic stagnation? And of course development economics is interested in human well-being in many dimensions and the well-being of the worst off, not just averages. Why did Indonesia nearly eliminate destitution (low-bar poverty) from the 1970s to 1996? Why is measured learning so much higher in Vietnam than in Peru? Why did life expectancy in Bangladesh increase in the 40 years from 1977 to 2007 almost as much as it had from 1800 to 1977? And we are also interested in the differences across individuals: Why do some people make so much more money than others? Why do some students perform well and others poorly? Why do some people escape poverty and others not?

These “causes of effects” questions are “y-centric”: there is some variable of intrinsic interest (Y) that we seek to understand by relating to its determinants, which, given the history of mathematical notation, we call “X” variables. A successful model is one that explains a great deal of Y, both in levels and in its dynamics, with a relatively parsimonious set of Xs. From a practical and pragmatic view, it is terrific if the Xs are susceptible to control by the purposive actions of some group of agents. A model that explains that some countries are poor because they are landlocked may be true but is of less practical interest than a model that explains that some countries are poor because their state chose a misguided economic policy, because one cannot be changed while the other can.

A different approach to research is “x-centric” or “effects of causes,” which is to start from an X that is under some agent’s active control and ask: “What is the impulse response function of Y with respect to purposive variations in X?” Will I lose weight if I shift from my current diet to diet X? Will this child learn more if provided with a textbook by an NGO? Will this firm’s profits grow if it is provided management training? Will a child attend more school if I threaten to take away his/her family’s cash transfer if he/she doesn’t attend?

The beauty of an x-centric approach is that a researcher can do empirical work in the complete absence of a model or theory of Y. All I need to do is randomize units into “treatment” and “control,” and do X to the treatments and not the controls, and I can trace out the (mean) impulse response function on Y of doing X by comparing the paths of the treatment and control. Any researcher can then make (seemingly) rigorous statements of the type “In the following context and background conditions, my research did X and the (average) impact on Y of doing X was ΔY.”

The danger of the x-centric approach is that a researcher can do empirical work in the complete absences of a model or theory of Y. This can be sold as an advantage and the search for “causes of effects” dismissed as irrelevant, or worse (e.g., Gelman and Imbens 2013). But without a model of Y, x-centric research can easily become eccentric, in many ways.

X-centric can become eccentric by becoming opportunistic.

Rather than asking “what would help understand the observed (or achievable) variation in Y in ways that allow maximal benefit?” the research asks “who will let me do X or evaluate the impact of their doing X?” There are good reasons to believe that there are massive biases in willingness to let X be randomly evaluated (see It pays to be ignorant for a model of that endogenizes willingness to allow a randomized evaluation) and research opportunism on X does not add up to a sensible set of research questions about Y.

X-centric can become eccentric by being driven by statistical power.

Suppose the reason people are poor is that they live in poor countries and poor countries are characterized by low productivity of all factors. But suppose the only level at which one can randomize enough units to generate statistical power to track the impact of X is at the individual (or firm) level. This will bias research away from useful answers about what generates variation in Y at the regional or country level to an eccentric research strategy that examines only that subset of determinants of Y that can be “powered.” Almost certainly little to nothing about why Denmark is Denmark and Haiti isn’t, or why China of 2017 isn’t China of 1977, can be “powered.” But whether conditional cash transfers induce more attendance at school can be individuated, and sample sizes of individuals can generate statistical power, so we have dozens and dozens of RCT studies of CCTs. This is the case even though we know and knew, for sure, both before and after the CCT research that CCTs were not a very big determinant of even school attendance (across countries, over time, or across individuals), much less learning, much less overall development.

X-centric can become eccentric by never asking how big.

A casually identified estimate of the impact of X on Y does not answer the question of how big the impact is relative to the magnitude to be explained. Using a large experiment, I could identify as statistically significant an impact that was miniscule in explaining the observed variation in Y. A field experiment using variations in X that are novel to the field experiment, such as a brand new technique, does nothing to explain the existing variation. Much of the x-centric literature is satisfied with showing “no impact” or a “positive impact” without any sense of “how big?” While one might argue that having a complete model of Y is unnecessary to knowing whether X is a cost-effective intervention, without some sense of “how big” it is impossible to know whether there are other, potentially much bigger Xs left unexplored.

X-centric can become eccentric by ignoring external validity.

 If the essence of science was doing experiments, there would be a Nobel Prize for alchemy. The essence of science is theory because only theory provides the framework within which individual empirical results can be evaluated and aggregated. That doing X had impact ΔY in a given context is, in the absence of theory, zero rigorous evidence about the likely impact of doing X in any other context. For that matter, it is only theory that can tell us what “context” even means. Without a theory of Y we cannot explain the why of the impact of X, and without that, there is no hope of application. It is easy to show that the simplest possible OLS evidence can (and does in some cases) provide a better prediction of policy impact than “rigorous” evidence from other contexts (Pritchett and Sandefur 2015).

X-centric can become eccentric by ignoring implementation feasibility.

What a researcher can do in a “field experiment” or an NGO might be willing and able to do with an impact evaluation built in might have nothing to do with what is scalable, particularly scalable by the public sector. So the policy/program/project “lessons” from “rigorous” evidence about doing X are irrelevant if the government cannot or will not do the X. The only rigorous evidence about the scalability of rigorous evidence says it isn’t (Bold et al 2013).

X-centric can become eccentric by ignoring rather than encompassing all other evidence about X.

To be progress, new evidence in science has to encompass all previous evidence into a more coherent whole. There is typically a large body of evidence about the (partial) correlations of various Xs with Y. The existence of a single study in a single context that provides an estimate of the impact of X does not “contradict” any of this previous evidence. It is widely ignored that there is no coherent guidance as to how a single “rigorous” study should change the Bayesian priors on the interpretation of existing evidence from other contexts (Pritchett and Sandefur 2014).

A typical development seminar invokes some variable of interest Y (growth, poverty, schooling, health, firm growth, gender empowerment, corruption). Then the ritual incantation: “what does ‘the evidence’ say?” This is followed by a description of the implementation and findings of one experiment in one small region or sector in one country. There is no return to the Y or why: how much has our understanding of the true variable of interest been enhanced by X? And do we understand the process of Y any better? For all its problems, the y-centric approach had its virtues, and for all its methodological strengths, the x-centric approach can easily become eccentric.