Topics:
Expertise
Migration and development, economic growth, aid effectiveness, economic history
Bio
Michael Clemens is co-director of migration, displacement, and humanitarian policy and a senior fellow at the Center for Global Development, where he studies the economic effects and causes of migration around the world. He has published on migration, development, economic history, and impact evaluation, in peer-reviewed academic journals including the American Economic Review, and his research has been awarded the Royal Economic Society Prize. He also serves as a Research Fellow at the IZA Institute of Labor Economics in Bonn, Germany, an Associate Editor of the Journal of Population Economics and World Development. He is the author of the book The Walls of Nations, forthcoming from Columbia University Press. Previously, Clemens has been an Affiliated Associate Professor of Public Policy at Georgetown University, a visiting scholar at New York University, and a consultant for the World Bank, Bain & Co., the Environmental Defense Fund, and the United Nations Development Program. He has lived and worked in Colombia, Brazil, and Turkey. He received his PhD from the Department of Economics at Harvard University, specializing in economic development, public finance, and economic history.
Click here for full CV
May 10, 2018
Event
During the early 1990s Germany received over half a million Yugoslavians fleeing war. By 2000, many of these refugees were repatriated. In their new paper, Dany Bahar and his co-authors exploit this episode to provide causal evidence on the role migrants play in contributing to productivity shifts in their home countries after their return, as explained by changes in comparative advantage.
February 20, 2018
Blog Post
UN Member States are gathering today in New York at the United Nations Headquarters for the first round of negotiations on the Global Compact on Migration zero draft. It is a once-a-generation chance to shape migration cooperatively, for mutual benefit. Global migration governance is, in its current form, unprepared and insufficient to manage future flows.
The negotiations on the first draft of the Global Compact are therefore critical—especially considering the very brief window of opportunity between the end of this month and the start of summer this year, when the negotiations period is expected to close. The drafting process has cast a wide net. The “zero draft” of the Compact includes the multitude of commitments, ideas, and issue areas heard from Member States and other stakeholders throughout the months-long consultation phase last year. But with all suggestions in this one kitchen sink—and a very technical one at that—the side effect is that many could find something to object to in the current document.
I’ve been at some of the key meetings in this process. Some of the presentations and conversations I’ve heard point to a handful of potential sticking points in the negotiations. Some could bite in the short term as the text is negotiated, others in the long term as it is implemented. Here are three.
Conflicting Member State desires on high-skill versus low-skill migrationAs Michael Clemens has articulated before, this tension is a crack at the heart of Global Compact negotiations. Richer, developed countries are wary of low-skill immigration, while poorer developing countries are skeptical of high-skill emigration, fearing a “brain drain.” In previous years, this crack has been largely patched over by promoting “circular” or temporary migration, in which low-skill workers are supposed not to stay forever and high-skill workers are supposed not to leave forever. But labor rights-focused organizations have been vocal within the Global Compact process about their objections to circular migration, raising a number of important concerns about migrants’ rights.
An alternative tool to address the same conflicting interests is Michael’s proposal for Global Skill Partnerships. A migrant-destination country supports vocational training for potential migrant workers before they migrate, in the country of origin, bundled with training for non-migrants. Developed countries win out via getting migrants with the skills to integrate and contribute right away, developing countries gain from increased human capital, and migrants have the opportunity to move along lawful channels to pursue better lives for themselves and their families.
New lawful channels—such as that laid out in the Global Skill Partnership example—are necessary given the current and projected demographic and economic pressures. But this discussion introduces a second potential sticking point: Are returns an obligation, and to what extent is the Global Compact going to be some sort of “grand bargain” on greater lawful channels conceded to get greater cooperation on returns?
To what extent are returns an obligation?I heard several times in Member State remarks at the Stocktaking meeting in Puerto Vallarta that, from the perspective of numerous receiving European states, developing countries have an obligation to accept back their returned nationals. The zero draft of the compact calls on states to “cooperate in facilitating dignified and sustainable return, readmission and reintegration.” While the draft never names the “obligation” of states to take back their failed asylum seekers, the question of returns and the extent to which there is a universal obligation for states to accept back their migrants may become a stumbling block in negotiations. There are indications some Member States are less willing to discuss the potential for new lawful migration channels until the unauthorized migrants within their borders are dealt with—marking a contradiction Michael highlighted previously.
However, as we lay out in a recent policy brief, returns and immigration enforcement do not need to be undertaken independent from and before creating new lawful channels. Rather, past experience indicates creating new lawful channels, when coupled with enhanced enforcement, can displace and suppress unauthorized migrant flows—effectively achieving the aims of developed countries to reduce irregular migration, while providing a viable and lawful alternative migration path. In the example of the Bracero program, the reduction in unauthorized immigration happened during a time when legal pathways were expanded and immigration enforcement was enhanced (albeit in ways that were controversial), which required partnership with the Mexican government and willingness to cooperate, including on issues of deportations. More research on the nexus between irregular migration, regular pathways, and enforcement is needed, but this one example could prove a starting point for productive considerations.
More broadly, the question of returns introduces what could be a third sticking point related to mixed flows—something that doesn’t necessarily seem like a sticking point significant enough to stall negotiations in the weeks to come, but could prove a point of conflict down the road, including during implementation.
Who are the people within mixed flows, and how is mixed migration governed?The de facto answer thus far has been to split the global compacts into two—one for migration, and a separate one for refugees—and proceed as if there is a bright line distinguishing forced displacement from non-forced migration. But migration flows are often “mixed,” with migrants seeking both security and opportunity. The Zero Draft of the Compact mentions mixed flows several times, with calls for protection and addressing vulnerabilities. But sticking points around mixed flows are likely to arise at some point unless Member States seriously tackle the question of what constitutes mixed flows and what governance mechanisms are needed.
Central America’s Northern Triangle is one illustration: recent CGD research shows the unaccompanied child migrants (UACs) have been fleeing a complex mix of economic factors and generalized violence. However, most are not considered refugees, as generalized violence is not covered under the 1951 Convention. Central American and other countries have previously pushed for greater attention and effort to address the child migrant crisis stemming from the region. The more likely substantial stalling could come in implementation versus negotiations. On the broader scale, Central American UACs are an example in part of a population that falls into the “migration” category and that is made up of people who are therefore missing the likely necessary protections afforded to refugees.
Calls for more and better data and research on the “root causes” of migration, as increasingly heeded, will reveal even more nuances in driving factors of mixed flows that fall well outside a simplified category of “economic migrant.” Unaddressed questions of mixed migration could eventually blur the dividing line enough that implementation of the Global Compact is hindered.
The window of opportunity to address these and other potential sticking points is a narrow one. Countries are set to meet in Morocco next December to formally agree on the final Global Compact on Migration, and the de-facto final text will be agreed to well before that. Overcoming these conflicts won’t be easy. But the alternatives—a bootless Compact or pure unilateralism—cannot rise to this historic challenge.
February 14, 2018
Blog Post
Richer countries are under pressure to respond to and suppress high levels of irregular migration reaching their borders. One prominent recommendation is for richer countries to expand opportunities for lawful or regular migration. Suppose they do. Will more regular migration simply raise migration overall, or will it substitute for and reduce irregular migration?
The question underlies discussions around the Global Compact for Migration, a future international agreement on migration governance now being negotiated by United Nations Member States. Country delegations will gather in New York next week at UN Headquarters for the first round of intergovernmental negotiations.
Many governments around the negotiating table are caught in a dilemma. It is politically difficult to say “yes” to expanding channels for regular migration when they are currently dealing with large numbers of irregular migrants who have already arrived. But many understand that overwhelming demographic pressures make some future migration inevitable, and closing regular migration channels too tightly might actually exacerbate irregular migration in years to come.
Those governments have rightly asked for clear evidence that regular migration channels can help suppress irregular migration. There has been very little such evidence in the policy debate.
In a new policy brief, we review evidence from one historical episode in which regular migration channels did clearly substitute for and suppress irregular migration. This is the last 76 years of policy change at the US-Mexico border, scene of one of the largest bilateral migration flows on earth.
We find:
Lawful migration channels are often suggested as a tool to reduce unlawful migration, but often without much evidence that they work.
There is evidence that lawful channels for migration between Mexico and the United States have suppressed unlawful migration, but only when combined with robust enforcement efforts.
Massive demographic pressures for migration between Africa and Europe will continue to resemble past pressures between Mexico and United States. The evidence from the US suggests that regular migration channels could be one critical tool for Europe, alongside enforcement, to suppress irregular migration.
Here is a glimpse of how regular migration channels (green) have displaced irregular migration from Mexico (red) over time—primarily during periods of robust enforcement:
For the details, and more data, read the full brief.
February 12, 2018
Blog Post
Migration is a volatile political issue worldwide. It will become explosive in decades to come, as demographic change raises pressure on the traditional ways of regulating migration. Right now there is a window of opportunity for countries to agree on how to manage migration better. That window will close in July.
A New BargainThe Global Compact on Migration is a non-binding agreement expected to be signed by most countries on earth near the end of 2018. They are negotiating the text now. This Compact is a once-in-a-generation chance for the world to strike a new bargain on migration. There will not be another chance soon to get this bargain right.
As it stands, the Zero Draft of the Global Compact is mostly not a bargain. It is mostly a detailed and inspirational list of general ways that countries can coordinate to advance their shared interests: Everyone wants migration to be safe and orderly. Everyone wants to share information and reduce racism. Everyone wants to collect better statistics and promote sustainable development. Let’s work together.
But the final draft of the Global Compact needs something else.
It must be a bargain. In a bargain, people don’t pretend their interests are aligned. They admit conflicting interests. And they partially concede, in order to come out better than they could by going it alone.
Right now, vast numbers of people have deep concerns about migration. That includes people in the countries that migrants leave. Politicians cannot go to them with a document touting the shared interests of all. Many see real conflicts in the interests of more developed and less-developed countries, and conflicts between migrants and non-migrants.
A core conflicting interest, to put it roughly and bluntly, is that many in richer countries want less low-skill migration, but many in poorer countries want less high-skill migration. This is a crack in the heart of the negotiations on the Global Compact.
A Compact that simply proposes “more” low-skill or “more” high-skill migration does not address these concerns. It is not a bargain. On the other hand, a compact that does not explore new lawful channels for migration—of some kind—ignores overwhelming demographic, economic, and geographic realities.
So the Compact needs three things: Bargains—with concessions. Specific innovations—because proposing “more” migration cannot rise to this challenge. And small step commitments—not committing to some “answer”. No one knows what the “answer” is, but countries can learn more by committing to pilot a range of approaches testing what can create tangible, visible, mutual benefits. A Way Forward: Global Skill PartnershipsThere are seeds for a bargain in the draft Compact already. Negotiators can cultivate them.
I was delighted to see, named in the Zero Draft of the Compact, one proposal that does rise to this challenge: a commitment to pilot Global Skill Partnerships. (It’s in part 32(e) of the Zero Draft.) I was involved in designing the Global Skill Partnership proposal. Today this proposal has worldwide momentum—last year it was a focus of the Global Forum on Migration and Development in Berlin, and this year was endorsed by the UN Secretary General.
A Global Skill Partnership is a bilateral agreement in which a migrant destination country gets directly involved in creating human capital among potential migrants in a country of origin—before they migrate—and non-migrants. Destination-country employers and governments provide technology and finance for vocational training inside the country of origin, and job placement at the destination—critically, paired with the training of other workers who remain in the origin country.
For example, think of a nursing school in central Banjul, in West Africa. The EU and European employers transfer finance and technology to support that school’s training of two kinds of people: specialized nurse assistants to work in Europe, and nurses to work in the Gambia. Migrants coming from such a program would have precisely the skills they need to integrate quickly and contribute maximally when they get to Europe. European governments would save money, not spend money: they need nurse assistants regardless, and training them the Gambia is enormously cheaper. The Gambian government gets more nurses, not fewer, and can cooperate as an equal partner without fears of “brain drain.” Young girls in Banjul could walk past that school and see that, growing up, they’d have concrete options other than risking their lives to migrate unlawfully as a sub-minimum wage maid or sex worker.
A Global Skill Partnership is a bargain because it means concessions. Richer countries concede to support origin-country training centers and offer labor-market access, but in return they get precisely the kind of immigrants they want. Poorer countries concede to more skilled migration, but in return they get support for training at home. This is a bargain. If either acts alone, they those.
It’s a wonderful sign that the specific bargain of Global Skill Partnerships is endorsed in the Zero Draft. It’s one of the best tools we have to mend the crack at the heart of the negotiations. The Compact needs this and other bargains if it is to address real and widespread concerns about migration.
As you think through what should be defended in the Zero Draft, and what should change, please push for the final draft to contain bargains with concessions to real concerns of conflicting interest, commit to specifically-named innovations, and commit to testing those innovations, not prematurely embracing them. Don’t describe trees; plant seeds.
One way to do all this is to insist that the final draft of the Compact retain the Zero Draft’s commitment to test out Global Skill Partnerships at a small scale. That’s just the beginning.
February 12, 2018
Publications
As waves of migrants have crossed the Mediterranean and the US Southwest border, development agencies have received a de facto mandate: to deter migration from poor countries. Will it work? Here we review the evidence on whether foreign aid has been directed toward these “root causes” in the past, whether it has deterred migration from poor countries, and whether it can do so.
February 12, 2018
Publications
In response to the recent migrant and refugee crisis, rich countries have redoubled policy efforts to deter future immigration from poor countries by addressing the “root causes” of migration. We review existing evidence on the extent and effectiveness of such efforts.
Pages
April 22, 2014
Blog Post
When opportunities for corrupt earnings rise, is there more corruption? This fundamental question is the subject of new, frontier-pushing research by two young stars of development economics: CGD alumnus Sandip Sukhtankar and his co-author Paul Niehaus. I was delighted to learn this week that their work (ungated) just won this year’s American Economic Association prize for best economic policy paper.
Want more from CGD? Sign up for our weekly newsletter
When officials can earn more from corrupt acts, you might expect there to be more corruption. But wait: Suppose that corrupt acts are risky and if you get caught you’re out of the game. In that case, greater opportunities for corrupt acts might reduce corrupt behavior going forward. The bigger pie makes officials want to get their slice, but to do that they need to take fewer risks to stick around longer. Sandip and Paul call this the “golden goose” effect, after the fable discouraging myopia. In economics these countervailing forces are called the “substitution effect” and the “income effect.” This might seem like mere speculation; Sandip and Paul have shown that—at least in India—it really happens.
They study India’s National Rural Employment Guarantee Scheme, a vast program that guarantees some minimal employment to jobless people in the countryside. Because the wages must travel from the central government all the way to rural workers, there are ample opportunities for skimming. Sandip and Paul show that when the scheme’s wages rose, the greater opportunities for skimming actually caused less skimming. They show this “golden goose” effect by comparing projects affected by the wage hike to similar but unaffected projects. Among other things they had to do prodigious data collection: they ingeniously scraped a massive dataset from a public website that was set up to release data about payments for individual projects, one project at a time. They then compared the scraped data to their own survey data on how much workers had actually received to measure how much money had been skimmed between the original project design and the actual delivery.
I’m proud that Sandip is one of several people who, after working here at CGD as research assistants or in other junior staff positions, have gone on to become prominent academic researchers. Amar Hamoudi, now of Duke University, helped to lay the organization’s foundation when it launched in 2001. Jessica Gottlieb of Texas A&M was once a junior staffer on the CGD health team, among other things writing for the major book Millions Saved. Three of my own former research assistants at CGD are now successful academics: Samuel Bazzi of Boston University, Rikhil Bhavnani of the University of Wisconsin-Madison, and Bilal Siddiqi, who just joined the World Bank’s superb Development Economics Research Group. Many other former junior staffers here are also stars outside of academia. One of CGD’s most precious assets is our ability to attract brilliant young people like these.
May 22, 2017
Blog Post
En español
Read Part Two: Answering Questions about Our Research
Update: On August 2, 2017, White House senior adviser Stephen Miller called for slashing U.S. immigration by half, citing the study that is discussed in this blog post. A more recent discussion of the flaws in that study is here, and the updated research paper is here.
Do immigrants from poor countries hurt native workers? A study by an influential immigration economist at Harvard University recently found that a famous flood of Cuban immigrants into Miami dramatically reduced the wages of native workers.
That study lit a small fire last year in The Atlantic, National Review, and New Yorker. Celebrity advocates of restricting immigration declared that the study had “nuked” their opponents’ views. The study, by George Borjas, is a centerpiece of his own mass-market book on immigration, We Wanted Workers, cited by the U.S. Attorney General as proving the economic harms of immigration.
But there’s a problem. The Borjas study had a critical flaw that makes the finding spurious. That flaw is explained in a new research paper that I co-authored with Jennifer Hunt, who is the James G. Cullen Professor of economics at Rutgers University. In this blogpost, I explain the flaw and why it reinforces earlier findings that the Mariel Boatlift influx of Cuban immigrants did not reduce wages of Miami workers.
The flaw was not known to the academic peers who reviewed the study for publication. The finding that the Cuban immigrants caused Miami wages to collapse is an artifact of how the study handles its data. The study is based on a wage survey from a sample of workers. The study focuses on a small group within that larger sample, a group where the sample shifted to include a lot more black male workers with relatively low wages—simultaneously with the Boatlift. This has the effect of sharply reducing the average wage of people in the sample, but this had nothing to do with the Cuban influx.
This is noteworthy because what happened in Miami is the one historical event that has most shaped how economists view immigration. To explain what went wrong, I have to pan out and describe the historical setting under study.
The Mariel BoatliftFor an economist there’s a straightforward way to study how low-skill immigration affects native workers. Find a large, sudden wave of low-skill immigrants arriving in one city only. Watch what happens to wages and employment for native workers in that city, and compare that to other cities where the immigrants didn’t go.
An ideal experiment like this actually happened in Miami in 1980. Over just a few months, 125,000 mostly low-skill immigrants arrived from Mariel Bay, Cuba. This vast seaborne exodus is known as the Mariel Boatlift. The workforce of Miami rose by 8%, the low-skill workforce shot up by 20%. If immigrants compete with native workers, this is exactly where you should see natives’ wages drop. Of course, if they did drop, you would want to make sure that it wasn’t just a regional or national trend. You would compare wage trends in Miami after 1980 to the same trends in other, similar cities that didn’t get a migrant surge.
Economist David Card of U.C. Berkeley did exactly this, in a massively influential study in 1990. Card’s work became one of the most cited economic studies of immigration. The design of the study was elegant and transparent. But even more than that, what made the study memorable was what Card found.
Nothing. The Card study found no difference in wage or employment trends in Miami, flooded with new low-skill workers, and other cities—not for any workers, including low-skill workers. That study concluded that “the Mariel immigration had essentially no effect on the wages or employment outcomes of non-Cuban workers in the Miami labor market.”
Economists ever since have tried to explain this remarkable result. Was it that the affected U.S. workers had simply moved away? Had low-skill Cubans made native Miamians more productive, stimulating the local economy? Was it that the Cubans’ own demand for goods and services had generated as many jobs as they filled? Was it that Miami employers shifted to production technologies that used more low-skill labor? Regardless, the real-life economy was evidently more complex than an ‘Econ 101’ model requiring wages to fall when immigrant labor arrives.
A critical flawThis is where Borjas’s study came in, in 2015. The new paper by Borjas claimed that Card’s earlier analysis had obscured a large fall in the wages of native workers by using too general a definition of ‘low-skill worker’. Card’s study had looked at the wages of U.S. workers whose education extended only to high school or less.
Borjas’s paper divided up this group. It separately measured the wages of two slices of that larger group: 1) people who never finished high school, and 2) people who finished high school but went no further. It found that in the less-than-high-school group, wages plummeted right in 1980, just the year when all those low-skill Cubans had arrived. That happened in Miami to a much greater degree than in other cities. The study estimated that the Mariel Boatlift had slashed the wages of U.S. workers with less than high school by somewhere between 10 and 30 percent. It seemed like a smoking gun.
But it was in that act of slicing the data that the spurious result was generated. It created data samples that, exactly in 1980, suddenly included far more low-wage black males—accounting for the whole wage decline in those samples relative to other cities. Understanding how that happened requires understanding the raw data.
Both the Card and Borjas papers use data from the Current Population Survey or CPS, a representative sample survey of U.S. workers. This survey happens every month; it’s where we get the estimates for the U.S. unemployment rate. Two datasets taken from the CPS also report workers’ wages. These two datasets are called the ‘March Supplement’ and the ‘Outgoing Rotation Group’ (ORG). These aren’t surveys of all workers, but of a small number of workers chosen randomly from within subsets of the population, so that their answers to questions about wages will be representative of the wages of others like them.
Right in 1980, the Census Bureau—which ran the CPS surveys—improved its survey methods to cover more low-skill black men. The 1970 census and again the 1980 census had greatly undercounted low-skill black men, both by failing to identify their residences and by failing to sufficiently probe survey respondents about marginal or itinerant household members. There was massive legislative and judicial pressure to count blacks better, particularly in Miami. Starting in the 1981 CPS, survey coverage of lower-skill black men shifted sharply to include relatively more black men with less-than-high-school, and relatively fewer black men who had completed high school.
You can see the sample shift sharply in the graph below. In all graphs here, the 1981 CPS is shown above the year 1980 (as in the Borjas paper) because the wages reported there were mostly earned in late 1980.
On the left is the fraction of black workers in the exact March CPS samples used in the Borjas paper: male, non-Hispanic employed workers age 25–59, with less than high school. On the right is the same ‘fraction black’ in the same dataset, using the broader definition of low-skill used the Card paper: high school or less. The black line shows workers in Miami.
On the left, that’s a huge jump in the composition of the Miami sample in 1980. The fraction of blacks in the Borjas paper’s sample goes from just one third of the sample to two thirds, exactly when the Boatlift happens. Those new workers being surveyed aren’t the Cubans themselves: remember these are the non-Hispanic workers. And in the years thereafter the black fraction rises still more, reaching 91% (!) in 1985. That doesn’t happen among the broader education category used in the Card paper, on the right.
Those graphs show the same trends in the ‘control’ cities. Remember that both researchers test the effects of the Boatlift by comparing wages in Miami to wages in other, comparable cities that didn’t receive an immigrant surge. The two researchers choose these cities in slightly different ways, both of them based on finding cities with employment trends similar to Miami’s trend around the same time. Card uses Atlanta, Los Angeles, Houston, and Tampa-St. Petersburg; Borjas uses Anaheim, Rochester, Nassau-Suffolk, and San Jose.
But the black fraction doesn’t jump up in the samples from either group of ‘control’ cities. In fact, the fraction black in the control cities that the Borjas paper focuses on falls after 1980, actually reaching zero in 1983. At some points the Borjas study is comparing a city whose survey dataset is massively shifting to cover more low-wage blacks to cities where the CPS didn’t interview a single black man at this skill level.
The spurious findingThis shift in sample coverage of blacks is what creates the fall in measured wages. Among men in Miami with less-than-high-school at this time, wages are much lower among blacks than non-black workers. So including more blacks in the sample would make the average wage in the sample fall, even if nothing happened to the wages of any workers in the real population of Miami where the sample was picked. Take the shift in blacks in the sample, multiply it by the black-nonblack wage gap, and you get the fall in wages that would appear in sample used in the Borjas study.
For example, the black fraction in the survey sample went up between 1979 and 1985 by 51 percentage points more in Miami than it did in the samples from the main control cities considered in the Borjas study. Multiply that by the 49% relative gap in wages between blacks and non-blacks among very low-skill workers in this time and place, and you get 25% lower wages by 1985 just from including more low-wage blacks in the survey. The graph on the left below does this for all the years after 1980:
The vertical axis there is roughly comparable to a percent change in the wage, where –0.1 is a 10% decline and –0.2 is a 20% decline (it’s the change in logarithm points). Looking at that graph you can see that the inclusion of more low-wage blacks in the sample would cause the average wage to fall drastically, a decline that hits about 25% a few years after 1980.
The effect of the Mariel Boatlift measured by Borjas is “between 10 and 30 percent”. In other words, the spurious decline in wages explains the entire effect estimated in Borjas’s study.
On the right, in the figure above, is the spurious wage decline that you would get if you studied wages in the Card paper’s broader education category of high-school-or-less. You saw above that there was a much smaller shift in the number of blacks surveyed in that broader group. There’s also a smaller gap between black and nonblack wages in that group. So the estimated wage drop following the Mariel Boatlift would be negligible using the grouping of workers in Card’s paper: just 2 or 3% for a couple of years, not enough to show up among the statistical noise. Card’s study was therefore essentially unaffected by the bias of shifting sample coverage. This can explain why the Card and Borjas studies reach such sharply different conclusions.
What I’m discussing here is just one of many ways that this change in the composition of the survey sample can explain discrepancies between earlier studies of the Boatlift. One of these is that a study by Giovanni Peri and Vasil Yasenov of U.C. Davis showed that the results in the Borjas study are sensitive to which dataset one uses. I mentioned above that the CPS has two wage surveys each year. The above graphs are for the March CPS that the Card paper used, and Borjas’s paper focuses on. Peri and Yasenov showed that the result is much smaller in the other wage survey (the ‘ORG’ survey), a result that is discussed in revised versions of the Borjas study. Specifically, the wage effect estimate in the Borjas study is three times larger in the March CPS survey data than it is in the ORG data. There is no clear reason why the true effect of an immigration wave would change that much between two different surveys. But our paper has a simple and definitive explanation for this result: the post-1980 increase in coverage of low-wage blacks is three times larger in the March CPS than in the ORG data. The glove fits, in this way and several others that we discuss in our paper.
The origin of this problem is the way that the survey data are sliced in the Borjas study. When the Census Bureau picks people to survey who will represent the population, they only choose them to represent broad categories of people. If researchers slice up the data too finely, the Census Bureau can’t promise that those slices will represent the population anymore. For example, if you narrowly sliced the data to include only 53 year-old female Peruvian-American surgeons, you might find that very few (or no) such people happened to have been included in the survey sample. But such a very small (or empty) group would not necessarily represent all people of that description in any given city.
A less extreme example is what happened in the March CPS data of Miami: under pressure to improve coverage of low-skill black men, their efforts led to a shift within their sample of black men with high school or less. Within that group, right after the Mariel Boatlift, they began covering far more black men with less than high school, and relatively fewer who had finished high school. That wouldn’t change the racial representativeness of the broader high-school-or-less group. By slicing that group too finely, the Borjas study created a sub-group where the representativeness of the population changed sharply, at precisely the moment that a wage decline would be inaccurately attributed to the Boatlift.
More light, less heatThe evidence from the Mariel Boatlift remains as found in David Card’s seminal research: there is no evidence that wages fell, or unemployment rose, among the least-skilled workers in Miami even after a sudden refugee wave raised the size of that workforce by 20%.
This does not by any means imply that large waves of low-skill immigration do not or could not displace any native workers, especially in the short term. But facile pronouncements in recent statements by politicians, that immigrants necessarily do harm native workers, must grapple with rigorously-studied real-world experiences to the contrary. The Mariel Boatlift remains one of the most enlightening experiences of this kind.
Read more in Part Two: Answering Questions about Our Research.
Read Part Two: Answering Questions about Our Research
Update: On August 2, 2017, White House senior adviser Stephen Miller called for slashing U.S. immigration by half, citing the study that is discussed in this blog post. A more recent discussion of the flaws in that study is here, and the updated research paper is here.
¿Los inmigrantes de países pobres afectan negativamente a los trabajadores locales? Un estudio de un influyente economista que trabaja en temas de inmigración de la Universidad de Harvard descubrió recientemente que una famosa entrada de inmigrantes cubanos en Miami redujo drásticamente los salarios de los trabajadores locales.
Ese estudio generó una discusión acalorada el año pasado en The Atlantic, National Review y New Yorker. Los defensores célebres de la restricción de la inmigración declararon que el estudio había "derrotado" a las opiniones de sus opositores. El estudio de George Borjas es una pieza central en su famoso libro sobre inmigración, We Wanted Workers, citado por el Fiscal General de EE.UU. como prueba de los efectos económicos negativos de la inmigración.
Pero hay un problema. El estudio de Borjas tiene un defecto crítico que hace que el hallazgo sea espurio. Ese defecto se explica en un nuevo trabajo de investigación que coescribí con Jennifer Hunt, profesora James G. Cullen de Economía en la Universidad de Rutgers. En este blog post, explico el defecto y por qué refuerza los hallazgos de que el influjo de inmigrantes cubanos a partir del Éxodo del Mariel no redujo los salarios de los trabajadores de Miami.
La falla no era conocida por los economistas que revisaron el estudio para su publicación. El hallazgo de que los inmigrantes cubanos causaron el colapso de los salarios de Miami es un artefacto sobre cómo el estudio maneja su data. El estudio se basa en una encuesta de salarios de una muestra de trabajadores. El estudio se centra en un pequeño grupo dentro de esa muestra más grande, el cual se modificó para incluir a muchos más trabajadores hombres de raza negra con salarios relativamente bajos--simultáneamente con el Éxodo del Mariel. Esto tiene el efecto de reducir drásticamente el salario medio de las personas en la muestra, pero esto no tiene nada que ver con el influjo de cubanos.
Esto es digno de atención pues lo que sucedió en Miami es el evento histórico que más ha determinado la manera en que los economistas estudian la inmigración. Para explicar lo que salió mal, tengo que explorar y describir el escenario histórico bajo estudio.
El Éxodo del MarielPara un economista hay una manera directa de estudiar cómo la inmigración de mano de obra poco calificada afecta a los trabajadores locales. Encuentre una entrada grande y repentina de inmigrantes de baja cualificación solo a una ciudad. Observe lo que sucede con los salarios y el empleo de los trabajadores locales en esa ciudad, y compare esto con otras ciudades donde los inmigrantes no llegaron.
Un experimento ideal como este ocurrió en Miami en 1980. En apenas unos meses, 125 000 inmigrantes, en su mayoría de baja cualificación, llegaron de la bahía de Mariel, Cuba. Este inmenso éxodo marítimo se conoce como el Éxodo del Mariel. La fuerza laboral de Miami aumentó en 8%, la mano de obra poco calificada se disparó en 20%. Si los inmigrantes compiten con los trabajadores locales, este es exactamente donde se debería ver una caída de los salarios de los locales. Por supuesto, si cayeron, usted quisiera asegurarse de que esta caída no fuese una tendencia regional o nacional. Compararía las tendencias salariales en Miami después de 1980 con las mismas tendencias en otras ciudades similares que no obtuvieron una entrada migratoria.
El economista David Card de U.C. Berkeley hizo exactamente esto, en un estudio de gran influencia en 1990. El trabajo de Card se convirtió en uno de los estudios económicos más citados sobre inmigración. El diseño del estudio fue elegante y transparente. Pero aún más que eso, lo que hizo que el estudio fuera memorable fue lo que encontró Card.
Nada. El estudio de Card no encontró ninguna diferencia en las tendencias de salarios o empleo en Miami, la cual recibió nuevos trabajadores poco calificados, y otras ciudades--no para cualquier trabajador, incluyendo trabajadores poco calificados. Ese estudio llegó a la conclusión de que "la inmigración del Mariel tuvo esencialmente ningún efecto sobre los salarios o el empleo de los trabajadores no cubanos en el mercado de trabajo de Miami".
Desde entonces, los economistas han intentado explicar este notable resultado ¿Acaso los trabajadores estadounidenses afectados simplemente se mudaron a otro lado? ¿Los cubanos poco calificados hicieron que a los nativos locales de Miami fueran más productivos, estimulando la economía local? ¿Fue que la demanda de bienes y servicios de los cubanos mismos generó tantos empleos como los que llenaron? ¿Fue que los empleadores de Miami migraron hacia tecnologías de producción que requieren más mano de obra poco calificada? Sin embargo, la economía en el mundo real era evidentemente más compleja que un modelo 'Econ 101' que requiere que los salarios caigan cuando llegan inmigrantes.
Un defecto críticoAquí es cuando entra el estudio de Borjas, en 2015. El nuevo artículo de Borjas afirmó que el análisis de Card había oscurecido una gran caída en los salarios de los trabajadores nativos al usar una definición demasiado general de "trabajador poco calificado". El estudio de Card había examinado los salarios de los trabajadores estadounidenses cuya educación sólo se extendía hasta la escuela secundaria o menos.
El trabajo de Borjas dividió a este grupo. Midió por separado los salarios de dos proporciones de ese grupo más grande: 1) personas que nunca terminaron la escuela secundaria, y 2) personas que terminaron la escuela secundaria pero no obtuvieron grados educativos mayores. Borjas encontró que en el grupo con educación menor a la secundaria, los salarios se desplomaron justo en 1980, el año en el que todos los cubanos poco calificados llegaron. Eso ocurrió en Miami en un grado mucho mayor en comparación en otras ciudades. El estudio estimó que el Éxodo del Mariel redujo los salarios de los trabajadores estadounidenses con educación menor a la secundaria entre 10 y 30 por ciento. Parecía un jaque mate.
Pero fue este acto de dividir la data que generó el resultado espurio. Esto creó muestras de datos que, exactamente en 1980, incluyó repentinamente muchos más hombres de raza negra con salarios más bajos--lo que explicó la disminución salarial total de esas muestras en relación con otras ciudades. Comprender cómo sucedió requiere entender los datos sin procesar.
Los documentos de Card y Borjas usan datos del Current Population Survey o CPS, una encuesta representativa de la muestra de trabajadores estadounidenses. Esta encuesta se lleva a cabo cada mes; es donde se obtienen los estimados de la tasa de desempleo en EE.UU. Dos conjuntos de datos extraídos de la CPS también reportan los salarios de los trabajadores. Estos dos conjuntos de datos se denominan "Suplemento de marzo" y "Outgoing Rotation Group" (ORG). Estas no son encuestas para todos los trabajadores, sino para un pequeño número de trabajadores elegidos al azar dentro de subconjuntos de la población, de manera que sus respuestas a las preguntas sobre salarios sean representativas de los salarios de otros como ellos.
Justo en 1980, la Oficina del Censo--la cual llevó a cabo las encuestas de la CPS--mejoró sus métodos de encuesta para cubrir a una mayor cantidad de hombres de raza negra poco calificados. El censo de 1970 y, nuevamente, el censo de 1980, subestimaron en gran medida la cantidad hombres de raza negra poco calificados, por no identificar sus residencias y por no sondear suficientemente a los encuestados acerca de los miembros marginales o itinerantes del hogar. Hubo una presión legislativa y judicial masiva para contar de mejor manera a las personas de raza negra, particularmente en Miami. A partir de la CPS de 1981, la cobertura de la encuesta de hombres de raza negra poco calificados cambió drásticamente para incluir a relativamente más hombres de raza negra con educación menor a la secundaria, y relativamente menos hombres de raza negra que completaron la escuela secundaria.
Usted puede ver el cambio brusco de la muestra en el gráfico de abajo. En todos los gráficos, la CPS de 1981 se muestra por encima de la del año 1980 (como en el trabajo de Borjas) porque los salarios reportados se obtuvieron en su mayoría a finales de 1980.
A la izquierda se encuentra la fracción de trabajadores de raza negra en las muestras exactas de CPS de marzo utilizadas en el trabajo de Borjas: hombres, trabajadores no hispanos de 25 a 59 años de edad, con educación menor a la secundaria. A la derecha está la misma fracción en el mismo conjunto de datos, usando la definición más amplia de poca calificación utilizada en el trabajo de Card: escuela secundaria o menos. La línea negra muestra a los trabajadores en Miami.
A la izquierda, ese es un enorme salto en la composición de la muestra de Miami en 1980. La fracción de personas de raza negra en la muestra del trabajo de Borjas va de apenas un tercio de la muestra a dos tercios, exactamente cuando ocurre el Éxodo. Los nuevos trabajadores que están siendo encuestados no son los propios cubanos: recuerde que estos son los trabajadores no hispanos. Y en los años siguientes la fracción de personas de raza negra se eleva aún más, alcanzando el 91% (!) en 1985. Eso no sucede en la categoría de educación más amplia utilizada en el trabajo de Card, a la derecha.
Esos gráficos muestran las mismas tendencias en las ciudades de "control". Recuerde que ambos investigadores prueban los efectos de la llegada del Mariel comparando los salarios en Miami con los salarios en otras ciudades comparables que no recibieron un influjo de inmigrantes. Los dos investigadores eligen estas ciudades de maneras ligeramente diferentes, ambas procurando encontrar ciudades con tendencias de empleo similares a la tendencia de Miami al mismo tiempo. Card utiliza Atlanta, Los Ángeles, Houston y Tampa-St. Petersburgo; Borjas utiliza Anaheim, Rochester, Nassau-Suffolk y San José.
Pero la fracción de personas de raza negra no aumenta en las muestras de ningún grupo de ciudades de "control". De hecho, la fracción de personas de raza negra en las ciudades de control en las que el trabajo de Borjas se centra cae después de 1980, llegando a cero en 1983. En algunos puntos el estudio de Borjas compara una ciudad cuyo conjunto de datos de la encuesta está cambiando drásticamente para cubrir más hombres de raza negra y de salarios más bajos donde la CPS no entrevistó a un solo hombre de raza negra en este nivel de habilidad.
El hallazgo espurioEste cambio en la cobertura de la muestra de personas de raza negra es lo que genera la caída en los salarios. Entre los hombres de Miami con educación menor a la secundaria en este momento, los salarios son mucho más bajos entre las personas de raza negra que los trabajadores que no son de raza negra. Por lo tanto, incluir a más personas de raza negra en la muestra haría que el salario promedio de la muestra caiga, incluso si no sucediera nada con los salarios de cualquier trabajador de la población de Miami donde se tomó la muestra. Considere el cambio en la composición de gente de raza negra de la muestra, multiplíquelo por la brecha salarial de personas de raza negra vs no negra y obtendrías la caída de los salarios que aparecería en la muestra utilizada en el estudio de Borjas.
Por ejemplo, la fracción de gente negra en la muestra de la encuesta aumentó entre 1979 y 1985 en 51 puntos porcentuales más en Miami que en las muestras de las principales ciudades de control consideradas en el estudio de Borjas. Multiplique eso por la brecha relativa de salarios de 49% entre personas de raza negra y no negra entre los trabajadores muy poco calificados en este tiempo y lugar, y en 1985 se obtiene un salario 25% más bajo solo por incluir a trabajadores de raza negra de salarios bajos en la encuesta. El gráfico de la izquierda de abajo hace esto para todos los años después de 1980:
El eje vertical es aproximadamente comparable a un cambio porcentual en el salario, donde -0.1 es un descenso de 10% y -0.2 es un descenso de 20% (es el cambio en los puntos logarítmicos). Observando ese gráfico, se puede ver que la inclusión de más trabajadores de raza negra con salarios bajos en la muestra haría que el salario promedio caiga drásticamente, una disminución que llega a unos 25% unos años después de 1980.
El efecto del Éxodo del Mariel medido por Borjas es "entre 10 y 30 por ciento". En otras palabras, la caída espuria de los salarios explica todo el efecto estimado en el estudio de Borjas.
A la derecha, en la figura anterior, se encuentra el decrecimiento salarial espurio que obtendría si estudiasen los salarios en la categoría de educación más amplia de educación secundaria o menor de Card. Usted vio arriba que hubo un cambio mucho más pequeño en el número de personas de raza negra encuestadas en ese grupo más amplio. También existe una brecha menor entre los salarios de trabajadores de raza negra y no negra en ese grupo. Por lo tanto, la caída del salario estimada después del Éxodo del Mariel sería insignificante utilizando la agrupación de trabajadores en trabajo de Card: sólo 2 o 3% durante un par de años, no lo suficiente como para aparecer del ruido estadístico. Por lo tanto, el estudio de Card no se vio afectado por el sesgo en el cambio en la muestra. Esto puede explicar por qué los estudios de Card y Borjas llegaron a conclusiones tan diferentes.
Lo que discuto aquí es sólo una de varias maneras en las que este cambio en la composición de la muestra de la encuesta puede explicar las discrepancias entre los estudios referidos al Éxodo del Mariel. Uno de ellos es un estudio de Giovanni Peri y Vasil Yasenov de U.C. Davis que mostró que los resultados en el estudio de Borjas son sensibles a qué conjunto de datos se utiliza. He mencionado anteriormente que la CPS tiene dos encuestas salariales cada año. Los gráficos anteriores corresponden al CPS de marzo que utilizó Card en su investigación, y en el cual Borjas se centra. Peri y Yasenov mostraron que el resultado es de mucho menor magnitud en la otra encuesta salarial (la encuesta 'ORG'), un resultado que se discute en versiones revisadas del estudio de Borjas. Específicamente, la estimación del efecto salarial en el estudio de Borjas es tres veces mayor en los datos de la encuesta de marzo de CPS que en los datos ORG. No hay una razón clara por la que el verdadero efecto de un influjo inmigratorio cambiaría tanto entre dos encuestas diferentes. Pero nuestro trabajo tiene una explicación sencilla y definitiva para este resultado: el aumento posterior a 1980 de la cobertura de trabajadores de raza negra de salarios bajos es tres veces mayor en la CPS de marzo que en los datos de ORG. La explicación tiene sentido de esta manera y en varios otros puntos que discutimos en nuestro artículo.
El origen de este problema es la forma en que los datos de la encuesta se dividen en el estudio de Borjas. Cuando la Oficina del Censo escoge a los encuestados que representarán a la población, sólo los elegirá para representar a categorías amplias de personas. Si los investigadores dividen los datos de manera demasiado fina, la Oficina del Censo no puede prometer que esas divisiones representarán a la población. Por ejemplo, si los datos se dividen para incluir a sólo cirujanas peruanas-estadounidenses de 53 años de edad, podrías encontrar que muy pocas (o ninguna) de esas personas fueron incluidas en la muestra de la encuesta. Pero un grupo tan pequeño (o vacío) no necesariamente representaría a todas las personas de esa descripción en una ciudad determinada.
Un ejemplo menos extremo es lo que sucedió en los datos de marzo de CPS en Miami: bajo la presión para mejorar la cobertura de hombres de raza negra poco calificados, sus esfuerzos condujeron a un cambio dentro de su muestra de hombres de raza negra con educación secundaria o menor. Dentro de ese grupo, justo después del Éxodo del Mariel, comenzaron a cubrir a muchos más hombres de raza negra con educación menor a la secundaria, y relativamente menos que habrían terminado la escuela secundaria. Eso no cambiaría la representatividad racial del grupo más amplio de educación secundaria o menos. Al dividir demasiado el grupo, el estudio de Borjas creó un subgrupo en el que la representatividad de la población cambió bruscamente, precisamente en el momento en que se atribuiría erróneamente un descenso salarial a partir del Éxodo del Mariel.
Mayor conocimiento, menos discusiones acaloradasLa evidencia a partir del Éxodo del Mariel se mantiene tal como se mostró en la investigación seminal de David Card: no hay evidencia sobre la caída de los salarios, o del aumento del desempleo entre los trabajadores menos calificados en Miami, incluso después de un influjo de refugiados que aumentó el tamaño de esa fuerza de trabajo en 20%.
Esto no implica de modo alguno que los grandes influjos de inmigración poco calificada no podrían desplazar a los trabajadores locales, especialmente en el corto plazo. Pero los pronunciamientos en las declaraciones recientes de políticos, argumentando que los inmigrantes necesariamente generan efectos negativos sobre los trabajadores locales, deben evaluarse frente a experiencias del mundo real rigurosamente estudiadas. El Éxodo del Mariel sigue siendo una de las experiencias más esclarecedoras de este tipo.
Lea más en la segunda parte: Respondiendo preguntas sobre nuestra investigación.
May 15, 2013
Publications
Using data collected by the North Carolina Growers’ Association (NCGA), the leading employer of workers with H-2 visas, Michael Clemens shows that foreign workers have almost no direct effect on the employment prospects of US workers in H-2 occupations. Instead, they actually a large and positive indirect effect on US employment by contributing to North Carolina’s economy.
September 10, 2015
Blog Post
Journalists have warned you that there are now 60 million refugees worldwide, and that Europe faces “the worst refugee crisis since World War II.” They’ve told you this again and again and again and again.
Well, fact check: It’s not true. Not even close.
First, most of the people displaced by violence, like the slow catastrophe in Syria, are not properly called refugees because most of them have not left their country. Second, Europe’s current refugee crisis is certainly not the most serious since World War II. The international community has handled much more, and they’ve done so recently. If Europe is unable to accommodate the current influx of people fleeing violence, it’s because of a failure of policies and institutions, not because of unprecedented numbers.
The Real NumbersThat 60 million number comes from the United Nations, and it has really taken on a life of its own. But the UN is referring to the total number of people forcibly displaced from their homes. The large majority of those displaced people, despite the horrors many of them face, don’t leave their own countries. Only people who are forcibly displaced to other countries are defined as refugees. Here is the breakdown of those 60 million forcibly displaced people, from 2014:
That is, the large majority of forcibly displaced people have not shown up on rich countries’ doorsteps. By the UN definition, they are not refugees. Last year there were 21 million refugees in the world.
And despite the shocking images in the press, the numbers of refugees do not come close to making this Europe’s biggest refugee crisis since World War II. Here are the real UN numbers on refugees in Europe, 1951–2014, plus a range that includes all plausible scenarios for 2015:
The blue line is the actual number of all refugees in the countries of today’s 28 European Union member states, through 2014. The orange area is a range of numbers that we’re likely to see by the end of 2015, from the lowest plausible to the highest plausible. That range depends not just on how many people show up in Europe claiming asylum, but how many are actually granted refugee status and benefits.
You can see in that figure that Europe as a whole can do and has done much more than it is now. In the mid-1990s, Europe was successfully hosting refugees to the tune of 0.5 percent of its population. In 2014, that number was much lower, just over 0.2 percent. Even with the surge of pressure this year, the numbers are very unlikely to reach the levels seen just 20 years back.
Individual European countries have handled much, much more than even the highest regional average. After the Hungarian revolution of 1956, Austria found itself with 2 percent of its population as refugees. That’s four times the highest level you see in the figure above. And that was at a time that Austria itself was still wrecked by war.
None of this is to diminish the challenges Europe currently faces. There is certainly a major crisis in the policies and institutions designed to receive, process, and assist people who arrive seeking refugee status.
Those institutions are overwhelmed because Europe and the world community have not decided how to share the responsibility to protect these people. But Europe itself, both its economy and its society, are nowhere close to being overwhelmed. As Oxford’s Alexander Betts succinctly puts it, this refugee crisis “isn’t necessarily a crisis of numbers, it’s been a crisis of politics.”
The answer, too, lies in politics: stronger institutions to encourage cooperation and greater receiving capacity. These flows are not a major departure from other recent flows into the region when considered in larger context. The EU has admitted approximately 3.5 million long-term immigrants each year since 2010. Assuming asylum applications continue on the same scale for the rest of 2015, the EU will receive approximately 720,000 petitions this year. Even if all petitions were somehow approved (which is impossible), this would represent only an approximately 20 percent increase over current inflows.
The cooperative infrastructure to handle that increase can and must be built, and should extend beyond Europe to the United States and other countries who long ago promised to protect refugees. Refugees are not drowning us, but inaction could.
———
The raw data for the figures above, and the calculation of the ‘high’ scenario are available here. The ‘high’ scenario is estimated as follows: We take the Frontex estimates of illegal crossings into Europe for the first two quarters of 2015, and take the ratio of these to the numbers for the first two quarters of 2014. Multiplying this ratio by the 2014 third-quarter figure gives an estimate for the 2015 third-quarter figure of about 300,000. This roughly agrees with the Frontex figure for July 2015, the first month of the third quarter: 107,500. We then sum these projected numbers for 2015: 720,187. To be conservative, we double that number to 1.44 million. To be even more conservative, we assume that all 1.44 million people apply for asylum. To be furthermore conservative, we assume that 50 percent of those applicants will be accepted as refugees (the current acceptance rate is just 4.1 percent, from Eurostat). This results in an implausibly high projection for the increase in refugees in Europe during 2015: 720,187. This is much higher than any public estimate of approved refugees for 2015 (not asylum-seekers) made by the UN or the EU. The ‘low’ scenario for EU refugees in 2015 is calculated from the 2015 average of new asylum applications to date (62,466 per month) and a 5 percent acceptance rate — slightly higher than 2014’s 4.1 percent (Eurostat).
May 22, 2014
Lawrence MacDonald
In a recent study, CGD senior fellow Michael Clemens found that, contrary to popular belief, development in poor countries actually fosters more migration, not less.
April 9, 2015
Publications
The welcome rise of replication tests in economics has not been accompanied by a single, clear definition of ‘replication’. A discrepant replication, in current usage of the term, can signal anything from an unremarkable disagreement over methods to scientific incompetence or misconduct. This paper proposes an unambiguous definition of replication, one that reflects currently common but unstandardized use. It contrasts this definition with decades of unsuccessful attempts to standardize terminology, and argues that many prominent results described as replication tests should not be described as such. Adopting this definition can improve incentives for researchers, encouraging more and better replication tests.
July 30, 2015
Blog Post
A quarter of all people on earth, horrendously, have worms living inside their bodies. That includes 600 million school-age children. They usually get worms from contact with things contaminated by the excrement of infected people, which means that these are diseases inflicted on poor people who lack basic sanitation.
There is a large campaign for “deworming,” the mass treatment of children across poor countries to either prevent or reverse worm infections. That campaign is something of a poster child for the effective altruism movement, which seeks to channel philanthropy toward things that have scientific evidence showing they work.
So it was a big deal when various media outlets declared last week that the evidence to support mass deworming had been “debunked.” The stories reported on critiques of a famous randomized experiment conducted in Kenya in the late 1990s, and a new summary of evidence from a scientific body known as the Cochrane Collaboration.
The unique feature of that original study from Kenya, by Michael Kremer and Ted Miguel, was that it found important effects of mass deworming on school attendance — not just for children that got deworming pills, but for other kids who didn’t get the pills, both in the same school and in nearby schools. That’s plausible because the more infected kids take the pills, the less infected excrement is in the nearby environment, so it could be that fewer nearby kids who didn’t take pills get infected in the first place. This “spillover” effect to other kids would tend to magnify the pills’ impact, so Kremer and Miguel concluded that mass deworming is “far cheaper than alternative ways of boosting school participation.”
That’s the core of the debate. The debate is not about whether children sick with worms should get treated (everyone says yes), but whether the mass treatment of all kids — including those not known to be infected, just in case — is a cost-effective way to raise school attendance.
The healthiest parts of the debate have been about the need for transparency, data sharing, and more replication in science, as Ben Goldacre emphasized in his Buzzfeed piece that kicked off a firestorm in social media. We endorse those points. Goldacre is a longtime champion of transparency in science.
Here, we’re going to focus on the narrower question of the evidence for mass deworming specifically, which is where some journalists — like the Guardian headline writer below — have gotten things quite wrong. If you're pressed for time, scroll down to the bottom for the takeaway.
Cutting to the chase: new information about the original deworming study qualifies its findings, but certainly does not “debunk,” “overturn,” or negate its findings. Donors should remain open to and encourage new evidence but should not shift their priorities on deworming in response to this episode.
Here’s what happened. The respected International Journal of Epidemiology just published two separate critique papers that revisited the original Kremer and Miguel results — and reached very different conclusions with the same data. The key to understanding what this means is understanding why there were two separate critique papers in the first place. That requires a detour, but we’ll get back to the story soon.
Replication Tests and Robustness Tests Are Very DifferentThe two critique papers were supported by a replication project at a research funding body called the International Initiative for Impact Evaluation or 3ie (which our employer helped create). 3ie’s “replication window” offers small grants for researchers to revisit influential papers in international development to see how solid their evidence base is. The Kremer and Miguel deworming paper was one of the first nominees, and a team from the London School of Hygiene and Tropical Medicine set off to revisit Miguel and Kremer's analysis, starting from the original data and computer code. They produced the two critiques. This division into two papers was required by 3ie, for reasons that will become clear.
The first critique paper contains a “pure replication”; the second critique paper contains “alternative analyses.” The goal of the “pure replication” is simply to see if the computer code and dataset match the findings reported in the paper. That’s a check against errors and fraud. If a finding in a paper fails a test like that, it’s appropriate to say that the finding “could not be replicated” or “failed a replication test.”
The goal of the second, “alternative analyses” paper by Davey et al. is to try ways of analyzing the data that the original authors did not. The authors did some sensible things in the re-analysis that weren’t common practice in economics in the 1990s when Miguel and Kremer were writing: they pre-registered their analysis plan to minimize the risk of fishing for a particular result, and they attempted to follow the guidelines for transparent reporting of randomized trials issued by CONSORT and endorsed by many leading medical journals. Additional analysis like this is valuable because the authors of any original study might have omitted important analyses. Adhering to contemporary (2015) standards for the clinical trials also makes it easier to put this trial in the context of other deworming trials.
However, if “alternative analyses” give a different result, it would not be right to say that the original result “could not be replicated.” The results of any scientific paper ever written can be changed with alternative analyses of the same data, a well-known problem in social science. For example, an experiment that had a detectable effect on an entire high school as a whole might not have a detectable effect on each class looked at separately; slicing up the analysis can change the signal-to-noise ratio in the data. In this example it would be right to say that the result is not robust to separating by class. But it would be wrong to say that the effect on the whole high school “can’t be replicated,” because there could be good arguments for doing it each way. Neither is clearly an error, and each result could be right on its own terms.
That is, the two critiques are doing two separate, totally different tests.
The first is a test for indisputable errors or fraud. The second is a test for different results from legitimately disputable choices. If the first kind of test gives different results, this often brings stinging opprobrium on a scientist; if the second kind of test gives different results, this is just the normal progression of science, as colleagues try different things and learn more.
One of us (Clemens) has argued that these two kinds of tests need unmistakably different names, and shown that many researchers recognize the need for this distinction. But right now we don’t have clear terms to separate them, so the public discussion ends up calling both kinds of studies “replication” tests, and any different results from either kind of test get described as a failed replication. That’s very bad, because science needs a lot more studies that check previous results, as Ben Goldacre says, and it harms that enterprise to make researchers facilitating fruitful robustness checks on their work feel that they’ll end up smeared by the undeserved association with incompetence or fraud.
So Did the Original Deworming Paper Fail a Replication Test?Okay, back to the story. For the deworming study, both of the critiques report results that are substantially different from the original study’s results. But only the first study (by Alexander Aiken et al.) even has the potential to “debunk” or “overturn” the original result. The second study (by Calum Davey et al.) does not, and its results cannot be described as a failed replication.
The second, “alternative analyses” study could help us learn important things about the world by showing what happens when researchers try different ways of approaching the data. For example, the Davey et al. study explores what happens when different years of data are analyzed separately, rather than considering all years at once. But unless it’s beyond reasonable dispute that the original analysis should have done what the critique does, the critique can’t claim to be uncovering errors or “debunking” the original. It can claim to be checking robustness to alternative analyses, period. There are highly qualified people disputing the choices in the second, Davey et al. paper (such as here and here). We don’t take any position on that dispute here; we just note that the choices in the “alternative analyses” paper are apparently not beyond dispute. The choices have advantages and disadvantages.
What we want to be clear about is that no “alternative analyses” can be said to objectively uncover errors in the original if the choices they alter are disputable choices. If a choice is objectively an error, it means that no reasonable person could make that choice.
That’s why everything below is about the first study, by Aiken et al. If that study were to find that the analysis performed for the original paper does not match what’s in the paper, those discrepancies are rightly called errors. If those errors were to greatly change the result, it would be right to say that Aiken et al. “failed to replicate” the result.
Bottom line: we see no reason to doubt the Aiken et al. study is correct on its own terms, and is executed competently in a good faith effort to test individual findings from the original paper. The Aiken et al. paper fails to replicate some of the statistical estimates in the original paper exactly as they were executed in that paper. It would be completely wrong, however, to say that the Aiken et al. study “debunks,” “overturns,” or otherwise negates the original study. The effect of primary interest that Kremer and Miguel claimed is clearly present in the data even after the corrections by Aiken et al., albeit at a somewhat different magnitude and for a slightly different set of schools than originally reported.
Here’s an analogy that aptly describes this situation: Suppose a chemistry lab claimed that when it mixed two chemicals, the mixture rose in temperature by 60 degrees. Later, a replication team reviewed the original calculations, found an error, and observed that the increase in temperature was only 40 degrees. It would be strictly correct for the replication team to announce, “We fail to replicate the original finding of 60 degrees.” That’s a true statement by itself, and it doesn’t fall within the strict purview of a pure replication to do additional tests to see whether the mix rose by 30 degrees, or 40 degrees, or whatever. But in this situation it would be excessive to claim that replication “debunks the finding of a rise in temperature,” because the temperature certainly did rise, by a somewhat different amount. This is basically what’s happened with the deworming replication, as we’ll explain.
The Technical Stuff: Why the Original Deworming Result is Not “Debunked”The original Miguel and Kremer paper found that deworming had a positive spillover effect on school attendance for children in nearby schools of +2%. The most contentious part of the Aiken et al. replication is their conclusion that “after correction for coding errors, there was little evidence of an indirect effect on school attendance among children in schools close to intervention schools.” They get an estimate of –1.7%. In their reply, Hicks, Kremer, and Miguel (2015) declare that “we interpret the evidence from the reanalysis as strongly supporting the findings of positive deworming treatment externalities and school participation impacts.”
How can two teams of very sophisticated researchers look at the same data and reach opposite conclusions? We’re going to take you back to high school geometry, where you learned that the area of a circle is proportional to the circle’s radius squared (A = πr2). So if you double a circle’s radius, you quadruple the area of the circle. That informs how you would analyze the spillover effects of a treatment within a certain radius of a treated school.
Suppose the spillover effects looked like they do in this schematic example:
Figure 1. Hypothetical benefits of deworming for neighboring schools
The school that got treated is the black house in the center. Each circle around the black house is some other school that didn’t get treated. The number on each of those other schools is the spillover effect from treatment at the school in the center. For example, the number could be the percentage increase in school attendance at each untreated school due to spillover effects from the treated school.
Looking at the map, in this schematic example, it’s obvious that there is a spillover effect from treatment. You don’t need any statistics to tell you that. Schools near the treated school have big increases in attendance, schools far away don’t. It’s obviously very unlikely that’s this pattern is just coincidence.
We can use this example to see how correcting a mistake in the original deworming paper could make it look like there are no spillover effects, even when there are.
The original paper arbitrarily set two concentric circles around each treated school: It measured short-range spillovers inside radius 3km (in green). In the schematic picture above, using the made-up numbers there, the average spillover effect inside the green circle is 1.6. Suppose that, due to statistical noise, we can only detect an effect above 1; so this short-range effect is easy to detect.
The original deworming paper also measured long-range spillovers for some of the schools between 3km and 6km away. Why not measure long-range spillovers for all the schools between 3km and 6km? This would give excess “weight” to the schools furthest away, where spillover effects would tend to be smallest. (You can see that in the picture: for example, there are a lot more schools around 5km than around 2km. As we talked about above, the outer ring between green and gold has triple the area of the inner green circle.) Giving excess weight to schools with the lowest spillovers could conceal the spillovers amidst the statistical noise.
Thus the analysis underlying original paper measured long-range spillovers only for a subset of the closest schools between the green and gold circles. The picture above shows that schematically: it’s roughly like considering long-range spillovers only for the schools between the green and red circles. In this example, the average spillover for the 11 schools in that narrow band is 1.1.
Here’s where the mistake happened in the original paper: the write-up of this analysis in the original paper said that it did the equivalent, in our schematic example, of measuring long-range spillovers for all 55 schools between the green and gold circles. That’s not what the original statistical analysis actually did, but it’s what the write-up said it had done. If you do that for our schematic example, the average effect in the 3km to 6km is only 0.25. That’s below our detectable threshold of 1, so we can’t distinguish it from zero. Furthermore, in this example, the average spillover effect at all 76 schools inside 6km is just 0.6 — a statistical goose egg.
How would you report a correction to this mistake? There are two ways you could do it, ways that would give opposite impressions of the true spillover effects.
You could simply state that when you correct the error, the average spillover effect on all 76 schools in the correct 6km radius is 0.6, which is indistinguishable from zero. That’s an accurate statement in isolation. This is essentially all that is done in the tables of the published version of the replication paper. On that basis you could conclude, as that paper does, that “there was little evidence of an indirect [spillover] effect on school attendance among children in schools close to intervention schools.” Strictly on its own terms, that is correct. That’s the average value in all the circles in that picture.
But wait a minute. Look back at our schematic picture. It’s obvious that there is a spillover effect. So something’s incomplete and unsatisfying about that portrayal. First of all, the average spillover inside the 3km green circle is 1.6, which in this example we can distinguish from zero. So it’s certainly not right to say there is “little evidence” of a spillover effect “close to” the treatment schools.
Second, there’s nothing magical about 3km and 6km. Neither the original study nor the replication had any theoretical reason to think that spillovers happen at 2km but don’t happen at 4km, nor did they have any reason to think that spillovers should happen within 6km but not within other ranges.
So how could you report this correction differently, in a way that shows the obvious spillover effect? Using the same hypothetical data from the figure above, you could show this:
Figure 2. Hypothetical example continued -- benefits of deworming for neighboring schools
This picture shows, again for our schematic example, the average cumulative spillover effect out to various distances from the treated school: all the schools out to 1km away, all the schools out to 2km, all the schools out to 3km, and so on.
Here, there’s a big spillover effect nearby the treated school. That effect peters out as you expand the radius. In this example, it gets undetectable (falls below 1) once you consider all the schools within 5km, because the overall average starts to include so many faraway, unaffected schools.
The authors of the original paper made the corresponding graph for the real data in their original reply to the critique (the graph isn’t in the journal-article version). It uses the real data incorporating all the corrections made by Aiken et al., and looks like this:
Figure 3. Actual benefits of deworming on neighboring schools from Hicks, Miguel, and Kremer 2015
Just like in the schematic example before, the cumulative average spillover effect within a given radius first rises and then starts to fall as the radius expands. When the radius gets to 5km, the average spillover effect across all schools inside that radius becomes indistinguishable from zero.
This figure makes it clear that in the real, corrected data, there certainly is a detectable spillover effect: within 2km of the treated school, and within 3km, and within 4km. This is strictly incompatible with interpreting the Aiken et al. corrections as negating, debunking, or overturning the original study’s finding of spillover effects.
An accurate summary of the Aiken et al. result would be “The statistical analysis underlying the original study found there to be positive spillover effects extending 4km away from treated schools. The write-up of the original study incorrectly reported that those spillover effects extended to 6km. Aiken et al. correct errors in the original study’s computer code. They do successfully replicate the spillover effects at 4km, but fail to replicate the spillover effects at 6km claimed by the original study.”
In other words, the Aiken et al. finding is correct on its own terms.
The corrected results also confirm the existence of spillover effects detected by the original paper, with a somewhat different magnitude. It is wrong for others reading Aiken et al.’s work to report that it debunks the original finding of spillovers.
If the corrected data show “little evidence” of spillovers in the Kremer and Miguel study, the same logic could imply the perverse conclusion that John Snow had little evidence of cholera near London’s Broad Street pump in 1854. Snow revolutionized public health by proving the link between contaminated water and cholera. He did it partly by showing that infections during one outbreak were higher close to a single water pump, near which cholera-infected diapers had been washed. Here is Snow’s legendary map, with the infected pump at the center of the circles we’ve drawn on it, and black rectangles showing cholera cases:
Figure 4. John Snow's iconic map of cholera cases around London’s Broad Street pump in 1854
If you were to count up the density of cholera infections inside the green circle, you’d find it to be above average, incriminating the pump. But if you counted the density inside the entire gold circle, which has four times the area, the density would be much lower. It might be hard to statistically detect any difference between the density of infections inside that big circle and the density in other parts of London. If you reported exclusively that finding, you’d state that Snow had been debunked and move on. But that would not be a correct and responsible interpretation of the evidence; it would be an artifact of where you placed the circle. If you step back and simply look at the map, it’s quite obvious that there are spillovers of cholera from that pump.
Well Then, Should Donors Invest in Mass Deworming?Contrary to some of the more sensational media coverage, we feel the policy case for mass deworming is largely unchanged after the latest kerfuffle.
Before we dive into the details of the policy debate, it’s useful to step back and examine the track record of one of the major policy initiatives built on the back of the Miguel and Kremer paper — Kenya’s National School-Based Deworming program, which administered deworming tablets to nearly 6 million children across more than 13,000 schools in 2012–13. Monitoring and evaluation results from the Ministries of Education and Health suggest that in the 20 districts where the program was active in its first year, worm loads declined from an average of 33% in Jan-Apr to just 9% after treatment in Sep-Oct. (Thanks to Willa Friedman for pointing us to these data.)
Regardless of whatever further impacts deworming may have on health and education, this is a huge welfare gain for over a million children who no longer have these parasites living inside them. There are clearly questions of causality here — seasonal effects and other interventions may be contributing to this decline. But we take this as important prima facie evidence of the Kenyan government’s willingness and ability to implement an effective deworming program at scale.
Returning to the issue at hand: Note that the key policy question up for debate in the worm wars is not whether deworming treatments are effective at reducing worm loads — everyone agrees deworming treatments reduce worm levels and are good for infected kids — but whether deworming brings enough additional health and education benefits to justify mass, preemptive treatment of entire populations of children. Essentially this is a debate over whether deworming should be rolled out preemptively like vaccination or withheld until you're sick, like antibiotics.
Everyone also agrees that no big policy question like this should hinge on a single study, and should reflect the totality of the evidence. The sticking point is over which evidence to admit into that totality.
Prior to the latest round of debate, the World Health Organization deemed mass deworming to be cost effective, and the nonprofit GiveWell ranked two deworming efforts — the Schistosomiasis Control Initiative and Deworm the World — as two of its (top four) most effective charities in terms of lives saved per dollar spent. In a comprehensive review of the current debate GiveWell has reiterated its earlier finding that donors should fund mass deworming. GiveWell has no reputational or financial stake in the outcome of this debate, and we find their assessment independent and rigorous. (Both GiveWell and our employer receive support from the philanthropy Good Ventures.)
Proponents of mass deworming rest their case mostly on the long-term educational benefits of deworming found in studies that track children over many years. Roughly a decade after the Miguel and Kremer trial, the authors returned to Western Kenya and found positive effects on the employment and income of kids who had been dewormed (Baird et al. 2011). Another follow-up analysis found significant gains in test scores for younger siblings of the original treatment group -- kids who were more likely to have been worm-free during the critical developmental period of infancy (Ozier 2014). Across the border in Uganda, researchers also found big gains in test scores among children who had benefitted from a separate RCT years earlier (Croke 2014). And that evidence coincided with historical evidence on the schooling benefits from the Rockefeller Foundation's big push to eradicate hookworm in the US south before World War I (Bleakley 2007). The researchers involved with this work are some of the most able on earth.
Skeptics don't contest this evidence, but have set criteria by which this evidence does not get a seat at the table. A systematic review of evidence by the Cochrane Collaboration concluded that mass deworming had no impact on nutrition, health, or learning outcomes — a finding reached by omitting all the studies cited above.
For the narrow goals of the Cochrane review, there’s good reason to omit the US South study, as it’s not a randomized trial. (Though policy makers could still consider the evidence, appropriately caveated.) The rationale given by the Cochrane review for excluding the Kenyan and Uganda studies is that the control group eventually received deworming treatment, and other deworming programs now exist in both countries. As Alexander Berger of GiveWell noted, by this criterion it seems that no long-term study will ever be admissible, because deworming programs are proliferating everywhere. It doesn't require fancy econometrics skills to recognize that if the control group in a long-term study eventually benefits from a different deworming campaign, this will only lead researchers to under-estimate the impact of the original deworming. Ignoring positive findings because they might underestimate effects, and then concluding there is no effect, is puzzling. Conservative criteria can be a good thing, but it is also possible to be too conservative.
We conclude that GiveWell struck a good balance here (they call spillover effects an “externality”):
We continue to believe that it is extremely valuable and important for authors to share their data and code, and we appreciate that Miguel and Kremer did so in this case. We’re also glad to see the record corrected regarding the 3-6km externality terms in Miguel and Kremer 2004. But our overall impression is that this is a case in which the replication process has brought more heat than light. We hope that the research community can develop stronger norms supporting data sharing and replication in the future.
Donors should proceed by encouraging and supporting ongoing, iterative evaluation and learning about the long-term effects of deworming. They should also limit their diet of overblown journalism.
Conclusion / TL;DR The latest academic debate over a famous paper on deworming in Kenya (Miguel and Kremer 2004) questions whether deworming in one school had benefits for student attendance for neighboring schools. The original authors made an error, defining “neighboring” in the underlying code more narrowly than it was defined in the text. Upon replication, it appears there are spillover benefits up to about 4km, but not out to 6km. We see this as a useful correction, made in good faith by a competent replication team, and the result verifies the original paper’s finding that effects spill over to untreated schools.The policy case for mass deworming rests largely on the long-term benefits to children's cognitive development. But a new report summarizing deworming evidence by the Cochrane Collaboration rules out consideration of almost all long-term impact studies. This exclusion seems hard to justify. Aggregators of the evidence should explore criteria that allow consideration, to some degree, of long-term studies. The existing long-term studies, while fewer in number than one might wish, appear to justify investment in deworming. Ongoing long-term follow-up is needed to continue learning about the effects of mass deworming.
------
Disclosure: The authors have institutional links both with the authors of the critique studies and with the authors of the original study. Clemens and Sandefur’s employer, the Center for Global Development, created the working group that founded 3ie, the organization that encouraged and funded the critique studies; Clemens and Sandefur also have been involved in consultations with 3ie on the design of the “replication window” that supported the critiques. In addition, the authors of the original study are affiliates of the Center for Global Development: Michael Kremer as non-resident fellow, and Ted Miguel as a member of the Advisory Group.
June 9, 2014
Publications
This paper was updated in March 2015.
Developing countries invest in training skilled workers and can lose part of their investment if those workers emigrate.
Pages
October 6, 2017
Publications
Workers from poor countries can find enormous economic opportunity by working temporarily in a rich country. But agencies that fight global poverty do little to facilitate guest work. This may be because guest workers are perceived to typically suffer negative side effects that outweigh the benefits. This paper uses a natural experiment to test several perceptions of harmful side-effects on Indian guest workers in the Gulf. The research shows little evidence that the harmful side-effects often ascribed to guest work are typical and systematic, though this does not contradict the occurrence of many individual cases of harmful side-effects.
July 27, 2017
Publications
This paper studies the relationship between violence in the Northern Triangle and child migration to the United States. It finds that one additional homicide per year in the region, sustained over the six-year period of study—that is, a cumulative total of six additional homicides—caused a cumulative total of 3.7 additional unaccompanied child apprehensions in the United States. The explanatory power of short-term increases in violence is roughly equal to the explanatory power of long-term economic characteristics like average income and poverty.
May 22, 2017
Publications
An influential strand of research has tested for the effects of immigration on natives’ wages and employment using exogenous refugee supply shocks as natural experiments. Several studies have reached conflicting conclusions about the effects of noted refugee waves such as the Mariel Boatlift in Miami and post-Soviet refugees to Israel. As a whole, the evidence from refugee waves reinforces the existing consensus that the impact of immigration on average native-born workers is small, and fails to substantiate claims of large detrimental impacts on workers with less than high school.
April 3, 2017
Publications
We study a natural experiment that excluded almost half a million Mexican ‘bracero’ seasonal agricultural workers from the United States, with the stated goal of raising wages and employment for domestic farm workers. We reject the wage effect of bracero exclusion required by the model in the absence of induced technical change, and fail to reject the hypothesis that exclusion had no effect on US agricultural wages or employment. Important mechanisms for this result include both adoption of less labor-intensive technologies and shifts in crop mix.
July 8, 2016
Publications
Large international differences in the price of labor can be sustained by differences between workers, or by natural and policy barriers to worker mobility. We use migrant selection theory and evidence to place lower bounds on the ad valorem equivalent of labor mobility barriers to the United States. Natural and policy barriers may each create annual global losses of trillions of dollars.
February 23, 2016
Publications
For decades, migration economics has stressed the effects of migration restrictions on income distribution in the host country. Recently the literature has taken a new direction by estimating the costs of migration restrictions to global economic efficiency. In contrast, a new strand of research posits that migration restrictions could be not only desirably redistributive, but in fact globally efficient. This is the new economic case for migration restrictions: empirically, a case against the stringency of current restrictions.
January 25, 2016
Publications
Many developing countries need the World Bank’s capital less and less. What role should the Bank play in the 21st century? This paper argues that many features of the Bank today reflect a new role. That role, resting on the economic theory of bargaining and public good provision, is to reduce extreme poverty. Donor subsidies to the Bank already reflect this role, which implies new ways to structure and evaluate the Bank’s work.
September 9, 2015
Publications
This paper critiques the last decade of research on the effects of high-skill emigration from developing countries, and proposes six new directions for fruitful research.
Commentary Menu