BLOG POST

Resolving the Deworming Paradox: Rethinking Deworming Campaigns for Children

The World Health Organization (WHO) has long recommended mass deworming of children in endemic areas on the grounds that it improves health and nutrient uptake. Mass deworming is recommended at least once per year if, based on population survey, over 20 percent of pre-school and school-age children have worms. Hundreds of millions of children are treated each year.

However, WHO’s recommendation and health impacts of mass deworming have been questioned in recent years. According to a Cochrane systematic review and meta-analysis from 2019, “the current evidence does not support large public health programmes of deworming in low- and middle‐income countries” because of non-significant nutritional health effects. Regular mass deworming “does not appear” to improve three out of four primary outcomes in that meta-analysis, namely height, haemoglobin or cognition (“probably little or no effect”), whereas for weight there is mixed evidence. These findings are broadly in agreement with an independent replication by Campbell Collaboration from 2016.

One thing that everyone agrees on is that all infected children should receive deworming treatment. This is obvious because the drugs are known to be effective, safe, and inexpensive. In fact, it’s several times cheaper to administer the drug to a child than to test for worms, which would leave many cases undetected anyway. 

However, this suggests a paradox in deworming meta-analyses: if infected children benefit from deworming, why would we not find smaller, but still meaningful effects from the mass deworming in the areas where worms are prevalent? In a new paper, “Meta-analysis and public policy: Reconciling the evidence on deworming,” together with colleagues, we aimed to address this paradox. This blog captures the paper’s main takeaways.

Deworming significantly improves health outcomes and is cost-effective

We update the meta-analyses by following Cochrane Handbook for Systematic Reviews of Interventions, aiming to improve precision of the estimates through using best statistical practices. We are able to include five additional studies not included in the latest Cochrane review and revise estimates in another eight. We then compare our results to other reviews and conduct cost-effectiveness analysis based on nutritional health outcomes alone.

We do this by, for example, calculating standard errors (where they were not reported) based on other available statistics, or using ANCOVA and difference-in-difference estimators to improve precision. We include study-by-study description of these decisions in the appendix of the paper. These are mainly small updates at the level of individual studies, but in aggregate they are consequential.

This gives us a total of 37 randomised control trials to analyse across four most commonly measured health outcomes: weight, height, mid-upper arm circumference (MUAC), and haemoglobin (Hb). As there is evidence of a lot of heterogeneity across trials, we combine evidence using a random-effects meta-analysis.

Our main model measures impacts in settings where WHO recommends mass deworming (>20 percent prevalence); we find statistically significant increases on child weight (0.15kg, 95% CI: 0.07, 0.24; P < 0.001), mid-upper arm circumference (0.20cm, 95 percent CI: 0.03, 0.37; P = 0.02), and height (0.09 cm, 95 percent CI: 0.01, 0.16; P = 0.02), but not on Hb (0.06 g/dL, 95 percent CI: −0.01, 0.14; P = 0.1), where there are the fewest trials. Standard publication bias checks do not find evidence of bias.

In cost-effectiveness analysis we compare mass deworming to school feeding, another widely implemented programme that targets similar health outcomes in similar populations. We find that investing in mass deworming (where infection prevalence exceeds 20 percent) provides gains over 23 times as large for weight, 50 times as large for mid-upper arm circumference, and 13 times as large for height. These results do not mean that school feeding is not a beneficial policy—both school feeding and mass deworming are cost-effective and deliver additional benefits beyond the health outcomes included in this analysis.

Deworming studies don’t disagree

Our new findings may seem reassuring to some in that they go some way toward resolving the paradox, but how can we square them against other meta-analyses? Does that mean our paper is “right” and others are “wrong”? Not at all.

Suppose our analysis did not find statistically significant results. Would we then conclude there is “probably little or no effect” of nutritional impacts, as the Cochrane review did? Of course, small effects are definitely a possibility, but are they probable? That depends on how likely we were to find the significant effects in the first place.

In the paper we conduct post-hoc statistical power calculations for all previous meta-analyses. We find that, considering conventional levels of 5 percent significance and 80 percent power, the existing analyses were only powered to detect effects that are considerably higher (usually a few times higher) than the effects that we find. This means that even if effects of deworming are large, combining studies is far from guaranteed to show significant results.

Previous analyses also focused on subgroup analyses. In general, cutting data can be a good idea, especially if we know that there is a subset of studies which is not relevant to decision-makers or where we would not expect to find effects. That is exactly why for our main analysis we looked at studies that match WHO’s recommendation of at least 20 percent prevalence. However, too many cuts to data (e.g., looking only at studies published after 2000, as in one of the meta-analyses) in general reduce sample size and hurt statistical power.[1]

These may seem like dry methodological critiques, but there is an important and somewhat positive conclusion: none of the meta-analyses actually disagree with each other. Yes, this new analysis yields statistically significant results whereas previous analyses tended not to. However, if instead of focusing on crossing an arbitrary threshold (significance) we consider the uncertainty in each model, we can immediately see that these are gradual differences. The best way to see this is to plot the results of different meta-analyses for the two outcomes on which we have most data: height and weight:

Line plot showing height and weight data comparisons from Taylor-Robinson et al., Welch, and current study

(Taylor-Robinson et al papers are two most recent versions of the Cochrane meta-analysis. Welch et al 2016 is the Campbell Collaboration meta-analysis.) 

In both cases we can see some increase in average effects in subsequent analyses, but also a considerable increase in precision (narrower intervals). But the confidence interval for all of these analyses tend to overlap.

This is a reassuring finding. Perhaps it also has an interesting parallel to other debates of “worm wars.” Whereas some of the past discussions on deworming sometimes used terms such as “no evidence” or spoke of “debunking” and “overturning” of results, we can see in one graph a process of gradual refinement of our understanding of health benefits of deworming. Viewed through this lens, there is no paradox, just healthy scientific progress.

Coda: Why focus on nutritional health outcomes?

As most readers will be aware, the overall benefits of deworming are believed by many to go beyond nutritional health effects. These include improved school attendance, labour market impacts, and spillover effects. All of them have been debated extensively: certainly too extensively to attempt a decent summary here. 

However, it’s still worthwhile to consider the nutritional health outcomes on their own, for three reasons. First, they are a crucial part of any causal chain of deworming’s benefits. Second, they are also the most commonly measured outcomes. Third, if the effects on nutritional outcomes are large, this may be sufficient to position deworming as a highly cost-effective intervention, while remaining agnostic about its downstream effects.


[1] To be precise on whether it’s worth conducting subgroup analyses: in general terms, if the studies included in meta-analysis are too heterogeneous (i.e. not comparable) for some discernible, systematic reason, then dividing them into two more groups could give more interpretable and more precise estimates of average effects. And to be clear, in the 2019 Cochrane review the authors  acknowledge the importance of prevalence. The potential issue there is that they categorise studies into low, medium, and high burden, and then conduct subgroup analyses. While focusing on prevalence is the right decision, using three categories creates a problem, because each subgroup has  few studies. Hence in the Cochrane review, the highest prevalence settings tend to show large impacts on average, but, as one would expect given small samples, they are also non-significant. 

Disclaimer

CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.


Image credit for social media/web: DFID/Vicki Francis