July 26, 2012
This is a joint post with Rachel Silverman and Victoria Fan.
This month, both Health Affairs and the Journal of Acquired Immune Deficiency Syndrome (JAIDS) released special thematic issues on the US President’s Emergency Plan for AIDS Relief (PEPFAR) in which the articles – mainly commentaries but some analyses – provide an exceptionally positive readout on PEPFAR’s past performance and future direction. In principle, this is great – any insights into PEPFAR are always welcome, and it’s clearly valuable to discuss and disseminate lessons learned from the program. If these articles were posted on the PEPFAR website, or released as official PEPFAR reports, we wouldn’t bat an eye. But within scientific, peer-reviewed journals, the articles read more like PEPFAR PR rather than commentary and analysis from independent, third-party observers and stakeholders. A quick skim of the titles in the table of contents illustrates this point (see word cloud of selected title excerpts), and a closer look at the contributors sheds some light on why this may be the case: most authors of the articles are somehow affiliated with PEPFAR or with organizations that have received money from the program.
For how many authors in these two issues did this hold true? To find out, we compiled a list of all the authors who contributed to either issue, and noted their affiliations as described in the articles. If an author had multiple affiliations, we made a judgment call as to his or her primary affiliation. Next, we cross-checked the list of affiliated institutions against a list of organizations receiving PEPFAR funding in FY2008, compiled from country operational plans (COPs). We also used internet research to check for more recent funding. You can see all of our work in an excel file here, as well as notes on data cleaning. Here’s a summary of our findings (which should be treated as estimates):
Table. Numbers of authors in Health Affairs and JAIDS special PEPFAR issues working for PEPFAR or organizations that have received funding from PEPFAR
It’s a great thing to see PEPFAR and their affiliates writing and publishing about the program, as it brings much needed discussion of issues that will undoubtedly improve the quality of programs, policy, and advocacy. But the dearth of independent voices on the program is concerning. More generally, we wonder: To what extent can researchers maintain independence and scientific integrity in assessing and evaluating a program if they are also salaried by the program?
Every single article in the JAIDS supplement included at least one co-author who was employed by the Office of the Global AIDS Coordinator, or by PEPFAR’s other implementing agencies within the US government. Health Affairs was substantially more balanced by this measure; only a third of its pieces included an author directly employed by the US government, and most of those articles were commentaries (full disclosure - Health Affairs also asked CGDs very own Mead Over to write a more critical piece on PEPFAR for the issue, but he was unable to do so). In addition, the Health Affairs special issue received direct financial support from PEPFAR. It also received funding from two of PEPFAR’s private-sector implementing partners: Merck, a leading provider of ARV medicines, and BD, a global medical technology company. It’s not clear whether JAIDS received any external financial support for its supplement.
A second related concern is on the role of journals in countering bias. According to the International Committee of Medial Journal Editors, authors are responsible for explicitly disclosing any conflicts of interest, including financial and personal relationships, that might bias their work. JAIDS articles disclose that “various authors have professional relationships with PEPFAR (either as employees of PEPFAR-supported US Government agencies or as grantees/contractors)” Most of the articles in the Health Affairs special issue do not include an explicit disclaimer for conflicts of interest, though some (roughly half) disclose at least some funding sources and/or affiliations. But when over 80 percent of the authors work for PEPFAR or an institution funded by or affiliated with PEPFAR, it begs the question: can the journals themselves experience conflicts of interest, and further exacerbate them? And is full disclosure, when it happens, sufficient to overcome such bias?
With PEPFAR, the close ties between analysts and implementers may be unavoidable, as the most knowledgeable experts on the subject are also likely to be working closely with the program, and to have exclusive access to unpublished program data. Still, there may be ways to mitigate bias, and to foster broader participation and analysis. One idea: journals could adopt a policy on full data disclosure, as we have done at CGD. Full disclosure of the underlying program data behind these articles would allow for duplication and verification of their results, and invite further analysis by a broader pool of stakeholders.
We have only kind words for the PEPFAR-affiliated contributors, and the insider-perspectives they’ve brought to the issues. And we recognize that global health, and the AIDS community more narrowly, is a small and interconnected network, making some kind of association between PEPFAR and experts inevitable. But it is the responsibility of journals to ensure balanced content that clearly discloses conflicts of interest and maintains scientific integrity.
What do you think?
The authors thank Mead Over and Jenny Ottenhoff for their helpful comments.
It’s a great thing to see PEPFAR and their affiliates writing and publishing about the program, as it brings much needed discussion of issues that will undoubtedly improve the quality of programs, policy, and advocacy. But the dearth of independent voices on the program is concerning. More generally, we wonder: To what extent can researchers maintain independence and scientific integrity in assessing and evaluating a program if they are also salaried by the program?
Every single article in the JAIDS supplement included at least one co-author who was employed by the Office of the Global AIDS Coordinator, or by PEPFAR’s other implementing agencies within the US government. Health Affairs was substantially more balanced by this measure; only a third of its pieces included an author directly employed by the US government, and most of those articles were commentaries (full disclosure - Health Affairs also asked CGDs very own Mead Over to write a more critical piece on PEPFAR for the issue, but he was unable to do so). In addition, the Health Affairs special issue received direct financial support from PEPFAR. It also received funding from two of PEPFAR’s private-sector implementing partners: Merck, a leading provider of ARV medicines, and BD, a global medical technology company. It’s not clear whether JAIDS received any external financial support for its supplement.
A second related concern is on the role of journals in countering bias. According to the International Committee of Medial Journal Editors, authors are responsible for explicitly disclosing any conflicts of interest, including financial and personal relationships, that might bias their work. JAIDS articles disclose that “various authors have professional relationships with PEPFAR (either as employees of PEPFAR-supported US Government agencies or as grantees/contractors)” Most of the articles in the Health Affairs special issue do not include an explicit disclaimer for conflicts of interest, though some (roughly half) disclose at least some funding sources and/or affiliations. But when over 80 percent of the authors work for PEPFAR or an institution funded by or affiliated with PEPFAR, it begs the question: can the journals themselves experience conflicts of interest, and further exacerbate them? And is full disclosure, when it happens, sufficient to overcome such bias?
With PEPFAR, the close ties between analysts and implementers may be unavoidable, as the most knowledgeable experts on the subject are also likely to be working closely with the program, and to have exclusive access to unpublished program data. Still, there may be ways to mitigate bias, and to foster broader participation and analysis. One idea: journals could adopt a policy on full data disclosure, as we have done at CGD. Full disclosure of the underlying program data behind these articles would allow for duplication and verification of their results, and invite further analysis by a broader pool of stakeholders.
We have only kind words for the PEPFAR-affiliated contributors, and the insider-perspectives they’ve brought to the issues. And we recognize that global health, and the AIDS community more narrowly, is a small and interconnected network, making some kind of association between PEPFAR and experts inevitable. But it is the responsibility of journals to ensure balanced content that clearly discloses conflicts of interest and maintains scientific integrity.
What do you think?
The authors thank Mead Over and Jenny Ottenhoff for their helpful comments.Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
