Improving PEPFAR’s Data Management and Disclosure

The US government spends about $6.4 billion a year on preventing and treating HIV/AIDS in the developing world, and 4.5 million AIDS patients depend mostly on US generosity each day for the AIDS medicines that keep them alive. The administration, and in particular Ambassador Eric  Goosby, the head of the President’s Emergency Program on AIDS Relief (PEPFAR) have a unique opportunity to make that money stretch farther and do more good, at very little cost to US taxpayers: release the reams of data that PEPFAR and its contractors have already collected, at substantial cost—perhaps as much as $500 million each year.  This would be a first step in what I hope will be 2013 drive to improve the efficiency, the quality and the accountability of the US’s most frequently praised foreign assistance program.BackgroundSince January, 2011, I have served on two of the working groups of PEPFAR’s Scientific Advisory Board , those on data and key populations.    As I described in a previous post, Ambassador Goosby formally constituted this board in January, 2011 under the auspices of the Federal Advisory Committee Act and requested it to advise him and PEPFAR “concerning scientific, implementation and policy issues related to the global response to HIV/AIDS.”  [From the SAB’s charter.]  PEPFAR has recently posted the 22 presentations from the recent meeting as a downloadable 28 MB zip file.  For those who are following the progress of PEPFAR, the US’s most prominent, most frequently praised and most costly foreign assistance effort, these presentations provide a wealth of fascinating perspective, information, opinion and advise.  Because of the packed agenda, the SAB as a whole had little opportunity to review all of the material or to discuss each of the topics in the depth they deserved.  Hopefully many of the issues raised will receive a fuller discussion in the public arena, including on blogs such as this one.The Data Working GroupSince January, 2011, I have served on two of the working groups, the ones on data and on key populations.  While I have long had an interest in improving the global response to the epidemic in key populations (an interest that was inspired by my early collaboration on an article with Peter Piot here and ungated here), my interest in data has only been instrumental.   I have appreciated data not for their own sake but only as a means to an analytical end.  But serving on the Data Working Group has opened my eyes to both the opportunities and the challenges that PEPFAR faces in the arena of data collection, management and disclosure.  In a series of meetings and conference calls over the past two years with other members of the DWG and with US government staff of PEPFAR and other agencies, I have learned that PEPFAR and its contractors are spending a great deal of money, perhaps $500 million per year or even more, collecting and managing data, but neither the intended beneficiaries (the AIDS patients and vulnerable populations of recipient countries) nor the US taxpayer is benefitting as much as they could from this effort.In our group deliberations, the members of the DWG based our recommendations on a set of “principles of data collection and management”:

1. Data should strengthen US government program management
2. Data are a public good (i.e. the consumption of data is neither “excludable” nor “rivalrous” and therefore its production and distribution require government support)
3. Data should further transparency and accountability of government programs
4. Data should be standardized for comparability through a common data “platform”
5. Data based analyses of the progress or problems of a US government program should be replicable by members of the public, who can also be a source of independent ideas (“Crowd sourcing”)

How do you, the readers of this blog, feel about these principles?The DWG’s Recommendations to PEPFAARProceeding from these principles, the Data Working Group arrived at a set of four recommendations which we delivered to Ambassador Goosby (the US’s Global AIDS Coordinator) in the form of a presentation that I presented at the recent October meeting.  These were that PEPFAR should:

1. Establish and maintain a PEPFAR public access knowledge portal
2. Strengthen, streamline and publicly disclose PEPFAR’s collection and management of key program indicators
3. Establish, collect and publicly disclose activity-based budget, expenditure and cost data
4. Require each future grantee and contractor to submit a “Data Management Plan”

Since PEPFAR already has a website and collects indicators, the first two of these recommendations may seem anodyne.  However, those who have attempted to retrieve more detailed quantitative information from the PEPFAR website already know how frustrating such an effort has been.  PEPFAR does well on collecting its mandated indicators, but lags behind other US agencies both in the scope of its reports to Congress and in its  ranking on the 2012 Aid Transparency Index.  PEPFAR’s indicators also need to be strengthened (e.g. to better capture the retention as well as the enrollment of AIDS patients) and also streamlined (to minimize the collection and reporting of unusable indicators).  See the full presentation for more details on both the strengths and the weaknesses of PEPFAR’s current efforts.Recommendation 3, the collection and disclosure of activity-based budget and expenditure data, may be the most surprising recommendation to the average US taxpayer, who wants to be sure US foreign assistance is wisely spent.  The taxpayer might well ask, “Aren’t the budget, expenditure and cost of PEPFAR funded activities already fully disclosed?”   Unfortunately long before PEPFAR was created the US government ceded to its contractors the “right” to declare their detailed activity-based budgets and expenditure reports to be, get this, “trade secrets”, which the contractors can refuse to divulge to the public.  These data that would intuitively seem to be the property of the citizens of any democratic country, are hidden behind so many layers of protection that sometimes even PEPFAR staff cannot access them.  See this enlightening blog post by Till Bruckner who attempted to extract such information from US aid contractors using a Freedom of Information Act request.The good news is that PEPFAR has taken aggressive steps to better understand the cost-effectiveness of its programs, but has not yet disclosed the underlying data to the public.  The DWG recommended that PEPFAR build on its recent progress in this area by:

1. Regularly repeating its surveys of the activity-linked unit cost of anti-retroviral treatment and disclose the properly anonymized underlying data
2. Expanding its activity-linked unit cost analysis to other PEPFAR supported HIV/AIDS services
3. Releasing the properly anonymized activity-linked budget, expenditure and  cost data at the unit of observation, which is the partner or program
4. Disclosing the anonymized activity-linked financial data on PEPFAR’s open web portal, freely browsable by the public

Recommendation 4 will be the most controversial in the PEPFAR research community.  Researchers have many reasons, both good and bad, for preferring not to share their data.  The good reasons are to protect the privacy of their patients or subjects and to respect the sovereignty of host countries (although one must ask whether the host countries preferences for data control should trump those of the American taxpayer and the global community in every case).  The bad reasons have to do with researchers’ ambitions to squeeze out all of the possible publications from the data before releasing them for use by other researchrs.  (As a researcher I have not been immune to this temptation.)  Unfortunately, the two sets of reasons can become easily confounded, since a researcher wishing to retain private use of data over a longer time can often persuade the host country government to request the data not be shared.Although I empathize with the researchers’ desire to have privileged access to data they have collected, at least for a while, as an economist I am aware that private incentives will always be slanted against optimal public disclosure of data.  In this fourth recommendation, we on the DWG advise PEPFAR to represent the public interest as a counterweight to the various private interests of host countries and researchers and support the principle that publicly funded data should eventually enter the public domain and that US government contractors must establish an approved timetable for archiving their properly anonymized data for public use.An opportunity for some quick progress.Assembled in a blog post by CGD’s Lawrence MacDonald, four colleagues and I propose that those in the Obama administration with attention to spare from the task of avoiding the fiscal cliff could make a few immediate, low cost or free changes that would substantially improve US policy support for poverty reduction in poor countries for years to come.  Building on what I’ve learned on PEPFAR’s SAB, my contribution to this list is to suggest that Ambassador Goosby and his staff move forward quickly towards greater public disclosure of existing data (DWG Recommendation #1) and towards mandating the Data Management Plans for contractors (DWG Recommendation #4).  Looking forward to 2013, we can all hope and expect that further improvements in PEPFAR’s collection, management and disclosure of key program data will support continued improvement in the quality, efficiency and accountability of PEPFAR’s efforts against the AIDS epidemic.