PEPFAR: Messy Measurement or Unrealistic Requirements?

With billions of dollars going to global HIV/AIDS programs, it's good policy to ensure we are measuring the results of that money. But measuring is incredibly complicated, and the latest proof comes from a USAID audit (.pdf) of its global AIDS programs, featured in the AP/Washington Times article "AIDS Program Loses Track of Funds, Results."

The audit examined programs in four countries and found multiple instances of inaccurate or poor quality reporting by USAID contractors and grantee organizations. So how did the U.S. government respond? According to the AP story, "The Bush administration says the situation has improved greatly because it has imposed tighter, standardized reporting systems on groups that receive U.S. money." Meanwhile, Congress has signaled its intention to investigate the problems.

Michael’s Comments: Improvements need to be made in the monitoring system and tightening up reporting is a fine start. But the audit reveals deeper problems in the system which should be addressed. These include:

• Staffing up. In its country offices, USAID has, at most, a handful of staff overseeing a portfolio of AIDS programs that disburses tens of millions of dollars annually. These same few people have to write strategies, oversee country-wide programs, prepare new grant agreements, coordinate with the government and other donors, and so forth. It is no surprise that some USAID officials complained to auditors that their heavy workload inhibited their ability to carefully monitor recipients' reports on results achieved. But the audit report doesn't deal with this problem. If we want to improve verification, the job needs dedicated people. I suggest expanding the pool of staff, or - dare I say - contracting out the job.
• Joint reporting requirements. Existing reporting requirements are unnecessarily bureaucratic. For example, the audit cited a Nigerian recipient for poor quality reporting because they made the "mistake" of measuring the number of people trained with both USAID and CDC money, not USAID funds alone. If two agencies from the same government won't permit joint reporting, how will we ever achieve the UNAIDS best practice of one monitoring and evaluation framework for a whole country?
• Predictable funding. In Tanzania, Guyana and Nigeria, USAID officials did not know when money for fiscal year 2005 would arrive, and they couldn't even be certain about how much of their expected funding would be approved by Congress. This wildly unpredictable funding pattern means that program officers are often shooting in the dark when they set their annual targets. But rather than serving as loose guidelines or goals, auditors rely on these "guesses" as firm targets against which to measure the program's success or failure. Can we really expect precise targets without simplifying funding procedures?

Improved monitoring of results doesn't only need to be tighter, it needs to be smarter too. If Congress does investigate the reporting shortfalls of USAID programs, it should avoid the temptation to impose simple or overly bureaucratic solutions (like mandating more paperwork) and find ways to deal with some of the problems underlying the whole system.