Community Programming, the Final Frontier: Going Where No World Bank Evaluation Has Gone Before

On February 8th, the World Bank released a two-page summary of an evaluation underway to identify the effectiveness of the Community Initiatives component of the World Bank MAP, and how, if at all, it adds value to the national response. The evaluation—conducted in collaboration with DFID and the UK NGO AIDS Consortium—hopes to garner enough evidence to definitively show whether community involvement enhances the national response, with the idea that going forward, governments and stakeholders will use these findings to better structure their design and implementation of community-based programs.While key questions remain—i.e., How is the evaluation being conducted? Will new data be collected in the communities or will it be just a review of project reports and secondary data? What and how many countries will be studied?—the evaluation comes as welcomed news. Communities and community health workers have long played a critical role in primary health care systems, and in World Bank programs. The World Bank’s MAP program in particular has a strong community focus, with a sizable amount of their funds and resources going to Community-Based Organizations (CBOs). However, while so much is going IN to the community response, there is little coming OUT of the World Bank in regards to reporting, monitoring, and impact evaluation. Donors and others in the development community are left wondering how exactly this money is spent, what programs the money focuses on, and how effective these programs are in curbing the epidemic.Two of the main focuses of the evaluation are 1) to track the flow of funds from national to community levels; and 2) to assess the effects of the community response on the course of the epidemic. These are both important steps in identifying the roles that communities have and can play in the HIV/AIDS response, and issues that the Monitor has already examined, particularly in the report Following the Funding—which analyzes the transparency and effectiveness of funding data and flows of top AIDS donors.There is no doubt that the World Bank MAP is a leader in channeling significant sums of money to CBOs at the community level—so much so that that MAP funds tend to be designed and reported around types of implementing agencies, rather than programmatic activities. In order for a country to qualify for funding, the government must first identify CBOs to serve as implementing recipients of funds. Given this emphasis on communities, it is particularly surprising that we don’t yet know the returns of these investments.Findings from the HIV/AIDS Monitor (here and in two forthcoming reports on Performance Based Funding and the Health Workforce) about the MAP program, raise several issues about the allocation of resources to community based organizations that could reduce the effectiveness of community interventions:

Funding Bottlenecks: While the MAP is quick to disburse funds to national governments, the flow faces several challenges and delays once in the hands of the National AIDS Councils (NAC). Because these funds must first travel through the government system and because they are accompanied by complex reporting requirements, bottlenecks in the system often create slow and unpredictable money flows to the community, constraining program implementation. It is critical that the evaluation examine these funding flows and provide solutions to reduce these bottleneck challenges. In our Following the Funding report, we suggest that aligning reporting with national systems and increasing individual disbursement amounts based on the achievement of programmatic results could help reduce administrative burdens and ease the flow of funding to communities. Hopefully the evaluation will be able to provide more insight into how the administration of World Bank grants effects project performance at the community level.Lack of Program Data—collection and reporting: At the national level, the World Bank collects and releases financial commitments by country and by broad program area. At the country level, while the World Bank collects disbursements to recipient organizations, this data is not shared publicly. At the community level, while individual countries may collect comprehensive data on the amounts of money committed or disbursed to different types of recipient and sub-recipient organizations, it is not required by the World Bank, and it is therefore not released publicly. Yet, since the bulk of MAP funding goes to communities, it would be beneficial to see and understand how this money is actually spent at the local level. This data should be both collected AND released because the information is vital for not only tracking recipient spending in the community, but also for identifying and evaluating the cost-effectiveness of these programs.In addition, programmatically, the current data collected and released by the World Bank does not allow us to truly understand the extent to which communities have assisted in the response or which types of community-level programs are most effective. For instance, in our report Moving Beyond Gender as Usual, we found that the MAP has introduced gender-capacity building activities for community initiatives, but it does not systematically report on programming that results from this capacity building. Also, because of the lack of financial data collected at the community level, there is no way to know how much funding actually went to gender programming at the community level. In order to understand the effectiveness of community interventions—like gender programming—programs must be consistently monitored and evaluated.

In their current forms, MAP supported community based programs may be hindering or enabling governments’ and other stakeholders’ efforts to use aid effectively. Learning from an evaluation will help countries make informed decisions to allocate precious resources to community based interventions that are working—those that are helping to prevent new HIV infections and reducing deaths due to AIDS.