The inaugural conference of the Transforming Evidence Network took place last month, bringing together researchers, policymakers, practitioners, and funders working on education, health, housing, environment, youth services, and more, all united by a passion for evidence use as an important research and practice area in its own right (though many people in this space have created distinct disciplinary homes, per the visual below). CGD’s recent working group on reinvigorating impact evaluation and related evidence ecosystems for global development took a similar approach in its examination of how to address remaining obstacles to the use and utility of evidence in policy. At the conference, speakers and participants reflected on evidence contexts, lenses, products, and processes spanning countries around the world, taking a step beyond the problematic dichotomies still too often upheld between regions.
Here’s a round-up of some of the conference’s main themes (sessions were organized around making, mobilizing, and using evidence), plus reflections on areas for further action, drawing from the final recommendations of CGD’s working group.
via @juliaskaufman
Making Evidence
Making Evidence sessions explored a question at the core of CGD’s working group: how can we produce evidence that is more useful and relevant to policy and practice? As Ruth Stewart, Chair of the Africa Evidence Network, emphasized, this means going beyond academic standards and hierarchies of what constitutes “good” evidence. In the experience of Frejus Thoto, Executive Director of ACED Benin, a “helpdesk” approach to support policymakers in identifying evidence needs, formulating questions, and finding new or existing answers has been a useful way to pique user interest and generate more demand over time.
John Lavis, Co-Lead of the Global Commission on Evidence, urged attendees not to overlook crafting bodies of evidence as a main way to respond to policymaker needs, in addition to producing and promoting individual studies. Others raised related concerns about whether research funding models that prioritize policy impact risk rewarding the loudest and boldest, as opposed to supporting systems that enable real-time, high-quality evidence support from a range of data sources.
Funders must strike a strategic balance between funding individual studies and supporting evidence infrastructures, synthesis capacities, and partnerships over time. CGD’s report suggests that funders use a “value-of-information” approach to proactively consider and prioritize evaluation and evidence investments with the greatest potential “returns” in the form of improved outcomes, e.g., programs that receive a large share of resources and could easily be evaluated but have not yet been. The report points to climate interventions as a potential high-value area; climate policy receives increasingly significant public and philanthropic resources, but there is little evidence on what these resources are achieving and which emission mitigation and adaptation strategies provide the best value. IDinsight is also using a value-of-information approach to direct donations towards high impact projects through a new catalytic fund.
Alongside supporting specific evaluations, CGD’s working group calls on philanthropies and government donors to channel more funding toward long-term, flexible institutional support. Flexible resources for locally-linked evidence organizations, in particular, enable them to go beyond short-term consultancies and time-limited projects towards sustained engagement with policymakers. In the words of William Penuel from the University of Colorado-Boulder, “project-based funding is incredibly hard for partnership building for longer-term research and implementation.” The encouraging news, flagged by Kim Dumont from the William T. Grant Foundation, is that there are growing examples of funding for relationship building, infrastructure strengthening, and research use. But to date, these funding opportunities tend to be concentrated in the United States.
Mobilizing Evidence
Mobilization refers to activities taking place between and around production and use, including embedded experts within government-based or adjacent units, co-production partnerships, and communication outputs in accessible, non-academic language. Mobilization is increasingly considered to be its own distinct field, meriting designated roles filled by those with a combination of specialized research and communication skills. Those who do this work are sometimes called knowledge brokers, knowledge translators, and boundary spanners, among other terms.
One relatively simple mobilization area continues to go overlooked: language. As Frejus Thoto flagged, the concept of “evidence” is not entirely straightforward in French. As a bare minimum, evidence use products should be routinely translated into languages other than English. A strong example is the Evidence Commission report, available in seven languages.
Zooming out, how do we know if evidence mobilization and partnership efforts are working? When considering returns on investment and making the case for more funding in this space, greater clarity on the impact of knowledge mobilization is especially important. Many experts surfaced cutting edge methods to gauge whether and how evidence is being used as a result of mobilization activities.
For example, Sarah Morton, Director of Matter of Focus, presented a pathway to impact framework to help knowledge brokers understand how they make a difference. Discussing how to evaluate research co-production between researchers and program stakeholders, Anita Kothari from the University of Western Ontario outlined a “social model of impact” that helps capture relatively subtle impacts on relationships, knowledge sharing, culture shifts, and capacity building.
Source: Matter of Focus via @kmftimm
Using Evidence
Sustained evidence use requires institutional infrastructure. The working group report specifically recommends strengthening evidence use through new institutional incentives and structures. This can take the form of leadership role modeling, shifts to professional rewards, or new project review and approval processes and requirements. In the case of the UK's Foreign, Commonwealth & Development Office, an empowered chief economist is responsible for technical oversight through a quality assurance unit that peer reviews larger spending proposals, internal consulting to help country programs assess and revise their strategies, and thought leadership with staff at all levels. Further, USAID has announced plans to establish a new behavioral sciences and experimental economics unit headed by the agency’s chief economist, offering a new pathway to systematize evidence use at the agency and complement the efforts of the Bureau for Policy, Planning, and Learning. Harsha Dayal, Director of Research and Knowledge Management at the Department of Planning, Monitoring, and Evaluation within the South African Presidency, reflected on the need for explicit infrastructure and formalization of existing relationships, embedded within existing policy spaces.
Dayal also stressed the importance of “methodological pluralism” for effective evidence use, invoking the importance of varied data sources and strong data systems as part of evidence infrastructure. This is a main point of progress and future recommendation area in the CGD working group report, which highlights new opportunities to harness technology for timely, lower-cost evidence. Mona Shah of the Robert Wood Johnson Foundation discussed how funders can play a significant role here, including funding communities and countries to generate and own their own data from the contexts they know best. You can hear working group members discuss the potential of the digital transformation for evidence use in more detail here:
What’s Next? Funding Shifts for Equity
Those of us committed to unleashing the power of evidence-to-policy partnerships and research co-production also have a responsibility to recognize and ameliorate the power imbalances in resources, influence, and authority that are inherent in this work. In one of the breakout sessions on power, politics, and equity, Kripa Jagannathan, a climate adaptation research scientist at UC Berkeley, outlined specific ways imbalances show up and how knowledge brokers and translators can help address them.
Source: Kripa Jagannathan via @ChhayaKolavalli
Further upstream, funders have the power to help shift resources and agenda-setting influence to researchers and evidence groups that have historically been overlooked or excluded—especially those who best understand local policy environments and entry points for evidence use. Indeed, CGD’s working group offers specific ways for a range of funders—philanthropies, multilateral organizations, and aid agencies—to complement and leverage government resources and take this agenda forward.
Thanks to the Transforming Evidence Network Conference, a nimble group of knowledge brokers, boundary spanners, and bridge builders across sectors and settings found each other and formed a new community. Now the work begins to leverage these connections, resources, and lessons for collaboration and action. I can’t wait to pick back up—hopefully far from where we left off—at the next meeting.
For a deeper dive, I recommend checking out collections of resources on evidence use from Transforming Evidence, the William T. Grant Foundation, and Rutgers University, the Evidence in Policy journal, the 2019 version of What Works Now, and the lively #TENconf2022 tweets.
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
