Cutting through the Noise: Identifying Smart Bets in AI Applications for Development

AI companies often tout how their models are rapidly surpassing benchmarks in topics like math and reasoning. While impressive, model gains don't automatically lead to improvements in development outcomes like lower infant mortality or improved literacy. Impact depends on how these powerful technologies are applied.

There are more than 450 AI applications targeting health or education outcomes in low- and middle-income countries, but not every application will prove cost-effective—we are already seeing programs with uneven impacts. A randomized controlled trial (RCT) in Nairobi found that a ChatGPT-powered business coach delivered via WhatsApp increased profits and revenue for successful entrepreneurs but led to a decline for struggling entrepreneurs. Another RCT in Turkey found that the use of generative AI without additional safeguards led to worse student performance when the AI tool was taken away.

How can the development sector cut through the noise and make smart bets when we don’t yet have a conclusive evidence base on what interventions work? Spending wisely is always important but deserves extra attention amidst shrinking aid budgets and AI’s tremendous hype. In this blog, we propose organizing investment in AI powered interventions into three buckets: (1) Supercharging proven solutions, (2) Tackling known problems lacking solutions, and (3) Exploring unknown unknowns. This blog intentionally excludes discussing investments in digital infrastructure, or digital public goods like public datasets, open models, and open-source tooling—those foundational elements merit a separate, deeper dive.

1. Supercharging proven solutions

One way to approach the question of which development interventions would deliver the most impact when enhanced with AI is to look at the last 20 years of impact evaluations conducted prior to the advent of AI. These studies help us understand which interventions work. When we think we have a good understanding of why a particular intervention works, we should use AI to reduce delivery costs, improve implementation fidelity, and scale delivery. For instance, studies show personalized, one-on-one numeracy tutoring significantly boosts learning, but scaling can be limited by teacher shortages and costs. Could an AI tutor address these barriers, and will it prove comparably cost-effective? How many other interventions could benefit from similar reasoning?

Of course, the AI addition may change the nature of the program, so further impact evaluation is likely necessary. Additionally, due to rapid AI advancements, interventions previously evaluated may require repeated assessment if technology changes significantly enough to transform user experience. Nevertheless, using AI to enhance proven mechanisms likely improves the chances of impact.

In addition to generative AI tools, other machine learning capabilities combined with technologies like remote sensing and computer vision can drive large improvements in areas like weather forecasting, disease detection, natural disaster anticipation, and needs assessments. Combining these capabilities with proven interventions such as cash transfers, medical imaging, and digital agriculture extension could significantly reduce cost and expand scale.

2. Tackling known problems lacking solutions

Beyond applying AI to problems with proven solutions, it is worth asking if AI can help development practitioners find solutions to persistent development challenges. For example, there are several unresolved problems facing low-income countries that are hampered by the slow pace of scientific breakthroughs. This includes discovering vaccines or developing more climate-resilient crops. Beyond requiring faster scientific advancements, other problems like corruption, poor tax collection, and low agricultural productivity have interventions that show inconsistent effectiveness across contexts, with incomplete understanding for why the inconsistency exists. Could AI help solve such persistent problems—for example, by contextualizing interventions to achieve more consistent impact or by identifying new relationships between variables we had not previously considered?

3. Discovering unknown unknowns

Focusing only on known problems and solutions limits the discovery of high-impact innovations. Exploring “unknown unknowns” can yield breakthroughs. For example, mobile phones weren’t initially envisioned for mobile money, yet innovations like M-Pesa—launched 24 years after mobile phones became commercially available—enabled many African countries to leapfrog traditional banking. How can we accelerate the emergence of such transformative applications?

The private sector successfully discovers unexpected innovations through venture-backed startups and accelerators like Y Combinator, regularly launching successful products consumers didn’t anticipate. For instance, smartphones were expected to improve existing functions like calling, image sharing, and portable music. However, in 2009, just two years after the first iPhone and one year after the launch of the Android operating system, Uber emerged and brought a fundamentally new form of personal transportation. If we believe that AI can be as foundational as personal computing or mobile devices, how can we stimulate the sector to more rapidly discover these groundbreaking applications in low- and middle-income countries?

Each approach is a very different kind of investment

Given AI’s early stage, impact isn't guaranteed, and each of the three approaches carries different levels of risk and potential benefit. For example, the clearest opportunities lie in bucket one—enhancing proven interventions. If evidence shows students learn best when taught at their skill level, AI could help deliver this tailored instruction cheaply and at scale. In bucket one, the key questions are about whether AI enhances the underlying mechanisms that drive impact and is less about the mechanisms themselves.

Bucket two addresses known problems lacking effective solutions. It relies on AI to overcome current limitations, making it riskier than bucket one. For example, we may have a hypothesis that agriculture extension programs perform inconsistently across contexts because it is challenging to incorporate new data sources and contextualize information delivery with fidelity, and that an AI system could help. As theoretically promising as AI powered contextualization may sound, both the current incarnation of the underlying intervention and its AI augmentation have not proven to be routinely cost-effective at boosting lagging agricultural productivity. AI could address other challenges, such as efficiently testing vast crop variety combinations to identify the highest-yielding, drought-resistant strains. While promising, these solutions might require larger research and development and computing budgets.

Bucket three has the highest level of uncertainty—how do we discover outsized value when the problem statement and potential solutions are undefined? Some innovation experts argue that inventing the unexpected requires investing in the people and processes that drive exploration. This can take the form of fellowships and incubators that embed social entrepreneurs in health facilities and schools, to workplace or academic breaks that allow technical experts to innovate outside of their daily tasks.

While impactful AI applications can come from any of these approaches, these buckets have increasing levels of uncertainty and require funders to make very different types of investments. Indeed, it is possible that as we move from bucket one through to bucket three we are moving from lower risk, lower return to higher risk, higher return. The risk is clear, but we won’t know about the returns until we try and then evaluate those attempts.

The sector should start aligning on how to make smart bets

There are at least two trends that should make the development sector extra thoughtful when investing in AI. The first is that aid budgets are shrinking, and that there are lots of unmet needs. Investing in AI is a bet that allocating a portion of scarce resources to this novel technology will prove cost-effective relative to the important needs of today. The second trend is that AI is the shiny new thing. Hype mixed with anxiety about being left behind can lead to a misallocation of resources.

Development funders and practitioners should act now, while the “AI for good” sector is still emerging, to shape its trajectory by building approaches to identify best bets and aligning on evaluation frameworks that ensure high impact spending. Over the coming months, we'll collaborate on detailed frameworks to help development organizations make smart investments in each area—stay tuned!