Donors Funding Technology: 10 Recommendations

Summary

Funding the global public good of technology is a useful way for donors to leverage the impact of their aid. Different types of technologies appear to be important to development progress, and to spread, in different ways. ‘Lab coat technologies’ (inventions) spread easily and improve quality of life, ‘process technologies’ (institutions) spread with difficulty and are important to economic growth. For all donor interventions, however, it appears that context matters—the same technology or investment has varied impact in different environments. While there still may be some universal ‘best practices,’ there is also a considerable heterogeneity in what works where. Donors should take the importance of context on board when designing their technology interventions.

Introduction: Aid’s Changing Role

Recent trends in the analysis of the growth impact of aid suggest it has been reasonably successful at its traditional role using traditional channels—promoting economic growth through investment.[1] At the same time, the systematically more rapid GDP growth of developing countries than donor countries over the past decades alongside a stagnating share of rich country GDP going to aid translates into an ever smaller aid flow in terms of recipient GDP or government. In 1992, ODA was equal to 1.9% of the GDP of low and middle income recipients. Today that is down to 0.6%.[2] For aid to continue to have a significant impact on development, it will have to be utilized in different ways.

Alongside approaches that use aid to leverage domestic resource mobilization, foreign private capital, or global public good provision, the use of aid to support innovation and the rollout of new technologies is one obvious way to achieve outsized impact. Technology is itself a global public good—this means both that it is under-supplied by the market and that small investments in technology have the potential to achieve very large impact. The creation and distribution of a stable smallpox vaccine is one oft-cited example.

As a result, there has been a sustained interest amongst donors in the utilization of technology for development, demonstrated not least in the creation and support of institutions including the Consultative Group on International Agricultural Research founded in 1971 and the World Bank infoDev (information for Development) program in 1995 and the recently-created Global Innovation Fund.[3]  Donors have used approaches including direct support for research and development as well as prizes and Advance Market Commitments to produce new technologies that respond to particular development challenges —including a pneumococcal vaccine market commitment proposed by a CGD Working Group and launched by GAVI in 2009.[4]

But along with successes like the pneumococcal vaccine, donors have supported a number of failed technology experiments. Hans Binswanger discussed donor-backed efforts to introduce advanced technology in agriculture almost 30 years ago: “high-yielding varieties and fertilizers are no precondition for mechanization, nor is mechanization—except, perhaps, the mechanization of irrigation- a precondition for the adoption of high yielding varieties and fertilizers…  The direction of technical change—that is, the emphasis on mechanical or biological technology—is determined by the factor endowments of an economy, and attempts to encourage both concurrently as equal parts of a technical package have generally failed.”  In particular, “attempts of governments and donor agencies to bypass the animal-traction stage for direct tractorization” have “repeatedly failed…” and “…the switch from animal traction to the tractor has little or no effect on yields.” [5] Tractorization was about labor substitution—and labor was what African farms had in abundance.

More recently, we have seen reasonably consistent negative evaluations of donor-backed efforts to roll out clean cook-stoves,[6] as well as computers in classrooms as a learning tool. Analysis of the One Laptop per Child program, which distributes computers loaded with learning applications, has shown particularly disappointing results. In Peru, Inter-American Development Bank researchers found no impact at all on math or language scores.[7] An evaluation of the program in Nepal suggested computer-assisted learning had “no impact or a negative impact on student learning.”  Again, in Costa Rica a program that gave children a laptop found they did considerably worse on math tests.[8]  These are three examples amongst many that suggest that for all that aid can play a powerful role in spurring the development and rollout of high-impact technologies, it is not always straightforward to achieve that.

This note looks at both macro and micro evidence on the role of technology in development and the record of donor-funded innovations in delivery in an attempt to uncover lessons for the design and rollout of technologies and technology programs that might have the outsized impact donors are looking to achieve.

Technology and Development: The Macro Picture

A considerable debate in the economic literature on growth is the role of technology in that process. It is clear that ‘total factor productivity’ (the bit of output per worker that can’t be explained by stocks of capital like factories, roads and educated people) is a major factor in determining incomes. But that is only to say ‘we know it isn’t all about investment.’  Total factor productivity may be related to culture or government institutions or the kind of technologies that people in lab coats invent with a ‘eureka.’

Diego Comin and colleagues study the historical evidence of the global diffusion of lab coat technologies from electric arc steel production through telephones to vaccines. They provide a reason to downplay the importance of lab coat technologies in income growth. They suggest that while the initial distribution of these technologies is even more unequal than global income distribution, rates of convergence in technology usage are far more rapid than we see with income—at 3.7% per year. For technologies invented since 1950 the speed of convergence is 6% a year. That means it takes only about 12 years for countries to halve the technology gap between leaders and laggards for post-war technologies like the mobile phone (see Figure 1).[9]  Convergence in incomes has been nowhere near as rapid.

Figures at the end of this paper look at levels of energy use, air transport passengers, railway passenger miles, mobile phone subscriptions, Internet users, and secure Internet servers per unit of GDP. They suggest these technologies are about as widespread poor countries as in rich on this measure—some are in fact more widespread at least at the extensive margin of access (Internet access and mobile phone subscriptions) thanks to saturation in rich countries. Some are less widespread than you might expect at the intensive margin of use (measures like the number of secure servers, or air travel miles). But with most of these measures along with electricity consumption, motor vehicles, televisions and computers, poor countries often have higher ‘stocks’ per unit of GDP than rich countries.[10]

Some technologies have become truly globally ubiquitous: as part of their examination of the World’s Middle Class, Abhijit Banerjee and Esther Duflo took an inventory of a general store in a village in rural Kanataka, India. The goods in the store included three jars of sweets, candies, chickpeas, magimix, bread, papadam, toast, two types of biscuits, incense, pan parag (betel nut and chewing tobacco), tea bags, haldi (turmeric) powder, cigarettes, lux soap, talcum powder, washing powder and shampoo, many in individual packets.[11] It is surprising they did not find matches as 97% of Indian households purchase match books on a monthly basis—with a total of 1.3 billion books a month.[12]  And along with matches, soap and washing powder, a range of technologies have disseminated very widely in the developing world—even to some of the poorest parts of the poorest countries: technologies including concrete flooring, bicycles, motor vehicles, lamps, antiseptics, plastic, wire, radio, mobile phones, condoms, and oral rehydration salts. The same is true of vaccines—we are approaching 90% coverage even in the poorest countries with use per unit of GDP orders of magnitude higher in the poorest countries.

For all this spread of technologies and convergence in access has difficulty in explaining income convergence (because until recently there wasn’t income convergence)[13] it may be one important factor behind the fact that nearly all measures of the quality of life from health through education to levels of violence to human rights appear to have also been converging for some time.[14]

Perhaps quality of life improvements have been driven by ideas and ‘lab coat technologies’—inventions like vaccines or plastics—while income growth requires innovations in ‘process technologies’ or institutions like functioning stock markets or advanced logistics systems.[15] Vaccines, radios, cars and plastic sheeting all work broadly the same across countries. Logistics systems may not. Process technologies may be more at the whim of local institutional and cultural constraints than lab coat technologies.  

Pritchett, Woolcock, and Andrews note that many developing countries have adopted the form of rich world institutions—public sector process technologies—without the function, a process they call ‘isomorphic mimicry.’[16] An example might be adopting a legal minimum wage; while in some countries that wage is actually enforced, in others, it is set so high compared to average wages that enforcement would be disastrous were it possible. Perhaps in some comparatively simple cases isomorphic mimicry actually ‘works’ in terms of delivering certain quality of life outcomes (school systems, even if ones that don’t teach very well, court systems, even if ones that don’t protect rights that much), but isomorphic mimicry does not do well enough in the case of institutions to support the complex transactions of the large, internationally competitive firms that are vital for rapid economic growth. This might be suggested by the huge gap between the formal institutional rules reported by the Doing Business reports and the actual practice of business regulation in developing countries, for example. Halliward Driemer and Pritcvhett report that following the regulations to the letter, it might make as many as 50 days to clear imports into Kazakhstan, but the average surveyed importing firm in the country suggests it only takes about 10 days to do so. This result generalizes across a range of regulations and countries: the higher the de jure regulatory burden the higher the gap between de jure and de facto outcomes.[17]

It should be noted that if lab coat technology is designed to be used in complex institutional settings, it too will be subjected to the issue of institutional context (e.g., good examples, discussed above, may be cookstoves and computers in schools). On a related note, Guan Gong and Wolfgang Keller review the literature on technology transfer between firms and conclude lab coat technology is to some extent tacit as well, involving face-to-face interaction.[18] That will help to explain the strong role of migration in speeding transfer, as well as the more rapid transfer of technologies between countries which share a common language.[19] More broadly, the impact of all technologies is context-specific to at least some extent. Inoculating children using a more efficient smallpox vaccine would have a zero impact on health outcomes if tested anywhere today. Inoculating children using a fully effective malaria vaccine would have a far larger impact in central Africa than Finland.

So the gap between the two kinds of technology—process and lab coat—is not as large as perhaps might be imagined, in particular when lab coat technology is embedded in significant physical capital (an expensive machine), requiring considerable human capital to use (such as a skilled operator) as well as an institutional setting that allows for its exploitation (a business environment that allows for efficient production and sales). That may suggest that at least some lab coat technologies do have a large role in growth outcomes. And it may also suggest in particular the importance of technologies embedded in limited capital and cultural-institutional capacity to broad-based progress[20]    

Regardless, context is going to matter more to lab coat technologies when use is particularly determined by cultural and institutional settings—when traditional cooking styles do not adapt easily to heating over an improved stove or the curriculum isn’t designed with computer instruction in mind, for example. This is not necessarily obvious ex ante (for example, comparing clean cook-stoves with a technology like solar lamps one would not immediately expect an easier adoption path for the second). But it is likely to be more of a problem when it provides a public good or relies heavily on public provision of a range of services (education, for example). Indeed, some of the most useful technologies in a developing country context may be those that provide a similar outcome while relying far less on public provision of services: the mobile phone, which allowed competitive private provision of telecommunications in the place of monopoly (often public) provision; or sugar-salt solutions which allowed private citizens to mix life-saving anti-diarrheal treatment instead of relying on medical staff with clean needles and bags of intravenous solutions.

Again, the technologies that are likely to have the most impact in developing countries are those that are embedded in little physical capital and do not require skilled operators or strong institutions. One role for donors may be to help support the development of such technologies, in order to make up for a global market failure: most private and government research and development is aimed at responding to rich world problems in rich world settings. The market GNI of high income countries is $51 trillion, compared to $634 billion for low income countries, so most technology developed by the private sector is developed with rich countries in mind. Governments only make this discrepancy worse: rich countries spend $1.2 trillion on research and development compared to around $300 billion in low and middle income countries. This matters because of the often very different needs of consumers in poor and rich countries. For example, out of the top 10 causes of death in high income and low income countries only three conditions appear on both lists. And when it comes to production technologies, the question for places with little capital is, ‘How do we effectively use a lot of people to make a world-class product?’ That is an irrelevance in rich countries where research is primarily about inventing a new machine that can save on employing a person.[21]

Innovation (and Aid) —The Micro Picture

That context matters to the introduction of technology should come as little surprise. Take Internet expert Mike Dertouzos, who travelled to Nepal in 1999:

A few of us techies got together with a colleague from Nepal, fully expecting to boost his nation’s economy by 20% through clever use of the information marketplace. Unfortunately, we quickly found out that even if we got the communications, hardware, software and training for free, we would still fall short of our goal: Only 27% of the Nepalese are literate. And of these, only a small fraction speak English. When we asked what services that smaller group could offer, we hit a brick wall. Many are not skilled, and those who are already busy running their nation’s businesses. Maybe we were too ambitious when we envisioned a future workforce in Nepal selling office services to New York and London via the Web. What if we focused instead on selling Nepal’s famous crafts, like custom-made rugs, on the Web? That got us into all sorts of other concerns about establishing trust among distant buyers and distributing the goods. The potential of the modern information age seemed overshadowed at every turn by the ancient forces that separate the rich from the poor.[22]

Much as the literature suggests context is likely to matter to the spread and impact of technologies, the same appears to be true of aid interventions in general. Eva Vivalt (2015) studies a large dataset of impact evaluation results to find the degree of variance in impact reported by evaluations of similar projects in different contexts, as it might be a program supporting free school lunches in Mexico to a similar program in Nigeria. She finds that for any two evaluations of the same intervention and outcome, the effect reported by the more positive evaluation is on average 114% larger than the effect reported in the less positive evaluation.[23] Similarly, Pritchett and Sandefur (2013) undertake an illustrative literature review of the impact of class size on learning outcomes as measured by randomized evaluations to argue that “probably the most notable feature of the distribution of class-size effects in the larger literature is not that it is centered around a small, statistically insignificant effect, but that it is widely varying across contexts.”[24]

Multiple contextual factors often appear to be very important. Nadel and Pritchett (2015) note an observation that they attribute to Karthik Muralidharan: the reasons given for the failure of similar projects is often very different. Looking at textbooks in schools, for example, failures to change learning outcomes have been blamed on the books being stored away from students, that the books merely substituted for learning materials previously purchased by parents, that they were worthless for illiterate students, or that they only worked if combined with teacher performance pay.[25] Why something works (or doesn’t) is a function of many different factors, all of which can vary by region. This applies even to medical technologies—the cost-effectiveness of vaccine programs is significantly impacted by parental take-up, for example, to the point that payments to parents to get their children vaccinated considerably reduced per-child costs of programs in parts of India.[26]

Seemingly small factors of design can also significantly impact outcomes. Nadel and Pritchett note that the concept of ‘nudges’ to influence behavior is predicated in precisely that—small program design changes having an outsized effect. A well-illustrated example of this phenomenon is the demand effect of moving from a very small price for a product to making it free—where demand increases out of all proportion to what might reasonably be expected.[27] All of this suggests that, for many interventions, even small changes in design or context might produce dramatically different outcomes, significantly limiting the predictive power of a rigorous evaluation result in one context regarding the success of a program that is similar but not identical or in a different setting or at a different time.

Bold et. al. (2012) provide another example. They scaled an intervention tested by Duflo, Dupas, and Kremer (2015) involving contract teacher program managed by an NGO. When still run by the NGO (so that the scaling was adding more districts), the learning results of the intervention remained broadly unchanged. But when the contract teacher program was run by government (more districts and a different implementing institution), the positive impact of the program disappeared.[28]

And because context matters, individual fixes may be designed to address the wrong—or at least the non-binding—constraint. We build toilets where the problem is habits[29] or schools where the problem is attendance.[30] We pay for attendance when the problem is learning,[31] or invest in teaching aids when the problem is the teacher doesn’t show up or doesn’t care to teach.

For example, giving glasses to children who need them in a Chinese study had an impact on learning equivalent to as much an extra half year in class[32]and a study in Northeast Brazil found children with bad eyesight but no corrective lenses are more likely to drop out of school and over 17% more likely to repeat a grade.[33] But even assuming that there is a widespread problem regarding eyesight and education, the right (binding) constraint may still vary. Perhaps it is the cost of the glasses themselves, or the need for skilled opticians to do the test or the demand for corrective vision in the first place. A new eye-test system developed by the MIT Media Lab uses a matchbox-size plastic device attached to a cell phone loaded with software. The snap-on plastic device costs about a dollar and requires no expert instruction to use.[34] In places with easy access to cheap eyewear, this solution may be adequate to significantly increase corrected vision.

In places without such access, the for profit model of an Indian company Essilor may have appeal: it supports vans which villages offering free eye exams and prescription plastic glasses which cost an average of about $5.00. A pilot van profitably operated while testing 65,000 patients and selling over 8,600 sets of glasses.[35] Another joint approach to the optician and eyewear problem is adaptive eyewear, which uses a lens filled with silicon oil that can be adjusted by the customer until it best corrects their vision—more fluid equals stronger prescription. A new technology uses two lenses that slide across each other to alter and may cost as little as $4.00.[36] But a final barrier may be demand: the trial of eyeglasses and education in China found nearly one third of children refused free glasses perhaps because of the mistaken view that wearing glasses further damages eyesight.

Despite the importance of context, donors tend to do poorly ensuring that evaluations report on it. Eva Vivalt notes “most [evaluation] papers declining to take on the tasks that would make their findings more useful, such as: specifying a model or ‘causal chain’ through which the intervention is supposed to work; reporting results for outcome variables that other studies also consider; or providing basic information about the context of the intervention.”[37] Pritchett and Sandefur (2013) argue that the importance of context also demands shorter feedback loops. “Integrating rigorous evidence into the ongoing organizational implementation and learning processes requires an approach that is lower cost per finding, has faster feedback loops, is more integrated in decision making cycles than the way ‘independent impact evaluation’ has traditionally been conceived,” they suggest.

According to Pritchett and Sandefur, the problem of context is ubiquitous—meaning ‘planning with rigorous evidence’ usually won’t work. They argue that vaccinations are part of a ‘vanishingly small’ set of solutions where it will (i.e., methods proven by randomized control trial in one location are best practices that can be scaled up globally). We have seen this may be too kind to vaccinations because the development impact of a similar vaccination campaign does depend on context (disease burden and parental demand for vaccinations, for example).

But this notion may be a little too harsh on other development solutions because, across a wide range of countries, similar approaches have worked to deliver development outcomes. Beyond other examples in health, there is competitive provision to roll out mobile telephony that has worked from Somalia to Sweden, double-entry book-keeping to control accounts, currency to make transactions, pens to write text, or the sale of matches to light fires. More arguably, independent central banks to control inflation, giving money to poor people to make them richer, and high marginal tax rates to reduce inequality have all worked across a range of countries and contexts.   

Again, Duflo et al (2015) report results from multiple randomized control trials across a number of countries run by different organizations in which beneficiaries are given a productive asset that they choose from a list, training and support for the asset they have chosen, as well as general life skills coaching, weekly consumption support for some fixed period, access to savings accounts and health information or services. They suggest “there is significant site-by-site variation” but “the program appears to have positive impacts on most indices for most countries.”[38] For all the world is complex and best practices may be chimerical, it isn’t so complex that broad strategies might work similarly across the world.

As suggested by our brief tour of the macro literature, the micro evidence suggests the kind of interventions that are likely to be most dependent on the broader context for success are those that are part of a complex of causal chains (i.e., learning is affected by multiple factors inside and outside the school, while immunity to measles is largely determined by either prior exposure or vaccination).

Because context matters, innovative approaches should be introduced with an emphasis on flexibility and rapid feedback. If the intervention simply doesn’t look like it is going to work, or needs to be redesigned, this process should not wait on the publication of an evaluation of the project in the Journal of Development Economics.

And because the context matters, consumers may be best placed to find the best uses (if any) of a technology. Paul van der Boor and colleagues study the source of innovation in mobile banking services and argue that of twenty innovations they investigate, from bank transfers through savings storage to salary disbursement, 85% originated in developing countries and 50% were first pioneered by users (usually before being adopted by phone companies and banks).[39]  They argue that “The type of user-innovation that was observed in our sample—novel applications of existing functionalities—is likely to be observed in other cases in which a technological platform is sufficiently pliable and in which modification does not require very advanced skills or costly resources.”

Lessons

The above analysis suggests a number of lessons for where and how donors might want to direct their technology spending:

1. Donors should have a preference for supporting ‘double disadvantage’ technologies—those that are designed to minimize capital and institutional requirements rather than labor inputs and which address public good issues. All else equal, these will be the technologies currently garnering the least attention because public goods are routinely underfunded and because this particular set of solutions will be of particularly limited interest to rich countries. Examples may include vaccines for neglected tropical diseases and governance technologies like identification but also methods for distributing services that are more labor intensive —scratch cards as opposed to post-pay using bank accounts for mobile phones for example.
2. Especially with technologies with many uses such as the mobile phone, users may be far better at finding innovative uses than suppliers or aid agencies. Consumers should be mined for innovative ideas.
3. Fixing the right problem may call for (testing) a suite of technologies rather than one. This suggests advantages to approaches that can do horse races —including the potential of using prizes for achieving final outcomes in a particular setting or results-based advance market commitments and development impact bonds to subsidize delivery of desired outcomes at the lowest cost.
4. Given lab coat technologies in particular often diffuse very rapidly, if a technology does not diffuse, the problem may be the technology rather than the diffusion. Advocates should have a compelling case for why we would expect their high-return intervention to have minimal appeal if it is struggling to gain acceptance (compare the mobile phone where adoption has been rapid in developing countries to mobile broadband, where it has been much slower).
5. Process technologies are less likely than lab coat technologies to translate easily across institutional settings. Put another way, technologies of the quality of life may more easily scale to other contexts than technologies of economic growth (vaccines do better than logistics systems). And technologies that appear ex-ante to be embedded in institutional structures should be subject to particularly rigorous reappraisal in other countries or sectors.
6. There is danger in assuming that a single rigorous evaluation in one context demonstrates universal applicability of a development solution. Similarly systematic reviews of multiple implementations of a development solution which produce a single ‘average effect size’ should also carefully explore variance in effect size and reasons for that variance.
7. Project evaluations should speak to causal issues and context. This will help potential replicators to understand why the intervention worked or failed and develop adaptations so similar projects can work (better) elsewhere.
8. Evaluation tools should be used as part of the learning process of development organizations rather than as stand-alone, ‘independent’ reviews. With regard to the funding of technology innovation in aid programs, this suggests the need for ongoing data gathering and analysis during design, testing and rollout of a technological solution rather than (or in addition to) independent evaluation of just one stage of the process. A well-designed process should be able to use the same data for both exercises.
9. At the same time, there is probably an ex-ante justification for believing in the potential global development impact of products such as (cheap) vaccines to widespread diseases, (cheap and reliable) sources of electricity and other technologies where there is considerable evidence of demand for similar existing products. Context is not always overwhelming.
10. Testing aid-funded technology interventions on a bespoke basis is inefficient. By preference, these interventions would be tested and adjusted as part of the project development process using information that is regularly collected as part of a broader government-based effort to collect data on development progress. And this suggests an additional area for donor support of innovation—supporting the technologies of low-cost monitoring including new satellite monitoring and mobile phone based survey approaches.

Note: CGD will host the event, “Innovation for Development: Why are we not Getting to Scale?” on Monday, June 13, 4:00-5:00pm ET, featuring Charles Kenny, Ann Mei Chang, and other data and technology experts.

Image

Image

Image

Image

Image

Image

Image