I am reading now for my chapter 8, which assesses microfinance from the point of view that the essence of economic development is the creative growth of new institutions---in this case, microfinance institutions. As I have probably written before, this perspective casts Muhammad Yunus as a Henry Ford, a visionary who helped spawn a global industry with novel techniques to mass produce a valued product. The Grameen Bank he founded employs thousands, serves millions, competes, and innovates. The histories of the United States and other rich nations can be seen as streams of such business successes, which together facilitated mass escapes from poverty. In this light, the success of big microfinance institutions from Bolivia to Indonesia is economic development. (On Yunus and other microfinance pioneers, see chapter 4.)
That's a big idea. So is "development as freedom," the evaluative perspective of chapter 7. I confess that both these ideas originated in a more mundane effort, my attempt to mentally organize great books I'd read. While working in 2008 on the Pitt & Khandker replication, which involved crunching a lot of numbers on a computer, I visited microfinance programs in Egypt and Bangladesh. As I wrote in chapter 6, seeing women crowd into a branch of the Lead Foundation in Cairo to get new loans forced me to confront this paradox: thanks to work on the computer back in the hotel room, I was concluding that there was little solid evidence that microcredit helps on average---yet who was I to tell these women what to do with their lives? Two months later I visited Bangladesh, first tagging along with Stuart Rutherford in and around Dhaka, then taking an overnight trip into Rangpur district courtesy of my generous host, Imran Matin at BRAC.
As I rode in the van along rutted roads snaking through rice fields, I realized that several notions of success were at play in the grand conversation about microfinance. The econometricians had theirs: statistical proof of average change in indicators of poverty. In my mind, Stuart represented another view, which connected with the "development as freedom" perspective of Amartya Sen, himself a native of Bengal territory. Chapter 7 began on that ride as my Stuart Rutherford chapter.
And at that moment, chapter 8 began as my Beth Rhyne chapter. Beth has long been an eloquent and energetic proponent of the institutional view of microfinance success. Here she is in an essay called "A New View of Finance Program Evaluation," first written in 1992, then published in a 1994 book (please comment on my use of extended quotes, an experiment for me):
This new view takes a radically different stance….It argues first that the causal links between receipt of credit by individual borrowers and their subsequent economic growth are indirect, credit being but one factor in an extremely complex process of decision making by enterprises and households. Financial services are valued more for their general enabling effects across whole economic sectors rather than for the direct change induced by individual loans. Second, the new view argues that it is far better to build the capacity of the financial system than to provide a substitute for its inadequacies. By providing finance directly, often on a noncommercial basis, old-view programs reduced incentives for commercial lenders to innovate in areas where donors or governments already operated. Finally, the new view recognizes that, in the long run, nationwide financial services cannot be funded from limited donor or government coffers; a healthy, sustainable financial system must be based on mobilization of resources from the citizens of the country itself and through commercial sources. In short, the new view of finance emphasizes building financial intermediation systems that offer savings and credit services on a commercially sustainable basis.
In light of this new view of finance, methods of evaluation need to be rethought. For the most part, evaluations of credit programs are still based on the old view's ideas about causality. They are centered on the presumption of a direct causal link between receipt of credit by individual borrowers and a particular desired economic response, for example, changed borrower income resulting directly from receipt of a particular loan. In essence, these are still sector program evaluations rather than evaluations of financial system development.
There are two problems with the old evaluation model. First, the focus on the direct causal response to individual loans misses much of the richness of the effects of finance, which are generally diffuse rather than strictly linear. Second, evaluations that try to measure that response are, in effect, attempting to prove that financial services are valuable, a task that is better left to more general research that explores the relationship between the presence of a healthy financial system and economic growth. On the basis of what is already known about the value of financial services, one can be confident that wider availability of credit and savings services benefits the users, their communities, and their economic sectors. Additional knowledge about the benefits of financial services is needed, but it is best gathered in the context of a rigorous research effort that is not necessarily focused on individual loan programs. If program evaluations can be freed from the burden of proving that finance matters---a task for which they are not well suited---they can concentrate on evaluating the quality of the services and their institutional setting.
She offers "a framework for evaluating finance projects," which looks at institutions and clients:
In keeping with the emphasis on building healthy financial institutions, the new view would consider market discipline to be one of the most reliable and relevant indicators of performance. Each of the two evaluation levels is associated with a strong commercial test. The service-client relationship is best measured by a market test of demand, or the willingness of clients to pay. If people pay full cost for a service on an ongoing basis, then evaluators can be sure that the service is valued at least as highly as its price. By their actions, clients reveal information on the value of benefits, which is more credible than verbal responses to questionnaires.
At the level of the institution, the strong commercial test is financial self-sufficiency, or the ability to cover all costs from program revenues. These two tests, which are interdependent, make finance programs "self-evaluating." The tests are easily verified and, if both are passed, evaluators can be confident that the program is successful.
At the client level, Beth advocates examining client outreach (who is being served), quality of service, and---connecting to Sen---"effects on clients' range of decisions":
Donors and sponsors of finance programs are always intensely interested in the effect their efforts have on clients. Their ultimate motivation for becoming involved in finance---the new view of finance notwithstanding---remains a desire to increase the economic activity or improve the quality of life for direct clients and their families, employees, or customers. Evaluations are placed under great pressure to demonstrate such changes. However, the path from financial sewice to changed economic performance or quality of life is full of curves, bumps, forks, and even dead ends. It cannot be easily traced. The path is best understood not by examining the end points but by taking the perspective of the client as he or she negotiates it.
Evaluators must recognize three things that appear obvious to clients. First, economic and quality-of-life decisions are affected by a wide range of factors, of which finance is only one. Competition, markets, health, weather, and many other factors can all have an overwhelming direct effect. The effect of finance is not likely to be as clear-cut as the effect of these other factors. Second, clients use finance for a variety of purposes, as described above: protection against bad times, facilitation of efficient business operations (liquidity), and financing of investments. Third, use of financial services is not a single event but an ongoing process involving a series of decisions and a range of alternative sources. Tracing the effect of a finance program through all these processes is difficult for any evaluation. Therefore, when attempting to measure the effect of finance programs on clients, evaluations must recognize the limited role of finance, and particularly of specific transactions.
It is, of course, possible to distinguish the effects of any given variable, such as finance, from among a variety of contributing factors. Statistical techniques have been designed explicitly for that purpose. These techniques must be rigorously applied if they are to yield reliable results. Their application requires collection of data at two or more points in time as well as the use of control groups. For the vast majority of donor-assisted finance programs, such data are too expensive to collect, and evaluators fall back on measurement of subjective responses from clients and reliance on client memories. Such subjective responses, assuming they are positive, are of little value except for public-relations purposes.
More importantly, techniques designed to control for other factors are of little value if the ultimate indicators being examined are not the most relevant ones. This happens when evaluations employ a model of the role of finance that does not reflect the way clients actually use it. Typically, the evaluation assumes that financial transactions will be used for investment and therefore measures changes in return on investment. In such cases, all other uses of finance tend to be overlooked.
In order to avoid these pitfalls while still examining the effect of finance on clients, evaluations should stay close to the direct uses of finance. The basic question to be pursued is: How has the availability of this financial service changed the clients' decisions? In other words: What can clients do now that was not possible without the service? The aim is to explain how the service changed the strategies and options available to the client, and how those financial decisions affected other economic decisions. Questions should stay close to the direct and immediate uses of the services and examine how the stated use was satisfied before the service was available. Examples might focus on such changes as ability to purchase raw materials at cheaper prices or accumulation of enough savings to survive a drought year. The line of questioning may have to be as open-ended as are the strategies available to clients. Rather than providing definitive "impact" results on predetermined indicators, such a line of questioning produces results that are indicative of the nature of the effect. Although they may seem less precise, such results are more likely to be accurate reflections of what the provision of financial services actually accomplishes.
17 years later, Beth's views remain similar.