This article, by Ehren Reed of the Skoll Foundation and Caroline Fiennes, first published in Forbes.

Some charities are better than others, so we should find the good ones. On that we can all agree. We should support the charities that will be most effective in addressing the world’s pressing problems. And understanding that effectiveness requires measurement. But a reliance on quantitative analysis, which is helpful in understanding some charities, could prevent us from finding ones that are doing important, system-changing work.

The charities which are easiest to measure are those whose work is proximate to the beneficiary. They distribute mosquito nets to families in sub-Saharan Africa or deliver wheelchairs to disabled children. Their theory of change – the link between their work and the intended benefit – is simple. The intervention is well understood, the outcome is predictable, and most of the variables are clear. From a funder’s perspective, the risk is low.

These interventions are like machines, and advertise themselves as such. Three dollars in = one bed net out. Five pounds in = one case of diarrhoea avoided. Cause and effect are clear. They operate within a system in which the relevant factors are known.

Working within complex systems

But a lot of important work done by charities is quite different. It involves trying to change legislation to outlaw discrimination; it’s research to uncover the human genome; it’s changing societal attitudes on same sex couples. These efforts aim to change the system. Here, success depends on factors which are unseen, often unknowable, and mainly beyond the charity’s control. The causal chains are long and uncertain.

{Added later: a UN survey of 10million people found that even in poor countries, people would prefer ‘honest and responsive government’ to affordable food. And ‘freedom from political oppression’ is more highly valued in poor countries than by any other group. Those are not what most easily-measurable philanthropy tries to provide.} 

Working on hard problems within highly complex systems, a charity’s results can take ages to materialize. Even then, the results may not be predictable, attributable, or even measurable.

Yet this is probably the most consequential work that a donor can enable. In 2006, the Institute for Philanthropy surveyed a thousand experts on UK philanthropy’s greatest achievements. The resulting list is dominated by system-changing work: campaigns which ended the slave trade, created the welfare state, and ensured universal education. This type of work generates effects that are much broader and more profound than delivering services to a limited group of recipients.

There’s frequently a trade-off. The more certainty a donor wants about results, the smaller they will be. If she’ll accept more uncertainty, by operating further away from beneficiaries and engaging more with the system around them, the ultimate effect may be greater. In other words, if donors limit their risk, they may simultaneously limit their return.

Furthermore, philanthropy is uniquely able to fund these kinds of system-changing efforts, since governments and companies are inherently more risk averse and less likely to support them.

Moneyball philanthropy?

It has become trendy to liken effective philanthropy to Moneyball, the strategy pioneered by the Oakland Athletics baseball team, which involved choosing players based on statistical analysis instead of experts’ intuition.

But the analogy doesn’t hold. In baseball, the playing field is bounded, the rules are clear, the results are immediately evident, and the variables are visible and knowable. The Moneyball approach worked because the system was reasonably simple. Certainly the same approach can help to analyse charities whose work is based on simple models of cause and effect. The charities recommended by analysts like GiveWell, for example, can all show what they achieve for £10, though none of them has much effect beyond their immediate beneficiaries.

But the Moneyball approach is hopeless for assessing charities trying to change the system. Take the work of Global Witness in exposing the economic networks behind conflict, corruption and environmental destruction around the world. The ultimate value of their work, in terms of lives saved and natural resources protected, is literally incalculable.

Unintended danger

The Moneyball approach – like much of the current debate in philanthropic sectors on how to define and measure impact – is dangerous because it leads donors to seek out only the most easily provable results. It pushes them towards interventions within the current system and beguiles them into thinking that the best charities must be able to produce simple cost-benefit figures.

As we’ve seen, this approach would have precluded some of philanthropy’s greatest successes. Many of us owe our liberty, our freedom of speech, and our education to such philanthropy. We’d be crazy to sacrifice these kinds of achievements in pursuit of an immediate “return on investment.”

Giving slow, not fast

Two factors complicate effective philanthropy. The first is that it involves making decisions under considerable uncertainty. Because donors have finite resources, they must decide between competing activities. Yet as we’ve seen, many determining factors are in principle unknowable when working within complex systems: basic medical research may be stellar or may find nothing; a campaign to ban handguns will rely on political will which may or may not materialise.

The second complicating factor is that human brains love shortcuts, and are much better at making decisions which don’t require much thought. Indeed, as Daniel Kahneman explains in Thinking, Fast and Slow, we often fail to notice that many decisions require proper thought and instead make them on the fly, leading to predictable errors. Charities whose work is based on simple theories of change don’t require much thought. Those whose work is more complex require that we take proper time to make good decisions.

In the end, this shouldn’t be surprising. Philanthropy is about making the world a better place. And making the world a better place is going to be a lot more difficult than winning a baseball game. Let’s not let an idea like Moneyball distract us from the challenge.

How donors change the system to increase evidence in government policies–>