What is evidence-based giving and why does it matter?

Interview with Caroline Fiennes, Director of Giving Evidence (16 mins):


This interview was made possible by the Skoll Foundation. It was recorded in Oxford in April 2015, in a single take and with no notice of the questions.

More videos & radio interviews are right here!

Posted in Impact & evaluation, Promoting giving | Leave a comment

Avoiding waste in medical research

This article first published in the Financial Times.

The news that Facebook founder Mark Zuckerberg and his wife Priscilla are putting $3bn towards “ending all disease” has renewed the spotlight on the giving of Silicon Valley’s wealthiest. They are not alone in their enthusiasm for medical research: it is also the most popular target of charitable giving in the UK. 

There is a problem, however: about 85 per cent of medical research is wasted. That’s equivalent to 22 of the 26 miles which people run in marathons to raise funds for research into cancer and other conditions. Globally, that waste costs around $170bn every year. Perhaps many of us suffer with conditions which could have been cured if those funds had been spent more productively.

The startling statistic comes from the academic work of health experts Sir Iain Chalmers and Paul Glasziou in Oxford and Eastern Australia respectively. They identify four main causes of waste and suggest a number of ways in which we might fix them.First, wrong question. Much research addresses questions to which the answer may already be known: fewer than half of medical trials appear to be informed by all the previous relevant research. Another slice of research asks questions which are of little or no interest to patients. For example, only 9 per cent of people with osteoarthritis in the knee want more research into drugs, though over 80 per cent of trials for that complaint are drug-focused. Such mismatches occur for many types of medical problem. And right across the board, the most researched problems are often not the most severe or most prevalent.

Second, unreliable research methods. Many trials fail to compare an intervention with anything — and in my last column we saw how misleading that can be — or are conducted on too few patients. The average trial tests an intervention on only 36 people — clearly way too small to say anything statistically robust. (Incidentally, Emily Sena of Edinburgh University suggests that people interested in reducing the number of animals in pre-clinical research should, ironically, push for trials with more animals. Many studies have too few animals to be useful — taking the animals’ lives to no end — whereas fewer studies with more animals might enable researchers to discover something useful.)

Third, research is not clearly reported. For example, clinical research does not always adequately explain what the intervention being examined actually was. Does the patient see the same psychotherapist in every session, or do they change around? How often are the sessions, and how long are they? What is the dosage of the medicine, and how often do people take it? Was the surgery carried out on frail old ladies or strong young men, because you’d expect their survival rates to be quite different? If the research purports to provide a “recipe” for creating the alleged results, any doctor reading it needs that recipe in some detail.

And fourth, often research isn’t reported. Anywhere, ever. About half of all clinical trials are literally never published. Health systems and taxpayers pay dearly for this. In 2009, for example, governments around the world were concerned about a flu epidemic and stockpiled the drug Tamiflu made by Swiss company Roche. The UK government alone spent £500m on it. A Japanese researcher noticed that of the 10 studies on Tamiflu, eight had not been published. After a long fight, enough data about those unpublished trials were released to show that Tamiflu may not reduce deaths and does not even reduce hospitalisations.

Worse, it is not just a random half which never see the light of day. Studies funded by pharmaceutical companies (which have a strong incentive to show that their products work) are four times less likely to show a negative result than are independent studies. Studies with negative findings tend to take a year longer to publish than those with positive findings. Studies funded by charities and private donors fare only marginally better:

[Graph from The Lancet series on research waste, ‘Increasing value, reducing waste 4‘]

Fixing all this is difficult. Health research is a “system”, where behaviour is driven by incentives, infrastructure and information. The incentives for academics and pharmaceutical companies do not always align with those of patients and the public.

But if you support medical research through charitable donations and foundations, you have some powerful levers. You can ensure that the medical research organisation is aware of these problems (they are well-documented in a series in The Lancet) and has systems in place to address them.

For example, there should be a process for ensuring that research addresses patients’ priorities. The James Lind Alliance, which is affiliated to the NHS, holds lists of patients’ priorities in asthma, Parkinson’s, depression, end-of-life care and other topics.

You can require researchers to publicly explain how they decided on their research question, their method and the sample size, and to “pre-register” (that is, specify publicly) their research plan and publication timetable —  this effectively commits them to publishing it.

And you can stipulate that research which you fund or participate in stands on the shoulders of everything that already exists and is under way — and helps others in turn to stand on its shoulders — by detailing the researchers’ actions and results, and ensuring all the resulting data are freely accessible.

These steps may sound blindingly obvious — but they don’t happen by magic and could save many lives.

Posted in Uncategorized | Leave a comment

What would have happened otherwise?

First published in the FT.

Did the charity make it happen, or would it have happened anyway?

The purpose of a charity’s work — and your support for it — is to create a benefit beyond what would have happened anyway. This, of course, sounds obvious but (as my home discipline of physics powerfully attests) it’s fine to state the obvious as it can throw up some unexpected insights.

Charities are under considerable pressure to “demonstrate their impact”, yet few examine or recount what would have happened without them. “What difference does your work make beyond what would have happened anyway?” is perhaps the single most useful question that donors or trustees can ask.

Continue reading

Posted in Impact & evaluation | Leave a comment

The evidence system in the mental health charity sector

UK non-profits delivering mental health services are not great at producing or using scientific evidence.

This is the main finding of a new study by Giving Evidence. We interviewed 12 such organisations to understand their ‘evidence system’, i.e., how evidence is:

  • Produced
  • Synthesized
  • Shared, both ‘outbound’ from them and ‘inbound’ to them – and stored.

Screen Shot 2016-07-08 at 14.19.59

Continue reading

Posted in Uncategorized | Leave a comment

Why do so few charities have their meetings in public?

All charities and charitable foundations exist to serve the public good. Most of them are Open Meetings coversubsidised by the public, through various tax breaks. Whereas any company must have a meeting at least annually at which the directors are held to account to the people whose capital they deploy, in over 15 years in this ‘industry’, we’d only encountered two charities /foundations which have meetings at which the people whose capital they deploy – the public – or the intended beneficiaries can what goes on. The 800-year-old City Bridge Trust lets anybody observe its decision-making meetings, and Global Giving UK has an AGM at which anybody can ask anything. Why don’t more?

It’s hard to be accountable to people, or to hear from people, if they’re not in the room. So we wondered how many charities and foundations have public meetings.

Continue reading

Posted in Uncategorized | 2 Comments

Be a flexible friend

This article first published in the FT.

Toilet roll seems an unlikely emblem of effective philanthropy. Yet I’ve heard of a donor
who specifically funds loo rolls in London museums and galleries on the simple grounds that this kind of donation is both necessary and tough to fund.

Many donors want to fund the glamorous stuff: front-line programmes, identifiable projects and new buildings being prime examples. To do so, many “restrict” their gifts to those items. The main problem with this approach is that donors generally all want to fund the same things — none of which include loos, core staff or rent.

Continue reading

Posted in Uncategorized | Leave a comment

Results of systematic review about outcomes for young people

Like those of many social programmes, the goals of Sail Training voyages are long-term: In this case, to improve young people’s life chances, involvement in employment and ST report covertraining, and sound mental health. However, many organisations which provide Sail Training cannot conduct or commission high-quality longitudinal studies that demonstrate an impact on these long-term outcomes, because of complexity and cost.

So Giving Evidence was delighted to be asked to identify short-term outcomes that, if ‘produced’ by an intervention, have a beneficial effect on key longer-term outcomes. If future research can show a link between the intervention(s) and certain short-term outcomes, and there is a known link between those short-term outcomes and particular longer-term outcomes, then one can make a coherent and evidence-informed claim about the long-term outcomes produced by the intervention.

Our findings are here.

diagramDiagram 2 Continue reading

Posted in Uncategorized | Leave a comment

How to give it: Why charity should begin in the science lab

This article first published in the Financial Times.

Not all charities are good causes. This may sound surprising, because we’re used to thinking of them all as being somehow virtuous, but they vary in their effectiveness. Smart donors know that good intentions alone aren’t enough.

Take, for example, the important goal of reducing cases of diarrhoea in Kenya — a major cause of child death. One solution is to provide chlorine at the water pump for people to add to their water when they collect it. Another is to deliver chlorine to households so people can add it there.

Both of these sound pretty sensible, but researchers at the Massachusetts Institute of Technology have found that providing chlorine at the water source prevents more than twice as many cases of diarrhoea for a given sum of money. Put another way, a lot of children will get a potentially fatal disease if donors make the wrong call about which programme to fund.

Continue reading

Posted in Donor behaviour & giving stats, Effective giving, Impact & evaluation | 2 Comments

Oops: we made the non-profit impact revolution go wrong

By Caroline Fiennes and Ken Berger, managing director of Algorhythm. 

The non-profit ‘impact revolution’ – over a decade’s work to increase the impact of non-profits – has gone in the wrong direction. As veterans and cheerleaders of the revolution, we are both part of that. Here we outline the problems, confess our faults, and offer suggestions for a new way forward.

Non-profits and their interventions vary in how good they are. The revolution was based on the premise that it would be a great idea to identify the good ones and get people to fund or implement those at the expense of the weaker ones. In other words, we would create a more rational non-profit sector in which funds are allocated based on impact. But the ‘whole impact thing’ went wrong because we asked the non-profits themselves to assess their own impact. Continue reading

Posted in Donor behaviour & giving stats, Effective giving, Impact & evaluation | Leave a comment

Science and philanthropy: podcast

A lovely interview on US radio station The Business Of Giving with Denver Frederick. We discussed lessons for philanthropy / nonprofits from Galileo, Zen and the Art of Motorcycle Maintenance; how nonprofits sometimes harm, how we don’t yet have good frameworks for analysing the (likely) effectiveness of (future) campaigns and other systemic work, and much else.


Posted in Uncategorized | Leave a comment