Giving Evidence

With Keystone Accountability, we recently worked for a funder who was relatively new to monitoring / evaluation / learning. We created for them a ‘primer’ to introduce some of the key concepts a ‘primer’ to introduce some of the key concepts, and are publishing it because we feel, and hope, that the material is useful for a wider audience of funders and implementers.

It is designed to explain what monitoring is, and what evaluation is, and how they differ. We structured our thinking into a four-level framework. This simply splits out the various questions about monitoring and evaluation (note that, as the document explains, monitoring and evaluation are two completely different things, even though they are often conflated):

• Level 1: dimensions of the grant; inputs (such as grant size) and grantee activities
• Level 2: tracking changes around the grantee, e.g., increase in number of jobs, change in grantee partner revenue, number of workshops run
• Level 3: evaluating grantees: i.e., establishing what of those changes result from (i.e., are attributable to) the grantee partner
• Level 4: evaluating a funder: i.e., establishing what of those changes result from (i.e., are attributable) to the funder

We present these four levels as a ladder, because the issues at Level 1 are simpler than those at Level 2, and so on, both in terms of the types of data / analysis needed and the conceptual complexity.

Download the primer here.Why charities should do much less evaluation–>

Various types of monitoring, evaluation and other information that funders need (2nd half of this talk) –>

Keynote talk and very spirited panel at the Philanthropic Foundations of Canada, Toronto, Oct 2018.

Notice the all-female panel 🙂

(We will eventually cut the slides into this so you can see what Caroline was talking about. For now, you can download them here and follow what’s going on.)

Caroline Fiennes gave a keynote presentation at the Global Evidence and Implementation Summit in Melbourne, October 2018. To watch, click on the photo and wait a second. You may need to log in – any email address is fine. Excuse the didgeridoo interruption!

Other talks from the conference are here.

Demanding a financial return often reduces the social benefit rather more often than impact investors let on

A version of this article first published in the Financial Times.

It is a beguiling offer — an investment that can produce a financial return and also a social / environmental benefit. Private benefit plus public benefit. It probably does happen sometimes but certainly not for every impact investment product. Investors must be on their guard.

The impact of something is the difference between the world in which that thing happens versus the world in which it does not. So assessing impact means considering what would have happened without it.

This is how to assess the impact of anything. For example what would have become of you if you had not been educated? What would have happened to Europe without the reparations demanded from Germany after World War I?

Establishing what would have happened otherwise, the counterfactual, is sometimes impossible as in the reparations example. In those cases we have to make reasonable conjectures based on everything else we know. Sometimes it is possible – though it may be complicated. This is a whole branch of social science. Continue reading →

Lack of diversity is a problem for foundations and grant-making committees

This article first published in the Financial Times.

Every donor who sets up a charitable foundation needs a board. And every company  starting a charitable programme needs to determine who will make the decisions about what it does and whom it supports.

There seems to be a major problem with these boards. In the UK, 99 per cent of foundation board members are white, according to data published this summer by the Association of Charitable Foundations. Only three per cent of foundation trustees are under 45 years old. Sixty per cent are retired. Two-thirds are male. They are “drawn from a narrow cross-section of society: white British, older and above average income,” the report says.

When a friend of mine began running a foundation outside Europe, she discovered that several people listed as trustees were, in fact, dead. The dead are weirdly important in philanthropy – for example, they are major donors – but they’re not meant to be making decisions. Continue reading →

We need to be scientific and fearless about assessing whether proposed solutions work

This article first published in the FT

The Bill & Melinda Gates Foundation seems to have wasted six years and $1bn, having initiated a programme to improve teaching effectiveness in US schools. An evaluation released last month showed that it had a negligible effect on its goals — some of them worsened — which included student achievement, access to effective teaching and dropout rates.

Much the same happened to Ark, the UK-based charity founded by the hedge fund industry. It created and co-funded a programme in 25,000 schools in the Indian state of Madhya Pradesh, supporting the government to improve school performance. It was based on sound research about how to reduce teacher absence, improve teaching and create more accountability through school inspections. Ark has worked on it since 2012. Continue reading →

This article first published in the Financial Times.

Anne Heller has done something that I had never previously seen in my 18 years in the non-profit sector. She identified a social problem, scoured academic literature to find a solution, and then set up a non-profit to implement it. That approach sounds jaw-droppingly obvious, but it is in fact very rare for a charity to design itself around existing evidence.

Ms Heller had worked for the city of New York when Michael Bloomberg was mayor, running shelters for homeless families. She noticed that about 10 per cent of the families who use the shelters returned repeatedly. In other words, the services which the shelters provided were not solving these people’s problems. Continue reading →

We are producing a map of the existing evidence about child abuse within organisations

New project! Giving Evidence is working to produce a map of the existing evidence (and gaps in it) about what is effective at reducing child maltreatment – particularly child sexual abuse – within organisations, such as youth clubs, sports clubs, churches etc. We’re working with the Centre for Evidence and Implementation, and Professor Aron Shlonsky of Monash University in Melbourne, who have undertaken considerable research around this topic. Continue reading →

Wide-ranging chat about charitable giving, with the Daily Mail.

What to give to, what to avoid, one of Caroline’s favourite charities, how to choose a charity… (~20 minutes) Please click on the picture below to watch the full video.

New report!

Giving Evidence has been consulting with UK charities and donors to identify the topics in which they would like further research (/ evidence / data) into charities and philanthropy. This was a large-scale and ground-breaking exercise, using a (variant of a) consultation method developed in medical / health research for eliciting from patients and their carers where they would like researchers to focus.  We are today (2nd July 2019) launching the findings, which you can download here. 

This sits alongside a separate project which we have done to map the existing research around strategic and operational management of charitable and philanthropic activities. A ‘review’ version of the findings and method of that ‘supply’ study are here.

The press release summarises the findings of the two studies. Continue reading →