Charities (gasp!) using and producing sound evidence

This article first published in the Financial Times In August 2018

Anne Heller has done something that I had never previously seen in my 18 years in the non-profit sector. She identified a social problem, scoured academic literature to find a solution, and then set up a non-profit to implement it. That approach sounds jaw-droppingly obvious, but it is in fact very rare for a charity to design itself around existing evidence.

Ms Heller had worked for the city of New York when Michael Bloomberg was mayor, running shelters for homeless families. She noticed that about 10 per cent of the families who use the shelters returned repeatedly. In other words, the services which the shelters provided were not solving these people’s problems.

She learnt from a search in academic reports that some of these problems are caused by stress and begin very early, resulting in poor cognitive development in early childhood. A good remedy is strengthening the parent-child bond. She searched for a programme which seemed effective, was evidence-based, accessible to families and could be implemented in a city the size of New York.

Using a simple internet search, she found one: a 10-week programme to coach parents of new babies, called Attachment and Biobehavioural Catch-Up programme (ABC), developed by Professor Mary Dozier of the University of Delaware. Ms Heller wrote to Dr Dozier and found she was interested in her findings being used in practice — which not all academics are — and had already supported a few small-scale implementations of the programme in the US.

Supported by Dr Dozier and her former graduate student Kristin Bernard, at Stony Brook University on nearby Long Island, Ms Heller partnered with Erasma Monticciolo, an expert in community engagement and set up a non-profit organisation called Power of Two, which delivers ABC in New York City.

Since it began nearly three years ago, Power of Two has served about 1,300 families. The programme tracks outcomes to ensure that families are benefiting from ABC as the research suggests they should. Indeed, parents show meaningful change in their sensitive parenting and children improve significantly in both their socio-emotional wellbeing and their biological regulation, as evidenced by healthier stress hormone levels. This predicts better long-term academic attainment, and physical and emotional health. This story is remarkable because most charities seem to start by identifying a problem and then just guessing at the solution.

This solution may or may not be very good. Or, worse, they start directly with the solution itself — “we provide mentoring to young people” — without a clear statement of the problem.

Academics are increasingly focused on people using their research. The New York example involved no specific “outreach” activities or communications by academics: rather, just a findable and intelligible research paper, and a willingness to get involved.

Happily, there is a clutch of charities based on good evidence. One is No Lean Season, a hunger-prevention programme I’ve written about before, which grew out of research in northern Bangladesh.

The Flemish Red Cross in Belgium has gone one better by generating decent evidence about its operational practices, as well as about the design of its programmes. It runs reception centres for asylum seekers at the request of the Belgian government. During the refugee influx in 2015, it had to more than double its number of centres (from 11 to 23) and quadruple its capacity in just three months. Sometimes it was only given a week’s notice that a new centre was needed.

There simply was not time to run the normal process of recruiting staff for the centres. So the Red Cross invented a stripped-down process. This involved screening CVs and two brief interviews, rather than the conventional longer interviews and detailed discussions of handling “cases” (people in particular circumstances).

The Flemish Red Cross is run by Dr Philippe Vandekerckhove, who realised that this was a “natural experiment”. Recently, he and a colleague compared the performance of the people recruited through the speedy process against that of people recruited normally. They used any objective metrics they could, such as sick days taken, disciplinary incidents and having to fire people.

There was no difference. The two groups performed similarly in terms of terminations, disciplining and long-term absence. There was a variation in short-term absence, which was higher among people recruited the fast way though, as Dr Vandekerckhove explains, “the refugee crisis was a crisis” and the workload exceptionally hard, so this is hardly surprising.

In short, the Flemish Red Cross found that conventional recruitment methods, which consume considerably more time and money, add no discernible benefit.

Because he is a scientist, Dr Vandekerckhove has built a team with a scientific approach even to management. They are soon publishing two academic papers about their findings. (And good for them, because they don’t have to. The “currency” of academic citations is worthless outside academia.) Meanwhile, they are integrating their findings about recruitment into their management practices.

These instances of using evidence-based practice don’t happen by magic. Trustees and donors are major determinants of what charities do. As ever, if we want charities to improve their work by basing it on decent evidence — about both what to do and how they do it — we should incentivise and reward it.

This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to Charities (gasp!) using and producing sound evidence

  1. Pingback: Why I’ve joined a board of the Flemish Red Cross | Giving Evidence

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s