Dreadful practice at the well-known charity Kids Company – around services, governance and evaluation – are exposed in The Spectator and by Genevieve Maitland Hudson here.  

Evaluations of a charity’s work are the main tool by which public donors and taxpayers can know if a charity is doing a good job. But these are normally conducted and/or funded by the charity itself which creates two major problems: first, charities have an obvious incentive to present themselves favourably; and second, most charities lack the research skills to run evaluations well.

This matters. The ‘answer’ from an evaluation depends markedly on how well the research was done. A National Audit Office study found that positive claims about government programmes often come from ropey evaluations whereas robust evaluations only allow for modest claims. And good  evaluations usually cost more than bad ones. So it’s hardly surprising that one of the few reviews of the reliability of charities’ evaluations – by the Paul Hamlyn Foundation – found that 70% are not ‘good’.

One complaint of the lady who sold her house to raise funds for Kids Company is that the evaluation report is completely unclear: “Five of its 11 pages were simply photographs of children”. This is why Giving Evidence is working to improve and standardise charities’ research reports, creating a ‘checklist’ of items that they should all contain. This idea comes from medicine where these ‘reporting checklists’ have dramatically improved doctors’ ability to make good decisions between treatments.

Indeed Giving Evidence suspects that many unflattering charity evaluations never get published at all. This ‘publication bias’ problem is rife, well-documented and fatal in medical research, and hence is getting fixed.

This isn’t solely charities’ fault. Charities are responding rationally to the badly-designed incentives they face. Incentives mainly designed and exercised by donors.

By analogy, medicine has made huge strides by improving the quality, clarity, findability, comparability and use of evaluations of drugs and devices. Work is underway to make analogous improvements in the charity world.

We – as donors and taxpayers – should demand, encourage and fund better evaluations.

The evidence-base about charities could get a lot better. Here’s how —>