Publishing the whole truth

This article first appeared in Third Sector.

The C&A Foundation – linked to the department store that closed in the UK but is flourishing elsewhere – joins a small clutch of non-profits this month that publicise the lows as well as the highs in their work.

It ran a programme in 18 garment factories in Asia designed to improve both working conditions and productivity. Some aspects worked in some factories, some aspects worked in others. Rather than taking the conventional option of reporting only the glories, in a kind of self-promoting publication bias, the C&A Foundation is publishing it all: data from each factory, correlation coefficients, statistical significance tests and operational challenges all feature in the report, which is called Frankly Speaking. (Disclosure: I am a co-author.)

Likewise, Engineers Without Borders, a Canadian NGO, has been publishing an annual failure report since 2008. In each one, volunteer engineers recount tales of an error they made in the field. Yet, despite the praise for EWB and the obvious value of hearing the whole truth, EWB remains an anomaly. To my knowledge, it’s the only operating charity that publishes so candidly. When I asked its chief executive why it discloses when others don’t, he said: “Well, if your bridge falls down, it’s pretty obvious.” Indeed. By contrast, plenty of social programmes appear to work but are shown by sophisticated analysis not to work. The crime-prevention programme Scared Straight and some micro-finance programmes are examples of this.

The C&A Foundation encountered something similar – but the opposite way round. When Giving Evidence got involved, it looked as though working conditions in the factories hadn’t improved much, but the inclusion of later data in the analysis showed that they had.

Medical doctor John Ioannidis, now of Stanford University in California, uses complex statistical tests to unearth often shocking stories within clinical trial data and says his work echoes a theme from the ancient literature of his native Greece that “you need to pursue the truth, no matter what the truth might be”.

Dogwood Initiative is an environmental campaign on Canada’s west coast. It was inspired by EWB to publish its own failure report and found an important issue in its internal culture. “Dogwood Initiative could change our relationship with failure,” says the report. “It involves piloting an idea, measuring results, figuring out what works and what failed, adapting and rebooting.” Giving colleagues that right to admit failure can take time. Dogwood Initiative’s first annual failure report took so long to agree internally that it ended up covering the next year too.

The World Bank also holds internal “failure fairs” and finds it needs rules to ensure the discussions are safe – each speaker makes a case but can’t blame anyone except themselves and can’t name anyone else.

The funding and competition in the charity sector undoubtedly discourage confessions of weakness, but if we don’t do it, we won’t learn from ourselves and we’ll bias the literature from which our peers – and funders – can also learn. EWB’s Admitting Failure website lets anybody upload stories.

Go on, I dare you.

This entry was posted in Impact & evaluation, transparency, Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s