This article first published in Third Sector Magazine.

It’s June, which brings the Queen’s official birthday, and perhaps this year you – like many charity sector people before you – will get lucky and appear in the Birthday Honours list. If so, arise, Sir or Dame Reader, for I have an important task for you.

This auspicious occasion presents an opportunity to find out whether Her Majesty’s gongs actually make any difference. We currently don’t know, despite all the sound and fury about them. 

When the health services researcher Iain Chalmers was knighted for his role in creating the Cochrane Collaboration, the leading source of rigorous evidence in healthcare, he wondered whether the honour would make people pay him more attention. So he asked a colleague, Mike Clarke, to carry out a randomised trial. Chalmers’s outgoing letters were, by random allocation, signed either “Iain Chalmers” or “Sir Iain Chalmers”: Clarke monitored the response rate and times. A clue to the findings is that the resulting paper is called Yes Sir, No Sir, Not Much Difference Sir.

This little story says a lot about evaluations and evidence. First, this trial was free. Randomised trials have a reputation for vast expense. This is wrong: there’s nothing inherently expensive about having a control group, nor populating it at random. This trial used data that was cheap to collect or being collected anyway. So do most of the new-ish breed of low-cost public sector randomised trials, such as those by the government’s nudge unit to assess how to get people to pay tax on time: HM Revenue & Customs is already collecting data on whether you pay tax.

Second, the trial was quick. Randomised trials are reputed to take ages, which is also untrue. They take as long as it takes for the outcome of interest to appear. No sensible evaluation could do otherwise. If you’re assessing whether learning a language at school affects a person’s lifetime earnings, then you can’t avoid a long wait. However, if you’re interested in whether – to cite a real study that we’ve seen before – including information about your charity’s results in your fundraising solicitations increases donations within two months, then you have to wait only two months.

Low-cost and rapid randomised trials could also be used much more to assess charities’ work. By not doing them, we’re missing fantastic opportunities to find out what works best.

And last, the “not much difference, Sir” effect may have occured because Chalmers was already so well-known and respected that people replied to him promptly anyway. What’s true for the mighty Iain Chalmers might not be true for you: this experiment’s results might not be “externally valid”, to use the jargon. A sample of one doesn’t tell us much, which is why science is fundamentally about repeatable results: nobody is impressed if you achieved cold fusion in your bathroom last Tuesday but can’t do it again. To my knowledge, nobody else has scientifically studied the effect of a gong. {People often say that RCTs are the ‘gold standard’. This is really unhelpful. A well done RCTs is the design for a single study to understand the effect of (an) intervention(s). RCTs are not even the ‘gold standard’ for understanding the effect of (an) intervention(s) -that’d be a systematic review or meta-analysis of a load of decent individual studies.}

This is where you come in, Sir or Dame Reader: you can help to find out. Look up this study, run it on your own correspondence and report the findings. I’ll happily compile the answers – and tell the Palace.

This article first published in Third Sector.