Some grant decisions should be made at random(!)

Don’t laugh. The notion that grants should be given at random rattled around when the National Lottery was set up over 20 years ago: the joke was that since prize-winners are chosen at random, maybe grant-winners should be too.

Perhaps we should resurrect the idea. The medics have studied it: Australian health economists looked at every grant application to the National Health and Medical Research Council of Australia in 2009 – all 2,085 of them – and analysed the scores given by the expert panels that assessed applications.

Now, if there’s one thing we know about experts it is that they’re not very good. For example, the US National Academy for Sciences showed that judges’ decisions about imprisonment varied dramatically and predictably, depending on whether the decision was made before or after lunch. The Nobel laureate Daniel Kahneman reported how, given the same picture on different days, radiologists contradict themselves 20 per cent of the time, as do stock market analysts, pathologists and many others. Extraneous factors often hold sway.

The Australian health economists put a margin of error around the experts’ scores to account for extraneous factors such as these. They found that 80 per cent of proposals were ‘never funded’ – those applications would get binned even with extraneous variations in their favour; only 9 per cent were ‘always funded’ – they scored highly enough for extraneous variations not to sabotage them. Nearly a third (29 per cent) were ‘sometimes funded’ – their fate depended on how supposedly-irrelevant factors happened to play out that day.

The health economists note that “strong and weak grant proposals should be identified consistently, but most are likely to occupy a tightly packed middle ground”. This study – the only one of its type I’ve seen – showed there was still what the authors called “a high degree of randomness”.

So let’s formally introduce randomness – just for the applications that are neither stars nor duds. It might save considerable time and, therefore, money. It is also honest: a 1998 US paper called for some random grants because “instead of dodging the fact that chance plays a big part in awarding money, the (random allocation) system will sanctify chance as the determining factor”. roulette-1

Grant-makers and their trustees and panels might hate this idea because it reduces their decision rights. But it might also reduce “grant rage” from rejected applicants and hence reduce aggravation in their jobs. It might have secondary benefits too, such as discouraging grant-seekers from wasting time hanging around at drinks receptions hoping to suck up to grant officers or trustees who might influence their case, because they’d know there was a good chance of a decision being random anyway.

We all know that grant funding is scarce. The experience of running randomised evaluations in less developed countries – in most of which some people get something and others don’t – is that people who are accustomed to scarcity often value the transparency of allocation being made at random. It is better than the usual allocation by patronage.

This article first published in Third Sector.

The day after I filed it, somebody at my tennis club happened to recount the uproar when, one year, the club’s batch of Wimbledon tickets were given to the chair-person, the ladies team captain, the men’s team captain, etc. It only subsided when those tickets were recalled and re-allocated via a ballot in the bar. It seems that we all appreciate the transparency of random allocation when valuable resources are scarce.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s