‘Ask an important question an answer it reliably’ is a central tenet of medical research. Yet  much ‘impact research’ in the charity and social sectors (including monitoring and evaluation’) isn’t like that. Instead, we ask lots of questions, and answer them somewhat unreliably because resources won’t stretch to answering them all properly. The selection of questions which get answered is often driven by funders, rather than by what’s important operationally or unknown.

Giving Evidence has thought for a while that it would be better if sectors identified their key unanswered questions, prioritised them, and then systematically worked down the list, corralling the available research resource to answer each important question reliably. That is, if we used sector-wide research agendas – rather than answering badly loads of ad hoc monitoring and evaluation questions.

Giving Evidence was therefore delighted to work with a funder to explore doing exactly this. The sector is Industrial Farm Animal Production (think battery chickens and pigs in cages), and the work is funded by a private US-based individual donor with a long history there. We created and ran an initiative to consult within that sector, to create and execute a sector-wide research agenda. The resulting prioritised list of research questions is here.

The unanswered questions could be of many types: where the intensive animal farming is, how it has arisen, who is doing what about it where, what is effective in what circumstances, ‘basic research’ about animal preferences, how the funding flows, what laws are in place and how /whether/ where they are enforced, etc. That is, the research could be ‘just’ gathering data, and/or it could be evaluating potential fixes.

Such a model to gather and prioritise research questions is, to our knowledge, untested in the social sector. However, it potentially could be very valuable – in avoiding waste on research which is duplicative or addresses unimportant questions, misleads by being poor quality, or is unduly influenced by vested interests. It also builds on relevant experience elsewhere.

For example, the James Lind Alliance is a UK not-for-profit set up because of the mismatch between the issues which medical researchers research and what patients and their carers want them to research. It seeks to ensure that medical research prioritises issues which concern patients. It run Priority Setting Partnerships (PSPs) within a specific area – such as asthma or Parkinson’s or end of life care – and brings ‘patients, carers and clinicians together to identify and prioritise the top 10 uncertainties, or ‘unanswered questions’, about the effects of treatments that they agree are most important.’ PSPs are essentially structured consultations to create a research agenda, often initiated by patient groups, or sometimes by doctors or hospitals interested in an area underrepresented in health research, such as acne. In asthma, the James Lind Alliance found that patients were keen to know if there is value in the breathing exercises they’re asked to do, which researchers had overlooked. In terms of getting that research agenda enacted, the JLA is still tiny but has recently been integrated into the research funding mechanism of UK’s National Health Service.

Of course, whereas health research can solicit opinions from people affected by medical conditions, it’s obviously not possible to solicit the views of animal directly (though actually there is some clever research which infers their preferences by observing their choices, e.g., whether cows choose to be in the field or the barn at night).

As ever, we are learning from initiatives in other sectors, rather than emulating them precisely.

A reasonable outcome of this initiative would be that players in the sector a) increasingly realise that there are critically important questions which are unanswered, b) commit to spending a reasonable amount of resource to identify those questions and to prioritise them, c) agree to work with others, for example to set standards for themselves or others when collecting evidence, doing research, etc.

We consulted about the concept, ran a prioritisation exercise, and the results are here.

A much better research system than the current M&E system–>