Below: A map of the existing rigorous ‘what works’ evidence about institutional responses to child abuse
Giving Evidence and partners have produced a map of the existing evidence (and gaps in it) about the effectiveness of all institutional responses to child maltreatment. We included: any study with a decent counterfactual (so RCTs and some quasi-experimental designs, plus systematic reviews), in settings such as organisations such as schools, youth clubs, sports clubs, churches, and residential care; studies from any country and published at any point in time, and included both academic and non-academic studies; and studies about prevention, encouraging disclosure, response (e.g., legal sanction), and treatment of survivors.
Below is our map. We ran a systematic search for the kinds of studies that we were including, screened them all for relevance, and then categorised them. We put them on a grid, in which the rows are interventions (from the top: prevention, disclosure, response, treatment), and the columns are the outcomes. Each study that we include is placed in the cell(s) to which it relates (e.g., if the study looked at the effect of Intervention 2 on Outcomes 3 and 4). This is the standard structure of an evidence gap map. As you can see, much of the map is blank, but there are a few marked concentrations.
- Only 82 studies. Only 72 once duplicates are removed. Three are protocols, so that’s only 69 completed studies
- The studies don’t at all match where the world’s population is: none from India, only two from China, and only three from Africa.
- The evidence is very concentrated: mainly in education-based prevention programmes, in early education and school settings. Fully 53 of the 61 primary studies look at that
- Most of the studies are about sexual abuse. Sexual abuse was considered by 56 of the primary studies.
- Only 12 studies reported on physical abuse, four on neglect, and three on emotional abuse. None had emotional abuse as a main focus.
- Most of the studies are about prevention. Prevention was examined in 59 primary studies (completed and on-going studies), and 10 systematic reviews.
- Treatment was studied in only two primary studies and two systematic reviews; and response was studied in three primary studies and five systematic reviews. On disclosure, we found no primary studies of interventions aiming to facilitate disclosure (!)
- No completed study has assessed interventions with adults to stop them offending within organisations
- Only few studies focus on children particularly at-risk
- No causal studies conducted in religious organisations
- Almost all the studies have appreciable risk of bias i.e., the ‘answers’ that they report may be wrong. The colours on the map above indicate risk of bias: red is high, and green is low.
- Only one study had educational attainment as an outcome
- There are no primary studies about treatment from the last nearly 30 years
- Institutional safeguarding practice was very little studied: only in seven primary studies
- Very few studies came from practitioners and non-profits
We also explored finding data about activity in child protection (e.g, what schools are doing, governments, youth clubs etc.) and mapping it to the EGM frame – to identify where there is lots of activity but no evidence (=priority for new evidence) and areas of little activity but where the evidence is clear that something works (=areas to encourage more activity). Basically, that didn’t work because the evidence about child protection activity is so scant (=important research gap!). Our report into this exploration is here.
Clearly, many nonprofits and other types of institution are working on improving safe-guarding and child safety, and the issue has gained prominence because of the allegations about aid workers in Haiti, etc. It therefore seems important to establish what is already known and what still isn’t known about what interventions are effective at improving child safety in various situations. That should enable (i) delivery organisations and funders to make evidence-based decisions, where there is sound evidence, and (ii)researchers to prioritise producing evidence in the important gaps which still exist. To be clear, our study is a review of the existing evidence: we did not produce fresh primary research.
The report is currently going through peer review with the Campbell Collaboration. The title is registered with the Campbell Collaboration, and that document contains the precise scope. The protocol is published by Campbell Collaboration here. which explains precisely what we did (search, screening, coding etc.)
What evidence & gap maps are
Evidence Gap Maps (EGMs) consolidate what we know and do not know about ’what works’ in a given area– in this case, child abuse within institutions. They show where there are systematic reviews and impact evaluations, and provide a graphical display of areas where evidence on this topic is plentiful, sparse or non-existent.
EGMs show what research already exists within your scope (above a certain quality threshold); they do not show what that research says. They’re like real maps, which show where the pubs are but not what they serve 😉
For example, you may want to ensure that the institutions that you fund have done and are doing everything possible to prevent child abuse. You’re aware that many organisations put in place safe-guarding policies. But you are not sure whether safe-guarding policies actually have any effect on the levels of abuse. This EGM did not find that out, but it does show whether anybody has yet looked at that question, and if they have, the geographies and types of institutions that they examined, and the research methods that they used (which matters since some research methods are more reliable than others). That may answer your question, or you may want to commission new research into it. The forthcoming ‘guidebook’ (see below) will show what the evidence says.
This makes EGMs useful for policymakers, funders and practitioners looking for evidence to inform policies and programmes. For donors and researchers, these maps can inform a strategic approach for commissioning and conducting research. EGMs are not intended to provide recommendations or guidelines for policy and practice but are meant to be sources that inform policy development and guidelines for practice.
This map looks only at studies of ‘what works’, i.e., the effect of some intervention(s) on some outcomes(s) for particular groups. Our map therefore does not include non-causal studies, such as studies of prevalence of abuse, attitudes, activities within organisations (eg., to reduce abuse or encourage disclosure).
This video explaining evidence and gap maps was produced by the Centre for Homelessness Impact, the UK’s ‘what works centre on homelessness:
The research was undertaken with the Centre for Evidence and Implementation, and Monash University.
EGMs show where there is evidence and where there isn’t. They do not investigate what the evidence says. Clearly funders, practitioners and others need to know that, so we are producing a ‘guidebook’ to the evidence, which will say what it says. We hope to publish that during the Summer / early Autumn 2020.