Bradford Research School: The tentative language of evidence

For most areas in education, we can make tentative statements based on the available evidence. We can weigh up evidence, and use that to help us decide what would be the most fruitful thing to focus on. We place our ​‘best bets’. It’s important that when we communicate these ​‘best bets’, we acknowledge that tentative nature of our claims.

Have a look at this, from the EEF’s Cognitive Science Approaches in the Classroom:

“There are a significant number of studies showing that spacing across days and lessons can have a small positive impact on learning outcomes.”

One enthusiastic reading of this could be: Spacing leads to positive outcomes. But the word ​‘can’ is crucial. They haven’t used ​‘will’. The next paragraph explains why:

“There are, however, some important limitations to the evidence. Substantial variation is found between study results with a significant number showing either no impact or a negative effect. Also, while spaced practice principles were tested in real classrooms within these studies, the approaches were rarely delivered by classroom teachers. Those that were designed and had the lesson materials provided by researchers.”

In the guidance, the word ​‘suggest’ is used 23 times, ​‘may’ 46 times, and ​‘might’ 41. ​‘Must’ only appears 5 times. This might come across as sitting on the fence, but it is a realistic presentation of the evidence, while trying to support teachers in implementing findings.

Every time we use words like ​‘may’ or ​‘can’, we are forced to reflect in a few questions:

• Under which circumstances was this effective?
• Under which circumstances was this ineffective?
• What can we say for sure?
• What don’t we know?

And being able to reflect on these questions is more beneficial to a nuanced understanding of the evidence than bolder claims. The questions signpost us towards reading further around the subject.

Research into practice

One of our challenges when writing our guidance on Working Memory (download here) was using the extensive evidence on working memory to make appropriate recommendations for classroom practice. Once we move from research into practice, we move further from the evidence. We used the analogy of a bridge in our talk at ResearchEd. On one side of the bridge is the evidence, on the other the classroom. Our goal is to remain faithful to the evidence, but we also need to help teachers make that more concrete.

So our approach was to share the research evidence. Then use this to outline broad principles in line with this evidence. Then use concrete examples informed by the principles. We know that we cannot say that x approach will definitely work, but it is a sensible recommendation for the kind of thing that is in line with the principles.

For example, in our section on memory aids, we write: ​“Children with reduced working memory ability benefit from the ability to offload aspects of task performance on to external memory aids. Transferring some of the processing requirements of a task in this way reduces the demands placed on working memory, releasing capacity to help with successful task completion.”

This is a sensible principle based on the evidence. We then try to exemplify:

“These aids are generally physical or visual aids. For example, it could be a readily available list of French verb endings, the formula for calculating circumference, or a list of sentence starters for a history essay. To aid with calculations, multiplication grids and number lines are simple examples of memory aids, particularly for more complex addition and subtraction.”

We frame these as reasonable approaches aligned with the evidence, and expect teachers to identify similar approaches from their own context.

In our upcoming webinar on Disciplinary Literacy, we will wrestle with this challenge of a lack of certainty. Daniel Willingham, in When Can You Trust the Experts, writes, ​“Indeed, when certainty is not available, a plausible guess is the best we’re going to do.” We might do a little better than guesswork, but unfortunately we won’t be definitive.