How to make Evidence-Informed teaching work

In this post I’m presenting a conceptual model for how the teaching profession can reboot its relationship with scientific research into learning.

“…the simple truth is that, in education, everything works somewhere and nothing works everywhere.” – Dylan William – TES – 10th April 2015

“…Working out what works also means working out what we mean by ‘works’, and where science, heart and wisdom overlap and where they don’t….Evidence-based education is dead – Long live evidence-informed education.” – Tom Bennett – TES Blog 11th April 2015

Two frustrating truths:

  1. Science always seems able to measure ‘better’ or ‘worse’ ways to do things.
  2. Yet… Science cannot tell us the best way to teach.

Why is this the case?

  • Because the unique combination of a particular teacher, particular pupil and particular set of circumstances can result in an unpredictable outcome.
  • Because, metaphorically, we only focus on the “brightly lit corner” of the educational room. Ponder these two diagrams…
  • The limits of proven techniquesThe Sea of Modifiers

The first diagram shows just how limited we can ever be with trying to adopt guaranteed scientific strategies in teaching. The second one shows a similar thing – demonstrating the impossibility of ever being able to create a ‘best practice’ guide for dealing with any one individual

Another two frustrating truths to add in to ponder:

  1. Teachers ‘on the ground’ are the very best way to detect and intelligently respond to the unique individual educational needs of a child from moment to moment
  2. Yet… All people can be ignorant, naïve and deluded regarding what they perceive and what they think they are doing

So… how do we harness research to make things the ‘best’ that they can be…?

How things currently tend to happen:

  • We rely on headline summaries of research into concrete strategies… “Feedback is good”… “Homework is bad”…
  • Then we look for ‘silver bullet’ actions we can take which can be reliably rolled-out across all children in our class, or all classes in our school.

As Tom Bennett explains in his blog response to Dylan Wiliam’s article, the detail which would be of most use to teachers tends to be buried in “page 2,3 and beyond” of research papers, as these details give us the context which in practice mediates the effects – the overall average of which tends to be summarised in the headline.

So, for example, as Professor Wiliam points out, the effectiveness of ‘feedback’ to learners can rely entirely on the relationship between the teacher and the pupil; just giving feedback doesn’t guarantee a pupil will use it to move forward. Does this then mean we need scientific research to help teachers understand interpersonal relationships better – so as to map-out yet greater parameters for teachers to calculate? As someone who focused heavily on the ‘science’ of interpersonal relationships during their first degree, I’m relieved that Wiliam doesn’t think so. It isn’t more research that is required for the particular teacher, but a different relationship with the data. As he bluntly states… “The best way to become a more effective teaching body is to build and trust the ‘practical wisdom’ of teachers.” Drawing on Polanyi he adds “We can know more than we can tell”.  In other words, teachers possess tacit knowledge of a situation which could never be expressed fully by language, and certainly not by rubrics.

Okay – Sounds crowd-pleasing! Let teachers get on with it and trust their professional judgement etc. Well, sort of. There is still that nagging first truth that I started with – which is still there – that science can tell us what works better and what works worse, and my fourth truth – all people can be deluded.

So…HOW CAN WE STRATEGICALLY IMPROVE OUR PROGRESSION AS AN EVIDENCE-INFORMED PROFESSION? 

Professor Wiliam’s general solution is to ensure teachers access a range of quality professional development opportunities, tailored to suit their particular needs.

Geoff Petty this week takes that further, laying out a comprehensive list of the different kinds of research that teachers could access, and the principles they should think about when viewing it. He particularly stresses the usefulness of teachers ‘triangulating’ between quantitative, qualitative and ‘field’ research (case studies).

Here I’m going to go further still and propose a conceptual framework which school leaders and individual teachers could use to better develop as an evidence-informed profession. This framework suggests:

  1. A shift to focusing on principles of practice, rather than headline strategies
  2. A shift from looking for isolated ‘silver bullets’ to comparing contextualised ‘golden glow’ areas of research and case studies.
  3. A move towards seeing genuinely useful individual techniques as ‘silver seeds’ of practice, which can grow into something personally powerful when adapted by the professional teacher.
  4. A shift towards empowering individual teachers as the ultimate arbiters of what goes on in their classroom, according to the process outlined in the table below.

The most powerful way of moving a learner forward is not the delivery of a strategy. It is the coherent, integrated, responsive, humanly authentic educational mechanism we call a teacher, and each one of these clever devices functions at their best in a unique way, seeing things through a combinatorial lens unlike any other.

As with Dylan Wiliam’s reference to Polanyi, ijstock has recently pondered the unique way in which a teacher experiences the learning process and interacts with the unobservable processes going on in children’s minds:

“Even the most dispassionate of scientists amongst us cannot escape the realities of the educational world: when we deal with (young) people, we are operating in the social and human domain, for which scientific approaches may just prove too technically precise; those soppy soft skills may actually be better… The inescapable fact is that the reference base-line is our own knowledge and experience of what it means to learn, as experienced by us and us alone… if we want to understand the nature of learning, we perhaps need to spend less time in conferences and webinars and more time looking inwards at our own experiences as successful learners.” How could we improve the blend between research and professional judgement-page-0 There are strong calls through the ResearchEd movement for more teacher-led research to go on in schools. Whilst I more than support that where it is feasible, I have a rather nuanced vision for how teachers should become involved in research:

  1. New teachers (and existing ones where possible) should be taught the key findings of cognitive psychology regarding how people learn (less so the developmental psychology theories which can never be used to go from the general to the particular)
  2. New teachers (and existing ones where possible) should also be inducted in the skills required to carry out systematic social science research. This is not so much so that they could embark on quantitative research projects which can be used to add to our global data-bank of statistical findings – I personally think that the art of doing unambiguous quantifiable research is demanding enough for professional researchers. Rather…
    • It will enable those who can to generate detailed in-house qualitative case studies which – without seeking to produce definitive recommendations in themselves – could add to a wider bank of resources which teachers could draw upon when attempting to triangulate their own approaches.
    • It will enable them to question more knowingly the scientific claims and fads which they can be confronted with throughout their careers, and help to immunise them against sweeping claims when it is announced that ‘it has been proven’.
    • It will enable them to gain the fullest benefit from their participation in the richest single source of research data available to them… their own personal careers. Of course, teachers already do gain wisdom from this day-to-day experience, but formal training in the practice of research would allow them to more carefully question their own perceptions of what goes on from lesson-to-lesson, and the various meanings which can be deduced from the efficacy – or otherwise – of their own practice.
Advertisements

13 thoughts on “How to make Evidence-Informed teaching work

  1. Research is a human activity. Researchers are humans. They have agendas, biases. They disagree.
    And educational and psychological research is a social science. When interpreting it or attempting to apply it, you have to attend to the context of application. The bigger the difference between the context in which the research was conducted and the context of application, the bigger the grain of salt you need to take.

    Take for example your first recommendation that teachers learn “the key findings of cognitive psychology regarding how people learn”. There is disagreement about which findings are key. And just like in politics, there is a large, vocal minority. But a large part of that disagreement comes from differences in the context of research. Folks arguing for memorization, direction instruction, and the like are largely basing their recommendations on research studies from the 1980s that were conducted outside the classroom, in labs (sequestered problem-solving), with volunteer undergraduate students, or else they were conducted using very short, rote learning tasks on school children. In many cases, when the findings of these studies are applied to real classrooms, the results are very weakened or even reversed. See for example the What Works Clearinghouse, which found that several “cognitive” based strategies and technologies had zero impact on learning in randomized, experimental classroom studies.

    In contrast, the “key findings” from other cognitive and educational research starts from, rather than ends at, realistic classroom activities and real-world learning contexts (such as design-based research studies in the learning sciences, and discipline-specific educational research). These are summarized in books like How People Learn and other publications from the National Academies Press, and in journals of discipline-specific educational research (physics education, math education, etc.). There they are finding the benefits of research-based instructional strategies like active learning, inquiry learning, problem-based learning, etc. Here is just a small summary of some of the thousands of studies that provide evidence for these strategies in real classrooms, rather than labs: https://edtechdev.wordpress.com/2015/04/03/evidence-for-various-research-based-instructional-strategies-countering-critiques/

    As to your second recommendation that teachers need to learn how to conduct their own research studies, I would argue that they should only to the extent that they feel it will help improve their own teaching practice. Research isn’t a panacea for teaching practice. Teachers are designers and engineers of learning environments and learning experiences, not scientists. And teachers, like engineers, have to deal with thousands of things that science hasn’t settled on or figured out yet. Medical professionals know this, too: https://edtechdev.wordpress.com/2014/06/03/healthcare/

    Like

    • Hi Doug,

      I broadly agree with your point that there will always be biases regarding emphases and interpretations in science. It’s an area I critiqued with good success during my psychology degree (so I know where you’re coming from with this). Ultimately we will ALWAYS have to take a stance regarding what we believe is the most compelling theory or finding.

      This post isn’t really meant to be about ‘Explicit Instruction’ vs ‘Constructivism’– I’ve got something you might be interested in regarding that coming up. For what it’s worth I think they both provide distinctive strengths. My reference to cognitive psychology was really focusing on the following areas, which I think remain constant whether you are advocating inquiry based or more direct routes to learning: Our inability to attend to more than one thing at a time; the limits of our working memory; the importance of transfer to long-term memory to enable skilled performance; the implications of the above on our ability to multitask, and the knock-on for all of this on our construction of lessons, lectures, research tasks etc. I believe that these ‘lab’ discovered findings are still robust in whatever learning environment you might construct.

      Yes, I was also meaning Bjork et al’s work on spaced learning, interleaving etc. although you might have alternative suggestions for how recall can equally well be improved. Exactly where the line is drawn will always be up for debate, but I think these are all things which can be applied with some certitude to the individual student.

      I think my suggestion that basic training in the principles of good scientific research does benefit teachers a little more than you suggest it might. For sure, application to personal practice I think is key, but also their ability to read more deeply into scientific studies which come their way (perhaps from your website!) and their ability to share the good things about their practice (as they inevitably already do) with a more critical eye regarding the myriad of things which might be having an effect.

      I think it’s pretty clear overall that I think the individual teacher should be the ultimate arbiter of what judgement calls they make.

      Thanks very much for commenting 

      Like

  2. Hi Doug; “Research is a human activity. Researchers are humans. They have agendas, biases.” This is similar to the comment I recently shared on the Hunting English blog (http://www.huntingenglish.com/2015/04/11/the-research-says/) though you’ve elaborated much more fully here. Good read.

    Chris: There is allot in this post that I like very much. For me this felt very balanced, thank you! I’ve had an email chat recently with a teacher frustrated that students aren’t interested in exploring alternatives once they’ve decided they believe etc. My ‘silver seed’? (love that by the way, it’s a great way for that silent majority to find a voice… to share silver seeds!) My advice? I’ve happened to stumble on high level corporate sales training as via my first job out of uni where I got a week long sales residential and followed up by reading on persuasion etc. I’ve not studied psychology in an academic environment but, following this training, have became hugely interested in the power of this field, because it is powerful. Sales people turn a ‘no’ into a ‘yes’, a ‘yes’ that is so strong that the other person then dedicates time/money/effort to the solution …that’s the sales person’s job. Whether this is used to empower other people to get out of their own way, or to manipulate them, is simply down to the character of the sales person. I wonder if some teachers might find that corporate sales training provides useful insights for the purposes of helping students get out of their own way? This wouldn’t be for everyone of course; I too believe the silver bullet is a myth but, from my own perspective, sales training provided me with many ‘human’ insights in a short space of time and has since empowered me in situations where I’m being ‘sold to’ (and don’t people sell to teachers all the time; policies, excuses, equipment, software, courses, resources…?) I wonder, given your academic background, what you think about this Chris?

    Liked by 1 person

    • Hi Leah – thanks very much for the thoughtful ideas!

      It’s interesting isn’t it how we draw on our diverse backgrounds to make insights into the teaching world. I myself had two very diverse career areas before doing my psychology degree – very different to sales (more about them another time perhaps) – and I can see how they influence my perspective on things.

      I’ve certainly been interested in the use of social psychology in the classroom, and I think I subtly draw on what I know about the area without really thinking about it strategically. I sped-read ‘Nudge’ recently and was thinking about how I could recommend some of the techniques in there to other teachers in a way which wasn’t just an overwhelming list of things they could do. I suppose my concerns regarding including the area here were, on the one hand, the huge scope of areas you could argue to be relevant – persuasion, stereotyping, group dynamics etc. etc. – and also a desire to only really prescribe psychological ideas which you know will firmly apply to each individual – which just isn’t the case for social psychology.

      …HOWEVER…! Then you come in with your background in sales, and what is striking there is that here is a world which has had the motivation (another useful psychological area!) to really work at polishing an area of social psychology into the most powerful, refined techniques and then training people in the use of them. Yes, it would be really interesting to see how a focused burst of training in sales techniques could provide a context to draw upon for any teacher. Or we could get Derren Brown in to devise something…

      Inevitably though the same case could possibly be made for a lot of other careers. For example, I also did a year’s counselling skills training, and can draw on that quite readily… and my brother is a policeman with a wide experience of responsibility for public order – it would be interesting to watch him in the classroom for a week!

      It would seriously be interesting to see a thread somewhere fishing for suggestions from people of very useful alternative career backgrounds that people have had before coming into teaching (other than obviously, engineers now teaching DT, or Lawyers teaching law etc). Perhaps either you or I could run a post like that…? Thanks again for the post 😀

      Liked by 1 person

      • Hi Chris… oohhh, love a joint project. Would you like to take this onto email for a while so we can shape it? I’ve lead sales training in the UK/US/Canada too. You’re right; the techniques have been highly refined. I have some ‘military’ type experience too which I’ve discussed elsewhere as many people have a strange conception of military and think it is the opposite to my whole ’empowerment’ thing going on but – based on my experience – it doesn’t have to be. I can’t find a way to email you from here but if you pop me a message from my website contact page I’ll get back to you that way…

        Liked by 1 person

  3. I’m flattered to be quoted in such distinguished company! 😉 Thank you. I think there is an underlying problem behind all of these issues: the tendency of society to trump up the outcomes it requires from processes such as education – while simultaneously refusing to accept the natural limits on the process.

    The judgements being made about educational methods still spring from views about what is desirable and what is not, that are simply too precise to be valid. ‘What worked’ in a given situation is, as you accept, entirely determined by what is considered to be success. If you want to find more success, then redraw the boundaries at a more appropriate position! There is no in-built guarantee that the things now being demanded from education are actually (reliably) deliverable – I would say not.

    The decision-making that is currently required from teachers regarding the adoption of strategies that provide ‘better outcomes’ therefore also relies on knowing what bad, good and better actually are – and being able to compare them in a situation where opportunity-cost type experimentation simply isn’t possible.

    And then it is amplified by expecting teachers to see the future and know where it leads, from the moment of their intervention into the far depths of an individual pupil’s future… Yet their decisions are made in real time, with often limited information. Expecting them to do anything more than ‘what seemed right at the time’ (albeit hopefully informed by professional knowledge, experience and even intuition) is to stretch the limits of credulity.

    It’s good to see that some of these limitations are being discussed – though I suspect that much of the educational establishment will find it hard to live with the conclusions, particularly the uncertainly they imply. I think your final chart is definitely heading in the right direction, though.

    Liked by 1 person

    • Thank you IJ – I had no qualms about taking a lengthy quote from you in the context of the rest of it 🙂

      I think what makes your voice special is that it has a purity which is rare to hear on educational blogs. Most of us either get too wrapped-up in the politics, tangled-up in the practicalities of the coal-face or fixated on the meaning of key data. You write with the vision of a professional philosopher, yet one who has had a long career (ongoing) as a teacher and therefore is able to talk thoroughly from the inside-out. What’s more, you perceptively value that most unfashionable of scientific methods – introspection – and the reality that the value of all education, whether in the mind of the teacher or the recipient, will always be relative to their personal experiences and priorities.

      Thanks again

      Liked by 1 person

  4. I wish I had the time to redo all of the charts into useful items by removing the ‘academic speak’ and putting them in plain English. The opening quote is the highlight of the piece. Essentially saying that everything works sometimes and nothing works all of the time. So we can boil this down to flexibility. Is the instructor willing to abandon her plan and go with what feels like a more productive pedagogy on the spot?

    If so, we have a teacher.

    Like

    • Excellent – my first heckler!

      Essentially – spot on – the teacher should have the flexibility to do whatever she (or he) judges to be the best – I think that is the clear bottom line.

      However… if we want a system where the wisdom of the rest of the profession and the scientific method counts for anything in the judgments we make, then we have to find a way of integrating that knowledge in a systematic way that doesn’t overwhelm things.

      I worried that I had made things too simple already for most people’s tastes. Please do feel free to exemplify any aspect which you have time for which manages to go beyond simply the instructor doing what feels right at the time. In what way should she be informed beyond her own limited experience…?

      Thanks,

      Chris

      Liked by 1 person

  5. Pingback: Can and should teachers be (viewed as) researchers? | the édu flâneuse

  6. Pingback: The Thirst for Mastery: Eternal CPD – Part 1 | Stepping Back a Little

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s