Dr Dylan Wiliam The Secret of Effective Feedback, such a tremendous resource; 51 minute webinar packed with wisdom.
Dr Dylan Wiliam The Secret of Effective Feedback, such a tremendous resource; 51 minute webinar packed with wisdom.
Last week i visited London’s faintly notorious Michaela School. A few days ago I wrote an overview framing the visit in the ‘big picture’ sense. Now I propose to begin sharing a series of posts–the Michaela Files–describing some of the useful things I learned there.
As I walked through London’s Michaela School I was struck suddenly by a strong intuition—call it fear maybe–that recalled my days as a teacher in a high performing urban school. I was in my late twenties then, and my colleagues and I were seeking, like Michaela, to engineer every moment for maximum student benefit. God, we loved those kids. And they needed every ounce of what they had and what we had to have a fighting chance. So we pushed them hard and worked long hours. We were going to do every single thing we could. But the hours were often unsustainably long I can now see.
So when I glimpsed the artful intentionality of every moment in student’s lives, the impeccable designed and executed systems, I feared that this was also true of Michaela. I knew this would challenge the long term sustainability of the school. Those brilliant teachers would tire. Many would leave. Organizational memory, the culture, the will to sustain it all would be diluted. The school would regress to the mean.
But happily like many intuitions which we presume are accurate, mine appears to have been in large part unfounded. And this is very good news, not just for Michaela but for every school seeking long –term, sustainable excellence—because one reason teachers at Michaela told me they leave by five is that they do something brilliant, simple and replicable to reduce workload without eroding outcomes. And you can copy or adapt it tomorrow.
Ready for it?
Teachers at Michaela do not mark student essays and other writing. They read the essays their students write in English and History and French and Religion and they grade them but the grade is all the written feedback students receive.
How could they? you ask- perhaps thinking of the personal dialogue you establish with your students when, late at night or on Sunday morning, you fill the margins of their essays with individualized comments—“Nice, use of evidence, David, but you’ve left out an important piece of evidence from page 62.” “I love this sentence Sarah. You capture Hurston’s vision perfectly.”
This task, as Joe Kirby and Jo Facer described it to me is ‘maximum effort; minimum impact.’ It takes hours. It almost always happens at home for teachers, blurring the lines of the workday and making you feel like you are never done as you stare guiltily at those accusing stacks of essays and writing pieces. Over time you perhaps assign less writing. Or you grade them when you can and give that artful feedback weeks after students wrote the papers—and hardly remember writing them. Or you toil away and just maybe work yourself towards unsustainability. And maybe once or twice you’ve even wondered- do they read them with as much intentness as you put into the writing of them? Do we know whether they read them at all in some cases?
“We seek the opposite,” Joe said, “Maximum impact at minimum effort.” So they re-worked marking. And the solution is pretty logical.
When students at Michaela write essays, their teachers read them all and take notes to inform re-teaching the next day. Instead of marking each paper that needs a better topic sentence, they jot down themes and maybe example: “better topic sentences” and “brilliant example of topic sentence from Gabi.” The next day show up in class and say: “Many of us are struggling with our topic sentences. So let’s look at how to write them better.” and then: “Let’s look at why Gabi’s was so good.” And then: “Now go re-write your topic sentences.” Or perhaps its content focused: “Many of us misunderstood what the results of the battle were. Let’s review the events and then you can rewrite.”
And there it is. Time spend writing comments becomes time spent re-teaching. And students then must be responsible for using and applying what they’ve learned. And the greatest amount of energy can be spent on what’s most important to the group. And then—without the disincentive to assign writing—students can write some more.
Honestly, it’s brilliant. And it actually draws on what many of us already do during class, when we circulate, observe work and then address common misunderstandings. But it asks the question-is a focus on always individualizing feedback for every student always the best use of our time? Or even usually? Especially if it’s in writing? When you consider alternative uses to a teacher’s time, the answer is probably no.
Take three hours spent writing comments, allocate 30 minutes to identifying trends and planning a short re-teach and then do other things, including some non-teaching things. This is hugely important because workload has historically been an Achilles heel in the high performing urban schools movement. To change the lives of students otherwise cut off from opportunity is an immense job and often presents a brutal choice. Do you reduce the hours for the adults—the adults you love and honor and who give deeply of themselves to help others—and know that the cost will be lesser outcomes for the students and families you also love and have dedicated you professional life to serving? Or do you push for maximum outcomes for kids and know that people you care about will sometimes—often—work too hard and suffer?
In the end the core social good—the expectation shifting inner-city school—will only be as expandable as we need it to be if we can unlock large scale efficiencies- ways to continue to get maximum value for kids at more sustainable cost for adults. Insights into game changing efficiencies and synergies are rare. But Michaela School is on to one. And, as I pointed out in my first post, you can borrow it no matter what your school’s approach.
At HGS we’ve been thinking hard about how to make sure teacher feedback has maximum impact and, recently, I’ve been revisiting some blog posts that continue to inform my thoughts on this important area: ‘Close the Gap’ Marking: a whole-school approach used at Saffron Walden High School, focusing all feedback on student response. Marking […]
A lovely example of how to reduce workload BUT improve the quality of feedback: Please find details and more info at https://mrthorntonteach.com/2016/04/08/marking-crib-sheet/ but I have pasted how he uses it below:
The crib sheet as a way to provide quicker feedback to the whole classroom rather than writing comments in each book, so reducing marking time from 2-3 hours per class to less than an hour. Now I actually really do miss writing comments, leaving questions and the other bits in their books but it really wasn’t a workload issue I could continue with (especially as I have my first child on the way!).
Therefore the crib sheet allows me to go through each students’ book and I make comments on the whole class sheet using the sections below.
The benefits are that it gives me a snapshot of the whole class’s progress, allows me to ‘fine tune’ my lesson planning and it also gives activities and tasks for students to complete within DIRT the next lesson.
Using this I do the following which we complete in lessons for 20-25 minutes, you can also get a feel of what it looks like in students books. Students’ are given an A5 copy to stick in under the title of ‘DIRT’ and using a red pen they review their SPaG, answer questions I have created from reading their books, finish any work or complete an extension activity. We always finish with a spelling test to hammer home those misspelt key words.
Pupils books look something like this – we do this every 2 weeks or so.
Now, that does not mean I use this for assessments – my focus is on providing more detailed and worthwhile assessment feedback, with this being reserved for standard lessons etc. For more on what I’m trying in assessments, check out the rest of my blog.
At Michaela, we have two exam sessions each year: in February and the end of June. Nonetheless, when completing a unit we do sometimes give pupils an assessment to see what they can do. Recently, our year 8s finished learning about Romantic Poetry. To really stretch them, we decided to give them a poem they had not seen before, and ask them to write on it. The responses were phenomenal, and you can read some below. But today, I want to focus on how we give feedback following such an assessment, using a specific example.
I visited Joe Kirby’s year 8 lesson, just at the moment he was testing them on the words they had misspelled. He tested them on the spellings (in the same way as I have written about previously here) and then went on to look at what else the pupils needed to do to improve their essays.
He began by looking at grammar, a key aspect of our English curriculum at Michaela. At Michaela, we focus on memory and automaticity, and we know pupils need to overlearn each aspect of writing in order to improve. If a couple of pupils are misusing the apostrophe, we know all pupils will benefit from overlearning this key ingredient of accuracy. Joe has written three sentences on the board which come from different pupils’ essays, and he asks them to write them correctly in the back of their books. He then goes over this as a whole class, leading pupils to articulate why each apostrophe is needed:
Following the focus on spelling and grammar, Joe goes into what not to do, using examples again lifted from the pupils’ essays, and helps them to see how to improve these by explaining from the front of the class:
Here are some more examples of ‘vague’ sentences, with Joe explaining what pupils need to do better:
He then goes on to explain what precision means, and gives concrete examples of how to be precise:
Joe then leads pupils through some of the most impressive insights from their essays. This was my favourite bit of the lesson, and something I tried with my own year 8 classes the following day. When reading their books, you put a tick in the margin of a sentence you found especially impressive, and note their name and a trigger word on your feedback sheet. You can then say, ‘Elena, can you read your sentence on alliteration?’ It is lovely to celebrate the impressive responses of pupils, while also helping others see what they ought to be writing about:
Following this, pupils read one of their classmate’s essays, again focusing on what precisely made it so effective:
After this, pupils re-wrote a paragraph in their books.
The above approach is simple, and requires no marking. The teacher reads the essays, noting down examples of great work and ‘non-examples’, or examples of what not to do. The teacher then structures the feedback in a clear way, for us beginning with accuracy, moving on to ‘non-examples,’ and finishing with exemplars.
Here are some further examples of the pupils’ writing. Remember, this was analysis of John Keats’ ‘This Living Hand,’ a poem they had never encountered before. Some sophisticated insights they have written include:
‘Keats keeps the poem following free verse and no rhyme scheme to perhaps inform readers that the possibilities and powers of the ambiguities, hidden meanings and unknown capabilities are not so easily understood and that the power is so strong that it breaks all form of rhythm and pattern.’
‘This poem could be about the relationship between the poet and poem and the emotion it gives the reader. Keats could be saying that poetry is capable of inflicting an outburst of emotion, which is recollected in “tranquillity.”’
‘At the beginning of the poem, “now warm and capable” is used combining life and death imagery to describe the transience of life in the present.’
‘The poet does not refer to an actual living hand in his poem, instead it is used to symbolise the poem itself, personifying it. He does this to illustrate that life may be transient however this poem shall be transcendent, otherwise “haunt” our “days” and “chill” our “dreaming nights.”’
A review by David Didau on the EFF report – some interesting view points but not a surprising conclusion…
Question: How important is it for teachers to provide written feedback on students’ work?
Answer: No one knows.
This is essentially the substance of the EEF’s long-awaited review on written marking.
The review begins with the following admission:
…the review found a striking disparity between the enormous amount of effort invested in marking books, and the very small number of robust studies that have been completed to date. While the evidence contains useful findings, it is simply not possible to provide definitive answers to all the questions teachers are rightly asking. [my emphasis]
But then they go and spoil it all by saying something stupid like:
Some findings do, however, emerge from the evidence that could aid school leaders and teachers aiming to create an effective, sustainable and time-efficient marking policy. These include that:
- Careless mistakes should be marked differently to errors resulting from misunderstanding. The latter may be best addressed by providing hints or questions which lead pupils to underlying principles; the former by simply marking the mistake as incorrect, without giving the right answer
- Awarding grades for every piece of work may reduce the impact of marking, particularly if pupils become preoccupied with grades at the expense of a consideration of teachers’ formative comments
- The use of targets to make marking as specific and actionable as possible is likely to increase pupil progress
- Pupils are unlikely to benefit from marking unless some time is set aside to enable pupils to consider and respond to marking
- Some forms of marking, including acknowledgement marking, are unlikely to enhance pupil progress. A mantra might be that schools should mark less in terms of the number of pieces of work marked, but mark better.
The only one of these statement that can reasonably be concluded from the flimsy research base the review’s authors unearthed is the one finding that awarding grades seems to undermine the effects of written feedback. All the rest is speculation at best and unexamined, biased assumption at worst.
Let’s consider each claim in turn.
1. “Careless mistakes should be marked differently to errors resulting from misunderstanding.”
This, in and of itself, is probably correct. I find the distinction between ‘errors’ (misconceptions) and ‘mistakes’ (typos & slip-ups) pleasing. Clearly, giving detailed written feedback on something students already know is a waste of time. The problem is how to distinguish between something a students doesn’t know and something a student doesn’t do. I’ve seen reams of work in which capital letters are missing but have encountered almost no students in mainstream secondary schools who do not conceptually understand the use and purpose of a capital letter. The fact they don’t use them isn’t down to ignorance, but habit. They have practised writing without capital letters and have, consequently, become superb at it: they do it effortlessly. The advice offered in the review is that teachers should “simply mark the mistake as incorrect, without giving the right answer.” There’s just no evidence for this. The only way to undo to undo this habit is to make it more onerous for students to continue making the same mistake than not. I’ve found it useful to refuse to mark work which students haven’t proofread: failure to spot mistakes which I know they know need to result in some sort of consequence.
2. “Awarding grades for every piece of work may reduce the impact of marking”
Lots of people are aware of Ruth Butler’s small-scale studies demonstrating the nugatory effects of grading, but these wouldn’t count for much on their own. Much more interesting is the research conducted in Sweden by Klapp et al.
During 12 years (1969 to 1982) Swedish municipalities decided themselves whether or not to grade their students and this natural setting makes it possible to investigate how grading affected students’ subsequent achievement. This natural setting caused some students in the 6th Grade in Sweden to obtain grades while others did not. This circumstance, in combination with the fact that a longitudinal cohort study included a large sample of students both with and without grades offers an opportunity to use a quasi-experimental longitudinal design in order to investigate how grades affect students’ later achievement.
The results are still somewhat equivocal, but it seems pretty clear that although grades might be useful (or even essential) for some purposes, they do seem to undermine many children’s academic performance.
My advice would be that if you really need to grade a piece of work, don’t then undermine your efforts by also writing feedback. Conversely, if you’ve spent time writing feedback, it’s probably not a good idea to also grade the piece of work.
3. “The use of targets to make marking as specific and actionable as possible is likely to increase pupil progress”
As the report says, “Very few studies appear to focus specifically on the impact of writing targets on work.” Unfortunately, instead of simply acknowledging this deficit and moving on, the review’s author decide to extrapolate from research on other forms of feedback to draw their conclusions. In a review on the evidence of written marking this is odd to say the least. It’s definitely the case that findings from other areas of research suggest that further research is desirable, but how can we reasonably conclude anything more beyond suggesting that setting specific targets might be a good idea. Or it might not. This is just guesswork.
4. “Pupils are unlikely to benefit from marking unless some time is set aside to enable pupils to consider and respond to marking”
This is, I think, the most controversial of the review’s assertions. The only evidence which currently exists are students surveys of whether students like responding to feedback. Apparently they do, at least in Higher Education settings. Well, so what? Students like Calypso ice pops, watching The Next Step and Snap Chatting each other inappropriate pictures. What students like is hardly qualification for making education policy. And what HE students like tells us precious little about what school students need. Again, the conclusion drawn by the review ought to have been that it might be a good idea to encourage students to respond to feedback, but equally, it might not.
5. “Some forms of marking, including acknowledgement marking, are unlikely to enhance pupil progress”
Well, maybe. It might be the case that tick’n’flick has little impact on students’ progress, but there’s a possibility that it could provide much-needed motivation. Also, teachers receiving feedback from students may actually be more important students receiving feedback from teachers. This marks a powerful change of perspective. John Hattie says in Visible Learning, “It was only when I discovered that feedback was most powerful when it is from the students to the teachers that I started to understand it better.” When we read students’ work we take feedback from them. We find out something about what they’re thinking. We shouldn’t be deceived into thinking that this is evidence of learning, but we should see it as useful information which gives us some indication about whether our teaching is having the effects we intend. Having taken feedback from our students, we are then in a better position to fine-tune our instruction, give whole class feedback on common errors and misconceptions, and talk to individuals about their work at quiet points in a lesson.
The only really useful findings the report has to offer is that “The quality of existing evidence focused specifically on written marking is low.” Without proper research we’re operating in the dark with guesswork and intuition. It could be that all the reviews recommendations are spot on. It could be the equivalent of encouraging teachers to use bloodletting to balance humours in their patients. We just don’t know enough to make reliable recommendations or draw meaningful conclusions. The authors are right to point this out as both surprising and concerning and the call for further study is welcome. The pages of speculation and guesswork are not.
Richard Farrow sums the situation up here:
This report is a living, breathing, example of why you should NEVER only read the executive summary. But that aside, the report has no evidence about anything useful (to do with written marking) and should never have been published. In fact, it could have been one paragraph saying the following: “we can’t find anything to look at so we are saying to the research community that they MUST research this. In the meantime we will not be publishing a report on this, just in case school leaders take it out of context.”
IF this report is quoted at you in your workplace to make you do something you feel is daft, say the following: “there is no evidence to back up any conclusions you draw from the report. We are still waiting for decent studies on this area of research.”
The post A marked decline? The EEF’s review of the evidence on written marking appeared first on David Didau: The Learning Spy.
*Ideas are the currency of the 21st century*
Churchill Academy and Sixth Form
Brain food for the thinking teacher
Working harder makes you smarter
...our directory of excellence
Developing successful habits of mind, body and pedagogy.
Fascinated by leading and learning
"There is nothing either good or bad, but thinking makes it so."
Finding & sharing teaching 'bright spots'
Education Writer, Researcher & Policy Nerd.
Most Influential Blog on Education in the UK
I want a life that's bigger than me!
Because learning is too important to be left to chance
Zest for Learning... into the rainforest of teaching and school leadership
Teaching in British schools
Reflections on Education: a blog by Chris Hildrew
How to implement a growth mindset culture......