‘Four Quarters Marking – a workload solution?

September 2, 2017 by Carl Hendrick

In our new book ‘What Does This Look Like in the Classroom?’ we interviewed Dylan Wiliam on how to implement research on assessment in the classroom.  

 

A central problem in the area of assessment in the classroom has been in the way we often confuse marking and feedback. As Dylan Wiliam points out in our discussion, there is an extraordinary amount of energy expended by teachers on marking and often very little to show for it in the way of student benefit. Although feedback is one of the most effective drivers of learning, one of the more surprising findings is that a lot of it actually has a negative effect on student achievement.

A set of marked books is traditionally seen as an effective proxy for good teaching but there is a lot of evidence to say that this might not always be the case. This problem is on a scale that might surprise a lot of people:

Dylan: I once estimated that, if you price teacher’s time appropriately, in England we spend about two and a half billion pounds a year on feedback and it has almost no effect on student achievement.

Certainly students need to know where they make misconceptions or spelling errors and correcting those is important. Doing so also provides a useful diagnostic for teachers to inform what they will teach next, but the written comments at the end of a piece of work are often both the most time-consuming and also the most ineffective. For example, taking the following typical comments on a GCSE English essay:

  •        Try to phrase your analysis of language using more sophisticated vocabulary and phrasing.
  •        Try to expand on your points with more complex analysis of Macbeth’s character.

This is a good example of certain assessment practices where the feedback mainly focuses on what was deficient about it, which as Douglas Reeve’s notes, is “more like a post-mortem than a medical.” The other thing is that it doesn’t really tell the student what they need to do to improve. What is more useful to the student here? receiving vague comments like these or actually seeing sophisticated vocabulary, phrasing and analysis in action? It’s very difficult to be excellent if you don’t know what excellent looks like.

Often, teachers give both a grade and comments like those above to students, hoping that they somehow improve by the time their next piece of writing comes around a week later and then berate the student when, lo and behold, they make the same mistakes again. Perhaps part of the problem here is that we have very low expectations of what students are willing to do in response to a piece of work and do not afford them the opportunity to engage in the kind of tasks that might really improve their learning.

To address this problem, Dylan advocates a much more streamlined model of marking that is not only more manageable for teachers, but also allows students to have more ownership over the process:

Dylan: I recommend what I call ‘four quarters marking.’ I think that teachers should mark in detail, 25% of what students do, should skim another 25%, students should then self-assess about 25% with teachers monitoring the quality of that and finally, peer assessment should be the other 25%. It’s a sort of balanced diet of different kinds of marking and assessment.

4 corner marking

After producing a piece of work, instead of using abstract skills based success criteria, it is probably more powerful for students to have access to a bank of exemplar essays or worked solutions to see concrete examples of success against which to self-assess their own work. Marking everything in sight and leaving detailed comments is an established cultural norm now but this practice doesn’t appear to be based on good evidence. We know for example that many students will look at a grade and not engage with the feedback but is that feedback always useful anyway?

As we discuss in the book, a common issue we see again and again in using research in the classroom is the ‘Chinese whisper effect’ where by the time evidence works its way down to the level of the classroom, it’s a pale imitation of its original form. This is especially prevalent in the area of marking where convoluted policies such as triple marking are enacted as a means of raising pupil achievement whereas all they are doing is often increasing teacher workload. As Dylan Wiliam reminds us, “feedback should be more work for the recipient than the donor,” but how do you change a culture that has traditionally been the opposite?

Dylan: In terms of what we do about this, I would say first of all, headteachers should lay down clear expectation to parents and say things like, “We are not going to give detailed feedback on more than 25% of what your child does. The reason for that is not because we’re lazy. It’s because there are better uses we could make of that time. We could mark everything your child does, but that would lead to lower quality teaching and then your child will learn less.”  Heads have to establish those cultural norms. If a teacher is marking everything your child does, it’s bad teaching. It is using time in a way that does not have the greatest benefit for students.

As a profession, we are too some extent, we are our own worst enemy. Using marking policies that have little impact on student achievement and a negative impact on teacher workload and morale makes little sense. By adopting an approach like four quarters marking, we might go some way to address this issue and at the same time, give students more ownership over their own learning.

‘What Does This Look Like in the Classroom?’ is out later this month. 

 

 

 

 

Advertisements

Workload – repost

DFE Workload reports…

I am glad that we have already embedded a lot of these proposals so we are in a really good position compared to other schools. However, we are keen to keep improving so any suggestions to me please. Tim Workload

I have put the three reports below (Data, Planning and Marking) with the recommendations below each hyperlink. Leadership will be working through these over the coming weeks and will discuss with staff our response and any actions needed. As always any thoughts on this please let me know. Tim

Eliminating-unnecessary-workload-associated-with-data-management

Recommendations

For EVERYBODY involved in data management:

  • Collect data that are purposeful, valid, and reliable. Use the principles in this report to decide what to collect and how to collect it.
  • Be prepared to stop collecting data if the burden of collection outweighs their use.
  • Do not reward ‘gold plating’. Excessive data collection and processing takes teachers, school leaders, and officials away from more productive tasks.
  • Use data in the format available. Do not ask for or duplicate collection of data collected elsewhere – ‘collect once, use many times.’
  • Take measures to understand the cumulative impact on workload of new initiatives and guidance before rolling them out and make proportionate and pragmatic demands.

For the DfE:

  • Ensure that officials, Regional Schools Commissioners, and system leaders supported by Government (e.g. NLEs) commit to the principles in this report.
  • Implement the common data standards developed by the Information Standards Board and modern data transport options under Data Exchange as quickly as possible.
  • Bring forward the release of both validated and unvalidated data to as early as possible in the cycle so it is available when decisions are taken to prevent unnecessary duplication by schools.
  • Reduce the number of different log-ins schools need to use simply to access and share information with DfE.
  • Consider including data management skills in national qualifications for school leaders.
  • Support the MIS market to develop and diversify, to respond better to school needs.

Ofsted:

  • Continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.
  • Continue to monitor inspection reports to ensure no particular methods of marking are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.
  • Monitor the impact of the revised inspection Framework on the practice of schools.

LAs, MATs and School Leaders:

  • Use software which adheres to common definitions and standards.
  • Conduct a regular audit of in-school data management procedures to ensure they remain robust, valid and effective, and manageable for staff.
  • Do not routinely collect formative assessment data.
  • Summative data should be collected only as frequently as essential to ensure appropriate action can be taken in between collections. Unless there are issues of performance to address and monitor, summative data should not normally be collected more than three times a year per pupil.
  • Review assessment which leads to data generation and consider a range of approaches (including standardised tasks/test items).
  • Make data accessible to all stakeholders in an appropriate form.
  • Do not collect data outside of agreed data collection points. Take a strategic view of the assessment demands throughout the school year and implement an assessment and data management calendar.

Governing Boards:

  • Do not request data in any other format than that which the school regularly and routinely presents.
  • Keep data requirements under review and challenge selves and leaders to collect the least amount of data possible.

ITT providers:

Ensure strategic use of data to inform teaching and learning, and understanding of assessment is part of any initial training.

Teachers:

  • Record data accurately and ensure it is correct first time.
  • If you do not understand why data is being collected, ask. Suggest alternative sources of data or processes if you think better ones exist.

Eliminating-unnecessary-workload-around-planning-and-teaching-resources

Recommendations For Government and its agencies:• DfE and its agencies should commit to sufficient lead-in times for changes for which the sector will have to undertake significant planning to implement. This includes releasing relevant materials in good time.

DfE should review the DfE protocol to ensure it is fit for purpose, and takes full regard of the workload implications of any change.

DfE should commit to using its influence to disseminate the principles and messages of this report through system leaders.

Ofsted should continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.

Ofsted should continue to monitor inspection reports to ensure no particular methods of planning are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.

For school leaders:• SLT should ensure there is ongoing work to develop a shared understanding of effective teaching to inform planning, underpinned by effective continuous professional development.

SLT should not automatically require the same planning format across the school.

SLT should review demands made on teachers in relation to planning to ensure that minimum requirements to be effective are made. Where more intensive plans are needed for pedagogical reasons, a review date is set.

Senior and middle leaders should ensure, as a default expectation that a fully resourced, collaboratively produced, scheme of work is in place for all teachers for the start of each term.

Senior and middle leaders should make clear who will be planning new schemes of work and associated resources, what time they will have available to do so, and how this will be made available to all staff in a timely fashion.

SLT should ensure that the highest quality resources are available, valuing professionally produced resources as much as those created in-house.

SLT should consider aggregating PPA into units of time which allow for substantial planning.

SLT should work with middle and subject leaders to identify alternative ways to evidence ‘effective teaching and planning’, emphasising teacher development.

Subject and phase leaders should lead discussions on quality assurance with SLT/governors to help them understand where a subject- or phase-specific approach may be most appropriate – and why the volume of paper plans may be an inadequate proxy.

Eliminating-unnecessary-workload-around-marking

Recommendations

DfE:

  • DfE should commit to using its influence to disseminate the principles and messages of this report through system leaders.

Ofsted:

  • Ofsted should continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.
  • Ofsted should continue to monitor inspection reports to ensure no particular methods of marking are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.
  • Ofsted should monitor the impact of the revised inspection Framework on the practice of schools.

LAs/MATs/RSCs/Governing Boards and School Leaders:

  • Use the three principles set out in this report to review the school’s marking practice as part of an overall and proportionate assessment policy in partnership with their teachers and governors.
  • Evaluate the time implications of any whole school marking and assessment policy for all teachers to ensure that the school policy does not make unreasonable demands on any particular members of staff.
  • In partnership with their teachers and governing boards, monitor their marking practice as part of their regular monitoring cycle and evaluate its effectiveness on pupil progress.
  • Challenge emerging fads that indirectly impose excessive marking practices on schools.

ITT providers:

  • Draw on research and make trainees aware of emerging findings and evidence.
  • Ensure requirements made of trainee teachers conform to the principles of this report.
  • Include a repertoire of assessment methods in training.

Teachers:

  • Seek to develop a range of assessment techniques to support their pedagogy.
  • Actively review current practice to ensure marking adheres to the three principles in this report.

Researchers:

  • Research current marking methods deployed in schools.
  • Work with schools to evaluate current marking and assessment practices in

schools to promote good practice

DFE Workload reports…

I have put the three reports below (Data, Planning and Marking) with the recommendations below each hyperlink. Leadership will be working through these over the coming weeks and will discuss with staff our response and any actions needed. As always any thoughts on this please let me know. Tim

Eliminating-unnecessary-workload-associated-with-data-management

Recommendations

For EVERYBODY involved in data management:

  • Collect data that are purposeful, valid, and reliable. Use the principles in this report to decide what to collect and how to collect it.
  • Be prepared to stop collecting data if the burden of collection outweighs their use.
  • Do not reward ‘gold plating’. Excessive data collection and processing takes teachers, school leaders, and officials away from more productive tasks.
  • Use data in the format available. Do not ask for or duplicate collection of data collected elsewhere – ‘collect once, use many times.’
  • Take measures to understand the cumulative impact on workload of new initiatives and guidance before rolling them out and make proportionate and pragmatic demands.

For the DfE:

  • Ensure that officials, Regional Schools Commissioners, and system leaders supported by Government (e.g. NLEs) commit to the principles in this report.
  • Implement the common data standards developed by the Information Standards Board and modern data transport options under Data Exchange as quickly as possible.
  • Bring forward the release of both validated and unvalidated data to as early as possible in the cycle so it is available when decisions are taken to prevent unnecessary duplication by schools.
  • Reduce the number of different log-ins schools need to use simply to access and share information with DfE.
  • Consider including data management skills in national qualifications for school leaders.
  • Support the MIS market to develop and diversify, to respond better to school needs.

Ofsted:

  • Continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.
  • Continue to monitor inspection reports to ensure no particular methods of marking are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.
  • Monitor the impact of the revised inspection Framework on the practice of schools.

LAs, MATs and School Leaders:

  • Use software which adheres to common definitions and standards.
  • Conduct a regular audit of in-school data management procedures to ensure they remain robust, valid and effective, and manageable for staff.
  • Do not routinely collect formative assessment data.
  • Summative data should be collected only as frequently as essential to ensure appropriate action can be taken in between collections. Unless there are issues of performance to address and monitor, summative data should not normally be collected more than three times a year per pupil.
  • Review assessment which leads to data generation and consider a range of approaches (including standardised tasks/test items).
  • Make data accessible to all stakeholders in an appropriate form.
  • Do not collect data outside of agreed data collection points. Take a strategic view of the assessment demands throughout the school year and implement an assessment and data management calendar.

Governing Boards:

  • Do not request data in any other format than that which the school regularly and routinely presents.
  • Keep data requirements under review and challenge selves and leaders to collect the least amount of data possible.

ITT providers:

Ensure strategic use of data to inform teaching and learning, and understanding of assessment is part of any initial training.

Teachers:

  • Record data accurately and ensure it is correct first time.
  • If you do not understand why data is being collected, ask. Suggest alternative sources of data or processes if you think better ones exist.

Eliminating-unnecessary-workload-around-planning-and-teaching-resources

Recommendations For Government and its agencies:

DfE and its agencies should commit to sufficient lead-in times for changes for which the sector will have to undertake significant planning to implement. This includes releasing relevant materials in good time.

DfE should review the DfE protocol to ensure it is fit for purpose, and takes full regard of the workload implications of any change.

DfE should commit to using its influence to disseminate the principles and messages of this report through system leaders.

Ofsted should continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.

Ofsted should continue to monitor inspection reports to ensure no particular methods of planning are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.

For school leaders:

SLT should ensure there is ongoing work to develop a shared understanding of effective teaching to inform planning, underpinned by effective continuous professional development.

SLT should not automatically require the same planning format across the school.

SLT should review demands made on teachers in relation to planning to ensure that minimum requirements to be effective are made. Where more intensive plans are needed for pedagogical reasons, a review date is set.

Senior and middle leaders should ensure, as a default expectation that a fully resourced, collaboratively produced, scheme of work is in place for all teachers for the start of each term.

Senior and middle leaders should make clear who will be planning new schemes of work and associated resources, what time they will have available to do so, and how this will be made available to all staff in a timely fashion.

SLT should ensure that the highest quality resources are available, valuing professionally produced resources as much as those created in-house.

SLT should consider aggregating PPA into units of time which allow for substantial planning.

SLT should work with middle and subject leaders to identify alternative ways to evidence ‘effective teaching and planning’, emphasising teacher development.

Subject and phase leaders should lead discussions on quality assurance with SLT/governors to help them understand where a subject- or phase-specific approach may be most appropriate – and why the volume of paper plans may be an inadequate proxy.

Eliminating-unnecessary-workload-around-marking

Recommendations

DfE:

  • DfE should commit to using its influence to disseminate the principles and messages of this report through system leaders.

Ofsted:

  • Ofsted should continue to communicate the clarification paragraphs in the inspection framework through updates and other relevant channels.
  • Ofsted should continue to monitor inspection reports to ensure no particular methods of marking are praised as exemplars and ensure training of inspectors emphasises the commitment in the framework.
  • Ofsted should monitor the impact of the revised inspection Framework on the practice of schools.

LAs/MATs/RSCs/Governing Boards and School Leaders:

  • Use the three principles set out in this report to review the school’s marking practice as part of an overall and proportionate assessment policy in partnership with their teachers and governors.
  • Evaluate the time implications of any whole school marking and assessment policy for all teachers to ensure that the school policy does not make unreasonable demands on any particular members of staff.
  • In partnership with their teachers and governing boards, monitor their marking practice as part of their regular monitoring cycle and evaluate its effectiveness on pupil progress.
  • Challenge emerging fads that indirectly impose excessive marking practices on schools.

ITT providers:

  • Draw on research and make trainees aware of emerging findings and evidence.
  • Ensure requirements made of trainee teachers conform to the principles of this report.
  • Include a repertoire of assessment methods in training.

Teachers:

  • Seek to develop a range of assessment techniques to support their pedagogy.
  • Actively review current practice to ensure marking adheres to the three principles in this report.

Researchers:

  • Research current marking methods deployed in schools.
  • Work with schools to evaluate current marking and assessment practices in

schools to promote good practice.

 

Workload and the Blind Man

The Workload Reports published over the Easter weekend was not so much a missed opportunity as one that never existed.  There is a certain amount of confirmation bias in making this statement as back in November 2015 I blogged Why Workload Working Groups Won’t Work.  The blind man of the title is each one of […]

http://leadinglearner.me/2016/03/30/workload-and-the-blind-man/