The final piece (number 3 but search the blog for the others) about how to use summative assessment in a knowledge based curriculum by Robert Peal:
Planning a knowledge-based scheme of work. Part 3: Summative Assessment
Like many teachers, I have spent the last week marking end of year exams for Key Stage 3. Having put some thought into the design of these exams, I have – perhaps for the first time – found this to be an instructive and, dare I say it, enjoyable process.
For the sake of this blog post, I am going to focus on our Year 8 exam, covering Early Modern Britain and the Age of Encounters. All our KS3 assessments share a similar format, and you can view them here, with examples from Year 7, Year 8 and Year 9.
In the past, I have struggled to find a satisfactory format for end of year exams, falling back on the unimaginative (and unhelpful) practice of mirroring GCSE examinations. Reading Daisy Christodoulou’s Making Good Progress, and talking to colleagues at the Historical Association Conference in May, helped me narrow my focus. At WLFS, the construct we want to assess in KS3 history boils down to three outcomes (four in the case of Year 9). Do pupils have:
- an accurate chronological framework of the period studied?
- a broad knowledge of the period studied?
- the ability to construct well-evidenced historical arguments?
- the ability to comment on the purpose and usefulness of historical sources? (Year 9)
Our end of year exams now mirror those outcomes. At Year 7 and 8, the exam consists of three sections:
- Section 1: Chronology test /5 marks.
- Section 2: Multiple choice quiz /20 marks
- Section 3: Essay /25 marks
Section 1: Chronology test
The chronology test for Year 8 involved linking 10 events with 10 dates. The events were chosen from a list of 25 dates included in the pupils’ revision guide, spanning from 1453 to 1721. We didn’t expect pupils to memorise all of the dates listed. But if they had a good understanding of the historical narrative, and knew some of the most important dates (such as 1588 and 1688), then they would – we hoped – be able to piece together the correct answer.
Pupils gained half a mark per correct answer. As a test item, the chronology test tended towards bifurcation: in all, 46% scored 5 out of 5, but with another a large percentage clumped towards the bottom end. Next year, we need to do more to ensure a strong chronological understanding amongst all our pupils. Perhaps our pupils should memorise all 25 dates?
Section 2: Multiple choice quiz
This quizzing portion of the exam has been designed to assess the whole domain of the Year 8 curriculum, in a way that the essay question could not.
In Making Good Progress, Daisy recommends using multiple choice questions for formative assessment. Though a good idea in principle, I have found MCQs too time-consuming to create, and too cumbersome to mark, on an ongoing basis. However, for our summative end of year exam, the investment in creating and MCQs was time well spent.
Once pupils completed their exams, our department entered all of the pupil answers into a question-level analysis spreadsheet (see here), so that we could see which questions pupils struggled with, and which questions pupils breezed through. Daisy suggests this is useful for highlighting pupil misconceptions, which it was. But I did wonder whether the varying success rates for different questions was more dependent on the design of the question, rather than the quality of pupil understanding.
For example, this was the most challenging question for our pupils.
4. Which Catholic martyr did Henry VIII execute for refusing to give up his religion?
a. Thomas More
b. Thomas Wolsey
c. Thomas Cromwell
d. Thomas Cranmer
Success rate: 39%
The low success rate clearly has a lot to do with the proximity of the distractors: parents at the end of the fifteenth-century really liked the name ‘Thomas’.
Question 4 tested an item of declarative knowledge, but the next most challenging question for our Year 8 pupils probed their understanding on a more conceptual level, in the way that Christodoulou argues MCQs are well equipped to do.
20. How was the power of Georgian Kings further limited by ‘Parliamentary government’?
a. The king was not allowed to be a Catholic
b. The king could only choose ministers who had the support of Parliament
c. The king could not start wars without Parliament’s permission
d. Parliament had the power to appoint and dismiss the king
Success rate: 44%
The question hinged on the word ‘further’, and required pupils to discriminate between the outcomes of the Glorious Revolution in 1688, and the outcomes of the development of Parliamentary Government under George I. At the other end of the scale, the question pupils found easiest did surprise me.
17. What title was Oliver Cromwell given to rule England in 1653?
a. Lord Protector
c. Prime Minister
d. Lord Chancellor
Success rate: 98%
I thought I was on to something, as many pupils had written about Cromwell as ‘King’ during the year. But by the time of the exam, not a single pupil chose that distractor. Three did choose ‘Lord Chancellor’. Again, the question with the second highest success rate was not one I thought particularly easy when writing it:
18. What did the Bill of Rights do?
a. secured the legal rights of Parliament and limited the monarch’s power
b. banned the monarchy, and establishing England as a Commonwealth
c. gave equal political rights to all people in England
d. united England and Scotland into a single Kingdom
Success rate: 94%
But, our analysis shows this question was simply too easy, and the distractors too dissimilar. Perhaps the most helpful outcome of this question level analysis has been to hone in on which questions worked well, and which did not – allowing us to refine the writing of the MCQs in years to come.
Section 3: Essay
Lastly, we set a mini-essay, with clear instructions that pupils were to write three paragraphs: two sides of an argument and a conclusion. With around half an hour to complete the essay, this seemed like a reasonable demand.
Throughout the year, our Year 8 pupils wrote five essays. They were on Henry VIII and the Reformation; The Age of Encounters; the Later Tudors; the English Civil War; and the late Stuarts/early Georgians. We chose two essays questions from these five units, based on the same enquiry as the earlier essay question. We were not interested in tripping up pupils with fiendishly difficult questions. Rather, we wanted straightforward essay questions that gave pupils the best chance of marshalling their knowledge to support a reasoned historical argument. The two questions pupils had the choice of answering were:
- ‘The invention of the Printing Press was the most important event that took place in Early Modern Europe.’ To what extent do you agree with this statement?
- ‘Charles I only had himself to blame for his execution in 1649’. To what extent do you agree with this statement?
To mark the essays, individual teachers grouped them according to whether they were A* to E in quality. We then met as a department, saw how consistent the judgements were, made some adjustments, and assigned a numerical mark out of 25 to each script. It was a low tech version of comparative judgement, which seemed to work pretty well.
The correlation between pupil outcomes in the multiple choice questions, with pupil outcomes in the essay, was 0.7. Most helpfully, this highlights for our department those pupils who understanding what we study, but still struggle with written work.
Next year, we will use a selection of this year’s essays as exemplification material for each grade band, replacing the need for a mark scheme.
So that you can see how WLFS pupils are getting on with a knowledge-based curriculum, here are the Year 8 exemplification essays we will use. Each grade band contains three exemplar essays. Having been written under timed conditions, during exam week, on an unknown question, the quality of did take a dip compared with the essays pupils have written throughout the year. However, I was still pleased with the way in which pupils were able to organise their knowledge into convincing historical arguments.
There is still much to work on (particularly on the explicit teaching of different lines of argument – more to follow), but I am happy that our KS3 curriculum is now equipping pupils with a deep well of powerful knowledge to inform their historical thinking.