Testing the Water - Pearson’s Response
By Rod Bristow
Education must be an essentially collaborative endeavour if it is to achieve the best outcomes for learners. That's perhaps one reason it attracts such controversy and debate, and it's also why we sponsored this report, on which we are proud to have collaborated with colleagues at LKMco. Testing and assessment are controversial, but often stakeholders seem to be talking at cross purposes and with varying levels of understanding about what is after all a highly technical, but incredibly high-impact activity. Given the undoubted impact of assessment on learning and on teachers, we wanted to provide a platform for a collaborative debate; a debate which would inform our own actions as well as perhaps, the actions of others. The report makes a number of recommendations and suggestions for how assessment in the UK can be improved. Some may be directed more toward some education stakeholders than others, but all are relevant to us at Pearson as one of the UK’s exam boards and a worldwide education company. We take our role in assessment seriously and so we are responding to this report with energy and commitment.
At a system level, we need to reassess the purpose and nature of assessment, to separate out what is there to enable good teaching and learning and what is there to hold schools to account.
Accountability measures are important, but we expect too many different things of individual assessments – a student’s performance in a GCSE exam for example, is expected to communicate different kinds of information to students, parents, teachers, schools, regulators, government, universities and employers. If we‘re to continuously improve the quality of assessments (as we should) each needs designing for a singular purpose. And if we‘re to reduce distracting pressures on teachers (as we surely must) it should be clear how that purpose serves the higher goal of improving teaching and learning.
To meet that higher goal we need much more than clarity of purpose. Teachers should also be provided with better support, training and resources founded on well-researched pedagogy, equipping them with a deeper and more technical understanding of assessment, as well as the tools to plan their curriculum, deliver high quality formative assessments, mark them, and use the insights to inform their teaching.
Our commitment, as part of what we hope will be a collaborative effort, is to do our utmost to help provide that support for deeper understanding. The only question, is how?
Increasing transparency demystifies assessment. At an SSAT meeting of head teachers a few years ago, I was asked why awarding bodies are ‘Kafka-esque’ when it comes to answering questions about assessment. One of the reasons I think, is a fear of being either misunderstood, or mis-represented - or perhaps inducing shock or indignation with the ‘revelation’ that no assessment is 100% reliable, as Dylan Wiliam points out. We’ve reached a point today however, where given the inevitable weight placed on assessment, we need to do better. We want to help demystify assessment by providing greater insight about what makes a good assessment - including how we match up, and how we’ll continuously improve assessment quality. We’ll act in four areas:
1. Improve transparency and training about assessment and its relationship to effective teaching and learning - The Pearson Assessment Charter
We will provide coherence and transparency in our work across formative, summative and high stakes assessments.
In 2018 we will be inviting students, parents, teachers and governors to work with us in the creation of a Pearson Assessment Charter. The Pearson Assessment Charter will lay out the requirements of good assessment - validity, reliability, comparability, manageability and minimising bias. We will be transparent in how our assessments meet these requirements and commit to continuous improvement in the quality and design of our assessments across all our ‘products’ and services.
We’ll offer more training on assessment to teachers and we’ll make training freely available to all those who work for us as markers.
2. Establish an assessment bank with supporting tools for building and using assessments that encourage and enable teaching beyond the requirements of high stakes exams
‘Testing the Water’ calls for teachers to have access to the best in class assessment materials and an understanding of how to design and use assessment effectively as a teaching tool. We have been talking to schools about the provision of a “build your own assessment” service, and will accelerate those plans. This will enable teachers to pick from a bank of assessment items with guidance about how to ensure the assessment provides appropriate differentiation and accessibility; for instance, for lower ability students or for undertaking the different kinds of assessments often necessary for creative subjects. The assessment items in the assessment bank will be as freely available to the teachers that we work with, as we can sustainably make them.
Technology can play an important role in addressing some of the workload challenges which can bedevil the use of good, formative and summative assessment. Where valid and reliable, our digital formative and summative assessments are auto-marked, and we are also investigating the provision of marking services for our printed formative assessments. This will allow teachers to tailor their teaching to support issues identified within assessments, rather than spending large amounts of time creating and marking assessments, not all of which may test the skills required.
We will link the data from formative and summative assessments that enable and encourage learning of individual building blocks of knowledge and empower teachers to teach more than just what might be tested in high stakes exams, so as to minimise ‘teaching to the test’. This will give teachers information about student performance in prerequisite curriculum areas as an integral part of their lesson planning and preparation. As part of our efforts to minimise teaching to the test, we will only give supporting textbooks for our high stakes exams our endorsement if they are authored to teach beyond what might be expected in the exam. Furthermore, to improve the experience that students have of assessment, and with better use of data to inform assessment construction, we will introduce adaptive testing over time.
3. Free Access To Scripts: we will extend indefinitely our popular free access to scripts service and add analytical tools to support better teaching
In the summer 2017 exam series we made every exam script available to teachers, at no cost to schools. Over 400,000 student scripts were accessed from the summer exams series. We want teachers to have a greater understanding of how the mark scheme is applied and used in conjunction with resources, such as the Examiner’s Report.
The next stage for this work will be to bring together the ‘micro’ script access with ‘macro’ data available on nationwide pupil performance on exam scripts and individual questions. We are finding ways to help teachers understand where their students performed well in exams, where they performed less well, and how to apply these insights to future teaching. Data can be a very useful tool for teachers if its purpose is to improve teaching, but teachers should never feel they are recording large volumes of data for its own sake.
We need to support teachers, and provide assessment opportunities; however, we appreciate that formative and summative assessments are different and our work with teachers needs to be clear about their forms and purposes.
4. Publish an Annual Monitoring Review on the reliability of assessments and comparability across subjects
Given the extent of our work globally, we produce and deliver internationally a very wide range of high quality assessments. We have carried out extensive international comparative studies to learn from best practice. We have conducted longitudinal randomised controlled trial research whereby we have observed students interacting with both our learning resources and formative assessments. Doing so has helped us to understand how our services impact student motivation, engagement, learning and, ultimately, outcomes.
The reliability of summative assessments is a key factor in ensuring confidence in the results that are issued. This is ever more important during a period of curriculum reform and the introduction of a new grading scale. Therefore we are carrying out research using knowledge and intelligence from the awarding of GCSE 9-1 Mathematics in summer 2017, for example, to understand comparability across the Foundation and Higher tiers for GCSE Science, French, Spanish and German in summer 2018.
As part of the programme of curriculum reform, we have committed to an annual Assessment Monitoring Review for all components of reformed qualifications. This annual review will use validity and reliability measures to continually improve our assessments over time, whilst ensuring that the tests remain true to the specification and do not represent a surprise for students and their teachers.
We will publish the Annual Monitoring Review, and unpack the meaning of it for teachers and their future teaching. We have already begun to do so following the first assessment of GCSE 9-1 Maths this summer, which has been well received.