Tens of millions of children and young adults are missing out on their education due to conflict, threat of attack or the after-effects of natural disaster, some for weeks and months, still more for years at a time. When world leaders came together at the UN in September 2015, this challenge was top of mind as discussion focused on the Syrian refugee crisis and the need for both immediate action; and also with the launch of the 17 Global Goals, long term sustainable solutions to the world's biggest humanitarian challenges.
At Pearson, we’ve chosen to work with Save the Children to pilot models of sustainable, quality schooling for children in conflict zones, but we also want to address the ongoing education crisis that can be less immediately apparent than that brought about by war - 59m primary-school-age children out of school and nearly 800m illiterate people across the world. For those learners who are in school, there are many other trenchant challenges that plague education systems in sections of the developing world: lack of teachers, poor teacher development, insufficient materials, out of date resources...the list goes on. As we focus on Global Goal 4 - to ensure inclusive and quality education for all and promote lifelong learning - we are looking at ways to ensure every learner has access to a high-quality, affordable education.
One of the ways we are looking to do this is through the Pearson Affordable Learning Fund (PALF), which invests in entrepreneurs who are helping to meet the demand for high-quality, low-cost education in the developing world. In PALF’s first annual letter, learn more about the impact and reach of our ten portfolio companies as they set out to improve the quality of education for people everywhere.
For most of the past century we have bundled a very complex set of disparate skills into a single role we call the ‘classroom teacher’. Teachers must have deep content knowledge to understand the scope and sequence of a curriculum, and pedagogical expertise to plan effective lessons and evaluate student comprehension and mastery. We also ask them to be charismatic presenters, a coach/mentor to provide support and motivation for students to persevere, and project managers able to keep track of each students academic progress.
It is incredibly difficult, and perhaps unrealistic, to expect to find such a diverse skill-set in a single individual. As a result the past few years has seen various attempts to “unbundle” the teacher. While much is made of the developed world’s experiments with unbundling, most notably flipped classrooms and MOOCs, some of the most interesting innovations are occurring in the developing world where the dual constraints of limited financial resources and a weak labor pool make the need for new solutions all the more pressing.
The Pearson Affordable Learning Fund has invested in some exceptional entrepreneurs that are tackling this challenge head-on.
For example, at SPARK, a school chain in South Africa, a highly trained teacher is in charge of the whole group and guided practice portions of the typical learning cycle, while the independent practice portion of the learning is done primarily with the aid of “e-learning labs.” Here students work to reinforce and extend classroom instruction with personalized computer programs overseen by a more junior assistant.
This allows the extremely valuable time of the master teacher to be dedicated to the more complex tasks of implementing best-in-class instructional methods and overseeing classroom management. As a result, the cost of delivering high-quality education is substantially lower, while quality is maintained.
Another example of the same trend is provided by Bridge International Academies, who dedicate the bulk of their six-week teacher training program to focusing on techniques for classroom management, student engagement, and checking for understanding, while a team of world-class educators based in Boston and Nairobi write a rigorous, student-focused lesson script which the teachers read on an e-book during class.
Visiting a Bridge classroom you will see students being pushed to perform more challenging cognitive tasks (for instance, instead of simply writing down a list of map symbols they will be using these symbols to draw a map of their own neighborhood) with teachers circulating the classroom carefully checking students work. Both of which are rarely found in a typical classroom in Kenya.
My prediction is that 2016 will see much more piloting, experimenting and testing of these new models. Some will be taken to scale, most obviously through new public-private partnerships that are able to see the value in moving away from the old model of a single, jack of all trades, teacher. This division of labour will allow expertise to be deployed where it is most needed, and where it can best be found - and the impact on learning will become increasingly visible.
Transforming anything needs bold people to kickstart it. And if you’re in any doubt that education needs some bold transformation, then you don’t have to search too far for the evidence. For example, the 101 million children not in school in Sub-Saharan Africa, or the 93 million women there classed as illiterate. Or how about the 14% of US adults who cannot read. In its latest report the OECD stated that “no country, no region in the world can claim in 2015 that all of its youth have attained at least a minimum proficiency in foundation skills.”
We can all play our part in kickstarting a transformation in education, because we all own the culture of education. Teachers, parents, students, governments, businesses… we all define the culture that sets the standards. The ‘right’ culture is one of the key explanations for the dominance of South-East Asian nations at the top of most education league tables; that believes that every learner can succeed rather than deciding at the outset that some are smart and others not; that as well as ambitious expectations has clear goalposts, high levels of community involvement, and a strong sense of accountability among all stakeholders. It’s what Hwy-Chang Moon, a dean of Seoul National University, calls “a mentality of the first-tier.”
Our education kickstarters are the people social scientists would refer to as the ‘innovators’, those on the left of the bell curve of adoption.
They are the minority that, if all goes well, morph into the majority. And to their far right, the laggards, the chorus of cynics saying “What’s all this nonsense of new ideas and technology!”. They will try to derail you, mud wrestle you into distractions by asking how you’re going to get ‘buy-in’ and ‘take people with you’. But you don’t win hearts and minds and then make the change; you make the change, and the hearts and minds will follow.
In an increasingly globalised world, a bold vision doesn’t just have to stand up to where you’re coming from, but where everyone else in the world is at. I was reminded of this on one of my visits to Punjab, Pakistan, where I have been working with the Chief Minister for a number of years (unrelated to my role at Pearson.) A government official was very proud to tell me how only 5% of kids cheated in exams, which was a huge improvement. I reminded him that in England the figure is 0.014%, 400 times better.
Obama called it “the audacity of hope”, but hope alone is not enough. Transformation is much more forensic than that. It needs a plan that spells out that this is where we are now, and this is where we can get to, and this is who’s going to need to do what, when, and how. And this is how we’ll know if it’s working.
Data will let you do that last one. And the closer to real-time that data is, the better. The world moves far too fast for data to have a shelf life. In Punjab we went from having no insight into what was happening in schools, to now collecting data against 16 indicators from 55,000 schools every month. Additionally we collect data from 25,000 schools in Khyber Pakhtunkhwa, the Province of Pakistan on the border with Afghanistan, prone to terrible winters, earthquakes, and terrorism. If you can do it there, you can do it anywhere.
The data will indicate where the plan is veering off, so you can decide how to get it back on track. Dig deeper, disaggregate it, find the hotspots where it’s working, where it isn’t, do more of the former and stop doing the latter.
When you marry all this together - the bold vision, the clear plan, the execution of that plan, and the real-time data that tells you how to adjust - that’s when you do real transformation. It’s how you move from small steps of incremental change to giant leaps of extraordinary outcomes.
What might an education transformation look like when it’s done? (Of course, it’s never ‘done’!) A school where technology is ubiquitous, classrooms have become wide open spaces, data is helping identify struggling students, progress is measured by grit and resilience as much as English and maths, and Artificial Intelligence diagnoses when a learner is bored, or frustrated, or confused before performing, with the help of an outstanding teacher, a well-designed intervention.
None of this is to say that education transformation isn’t already happening. Some countries are on the path - Poland and Singapore to name just two; and individuals too, such as, CP Viswanath, founder of Karadi Path. From small beginnings in the slums of India the company now has 800 English language schools across the country. Led by where the evidence has taken them, their students outperform ‘traditional’ classrooms by between 20-60%.
Last week I was in Davos at the World Economic Forum, where the major challenges facing the world - extreme poverty, civil rights, clean energy, gender equality, affordable healthcare were examined. We don’t have a hope of transforming any of these if we don’t first transform education.
Michael Barber is our chief education advisor. Connect with him on Twitter - @MichaelBarber9
This article is a response to a piece we published on this blog by Dr Ben Williamson of Stirling University, in which he explores an “emerging criticism of Pearson among education researchers”. In his piece Dr Williamson refers to two activities of Pearson: the Centre for Digital Data, Analytics and Adaptive Learning and The Learning Curve. Dr. Kristen DiCerbo and Dr. John Behrens of the Centre jointly author the response on the former; and Dominic Collard responds on the points made in relation to The Learning Curve.
We would like to thank Dr. Williamson for his interest in Pearson’s use of data and our research efforts. We believe strongly that open dialogue is key to the world making progress in education.
Dr. Kristen DiCerbo - (Centre for Digital Data, Analytics and Adaptive Learning.)
As one of the questions Dr. Williamson asks is “Who at Pearson is collecting the data, designing the algorithms to analyse it, and checking the analytics for their accuracy?”, I’d thought that’d be a good place to begin. I have written this with my colleague John Behrens, who is referenced with me in Dr. Williamson’s article.
One of our great delights in our years at Pearson is how much the company values our varied backgrounds as social science researcher-practitioners and supports our involvement in public and academic discourse. My Ph.D. is in educational psychology; after completing a school psychology program and becoming a certified psychologist, I worked in schools as a school psychologist and continued on to a research career that included observing classroom instruction around the world. John was a social worker before obtaining degrees in special education and educational psychology with cognates in instruction and cognition, as well as measurement, statistics & methodological studies. He was a professor for 10 years and gave up tenure to work in contexts in which he could apply learning and analytics at scale.
So when Dr. Williamson writes that Pearson is, “beginning to challenge the existing authority of social scientists and psychologists to study, understand and produce new knowledge about key aspects of education such as assessment and learning.” we find this a surprising conclusion, since we in fact identify as social scientists and psychologists (and technologists and educators).
Like Dr. Williamson, we believe the digital revolution is a remarkable event in the evolution of human interaction. We study this phenomenon and participate in academic communities to reflect, discuss, and have broad interchanges about these societal changes. We, and other colleagues at Pearson, publish papers, present in open scientific forums, and provide ongoing community service through external advisory boards, editorial boards, and support of journals with peer review, among other activities. We support graduate student training with internships and mentoring and have had a broad range of collaborations with academics in fields related to our interests. This provides future scholars with unique access to both the processes and challenges of research and innovation in business environments.
We are, however, not just researchers, but practitioners as well. A common concern for researchers and sponsors of research is the lack of mechanism for translating learning science research into practice. An embedded research group is one way to make that happen. We feel privileged to work side by side with product design and engineering teams to help build the most efficacious products and services we can for our customers. This means not just studying corporations and other loci of innovation and development, but working within them. We believe that we have a responsibility as stewards of educational data to conduct research to further our understanding of both learning and data methodology. There is no reason this activity should be the sole province of academia or organizations based on their tax status. Education is a complex endeavor and as Dr. Williamson points out, requires many actors and perspectives.
Our theory of action
So, what exactly is Pearson trying to accomplish with the funding of data and learning science research? Dr. Williamson asks, “why is Pearson investing in such a massive effort to conduct educational data science?” Our answer is simple: we want to serve students, parents, teachers, and administrators in the best possible way, by considering all the tools that can be fruitfully brought to bear.
Like our goal, our theory of action is simple: Better data analysis → better understanding of students’ attributes/curriculum/learning trajectories → better instructional decisions → improved learner outcomes.
By using better data analysis techniques applied to data captured from better designed activities, we hope to build more complete and accurate models of learners’ knowledge, skills, and attributes that will provide better information to teachers and learners and provide systems that are relevant to each student’s individual proficiency levels, interests, and current states. As we discussed in Impacts of the Digital Ocean on Education (DiCerbo & Behrens, 2014), our starting point on this journey is not that we should make the natural activity of society more digital, but rather that, as it is already happening organically, the educational community needs to understand the opportunities and challenges that emerge. If students are working in digital systems throughout the year, we think it essential to give them feedback along the way, and irresponsible to ignore the opportunity. Indeed it is our hope that increased awareness about learner progress throughout the year can change the balance of need for the much-maligned annual test. We are proud to work at a company that emphasizes learner outcomes (see our efficacy efforts for more on this) and whose results can be accepted or rejected by the consumer.
Our belief system
Dr. Williamson states that one of his main concerns is that our work is, “premised on a kind of big data belief system which assumes that massive quantities of data can reveal truthful and meaningful patterns about the reality they’re taken from—that the data can speak for themselves free of human bias.” While this is a common characterization of modern analytics writ large, a simple review of our writing suggests a different stance. Way back in 1997 John wrote (with Mary Lee Smith in the Handbook of Educational Psychology) that data analysis must be understood in the “context of history, the context of application, the context of practice and the context of alternative methods” (p. 945). More recently he advised the Learning Analytics community that “The successful learning analyst will avoid two common errors: Failure to understand the context and failure to become intimately familiar with the data.” (Learning Analytics & Knowledge Conference, 2013).
In the Impacts of the Digital Ocean on Education paper, the following figure is one of our favorites:
In the paper, we write, “Data is only a representation or symbol of what happens in the world. In most contexts, the goal of data collection and analysis is to provide insight and inform decisions. Accordingly, there is a long chain of reasoning that needs to be considered.” We recognize that data is a representation of the world and like all representations, it is an imperfect system which will not perfectly capture the detail of the world. We also believe that all of the activity coming after that (analysis, interpretation, etc.) is a human endeavor, involving all the benefits and challenges that implies. This view of data analysis as human process that requires understanding of meaningfulness of context and social negotiation is, in fact, a consistent theme over our careers as reflected in such works as Why People Are the Real Power Behind Big Data, Technological Implications for Assessment Ecosystems, and Activity Theory and Assessment Theory in the Design and Understanding of the Packet Tracer Ecosystem. Finally, interested readers can read more about how to avoid being “fooled by data” in our writings on exploratory data analysis (here, here, and here, for example).
We hope that Dr. Williamson is correct that we are well-positioned to create new knowledge and methods. Pearson is a dynamic and evolving company working in a dynamic and evolving set of social, technological, political and economic contexts. We are energized by the opportunity to serve the global community of learners and educators, and to work at the intersection of academic exploration and end-user service.
Dr. Williamson asks about what our work looks like “from the inside.” Given our experiences across a variety of research settings, we think he would be surprised to see how much the work we do looks just like work done in education research labs everywhere, with the added component that we are directly implementing our findings to impact the lives of learners. Just as with anyone else interested in what we do, we would be delighted to take him through our work in more detail.
The Learning Curve (Dominic Collard)
At Pearson, we believe that data helps unlock the secrets of learning. That alongside the know-how and experience of teachers and educators, data can reveal things that are invisible to the human eye and the human brain, and so help us all make better decisions.
It’s a belief that requires data to be not just robust, but also seen and used. The professional researcher may be comfortable navigating through labyrinths of numbers, but most of the rest of us are not. Teachers, parents, government officials… anyone interested in what is working well in education - most of us probably don’t have the time, the skills or the inclination to get really deep into the data.
The Learning Curve - essentially a collection of thousands of education data points collected from all over the world over the last 25 years - is one attempt to make data seen and used by more people. We want people to discover their own conclusions and draw their own correlations between education inputs (ie spend, teacher salaries, class sizes) and education and socioeconomic outcomes (ie literacy levels, graduation rates, crime and unemployment.)
None of the data on The Learning Curve ‘belongs’ to Pearson. The Economist Intelligence Unit gathers it all for us, from sources such as the OECD, UNESCO, The World Bank and the International Labour Organisation, to name just a few. Dr Williamson is correct that the EIU is an independent business within The Economist Group, which until recently Pearson had a stake. And it is equally true that few other organisations could manage the systematic and regular collection of the wide range of data that The Learning Curve demands.
All that data is then presented via a range of interactive visualisations, designed so the user is able to control the parameters of what they are seeing. For instance, you may like to know how the US and the UK compared in 2001 for public expenditure per pupil as a % of GDP. Or you may like to play that comparison out for all countries across 25 years. At a few touches of a button you can do both, and everything inbetween. Or, if you are confident using large spreadsheets of data, then we also give you the option of downloading everything to an Excel file. The Learning Curve has been specifically designed so nobody has to second guess what the user wants to understand, or the method they want to discover it.
There is another section of the site - the Index - which I suspect Dr Williamson is referring to when he argues it “limits what kinds of analyses can be done and what can be said about the data because it has been designed to prioritize the measurement and comparison of ‘effective’ education…”. The Index is an attempt to rank countries based on their overall education performance - a global league table of education standards. We think the way we have calculated where countries come stands up to scrutiny (and we provide a full explanation of the methodology on the site so people can judge for themselves) - but we also know that you could legitimately calculate this in many, many other ways. We have never suggested the Index should be seen as the final say, and have always gone to great lengths to explain that it is just one interpretation, whose value reduces the more you read it in isolation. Pearson would absolutely agree with Dr Williamson on the importance of understanding “...social and cultural context, emotional complexity, and the qualitative dimensions of human relations” in education systems. The truth is though, for now these things are much harder to measure and collect data on. That’s why we see The Learning Curve as the start of the conversation, not the end.
There is one more point I would like to make about The Learning Curve, that I appreciate is not brought up by Dr Williamson. It is free. As long as you have an internet connection and a device to access it, you can spend as long as you like exploring what it has to reveal; ¾ million people worldwide have done so.
The Learning Curve is not a modest undertaking for Pearson - in terms of cost or time - and there is no immediate revenue incentive for us either. Of course, we hope that it helps our reputation and so our ability to take part in the conversations that shape education. And, yes, that should then help our commercial performance in the long-run. But the absolute opposite will be the case if The Learning Curve somehow doesn’t stand up; if somehow we are using it to steer people away from the evidence and towards something we’d wish they’d believe, if only it were true.
Like my colleagues Kristen and John, I’d be delighted to spend time with Dr Williamson to show him behind the scenes of The Learning Curve , and of course get his view on where we might be able to improve things.
Earlier this month we came across an article in the European Educational Research Journal analysing Pearson's role in education research. In the spirit of open dialogue, we invited the author, Dr Ben Williamson of Stirling University in the UK, to summarise his points, which he does in the following article. You can also read our response to this piece.
Pearson has recently become the subject of several major research studies. These studies have sounded a largely critical note about Pearson, particularly around its business ambitions and its political influences. One of the reasons for the emerging criticism of Pearson among education researchers, I believe, is that Pearson is beginning to challenge the existing authority of social scientists and psychologists to study, understand and produce new knowledge about key aspects of education such as assessment and learning.
I recently published a research article in the European Educational Research Journal on what I described as Pearson’s ‘digital methods.’ The research tried to identify some of the many research methods that Pearson is using to make sense of education, and specifically looked into the statistical methods and the data visualization techniques behind Pearson’s The Learning Curve, and the data science methods used by Pearson’s Centre for Digital Data, Analytics and Adaptive Learning.
My argument was that Pearson is becoming a methodological gatekeeper with the capacity to carry out new forms of educational research using large-scale datasets, big data and data science methods. These are approaches that many educational researchers working in higher education institutions are ill-equipped to carry out, which puts Pearson at an advantage as more and more digital data is produced about learning and assessment. As a result, a research centre like Pearson’s Centre for Digital Data, Analytics and Adaptive Learning looks from the outside like a seriously-resourced laboratory for educational research and knowledge production that challenges the existing methods, knowledge and theories of educational sociology, philosophy and psychology.
For example, John Behrens, the director of the Centre for Digital Data, Analytics and Adaptive Learning has claimed that data-mining ‘the billions of bits of digital data generated by students’ interactions with online lessons as well as everyday digital activities’ will challenge current theoretical frameworks in education, as ‘new forms of data and experience will create a theory gap between the dramatic increase in data-based results and the theory base to integrate them.’ In a report co-authored with Kristen DiCerbo (also of Pearson), it is noted that ‘we need further research that brings together learning science and data science to create the new knowledge, processes, and systems this vision requires.’
The ambition to devise new data science methods together with learning science approaches, and then to use these to identify a ‘theory gap’ could cause disquiet among some education researchers. Of course, it’s intellectually healthy to challenge old theories, otherwise we would still be trying to construct behaviourist ‘teaching machines’ like those of Sidney Pressey a century ago. But for a big company like Pearson to position itself in a way which suggests it has the capacity to address the theory gap using its massive data analytic capacity could be seen as a little troubling. Here are two reasons.
First, Pearson promotes The Learning Curve as an ‘open and living database’ that will encourage ‘evidence-informed education policy’ and help ‘identify the common elements of effective education.’ What is less clear to the user is that The Learning Curve was constructed by the Economist Intelligence Unit (until recently owned by Pearson) whose expertise is in economic forecasting, business intelligence and national comparison. Although The Learning Curve invites the user to engage with the data through an interactive visual interface, ultimately it limits what kinds of analyses can be done and what can be said about the data because it has been designed to prioritize the measurement and comparison of ‘effective’ education according to the methodological preferences of the EIU. What Pearson says is ‘effective education,’ or rather what the EIU measures as ‘effective education,’ or indeed, what data can be included about ‘effective education’ in The Learning Curve in the first place, all point towards its limitations as an impartial, neutral and objective visual and numerical representation of education around the world. The methodological appendix to The Learning Curve even admits as much, stating that ‘because indexes aggregate different data sets on different scales from different sources, building them invariably requires making a number of subjective decisions.’ There is subjectivity to the objectivity offered by The Learning Curve.
For me as an education researcher with a sociological tendency, this makes me ask questions about the ‘who’ behind the data—who selected it, from where, what did they do to prepare it for inclusion, how did they clean it up, how has it been tweaked, how has it been presented, and, crucially, how much interpretation has been done by the designers of The Learning Curve in advance of its presentation on the site?
Second, Pearson’s Centre for Digital Data, Analytics and Adaptive Learning is premised on a kind of big data belief system which assumes that massive quantities of data can reveal truthful and meaningful patterns about the reality they’re taken from—that the data can speak for themselves free of human bias. Yet as many researchers of big data have pointed out, data do not exist naturally as a ‘raw’ or truthful representation of an underlying reality—they have to be brought into being through human, social, methodological and technical practices, and are constantly reshaped as they move between human actors, software platforms, and institutional structures and settings, all framed by social, political and economic contexts. Again, human hands, minds and biases, as well as technical platforms and business plans, can all affect the ways in which data are collected, calculated, and communicated back out into the world.
These examples are significant because Pearson is claiming to be opening up a ‘theory gap’ in our understanding of effective education and learning, and at the same time working on new digital methods and data scientific approaches that might produce new knowledge to fill that gap. As a global educational media company and increasingly a policy influencer, it is then very well positioned to use the insights it gains from the data to come up with new kinds of solutions in the shape of new software products for schools, or even new policy solutions for governments.
You can see why some critically-minded education researchers would be sceptical—Pearson’s identifying problems for which it might sell solutions! Others might point out that numerical data (no matter how big) and its visualization as heatmaps, time series graphs and so on are only part of the educational picture—that they don’t capture social and cultural context, emotional complexity, and the qualitative dimensions of human relations in classrooms.
My own critique is different. Instead, my emphasis is on acknowledging the human and social practices that go into the generation of data at Pearson as a new source of knowledge production, and on asking questions about how its new digital methods and data scientific approaches might be challenging the long history of educational theorizing, empirical investigation, and knowledge production. Pearson is positioning itself as a major source of methodological expertise in educational research, driven by ambitions to reconceptualise education and learning, and it has significant global power to influence policymakers, politicians and practitioners alike that its data provides the numerical and visualized facts that can fill the theory gap.
There is an exciting line of sociological inquiry into the ‘social life of methods’ to draw from here which treats research methods as the object of social scientific inquiry. Those of us trying to understand Pearson from the outside know little about the ‘social life’ of the methodological work being done inside Pearson’s research centres.
The necessary response, I think, is for education researchers to try to understand the ‘who,’ the ‘how’ and the ‘why’ of Pearson’s current digital ambitions. Who at Pearson is collecting the data, designing the algorithms to analyse it, and checking the analytics for their accuracy—and according to whose policy ambitions, business plans and personal objectives? How are the datasets that Pearson possesses selected, interpreted and presented, and how is the visualization of its data on platforms like The Learning Curve designed in such a way as to shape the possible interpretations that audiences can make? And why is Pearson investing in such a massive effort to conduct educational data science—to identify new market niches for itself, to displace higher education institutions, and to position itself as the dominant global centre of educational expertise and knowledge production?
Answering these questions may require researchers with a more critical set of methodologies and theories to engage in a dialogue with researchers within Pearson, and to understand Pearson from the inside as a new source of methodological expertise and knowledge production rather than criticising it from the outside as a commercial monster. There is an empirical gap in our understanding of how Pearson is approaching the theory gap in educational research.