Our efficacy process
In 2013, we made a commitment to put the learner at the heart of Pearson’s strategy and to improve outcomes for learners. We are honoring this commitment by
- identifying the outcomes that matter most to learners and educators,
- designing products based on evidence of what works to improve outcomes,
- evaluating using exploratory and rigorous confirmatory efficacy studies,
- and continuously improving.
One part of our commitment was to be open and transparent by publishing audited research on the impact of our products on learner outcomes. In April 2018, Pearson published our first set of externally audited and independently reviewed reports. We were the first education company in the world to do so. We now publish these reports annually.
Each year we add new product reports and incorporate new evidence into existing reports.
Regularly reporting on the efficacy of our products allows us to
- illustrate the relationship between the use of our existing products and the outcomes that matter most to students and educators,
- share the evidence underpinning the design and development of existing and new products,
- galvanize other learning companies to follow our example.
The research studies we have conducted for our audited efficacy reports (and the efficacy statements produced as a result) are designed based on academic best practices—including practices established by the American Education Research Association and the What Works Clearinghouse.
We know we can’t do improve learner outcomes alone. We partner with educators, researchers, and others to foster a community that is committed to improving outcomes by applying and generating evidence about what works in terms of learning.
At all times, we keep our focus on helping more learners learn more.
The efficacy reporting process
We refer to our approach to efficacy reporting as the Pearson Efficacy Reporting Framework. This framework begins with the determination of learner outcomes and ends with the creation of the efficacy reports.
The four-part process
- Learner outcomes: We define outcomes that matter to learners and determine how we should measure these outcomes.
- Study preparation: We design impact evaluation studies that will provide evidence about how our products are affecting learner outcomes.
- Study, write-up and efficacy statements: Our proposed studies are independently reviewed by the education research organization SRI International. This review step ensures that our studies are rigorous and can stand up to academic scrutiny. We conduct impact evaluation studies that are reviewed by a third-party research organization, which ensures that our findings and statements are accurately backed by evidence. We write-up the study and efficacy statements.
- Efficacy reports: We synthesize all our findings and present them in a variety of formats so that the material is accessible and understandable to a range of different stakeholders.
Our Efficacy Reporting Framework document delivers detailed information about the standards and practices that we hold ourselves accountable to in our impact evaluation research and reporting.
Efficacy statements
Based on our new impact evaluation research and on existing published research that meets the standards of the audit, we generate one or more statements about the use of each product and the impact for learners.
These efficacy statements help our customers and researchers judge the efficacy of using our products. However, they cannot be separated from the context in which the product was used in the underlying research study.
We also recognize that undertaking impact evaluation research and reporting on our own products may raise questions about the validity and reliability of our efficacy statements. Our Efficacy Reporting Framework and the audit are designed to provide assurance about the integrity of our efficacy statements.
The efficacy statements in our Efficacy Reports have been externally audited by PwC.
Ensuring transparency in our process
Our commitment to transparency is unparalleled in the commercial education sector. Because Pearson is dedicated to earning learners’ trust, we chose to have our reports audited and reviewed by third parties.
Pearson enlists professional services firm PwC to carry out third-party audits on all of our efficacy reports, our efficacy statements, and our efficacy reporting framework. Their independent assurance includes tests of controls over the design and execution of research studies, reviews of the data integrity and statistical conclusions, and assessment of the distillation of those studies into efficacy statements.
A team of researchers from SRI Education with expertise in study design and quantitative methodology performed an independent review of Pearson’s research and efficacy statements relating to the use of the product and its impact on learners and learning.
First, the team helped Pearson codify its research quality criteria into tiered levels of evidence to reflect best practices in the field. Criteria for these tiered levels of evidence drew both from the tiered evidence levels in the Every Student Succeeds Act (ESSA) and the 2006 American Educational Research Association's Standards for Reporting on Empirical Social Science Research.
Then the team assessed whether Pearson’s analyses, technical reporting, and associated efficacy statements complied with these research quality criteria. This year’s reviews applied Pearson’s comparative levels of evidence, which draw from ESSA’s “promising” level of evidence, and correlational level of evidence, which adopt the standards for studies that do not include a comparison group.
Outcomes-focused, evidence-based product design
When it comes to product development, we take a “backwards design” approach: we start with the outcome we are trying to improve and then apply evidence from the learning sciences to the design and development of the product, including designing appropriate pedagogical models to support its integration into teaching and learning.
Teaching & Learning Research and Design
Outcomes aren’t achieved by accident. They are designed and informed by learning science and research.