LearnED is a place to learn about learning—because great learning can lead to great opportunities, and great opportunities can lead you and your family wherever you want to go.

What You Should Know About ‘Efficacy’ in Learning, Starting With: What the Heck Does It Mean?

Teacher with two school kids

“It’s our work to understand how learning tools help learners achieve positive outcomes.”

 

Efficacy In a Nutshell

It’s one thing to design a super cool learning tool.

It’s another thing entirely to design a super cool learning tool that actually makes a (positive) difference in a student’s learning.

That, in a nutshell, is EFFICACY.

It’s a fancy word tossed around in learning circles.

And it’s something that Carmen Arroyo knows a lot about.

The Work of Efficacy

Carmen is Vice President of Impact Evaluation at Pearson and she applies efficacy research to learning products around the world.

She explains the company’s commitment to efficacy in this way: “It’s our work to understand how learning tools help learners achieve positive outcomes.”

With her colleagues, Carmen studies things like skills competencies, conceptual understanding, grades in school, and standardized test scores.

Carmen Arroyo

Carmen Arroyo

“The whole discipline is rooted in the scientific method,” she says. “We apply the most rigorous standards of research when we do our work with learning tools.”

“Then, we look at whether it’s effective in an imperfect world,” she says.

“Learning is not just about what happens in a perfect setting—it’s about a school’s resources, a teacher’s experience level and approach to teaching, and the characteristics of a student,” she says.

Put it all together, and you have the work of a Ph.D. in efficacy.

“I decided I wanted to do something that combined my love of research rigor and my desire to not just interact but help other people,” Carmen says. So she made the switch to education.

A Case Study

Carmen’s team recently completed a study in Brazil, where she and her colleagues mapped schools that used a full suite of educational and academic resources designed by Pearson (called Sistemas SoC) against those that didn’t.

They found—across all measures of achievement—that students in the Sistemas schools scored higher than students in non-Sistemas schools.

“For studies like this to work,” she says, “we ensure students in each group have similar characteristics and environments.”

“It’s how we know the differences in learning outcomes can be attributed to the learning tools and not to something else,” she says.

An Early Love

Carmen fell in love with research as a teenager.

She worked for three summers in a biochemistry lab at the Yale University Medical School.

It was part of a program to encourage high school students to become researchers.

“We were working on cancer drugs,” she says, “but the whole experience was so non-social.”

“I decided I wanted to do something that combined my love of research rigor and my desire to not just interact—but help other people,” Carmen says.

So she made the switch to education.

A Pioneer and Mentor

Carmen laughs at the suggestion that there might not be many people in the world who have a deep passion for both science and social interaction.

“My graduate advisor helped lead me down that path,” she says.

Dr. Edward Zigler was a professor of psychology and director of the Child Study Center at Yale University.

His research was instrumental in the formation of Head Start, a wide range of school readiness programs for young children from low-income families.

It was a pioneering method to build learning curricula that considered a child’s community as much as his ability in the classroom.

“He showed me how to apply the sciences to vexing social problems,” Carmen says.

“Today, this means building learning tools that help prepare many low achieving community college students for college-level courses,” she says. “Through efficacy work, we can build effective tools that teach basic college skills.”

“It’s more than understanding a learning tool,” Carmen says. “We really need to know how it works and for whom.”

Researching Community Colleges

Carmen and her team are also looking at skills disparities among community college students.

Some of the students were using an online learning tool called “MyWritingLab.” It’s designed to accommodate different learning styles with personalized, adaptive feedback.

“Students who completed MyWritingLab tutorials increased their final exam scores by 14 percentage points,” Carmen says. “That can mean the difference between passing and failing.”

Positive Outcomes

The work of efficacy research is complex.

Carmen and her colleagues conduct interviews with teachers. They pore over survey and test data. They sit down with students in focus groups. They ask hard questions.

“It’s more than understanding a learning tool,” Carmen says. “We really need to know how it works and for whom.”

“We’re so committed to doing this right,” she says, “so learners can benefit from positive outcomes.”