Research Blog

Latest posts

  • How Technology Can Boost Student Engagement

    Kids interacting with a table touch screen

    Gallup’s 2013 State of America’s Schools reported that 55% of US K-12 students are “engaged” in the learning process, while 28% are “not engaged,” and 17% are “actively disengaged.”

    Technology may be one of the keys to increasing the number of engaged students in America’s classrooms. In our multi-phase Teaching in a Digital Age study, we are working with many partners to research digital teaching strategies and how they positively affect student learning. One of these positive effects reported by educators is the increased intensity of student engagement that occurs when technology is integrated into the classroom.

    Technology as a tool helps teachers create and present content and instruction that is interesting and relevant to students. When learning is relevant to students, then they become engaged, active learners. How does this happen?

    With increased access to learning resources, tools and information, students are drawn deeper into a topic than ever before. They can even direct their own learning. In fact, when done well, students don’t just learn with technology- they create. One educator noted:

    “When students have this technology, they can create things. They can innovate things…. When they have Photoshop in front of them and I say do this, this, and this, what they can create is always going to be completely, uniquely different. And, they become artists with that or they become filmmakers, or they become web designers. Like they can take on a lot of really advanced roles, and I think that’s something that technology does uniquely provide, because you can’t be a web designer without that technology. You can’t create a film without that technology. And, I feel like that’s really different than a textbook…let me let you take your creativity, and using this technology, create something I would have never made.”

    Educators in Meridian, Idaho noted the misconception that students are only engaged individually with technology. Their classrooms don’t look like separate students glued to a screen. Instead, educators can direct students to engage collaboratively with the use of technology. With technology, collaboration among students is easier and broader. It also opens doors to widen the audience and purpose of student work, giving meaning to the schoolwork.

    And, with increased student engagement, comes increased learning. There is a strong research base that describes how technology strengthens student engagement and learning. For example, active learning is associated with improved student academic performance (Hake, 1998; Knight & Wood, 2005; Michael, 2006; Freeman, et al., 2007; Chaplin, 2009), and increased student engagement, critical thinking, and better attitudes toward learning (O’Dowd & Aguilar-Roca, 2009). Read more in my paper Teaching in a Digital Age.

    If technology supports teachers’ efforts to focus on effective practices that engage students, then we have another tool to engage that half of US students who aren’t currently engaged.

    read more
  • Part 2: How learning progressions can help with both monitoring progress and progress monitoring

    Girl in Chemistry class

    In my previous blog, I talked about the difference between progress monitoring and monitoring progress. Today, I share my ideas of how learning progressions can inform both.

    The key to monitoring progress is understanding what students know and don’t know at any given time. Learning progressions use research on how students learn to clearly define the learning pathway and conceptual milestones along that pathway. For example, my fourth grader’s teacher could compare his work to learning progressions so that she understands more clearly what he knows, and what she can do to move him most efficiently from his “check-minuses” to “check-plusses”.

    In progress monitoring, teachers use data on a regular basis to understand students’ learning rates, but it is up to the teacher to formulate an instructional response. If the CBM slope is flat, the instructional next steps may not be entirely clear. If CBMs were linked to learning progressions, it could enhance progress monitoring by making clear how students are approaching problems and what misconceptions are preventing their progress.

    The National Association of State Directors of Special Education (NASDSE) states that effective progress monitoring (NRCLD, 2006, p. 22):

    1. Assesses the specific skills represented in state and local academic standards.
    2. Assesses marker variables that have been demonstrated to lead to the ultimate instructional targets.
    3. Is sensitive to small increments of growth over time.
    4. Is administered efficiently over short periods.
    5. Is administered repeatedly (using multiple forms).
    6. Results in data that can be summarized in teacher-friendly data displays.
    7. Is comparable across students.
    8. Is applicable for monitoring an individual student’s progress over time.
    9. Is relevant to the development of instructional strategies and use of appropriate curriculum that address the area of need.

    These characteristics are quite similar to some of the features of learning progressions:

    • Many learning progressions have been linked to standards, such as the Common Core State Standards and the Next Generation Science Standards (#1).
    • What the NRCLD refers to as “marker variables” are commonly referred to in learning progressions as “levels of achievement,” or the conceptual milestones that students pass through as they are learning in a particular domain (#2).
    • The sensitivity to small increments of growth over time is related to the grain size of a learning progression; to be useful for formative assessments learning progressions usually need to have a relatively fine grain size (#3).
    • Formative assessments based on learning progressions should also be administered efficiently and repeatedly, and should be useful for monitoring students’ progress over time (#4 ,#5, and #8).
    • Because learning progressions are based on the scientific literature describing how typical students learn, assessments based on learning progressions should be comparable for most students, although it is necessary to collect empirical evidence that particular subgroups of students follow the same learning pathways (#7).
    • One of the most promising aspects of learning progressions is the potential for providing teachers with instructionally actionable information in the form of “teacher-friendly” student and classroom performance reports and instructional tools and resources that are aligned to the learning progression (#6 and #9). We are engaging in research to learn about the inferences that teachers make from learning progression-based assessment reports. Stay tuned to learn more about these efforts as the year unfolds.

    Can learning progressions live up to their promise and really help educators monitor progress and conduct progress monitoring? It is still too early to tell, but there is some encouraging research showing that with ample training and support, teachers can use learning progressions as a framework for their formative assessment and instruction and by doing so, they come to better understand their students’ learning pathways.

    read more
  • Part 1: Monitoring Progress, Progress Monitoring and Learning Progressions–What IS the difference?

    Boy in Chemistry class

    What do these three terms have in common? Progress, of course. Educators and parents across the globe all want to enable their students to make progress. When my fourth grader’s teacher sends home a weekly folder with his work samples and tests, a “check” or a “check-plus” tells me that he gets it, or he pretty much gets it, and a “check-minus” gives me the impression that he has more work to do, but I don’t know what pathway he needs to take to move from the “check-minus” to the “check-plus”, and what is the best way to get him there.

    Currently, educators frequently measure what students know and what they don’t know, but this “mastery measurement” does not provide information on students’ progress or learning pace so that they can ultimately meet the standards we set for them. Monitoring is an integral part of ensuring that students make progress, but what is the difference between monitoring progress and progress monitoring? They sound like they’re the same, don’t they? And how do learning progressions fit in? In previous posts I defined and described learning progressions and why the Research & Innovation Network thinks they have promise. In today’s post (Part 1) I will distinguish monitoring progress from progress monitoring. In Part 2, I’ll share ideas of how I think learning progressions can inform both.

    Monitoring progress is a core instructional practice that includes formative assessment, questioning, providing feedback, and similar strategies. All teachers monitor their students’ progress throughout the year, using a variety of strategies, but these strategies are not standardized and vary greatly in quantity and quality. Formative assessment plays an important role in monitoring progress, but some teachers are more comfortable with formative assessment than others, and all teachers could use tools and resources that would make conducting formative assessment easier.
    Monitoring progress is a core instructional practice that includes formative assessment, questioning, providing feedback, and similar strategies.

    Progress monitoring is a term used to describe a formal part of Response to Intervention (RTI); it is a scientifically based practice used to assess students’ academic performance and evaluate the effectiveness of instruction. It was originally designed for use in individualized special education, but is now seen as a useful approach for many different types of students (Safer & Fleischman, 2005). Teachers are trained to use student performance data to continually evaluate the effectiveness of their instruction. Students’ current levels of performance are determined and measured on a regular basis. Progress toward meeting goals is measured by comparing expected and actual rates of learning, and teachers are prompted to adjust their instruction based on these measurements.

    Curriculum-based measurement (CBM) is one type of progress monitoring. A CBM test assesses all of the skills covered in a curriculum over the course of a school year. Each weekly test is an alternate form (with different test items but of equivalent difficulty) so that scores can be compared over the school year. Students’ scores are graphed over time to show their progress (see examples here); scores are expected to rise as students are learning and are exposed to the curriculum. The rate of weekly improvement is quantified as the slope of the line, which teachers can compare to normative data. If scores are flat, it signals the need for additional intervention.

    How can learning progressions help with both monitoring progress and progress monitoring? Stay tuned for ideas in my next blog.

    read more
  • What does Proficient mean?

    by Katie McClarty

    Man and girl playing chess

    The end of the school year is a time for field trips, class parties, and final report cards. The iconic report card lets parents know how their student did that year and typically reflects attendance, participation, and performance in class. Parents generally understand how to interpret report card grades: A (great), C (average), or F (failing).

    The end of the year also is the time when many parents receive their child’s standardized test scores. These results, however, are not as easy to interpret. For example, in Massachusetts, a student who earns a score of 250 on the state test is considered proficient. In Washington, it takes a score of 400. Each state has its own assessments, and each defines proficiency differently.

    Now, however, nearly all the states have agreed to adopt the Common Core State Standards as an outline of what students should be taught in mathematics and English language arts. Educators will use instructional materials appropriate for teaching students the knowledge, skills, and practices laid out in these documents. That should produce less variability in instruction state to state and district to district.

    In order to monitor how well students are learning this material, most of the states also have agreed to use one of two Common Core assessments that are being developed. That will make it possible for states to report results on a common scale: a 400 in English in Tennessee, for example, would be the same as a 400 in Florida. But the question remains: is 400 good enough?

    To answer that question, states set performance standards. Typically, this is done by educators and other experts who get together and look at assessments and agree on which questions or tasks a proficient (or advanced or in need of improvement) student should be expected to answer or complete. That information is then translated into a specific score. The same process can be used to analyze the quality of examples of student work.

    More recently, it’s become possible to answer the question of what is good enough more precisely, based not just on expert judgment but also on data. If by proficiency we mean that a student has learned enough in one grade to be ready to do well in the next one, we can test that definition by tracking how students actually perform. We can look at how well a group of students performs on a 5th grade math test and then look back at how those same kids had done on the 4th grade math test. Using statistics, we can then more accurately define what it means to be proficient in the 4th grade.

    This process is called Evidence Based Standard Setting, and because scores can be linked to future performance, it can give parents confidence that if their child is proficient, he or she has not only mastered an important set of knowledge and skills, but also is likely to be successful in the next grade. It can even give students and their parents a sense of whether they’re on track to do well after high school in college or in demanding career training programs. The scores can also help identify students who need extra help before it becomes too late, and parents, using this information, can advocate on their children’s behalf to make sure they receive that help.

    The familiar report card is but one source of information about how well students are doing in school. Test results linked to important future outcomes can provide another critical piece of information to teachers, parents, and students.

    read more
  • Rigor and Readiness: Measuring the Impact of Algebra II

    by Katie McClarty

    Students in a lecture

    There has been a lot of discussion lately about the role of advanced high-school mathematics courses — in particular, Algebra II — in promoting college and career readiness. On one side of the debate, the champions of Algebra II cite research demonstrating that completing the course leads to success in higher education and to higher earnings (Adelman, 2006; Carnevale & Desrochers, 2003). Achieve has been one of the leading advocates for including advanced mathematics in required high school curricula, suggesting there are not only practical advantages (e.g., prerequisites for future study), but also benefits to students’ general academic development. Skills acquired through Algebra II (including but not limited to logical thinking, cognitive capacity, and complex problem solving) can support success in areas far beyond a day-to-day work environment.

    This isn’t to say the debate is settled. A recent report from the National Center for Education and the Economy (NCEE) found that the skills most important for succeeding in community college math courses were those introduced in middle school. By analyzing textbooks, assignments, and tests at seven community colleges, the researchers concluded that few students need to master advanced algebra to be successful. The NCEE report comes at a time when several states (e.g., Florida, Texas) are changing graduation requirements to make Algebra II optional, provide more flexible pathways toward high school graduation, and create space in students’ schedules for more vocational training.

    Isolating the causal effect of taking Algebra II on future outcomes is a serious challenge, thanks to selection bias. It is likely that students who choose to take Algebra II in high school are higher performing and more motivated than many of their peers and thus more likely to attend and do well in college. In other words, it’s something about the type of students that take Algebra II, rather than completing the course itself, that leads to better student outcomes.

    In a recent research study, my co-authors and I set about tackling this thorny issue — separating selection effects from Algebra II’s true causal effects. We will be presenting our work next week at the Association for Institutional Research Annual Forum in Long Beach, CA. We used national datasets spanning multiple decades and sophisticated econometric techniques to isolate cause-and-effect relationships between completing Algebra II in high school and subsequent college and career outcomes.

    The verdict? Algebra II seems to matter more for college outcomes (including community colleges, technical colleges, and four-year institutions) than for career outcomes. Compared to their counterparts who didn’t finish Algebra II, those who did were more likely to be admitted to selective colleges, maintain higher college GPAs, stay in school, and graduate. Conversely, for students who did not apply to college after high school, completing Algebra II was not related to finding a job immediately after high school, initial occupational prestige, earnings, or career advancement.

    This research indicates that students not planning to attend any college (two-year or four-year) may not benefit substantially from finishing Algebra II. That said, it’s important to highlight one caveat: Algebra II does not seem to negatively impact any career outcomes. In that respect, completing the course will keep doors open to college for the many students who do not solidify their postsecondary plans before enrolling in high school courses or starting their mathematics sequence. Some of our other interesting findings from this study will be the topic of future blog posts.

    read more