Assessing Reading Comprehension in the Real World

Yet testing a child’s reading ability remains a linear experience. We ask them to read a paragraph, maybe two, and answer multiple-choice questions on a printed page. That’s hardly the way people read, learn and process to make decisions today.

A team of ETS researchers led by John Sabatini and Tenaha O’Reilly believe there is a better way to see whether a student possesses the skill to comprehend what they read — no matter the format. Their work is innovating how we measure a student’s reading proficiency, while challenging the very idea of the role a test can play.

Goal-oriented testing

“In the real world, when you go to buy a cell phone, maybe [choosing] between an Android^® phone or an iPhone, you have a goal,” and that goal serves as motivation, O’Reilly says. You want to learn as much as you can before buying a cell phone so you don’t make a wrong choice, such as spending too much money or buying a phone that won’t link to your current devices.

“You go on the Internet to compare prices and features, read reviews, and then make a decision,” he says. “That is a different thing than going to a test and answering detailed questions.”

O’Reilly, Sabatini and their research team have spent the past five years studying how students across 28 states absorb and digest what they read, administering over 100,000 pilot tests during that time as part of their research. The ETS team is one of six teams funded by the federal Reading for Understanding (RfU) research initiative, meant to bring reading intervention and assessment into the 21st century.

Some teams are looking at how students learn to read, others are looking at ways to support struggling readers in the digital age. The ETS research team is studying how to measure how well students read and understand information they see in their everyday lives — not just what appears on a test.

Their findings? That assessments can be transformed into an opportunity for learning and discovery, rather than a snapshot and a score.

Stress reliever

The best learning happens when students are allowed to make mistakes. Rather than worrying about being penalized for errors, they actually learn from, and build on, them instead. Built into the pilot tests the ETS research team has developed for RfU are supports, which involve modeling good habits of mind and allowing students to fix mistakes during the test, O’Reilly says.

For example, when students choose an answer, then learn more information, they will be asked whether, given these new details, they would like to revise their answer. Or students may be asked to write a summary — which can be difficult for some children. Those struggling on this task might encounter a virtual peer — someone who can act as a guide for them during the assessment. Students would write their own summary, then look at the virtual student’s work. The virtual peer then is sequencing, or showing the students what needs to be done, with an example of how the completed work should look.

“In the real world you collaborate and work with people,” Sabatini says. “We’ve added simulated students who come in at different points to help you out. This reduces stress and makes the testing experience more social.”

Sabatini and O’Reilly believe these changes in the way assessments are constructed could change not only a final score, but the way educators approach testing in the first place.

“Everyone makes errors,” says O’Reilly. “But how we recover from those errors really matters.”