English
Log in
(0)

Case study

Rutgers University Programme - America

The TOEFL ITP Assessment Series helps Rutgers University programme in American Language Studies earn a reputation as objective, systematic, data driven and research based

When the Rutgers University programme in American Language Studies (PALS) underwent a change in directorship four years ago, Assistant Director Peter Chang saw an opportunity to enhance the programme’s objectivity and increase student confidence in its assessments of their English ability. Rather than update the programme’s 1983 in-house admissions instrument, he told the new director that PALS should become a consumer of research and adopt the TOEFL ITP® Assessment Series.

“Developing something in-house requires extensive research to be valid,” Chang noted. “Why not go with an exam that already has research — that has made these connections for us in terms of the Common European Framework of Reference for Languages (CEFR)?”

He also stressed to his colleague that the TOEFL ITP test’s objective, high-quality descriptions of proficiency levels would be invaluable for students and teaching — especially when used in concert with those of the TOEFL iBT® test, which is the learning target most PALS students work toward. “Those were the merits I saw in using the TOEFL ITP test,” Chang said.

Looking back, Chang said PALS has benefitted from the change in exactly the ways he imagined. “On the student end, it became very easy to talk about their proficiency or their levels using the TOEFL ITP test,” he said. “Before that, we always had contentions with students. A lot of times, students felt that their levels were inaccurate. With a TOEFL ITP score, we never have arguments anymore.”

“… the TOEFL ITP test made our programme more systematic, more data driven … The TOEFL ITP® test has had a very transformative effect for our programme. We’re now based on research. We’re now based on analysis. The TOEFL® placement test sets the tone for the programme.“ Peter Chang, Assistant Director, American Language Studies, Rutgers University. “In terms of teaching, the TOEFL ITP test made our programme more systematic, more data driven,” he added. “When we’re looking at students’ abilities and levels, we no longer make subjective statements. We look at the numbers. The numbers give us a sense of objectivity. The TOEFL ITP test allows us a more analytical perspective. It develops the ability of our faculty to analyse.”

PALS has also benefited in other ways. Since most PALS students advance between the levels of English ability measured by the TOEFL ITP and TOEFL iBT tests, Chang and his faculty incorporated TOEFL iBT* rubrics into the curriculum. While teachers primarily use these and TOEFL ITP performance data to focus and drive student learning, analysing the tools and data has fostered deep faculty understanding of English-proficiency levels. This change helps colleagues across departments understand their students’ sometimes-puzzling English abilities. “We now can talk to departments about how students are developing or where they need to be,” he said.

"The TOEFL ITP test has had a very transformative effect for our programme,” Chang continued. “We’re now based on research. We’re now based on analysis. The TOEFL® placement test sets the tone for the programme.”

Here’s a closer look at how TOEFL tests and tools have transformed this English-language programme.

TOEFL ITP performance data, combined with other assessment information, create a robust initial portrait of students’ English proficiency for admissions and placement

According to Chang, PALS gathers a range of assessment information during admissions, which the faculty analyses collectively to get a robust understanding of each student’s current English proficiency. An initial Skype™ interview informally assesses potential students’ conversational skills. Then, when applicants arrive onsite to take the TOEFL ITP test, teachers videotape more formal interviews using a set of questions that vary in level. Students also write brief essays on the same day. Since most PALS students aspire to use the level of English measured by the TOEFL iBT test, instructors assess the essays and interviews with the TOEFL iBT writing and speaking rubrics.

Teachers work as a team to interpret the profiles and place incoming students in courses. It is conversations such as these that have helped hone the analytical expertise of PALS faculty, Chang said. Like TOEFL ITP scores, PALS classes are aligned with the Common European Framework of Reference for Languages (CEFR), so this serves as an initial guide to course placement. However, when students fall at the boundaries of score ranges, the other measures of proficiency help staff decide which aspects of students’ profiles should carry more weight.

Deep understanding of the English proficiency levels measured by the TOEFL ITP and TOEFL iBT tests help PALS faculty explain student ability across departments

Using the proficiency-level descriptions, rubrics and performance reports from both the TOEFL ITP and TOEFL iBT tests throughout the English-language programme has helped PALS faculty develop a finely grained understanding of developing English skill, which they, in turn, use to help colleagues from other departments understand their students’ unique abilities. The process has had enormous educational value across departments, Chang said.

“In our conversations with various departments, how to understand the TOEFL tests has become pivotal in our discussions,” he explained. For example, when an instructor questions why a student who has attained a particular score on a test still struggles with certain assignments, PALS faculty point to differences between overall scores and subscores and “we show them what type of readings students see on each of the tests,” Chang said.

“The TOEFL ITP exam has allowed us to talk more with the departments to educate them about the various shades of TOEFL tests — [including] what is the TOEFL [test] and what does the TOEFL [test] look like?”

TOEFL ITP tests help prepare students for college-level reading and critical analysis

For PALS students seeking entry into the Rutgers undergraduate programmes, the university accepts a TOEFL ITP score of 570 or higher as a measure of English ability that is sufficient for conditional enrollment, Chang said. Students who are admitted by this route undertake a year of pre-college courses, including one called English for Academic Discourse (EAD). “The class is heavy on reading, but also requires them to write and prepares them for English 101,” he said.

The TOEFL ITP test “introduces the idea that reading is an analytic process,” Chang added and PALS students who achieve this score are able to transfer that understanding to this course.

“At that particular score point, the students going into EAD courses are able to adjust better. We worked with students at lower levels. When they go into the classes, they cannot get used to the readings. But for anyone who gets 570, the transition is very smooth,” he said.

“It’s the complexity of the reading — the analytical nature of the reading — and of some of the questions that are asked about the reading on the TOEFL ITP test, like what does this infer [and] what is the author’s attitude?” that make the difference, he explained. “Those are very basic things that are being discussed in EAD courses. When students aren’t used to thinking in this way, they’re not able to perform. It’s kind of an interesting link between the design of the TOEFL ITP questions and transitioning into actual classroom usage.”

TOEFL ITP score reports help students better understand their current abilities

When students see their TOEFL ITP performance reports, they are often surprised to find that the scores and explanations help them understand their ability in reading, listening and grammar, Chang said. “These three areas are very easily understood by them. Their scores allow them to understand exactly where they are. They may have questions — what does the next level look like or what does advanced look like? — but it allows us to talk about where they are now in very concrete terms."

“My students just naturally digest it,” he continued. “I never have to argue with them about what their level is. And in my job — where I have to advise students on their courses, on what to take for the next step and how to develop their language skills — not arguing, but simply looking at data and being objective about something is a gift.”

The TOEFL ITP tests ease the burden of the accreditation process

Use of the TOEFL ITP Assessment Series recently helped PALS complete a programme-level, five-year accreditation cycle, Chang said. PALS’s enhanced research basis, together with the TOEFL ITP test’s alignment with the CEFR, helped make accreditation a very clear process.

“The evaluators felt that the programme was organised. It was systematic. We were able to provide rationales for any placements. However they looked at it, it was linked to data,” he said. As a result, “we’re hoping we’ll get our 10-year accreditation."

TOEFL ITP performance data can signal a need for programmatic changes

PALS administrators are watching incoming students’ TOEFL ITP performance data to determine whether to make any substantial changes to the programme. “We see different patterns,” Chang said. “For example, we see what our recruitment limitations are. As much as we try to find betterprepared students to join our programme, it’s very rare for us to receive a student who scores above 550 points on the TOEFL ITP test.”

While about 10 percent of applicants score between 500 and 550, he noted, the average score of higher-level PALS cohorts is usually around 450. “Otherwise, usually we’re in the 420s to 430s.”

The number of students coming in with lower levels of English proficiency may, in time, point to a need to build up the curriculum at that level, Chang said, but “we haven’t gone about doing that yet.”

However, because the data shows students are increasingly unable to achieve the minimum TOEFL ITP score of 480 required for admission into PALS’ TOEFL iBT preparation course, he recently added a foundations-level TOEFL iBT preparation course to the programme. The goal of the new course is to better prepare students for the work required in the higher-level class.

TOEFL ITP Assessment Series certificates matter to students

As someone whose own path to higher education mirrors that of his students, Chang understands why TOEFL ITP scores and certificates mean so much to them. “I, too, come from a country that doesn’t speak English as its primary language. I was not educated primarily in English,” he said. “My students who go overseas understand that they can take this TOEFL ITP score and it means something somewhere else, especially when it’s from ETS. Coming from overseas, I understand the climate there and the employment requirements. The certificate is something employers look for. I’m very certain that when they go into the market, they will find that certificate very useful."

Programme Snapshot

  • Website: http://pals.rutgers.edu/
  • One- to three-year, pre-college programme at the Rutgers Livingston campus (average: one year)
  • Serves 150–200 students per year; most students have been in the United States 10 years or less
  • Primarily provides English-language instruction to nonmatriculated international (95%) and domestic (5%) students working toward admission to undergraduate or graduate school
  • Teaching focus is academic English-language skills • Provides some language support to conditionally admitted Rutgers students
  • International students hail largely from Asia (75%) and South America (25%)
  • Conducts a Skype interview to assess conversational skill prior to admission
  • Administers TOEFL ITP tests onsite for admission (minimum score: 400–450)
  • Uses TOEFL ITP performance data, a brief recorded onsite interview and an essay written onsite for placement