News
Relationship between candidate profile and test results
Controlling the Quality of the TOEIC Test by Using the Relationship Between Examinees’ Background and Their Test Performance
As many testing programmes, the TOEIC® test collects examinees’ background during the registration and administration of the test, mostly because researches show an existing relationship between examinees’ background and their test results.
Researchers suggested that exploring the statistical relationship between examinees’ background information and their scores should be part of the quality control procedure for a testing programme. Previous analysis study found that examinees’ background information has potential for predicting test scores, even though this relationship has not been fully identified yet. Using data collected from the TOEIC® Listening and Reading test, ETS researchers were therefore able to control the quality of the TOEIC® test through the relationship between examinees’ background and their test performance.
Collecting Examinees’ Background Data
A background questionnaire is used to collect information on examinees’ education and work-related background, English-language experience and test taking experience. The study, conducted by ETS, was solely based on the data of examinees who had taken the TOEIC Listening and Reading test for the first time, as the inclusion of repeaters’ data might have violated the assumption of independence of observations. Overall, the data of about 1,5 million examinees’ test scores over six years was used for this research.
Results and Conclusion
In this study, the stronger the relationship between the examinees’ background information and their test performances, the more useful the background questionnaire seems to be. For the TOEIC® programme used in this study, the predictive power found is very strong while the prediction error is very small therefore the prediction model is effectively validated. Consequently, the examinees’ background information will be very useful for the operational work (the work that is ongoing, repetitive, meant to sustain the business). The predicted scale score means based on the background information can not only be used to predict and understand test performance before and after test scoring.
To conclude, the results from this study is very promising and positive. The strong prediction models for scale score means based on examinees’ background information have significant potential in the quality control of the TOEIC Listening and Reading test. However, there are some limitations in this study and future research may address these limitations.