Validating a Simulated Test of CET 4
Title
Validating a Simulated Test of CET 4
Validating a Simulated Test of CET 4
Keywords: Messick’s validation framework: Reliability; Validity; Test practice
Authors
Yang Miao
Shantou University Medical College, China
Yang Miao
Shantou University Medical College, China
Abstract
A simulated test of CET4 (College English Test, Band 4) was validated to check if it served the specific purposes of predicting and diagnosing. The study data came from a CET 4 simulated test sat by a class of sophomores who were to take a CET 4 test one month later. Based on Messick’s framework of validation, the test’s content coverage and representativeness were checked, and correlation analyses including inter-consistency reliability, item correlation, factor analysis and item analysis were computed. The analysis results show that the test is of modest reliability and validity, with the most serious problem in the reading section, which had too many misfit items and failed to effectively test the candidates’ discourse reading ability. The contextual difficulties or inadequacy of efforts in other aspects of the validation framework implied a very unsatisfactory situation of simulated test practice in the Chinese context. It is stated that for a simulated test to effectively fulfill its purposes of predicting and diagnosing, trial tests and post hoc analysis are essential, and empirical investigations into the process of test taking, the effects of coaching and practice and the motivation problems should be advocated, and effective remedial support should be provided afterwards to ensure the positive washback of such a test.
A simulated test of CET4 (College English Test, Band 4) was validated to check if it served the specific purposes of predicting and diagnosing. The study data came from a CET 4 simulated test sat by a class of sophomores who were to take a CET 4 test one month later. Based on Messick’s framework of validation, the test’s content coverage and representativeness were checked, and correlation analyses including inter-consistency reliability, item correlation, factor analysis and item analysis were computed. The analysis results show that the test is of modest reliability and validity, with the most serious problem in the reading section, which had too many misfit items and failed to effectively test the candidates’ discourse reading ability. The contextual difficulties or inadequacy of efforts in other aspects of the validation framework implied a very unsatisfactory situation of simulated test practice in the Chinese context. It is stated that for a simulated test to effectively fulfill its purposes of predicting and diagnosing, trial tests and post hoc analysis are essential, and empirical investigations into the process of test taking, the effects of coaching and practice and the motivation problems should be advocated, and effective remedial support should be provided afterwards to ensure the positive washback of such a test.
[private]
[/private]
Category: Monthly Editions, Volume 12
