205 / 2018-08-15 19:18:00

Validating the cognitive aspect of the listening subscale in CSE: An integrated approach of cognitive diagnostic modelling and standard setting

validating, cognitive-based, CSE, listening

全体主题 > 语言能力标准在教学和测试中的应用

One identifiable limitation of CEFR is its failure to take the cognitive aspect of language ability into consideration (Alderson 2007; He & Chen 2017), which is important in language comprehension and production. Developers of China’s Standards of English (CSE) have realized the problem and therefore created a scale unveiling the cognitive features at different proficiency levels. Taking listening subscale as an example, this paper attempts to explore the cognitive embodiment in CSE by combining standard setting and cognitive diagnostic modelling (CDM) approaches. Two research questions are addressed in this study: 1) To what extent do the cognitive features at different CSE levels match the actual cognitive status of target students? 2) How do the cognitive features embedded in CSE descriptors benefit panelist members in the standard setting process? Thirteen panelists participated in standard setting and four domain experts got involved in CDM attributes-defining stage. The test aligned is a large-scale in-house exit test administered in a prestigious university in China. Results show that the cognitive performances described at different CSE levels basically conform with the cognitive ability diagnosis of target groups. More proficient students are more likely to fulfill diverse cognitive activities. Except “inferencing”, the mastery probability of cognitive attributes surges greatly from level 3-5 but keeps relatively stable from level 5-7. Moreover, the cognitive features embedded helped panelist members to make decisions in standard setting, with first-round judgment unconventionally homogeneous, revealed by interview data and FACETS analyses. Serving as a good exemplar of cognitive-based scale, with which score users can realize what cognitive activities students can do, CSE is helpful indeed for the practice of language teaching, learning and assessment.