212 / 2018-08-16 00:13:34

Aligning School-based English Proficiency Tests with China’s Standards of English Language Ability (CSE): A Case Study and Reflections on the Aligning Process 校本英语水平考试与中国英语能力等级量表的对接研究

School-based English Proficiency Tests,CSE,Standard Setting,Validation,校本英语水平考试,中国英语能力等级量表,标准设定,效度验证

全体主题 > 语言测试与语言能力标准的对接研究

Abstract:
China’s Standards of English Language Ability was put forward in 2018, providing an objective, stable and relatively unified description of English proficiency. As a yardstick, CSE will play an important role in the connection and positioning of various English tests at home and abroad, so the study on aligning various English tests with CSE is necessary, which will become a hot topic in recent years. The relevant aligning projects in the past usually referred to Manual for Relating Language Examination to CEFR, but it’s incipient to apply the methodology in the manual to CSE, which needs to be further explored and verified. In addition, the aligning studies will also contribute to the further revision of CSE, which is also the key point of researches since CSE’s first proposed. Therefore, to position the school-based English proficiency test according to CSE and to provide experience and evidence for the subsequent series of CSE aligning researches, this paper aims to report on a research aligning the reading part of a school-based English proficiency test with CSE, as well as the reflections on the entire aligning process.
This school-based English proficiency test is a high-stake test, which checks students' ability to master and use English and undergraduates of all majors (except English) cannot get degree certificate if not passing the examination. The reading part consists of two essays (followed by 20 multiple-choice questions) and one cloze (choose right answers from 15 options to fill in 10 blanks).
This study was conducted under the guidance of the suggested procedures and approaches set out in Manual for Relating Language Examination to CEFR, including four steps—familiarization, specification, standard setting and validation, and was optimized according to the previous relevant aligning researches. In the standard setting stage, Basket Method and Modified Angoff Method were used respectively to judge the test’s CSE level, each conducted two rounds, and then were further compared and analyzed. Also, this study discussed the influencing factors of panelists’ judgments (panelists’ discussion; the provision of the examinee's actual accuracy on each item), and verified the importance of the organizational competence scale in CSE in the aligning process. In addition, this study paid close attention to the validation part, including procedural, internal and external validation.
Finally, combined with the questionnaire survey and interviews and using some software such as SPSS and Facets for data analysis, the study found that the reading part of this school-based English proficiency test matched Level 5 in CSE, and offered suggestions on CSE’s aligning studies. On the one hand, this study is instructive in English testing, especially in English teaching and learning; on the other hand, it is helpful for the further revision of CSE and provides some evidence for the optimization of aligning methods in subsequent studies.
摘要：
2018年，中国英语能力等级量表（CSE）基于我国国情应运而生，其提供了客观、稳定且相对统一的英语能力描述。作为标尺，CSE将在国内外各类英语考试的衔接定位方面起到重要作用，故各类英语考试与CSE的对接研究势在必行且将成为近几年热门的研究话题。先前的各类对接研究多参考《欧洲语言共同参考框架对接手册》，而将手册中的对接方法应用于CSE的对接是一个全新的话题，且有待进一步验证与探索。另外，CSE作为中国首次提出的英语能力等级量表，对其的进一步修订也是学者们关注的重点，而对接研究过程也将有助于CSE的修订。因此，为依据CSE将校本英语水平考试进行定位，以及为此后系列CSE对接研究提供经验与依据，本文旨在报告一项校本英语水平考试的阅读测试部分与CSE间的对接研究，以及对整个对接过程的思考。
该校本英语水平考试是一项高风险考试，考核学生掌握和运用英语的能力，规定各专业（除英语）学生本科阶段通过该考试才可获得学位证。其阅读部分包括两篇短文（二十道多项选择题），及一篇完形填空（十五个备选项中，选择十项补全全文）。
此次研究主要依据《欧洲语言共同参考框架对接手册》中所设定的四个步骤（即熟悉量表、分析测验内容、标准设定、效度研究）进行对接，并依据先前相关对接研究进行优化。其中，标准设定阶段采用“投篮法”及“改良Angoff法”分别判断了试题的对应等级，此过程各进行两轮，研究进一步对两种方法进行了对比分析。其次，此次研究还探讨了专家讨论及考生实际答题情况的提供对标准设定判断的影响情况，并验证CSE中组构能力量表在对接过程中的重要性。另外，此次研究注重对接的效度验证，包括对接过程效度，内部效度及外部效度的验证。
最终，结合问卷调查和访谈，运用SPSS、Facets等软件进行数据分析，研究发现该校本英语水平考试阅读部分与CSE的等级五相匹配，并且对CSE量表及对接过程提出了相关建议。此项研究一方面在英语测试、尤其是在英语教学和学习方面具有指导性意义，另一方面则有助于CSE量表内容的进一步修正，以及为此后相关系列研究中对接方法的优化提供了依据。