Current Research

The CI-UIUC research team has successfully developed an automatic test assembly (ATA) design and a web-based computerized adaptive testing (CAT) system for the HSK test. In the future, the research team will incorporate cognitive diagnosis into the web-based CAT system to facilitate language learning.

Research topics 

Adaptive Item Selection in Computerized Adaptive Testing
CAT tailors the test to each individual examinee. The item selection algorithm automatically selects the next item based on the examinee’s responses to previous items. In plain language, the algorithm mimics what a wise examiner would do: if the examinee answers an item correctly, the algorithm gives a harder item; if the examinee answers an item incorrectly, it gives an easier item. A plethora of item selection methods for CAT have been developed. Examples of the widely used item selection methods are the Maximum Fisher Information method, the Maximum Kullback-Liebler Information method, and the α-Stratified method. Many methods have also been developed to satisfy multiple non-statistical constraints (e.g., content balancing, word count, and answer key balancing), such as the Weighted Deviation method, the Maximum Priority Index, and the Shadow Test method. Item exposure can also be controlled by various item selection algorithms. The research team at CI-UIUC can engineer all different methods in the CAT system for the Chinese language proficiency tests.

Item Bank Management
CAT relies on an item bank from which the appropriate items are selected for individual tests, and thus, the composition of the item bank needs to be carefully maintained. The supply of different types of items can be adjusted using the test blueprint, test assembly requirements, characteristic of the item selection algorithm, and characteristics of different types of items. The goal is to balance the usage of all items in the item bank, so that there won’t be items that are over-exposed or wasted. One critical task in item bank management is to replenish the item bank in an efficient manner. A cutting-edge technique for this called Online Calibration in which new items can be embedded into operational tests without separate scaling procedures. The researchers at UIUC are currently studying the aspects of Online Calibration and have proposed new methods.

Automated Test Assembly
In the traditional linear tests and multistage tests, test forms are usually preassembled before administration. Traditionally, tests are assembled manually; nowadays, Automated Test Assembly (ATA) can be easily implemented by computer programs. Given the item bank, ATA programs can simultaneously assemble multiple parallel test forms that target the chosen difficulty distribution and meet multifold non-statistical requirements.

Cognitive Diagnosis
Cognitive Diagnosis methods can be applied to provide feedback on test takers’ strengths and weaknesses on language acquisition. These methods can help understand examinees’ strategies of solving particular tasks and can further provide remedial suggestions on the aspects that the examinee has not mastered. Administering intermediate Cognitive Diagnostic tests along the course of the language learning programs can assist instructors and policy makers to adjust  teaching contents, strategies, and emphases to better assist students learning.