Item Response Time Analysis Using ex-Gaussian Distribution for Disengagement Detection in Online Low-stakes Tests

Abstract

This study addresses the problem of detecting disengagement in online low-stakes tests used in blended learning within higher education. The detection method was developed based on an analysis of item responses and associated response times. The method applied the ex-Gaussian mixture model to response times, rather than the conventional lognormal model. The mixture component with the smallest Gaussian mean was chosen to represent the response times distribution of early correct responses. The selected mixture component was used to obtain the model’s mode, which then served as the threshold for classifying item responses into early and subsequent response groups. Based on the two classified groups, descriptive statistics and graphical visualizations were introduced to support manual inspection and provide insight into item- and person-level characteristics. A test statistic for disengagement detection was formulated based on the distribution of the number of early responses. Drawing on prior knowledge of the success probabilities associated with disengaged responses, two detection boundaries were defined to classify item-preknowledge and rapid-guessing behaviors. Unlike existing model-based methods for rapid guessing and item preknowledge behavior detections, the proposed non-parametric method does not require prior knowledge of item or person parameters, nor does it involve modeling or estimating such characteristics. The method's performance was assessed using both real and simulated data, and results for true positive rates and false positive rates were reported under various test conditions. The findings indicate that the method's performance improves with an increasing number of test items and a higher proportion of disengaged responses. Simulation results further demonstrated the method’s robustness to measurement error and small variations in response times, in contrast to the person-level adaptation of the NT10 and CUMP methods, whose performance varied significantly under the same conditions.