An IRT Model-Based Reliable Change Index With Empirical Priors: An Extension Using A Multiple Group Approach With Finite Sample Sizes

Abstract

The reliable change index (RCI; Jacobson & Truax, 1991) is a popular tool for assessing whether individuals have changed between treatments. Recently, an Item Response Theory (IRT)-based RCI that incorporates group mean information through the use of expected a posteriori (EAP) estimation has been adopted, showing promising results (Chalmers & Campbell, 2025). This paper extends the previous RCI-IRT work by (1) using finite sample sizes for model calibration and parameter estimation and (2) adopting a multiple group (MG) approach to modelling sample data. Results showed that even with slight methodological changes, the results are similar to the previous studies, in that incorporating empirical priors improves rates of detecting individual change when true change is present. Larger calibration sample size has an impact on model parameter recovery, but not person parameter recovery. Finally, results favour the use of the MG approach with EAP group-informed priors when underlying group heterogeneity is expected.