Population invariance in composite-score equating with the random groups design

In the context of educational assessment, composite score conveys the overall performance of an examinee on a test and is oftentimes used to make high-stakes decisions such as certification, licensure, and college admission. In order to make a valid interpretation of test scores as well as to make a fair judgement in these matters, all the reported composite scores should be comparable across multiple test forms. Composite-score equating is a statistical process used to adjust for differences in difficulty among test forms so that composite scores on the forms can be used interchangeably.

To produce interchangeable test scores from an equating process, multiple essential properties are desired to be met including the property of population invariance. Population invariance of equating exists when the score transformation function is not dependent on the subpopulations of examinees used in the equating process. If the population invariance requirement does not hold, the resulting equating relationship leads to an expected advantage for one or more subpopulations of examinees and hence poses a threat on test fairness and equity. The purpose of this dissertation was to investigate how well ten different composite equating procedures preserve the population invariance property under the random groups data collection design.

Results indicated that when test forms were very similar in their content and statistical specifications, population invariance of equating would not be compromised, regardless of which composite equating procedure was used. However, when test forms were not strictly parallel, the performance of the studied equating procedures with respect to population invariance varied slightly depending on which equating method was used, which composite equating framework was chosen, degree of dimensionality, and test and sample characteristics.