Equating with bivariate log-linear presmoothing under the common-item nonequivalent groups design: structural zeros and their implications

In equating contexts, when common items are internal and scoring is in terms of the number of correct items, for a bivariate distribution of total scores (X) and scores on the common items (V), some pairs of X and V can never be observed; these are called structural zeros (Bishop, Fienberg, & Holland, 2007; Holland & Wang, 1987). The primary purpose of this dissertation was to investigate how different approaches to handling structural zeros give different equating results. The following five approaches to handling structural zeros were compared: (1) no smoothing approach, (2) internal approach, (3) external approach, (4) adjusted external approach, and (5) univariate frequency estimation approach. Furthermore, this dissertation examined how the relationship between equating results and approaches to handling structural zeros changes as the proportion of common items changes.

Two types of data were considered: operational data and simulated data. The operational test analyses used five operational data sets that had differences in content areas, proportion of common items, group performance levels, and sample sizes. Three study factors were considered as follows: (1) approaches to handling structural zeros, (2) equating methods, and (3) degrees of smoothing. For the simulated test analyses, four additional study factors were considered: (1) proportion of common items, (2) test length, (3) examinee ability effect size, and (4) sample sizes.

There were four main findings from the operational and simulated test analyses: (1) the external approach generally gave the worst results; (2) for relatively small effect sizes, the internal approach generally gave the smallest overall error; (3) for relatively large effect sizes, the adjusted approach generally gave the smallest overall error; and (4) if interest focused on reducing bias, only, the adjusted approach was generally preferable. These results suggest that, under the conditions studies here, when a set of common items is an internal anchor and bivariate smoothing is performed, the internal approach which maintains structural zeros is generally preferable.