Unpacking student growth percentiles: statistical properties of regression-based approaches with implications for student and school classifications

The measurement of achievement growth raises many challenges, including how to define "growth" and select or develop a growth measure that captures that definition. Despite these complications, current federal educational policies focus on student growth measures for accountability purposes. Student growth percentiles (SGPs) are one metric developed under these policies. They use quantile regression to produce normative growth interpretations: They describe how much a student has grown relative to students with similar past test scores. SGPs are increasingly popular, but there are gaps in the literature concerning their performance for small sample sizes and the number of prior years of test scores included in the model, as well as their invariance to transformations of the test scale.

This study proposes an ordinary least squares analog, the percentile rank of residuals (PRRs). PRRs are the percentile rank of the residuals found by regressing the current grade-level assessment score on past grade-level assessment scores. PRRs may be a more robust alternative to SGPs, especially for small samples. They also stem from a wide array of regression based metrics in education and only require estimation of one regression line, as opposed to the 100 regression lines estimated for SGPs.

This dissertation first places the growth metrics of interest in a framework anchored by four key contrasts in growth interpretations: (1) absolute versus normative, (2) unconditional normative versus conditional normative, (3) student- versus group-level, and (4) aggregated individual growth versus growth of aggregated-individuals. SGPs and PRRs afford normative conditional growth interpretations. They are investigated at the student level using simulated multivariate normal data and two statewide empirical datasets. These student-level analyses assess the accuracy of SGPs and PRRs by their recovery of benchmark growth percentiles under multivariate normality, or normal conditional growth percentiles (NCGPs), their robustness to scale transformations, their comparability to each other under varying conditions, and their stability over different sample sizes and numbers of prior years included in the models. SGPs and PRRs are also investigated at the group level by aggregating them with the mean and median functions. The robustness of the aggregated growth percentiles to test scale transformations is also assessed. Finally, the aggregated growth percentiles are contrasted against group effects from a simple layered value-added model (VAM).

The analyses found that PRRs better recover expected growth percentiles under multivariate normality and are more accurate and stable for small samples, whereas SGPs are substantially more robust to test scale transformations. However, estimation issues with the SGPs can cause students with extreme initial statuses to obtain substantially different SGPs under transformations of the data. At the aggregate level, there is little distinction in how robust SGPs and PRRs are to scale transformations of the test score data. The mean SGPs and mean PRRs are consistently more robust to scale transformations of the test score data then their median counterparts. They are also the most highly correlated and rank order the groups more similarly to the value-added school effects than the median SGPs and PRRs.