Although much educational research has investigated the relative effectiveness of different educational interventions and policies, little is known about the absolute net benefits of K-12 schooling independent of growth due to chronological age and out-of-school experience. The nearly universal policy of age tracking in schools makes this a difficult topic to investigate. However, a quasi-experimental regression discontinuity design can be used to separate observed test score differences between grades into independent age and schooling components, yielding an estimate of the net effects of school exposure at each grade level. In this study, a multilevel version of this design was applied to scores on 22 common ability and achievement tests from two major standardized test batteries. The ability battery contained 9 measures of Verbal, Quantitative, and Figural reasoning. The achievement battery contained 13 measures in the areas of Language, Mathematics, Reading, Social Studies, Science, and Sources of Information. The analysis was based on sample of over 20,000 students selected from a longitudinal database collected by a large U.S. parochial school system.
The theory of fluid (Gf) and crystallized (Gc) intelligence predicts that these tests will show systematically different levels of sensitivity to schooling. Indeed, the achievement (Gc) tests were found to be three times more sensitive to schooling than they were to aging (one-year effect sizes of .41 versus .15), whereas the ability (Gf) tests were equally influenced by age (.18) and schooling (.19). Nonetheless, the schooling effect on most Gf tests was substantial, especially when the compounding over a typical school career is considered. This replicates the results of previous investigations of age and schooling using regression discontinuity methods and once again contradicts common interpretations of fluid ability.
Different measures of a construct often exhibited varying levels of school sensitivity. Those tests that were less sensitive to schooling generally required reading, reasoning, transfer, synthesis, or translation; posed a wider range of questions; and/or presented problems in an unfamiliar format. Quantitative reasoning tests showed more sensitivity to schooling than figural reasoning tests, while verbal reasoning tests occupied a middle ground between the two. Schooling had the most impact on basic arithmetic skills and mathematical concepts, and a significantly weaker impact on the solution of math word problems. School-related gains on isolated language skills were much larger than gains on solving grammar problems in context. The weakest schooling impact overall was on reading comprehension where effects were no larger than those on verbal ability measures.
An interesting dichotomy was found between spelling and paper folding (a measure of figural and spatial reasoning). Spelling skills showed robust schooling effects but a consistently negative age slope, a puzzling result which indicates that younger students in each group outperformed older students. Paper folding showed the opposite pattern, a large age effect and a small but consistently negative schooling effect.
Results serve to rebut skepticism about both the impact of schooling on test scores and the validity of distinctions between ability and achievement. It is argued that the regression discontinuity design has great potential in the measurement of school effectiveness, while also offering a source of validity evidence for test developers and test users. Implications for theories of cognitive ability and future research on schooling effects are discussed.