Report
Generalizability Theory: A New Approach To Analyze Non-Crossed Performance Assessment Data
Distributed by ERIC Clearinghouse
1997
Abstract
Unstable, and potentially invalid, variance component estimates may result from using only a limited portion of available data from operational performance assessments. However, missing observations are common in these settings because of the nature of the assessment design. This paper describes a procedure for overcoming the computational and technological limitations in analyzing data with missing observations by extracting data from a sparsely filled data set into analyzable smaller subsets of data. This parsing is accomplished by creating data sets that exhibit structural designs that are common in generalizability analyses, namely the crossed, mixed, and nested designs. An example of how to perform the procedure is given. Data are from a large-scale college writing assessment in which each of 5,905 examinees responded to 2 essay prompts. Results show that the sparsely filled performance assessment data sets can be restructured into analyzable smaller subsets of data. Results suggest that the crossed, mixed, and nested methods are comparable, but more study is needed to determine whether the methods generalize to other data sets with more than two facets. (Contains 3 figures, 9 tables, and 17 references.) (Author/SLD)
Details
- Title: Subtitle
- Generalizability Theory: A New Approach To Analyze Non-Crossed Performance Assessment Data
- Creators
- Chris W. T ChiuEdward W Wolfe
- Resource Type
- Report
- Publisher
- Distributed by ERIC Clearinghouse
- Number of pages
- 38 pages
- Language
- English
- Date published
- 1997
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9985134646302771
Metrics
1 Record Views