Dissertation
Non-cognitive skills as predictors of academic performance: how differential item functioning affects prediction of college enrollment
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Autumn 2021
DOI: 10.17077/etd.006255
Abstract
ABSTRACTThe 2015 Every Student Succeeds Act (ESSA) emphasized that students, regardless of group membership, should graduate ready for college or work. The economic impact of not being ready is significantly negative. Non-cognitive skills (NCSs) assessments have been used to identify students at risk and intervene to help prevent undesirable outcomes.
This dissertation intended to answer three main research questions. The first investigated if items on an instrument of NCSs showed significant differential item functioning (DIF) for relevant groups of students. DIF analyses were performed to verify if items are functioning appropriately for different ethnicity (Hispanic and non-Hispanic) and race (African American and White) groups. Results suggested that the 23 out of 45 polytomous items selected from the NCSs instrument showed significant DIF for the groups compared.
The second research question investigated how NCSs would increment the prediction of academic performance (measured by ACT composite score and college enrollment). Several models were compared using linear and logistic regression. The results suggested that NCSs assessments do not improve the amount of variance explained on the outcome variable (ACT composite score) or the quality of the model fit. For the logistic models used for predicting college enrollment, the NCSs did not improve the ability of the model in predicting the outcome.
Last, the third research question investigated if the prediction of academic outcomes would change if the items flagged for DIF were removed from the NCSs instrument when performing the regression analyses. The parameters used to compare the models, R square and RMSE (for linear regression) and c-statistic and Akaike Information Criterion were nearly identical to when the items have not been removed from the instrument. The results suggested that the removal of the items flagged for DIF from the NCSs instrument did not affect the prediction of academic performance.
Details
- Title: Subtitle
- Non-cognitive skills as predictors of academic performance: how differential item functioning affects prediction of college enrollment
- Creators
- Juliana Cerentini Pacico
- Contributors
- Catherine J. Welch (Advisor)Stephen Dunbar (Advisor)Alex Casillas (Committee Member)Ryan Carnahan (Committee Member)Terry Ackerman (Committee Member)Walter Vispoel (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Psychological and Quantitative Foundations
- Date degree season
- Autumn 2021
- DOI
- 10.17077/etd.006255
- Publisher
- University of Iowa
- Number of pages
- xvi, 202 pages
- Copyright
- Copyright 2021 Juliana Cerentini Pacico
- Language
- English
- Description illustrations
- color illustrations
- Description bibliographic
- Includes bibliographical references (pages 187-202)
- Public Abstract (ETD)
This dissertation examined the psychometric properties of an instrument measuring non-cognitive skills (NCSs). Its primary purpose was to identify differential item functioning (DIF) for race and ethnicity groups on selected items of the NCSs instrument. A second purpose was to investigate the ability of the NCSs instrument to predict academic outcomes. Last, it investigated how the prediction of academic performance would change when items flagged for DIF were removed from the instrument.
The results provided evidence that the items were functioning differently for the groups compared. Additionally, the NCSs did not improve the prediction of academic outcomes. When the items flagged for DIF were removed from the instruments, the prediction did not change. DIF flags for fairness issues, which might impact validity. One of the contributions of this study was to raise awareness of the importance of conducting fairness review in the assessment of NCSs.
- Academic Unit
- Psychological and Quantitative Foundations
- Record Identifier
- 9984210526602771
Metrics
248 File views/ downloads
95 Record Views