Differential item and distractor functioning between computer-based and paper-and-pencil testing within demographic groups on a Statewide Mathematics Assessment

This dissertation analyzed mathematics items across three grade levels of a statewide assessment for evidence of differential item functioning (DIF) to determine whether examinees who took the test on computers were more or less likely to answer the items correctly than score-matched examinees who took the test on paper. These DIF analyses were conducted within the total population of examinees, then within selected demographic groups. Following this, the items were analyzed for differential distractor functioning (DDF) to determine whether computer-based examinees or paper-based examinees were more likely to choose a specific distractor of an item, whether within the total examinee population or within demographic groups.

The results showed that while only one item showed evidence of DIF between the two test delivery modes within the total population, more items showed evidence for DIF within demographic groups. Furthermore, even more items showed evidence of DDF between test delivery modes. When the items flagged for DIF and DDF were examined for trends, evidence was found showing that paper-and-pencil examinees were significantly more likely to incorrectly select the last option of an item compared to score-matched computer-based examinees. Implications from the results and how these results can help test developers were also discussed.