Bohack, K. (2021). The effect of computer familiarity on computer-based assessment in mathematics (Publication No. 28317900) [Doctoral dissertation, Hofstra University]. ProQuest Dissertations and Theses Global. https://www.proquest.com/docview/2494876201

Hofstra University (Hempstead, NY); ProQuest document ID: 2494876201

Student performance on an algebra computer-based test (CBT) and paper-based test (PBT) were compared; further, students' familiarity with computers was explored as a factor influencing test administration differences. The CBT had embedded features including answer elimination, flagging questions for further review, notepad for each test item, and a test review tool.

Fifty-three (n=53) high school geometry students in grades 9 through 11 participated in the study. Students with an IEP or 504 plan comprised 18.9% of the sample (n=10). Other student demographic information reported, including race, gender, socio-economic status.

The computer-based test (CBT) and the paper-based test (PBT) were constructed from two recent state exams in algebra. The CBT was comprised of 13 questions selected from the January 2020 state algebra exam and the PBT was comprised of 13 questions from the August 2019 state algebra test. Questions on the two test versions were aligned and each of the respective questions tested the same concepts. The tests were composed of eight multiple choice questions and five written response questions and were to be completed within a 43 minute class period. Because the researcher determined that the Computer Aversion, Attitudes, and Familiarity Index (CAAFI; Schulenberg, 2002) focused more on computer hardware and aspects of computers from the 2000s, a different set of 10 computer familiarity questions was developed, yet using a similar scoring system as the CAAFI: a rating scale of 1–7, with 4 being neutral and higher scores indicating less aversion, more positive attitude, and greater familiarity. The dependent variables were the PBT, CBT, difference in scores (PBT-CBT), computer familiarity survey, and prior achievement. A single group alternating treatment design was used: all students completed the survey on the first day, and then half the students took the CBT and half took the PBT on the second day, and on the third day the students switched conditions.

The analysis showed no significant mean differences in scores between the computer-based test and the paper-based test. Further, no patterns were found when examining placement level, gender, and other factors in general. Scores between formats were deemed comparable and the formats were interchangeable. No significant correlation was found between computer familiarity and CBT scores, and no significant correlations were observed between computer familiarity and CBT scores, PBT scores, differences in scores, or prior achievement. Thus, degree of familiarity with computers was not a factor influencing performance. The implication was that implementing computer-based assessments was deemed appropriate, as there were no indications that this administration format was detrimental to student participants. Limitations of the study were reported, and future research possibilities were suggested.