Puhan, G., Boughton, K., & Kim, S. (2007). Examining differences in examinee performance in paper and pencil and computerized testing . The Journal of Technology, Learning, and Assessment , 6 (3). https://ejournals.bc.edu/ojs/index.php/jtla/issue/archive

[no doi located]; Also downloadable from ERIC online database: https://eric.ed.gov/?id=EJ838613

The comparability of two versions of a certification test—a paper-and-pencil test (PPT) and computer-based test (CBT)—were examined.

A total of 1,122 examinees in reading, 1,050 examinees in writing, and 1,136 examinees in mathematics comprised an extent data set of postsecondary students completing the computer-based test, and were compared with an equal number of test-takers completing the paper-and-pencil version. Participants formed samples of test scores from the nationwide (U.S.) population of test-takers of the Praxis™ teacher certification test in 2003. No disabilities were reported for participants.

A large scale certification test from the Praxis™ program measured basic proficiency in reading, writing, and mathematics and has been used for entrance into teaching programs.

The effect sizes were small (d < 0.20) and not statistically significant (p > 0.05), suggesting no substantial difference between the two test versions. Moreover, DIF analysis revealed that reading and mathematics items were comparable for both versions. However, three writing items were flagged for DIF. Limitations of the study were reported, and future research possibilities were suggested.