Wise, S. L., Kuhfeld, M. R., & Soland, J. (2019). The effects of effort monitoring with proctor notification on test-taking engagement, test performance, and validity . Applied Measurement in Education , 32 (2), 183–192. https://doi.org/10.1080/08957347.2019.1577248

The test proctors prompted student participants to reengage in completing test items when the computer-based test's software detected a response pattern indicating that test-takers were becoming disengaged, and the testing system notified test proctors of test-takers' behavior (a feature called "proctor notification" on "effort monitoring").

An extant data set of the Portland, OR school district (U.S.) K–12 student population was accessed for the two testing years, Fall 2016 and Fall 2017. The sample of students that met criteria for the pattern of disengagement in reading and math totaled 10,353: 2016 math, 1,470; 2016 reading, 2,804; 2017 math, 1,872; 2017 reading, 4,207. Demographic data were reported for each test event analyzed; no details on disabilities (if any) or special education status were indicated. 

District-wide computerized adaptive testing of interim achievement in math and reading was administered in fall 2016 and fall 2017. Comparisons between these test events were not through matching individual student scores, but rather with groupwise means, checked with proxy variables, and researchers deemed them reasonably equivalent. Observed test item response times were documented. Further, estimates of rapid-guessing thresholds were made by test item, drawing from an author’s independent research (Wise & DeMars, 2006) and seeking to serve as an indicator of test effort monitoring and engagement, addressing motivation.

Higher test-taking engagement and higher test performance resulted when test engagement monitoring (i.e., effort monitoring) and proctor notification were used. These effects were stronger for computer-based reading tests than for computer-based math tests. In addition, correlations between beginning and final test scores were higher when test engagement monitoring and proctor notification were used than when they were not used, supporting convergent validation. Results from a sample of test proctors suggested that, in general, test proctors viewed the proctor notification feature as valuable. Limitations of the study were reported.