Mastergeorge, A. M., & Martinez, J. F. (2010). Rating performance assessments of students with disabilities: A study of reliability and bias . Journal of Psychoeducational Assessment , 28 (6), 536–550. https://doi.org/10.1177/0734282909351022

Accommodations were not specified; teacher bias in rating performance was investigated. Any and all accommodations provided for students completing the Standards Test to Evaluate the Performance of Students (STEPS) in a school district were involved.

Data from raters of the performance assessments of students across a large public school district in the Southwest (U.S.) served as the participants for the specific part of the study pertaining to rating bias. Specific demographics were not made available on these raters, other than that they were current teachers at the three grade levels (grade 3, grade 7, and grade 9) and academic subjects, and varied in number between 4 and 13; consequently, 4 raters from each grade level were sampled for the bias analysis, totaling 12 participants. The raters scored a sample of test results from 30 students without disabilities and 30 students with disabilities receiving accommodations.

The dependent variables in the study as a whole included language arts and mathematics performance, as assessed by a large-scale test called the Standards Test to Evaluate the Performance of Students (STEPS), as well as teacher perceptions regarding rater bias, as measured by a survey. The part of the study associated with accommodations drew from teacher survey items regarding equity issues.

The results indicate that teachers did not introduce bias into the ratings of students with disabilities who had received accommodations, in comparison with students without disabilities not receiving accommodations. Interestingly, reliability decreased between a condition in which raters were unaware whether tests were those of students with disabilities receiving accommodations, to a condition when raters became aware of this fact. That is, measurement error increased between these two conditions; however, this error was not systematic, and the scores neither increased nor decreased consistently for these students. Further, raters indicated that teachers should not permit differing treatment of test results from students with disabilities receiving accommodations. Limitations of this study were reported, and recommendations for future studies were offered.