Higgins, J. A., Famularo, L., Cawthon, S. W., Kurz, C. A., Reis, J. E., & Moers, L. M. (2016). Development of American Sign Language guidelines for K–12 academic assessments . Journal of Deaf Studies and Deaf Education , 21 (4), 383–393. https://doi.org/10.1093/deafed/enw051

The researchers investigated the impact and perceptions of two versions of the American Sign Language (ASL) accommodation as presented through embedded videos incorporated into digital online assessments. Also incorporated in this accommodation was the potential repetition of directions and items, in the control of test-takers. Related issues including the presentation of math notation, the finger-spelling of key terms, use of space, and the expression of plurality by interpreter were examined.

The randomized controlled trial involved analyzing math assessment results from 279 students who were deaf and used ASL for communication from across 17 U.S. states, including 54 elementary students, 99 in middle school, and 126 in high school. Teachers reported the students' reading and math abilities, to avoid additional screening test burden, allowing students to be sorted into groups of students with combinations of low and average or above ability. The cognitive lab interview protocol engaged 46 students who were deaf and used ASL for communication participated from across five different states, including 16 elementary students, 14 in middle school, and 16 in high school.

The math assessments were composed of 18 grade-level-appropriate test items drawn from publicly-released items linked to college and career ready learning standards for students in the three school levels (elementary, middle, and high school). Items comprised various response types, including multiple choice, constructed response, and technology enhanced items. The design of the sets of test items were organized around three different considerations, with two items for each; three different item conditions were formulated: one ASL version, another ASL version, and a standard/control condition without an ASL accommodation (3x2x3=18 items). An example is the consideration of how terminology is presented—with the ASL sign, with the term "finger-spelled" out in letters, or the item presented in print only—would be addressed through the formulation of these six items for one school level test. Qualitative data were gathered from comments of 46 students who were deaf and used ASL for communication who participated from across five different states, including 16 elementary students, 14 in middle school, and 16 in high school. Researchers followed a cognitive lab—also called "think aloud"—interview protocol, and documented students' statements and the interviewers' observations of students during this interview process.

The math test results analyses require a detailed description. While controlling for student ability variation, the comparison of performance on math items with and without ASL accommodations yielded that students on average scored consistently and significantly higher when using the accommodations, across all three school levels. The score comparisons between the two versions of ASL conditions were more complex in process, yet yielded similarly consistent but non-significant results. Specifically, there were no significant differences in scoring between sets of items that had equations and images signed or not signed, finger-spelled only or finger-spelled and signed, diamond item structure or not, reduplication of plurality or showing action and number of times, or graphic presented on signers' hands or in front of signers' bodies. Students' responses to the cognitive lab interview protocol were categorized into three themes. The commonality among the themes is that students who were deaf preferred receiving more information rather than less, and preferred receiving communication more akin to American Sign Language than English. For instance, students preferred when math notation and graphics were presented in ASL rather than not presented in ASL—that is, the student was left to read the notation or graphic without support. The elementary students indicated that the non-signed version was confusing; the older students explained that these math symbols and graphics would typically be signed during instruction. Similarly, finger-spelling provided additional information to students, and use of the diamond structure for items was the common communication pattern in ASL and more familiar to native ASL signers. The researchers also reflected on the process of developing ASL videos for online math assessments and offered recommendations, including the value of a team-based approach to translation, including academic content and ASL experts. Limitations of the study were reported, and future research directions were suggested.