Hansen, E. G., Loew, R. C., Laitusis, C. C., Kushalnagar, P., Pagliaro, C. M., & Kurz, C. (2018). Usability of American Sign Language videos for presenting mathematics assessment content . Journal of Deaf Studies and Deaf Education , 23 (3), 284–294. https://doi.org/10.1093/deafed/eny008

American Sign Language (ASL) presented by an avatar (i.e., an animated person) was compared to ASL presented by a video-recorded human signer on performance on a computer-based math assessment, and also compared to student performance on items presented in English written format. In all three conditions, participants responded by selecting from multiple answer choices, typically numerical options.

Students with hearing impairments or deafness who were both American Sign Language (ASL) users and written English users participated. Of the 31 participants, 12 were high school students and 19 were postsecondary students, all recruited in the western part of the state of New York (U.S.). All participants had already completed their first high school algebra course and received at least a passing grade ("C" or better). Demographic data including gender and race-ethnicity were reported, as were their ages when first learning ASL.

Mathematics test items were drawn from a set of practice items for preparing for a standardized math test, and were linked to standards expected of a student at the middle school or early high school level. The researchers noted that five of the 10 items were selected based on ASL educators' judgments that they were "'easy' to translate from English into ASL" (p. 288), and that the other five were judged to be "'hard' to translate" (p. 288). Three math test items included graphics. Participants also responded (via ASL) to a series of interview questions on their experience completing the test items using different ASL signers (human vs. avatar), including their perceptions about clarity of the signing, as well as their preferences for either signer version, and their suggestions for improving usability.

Participants' scores were not significantly different between the two ways that ASL was presented, by the video recording of a human signer versus the simulated signing by an avatar. When subsequently presented with the English version of the math questions, participants' average scores were significantly higher than the two ASL versions, with more participants scoring correctly on seven of the nine full English items than the signed items. The math performance scores and the accuracy of their translations of the ASL into English were highly correlated with one another, but the signed version and translation scores were not significantly correlated with one another. The researchers stated, "Thus, neither mathematics scores nor translation scores provided significant evidence that one version was more comprehensible than the other" (p. 292). The variation in age at which they started learning ASL did not appear to be associated with math performance or translation scores. On a five-point scale, participants rated the signing higher in quality for the human signer over the avatar signer, with about 60% indicating the human signing was 'very good' or 'excellent', and about 10% indicating those same ratings for the avatar signing. Of the 31 participants, 29 preferred the ASL by the human, one preferred the avatar, and one had no preference. Participants apparently did not perceive the avatar's facial expressions, mouthing of the words, and body movement as sufficiently communicative. One participant noted that the human signer did not pause enough. Nearly all participants indicated their impressions that the English version of the math test would have been sufficient for them to answer the items correctly. Participants suggested various improvements for the ASL versions of the math test, with many suggesting adding English captions for test items, and some requesting that direct signing of English, also called "Signed Exact English," would be used instead of ASL. Limitations of the study were reported, and future research directions were suggested.