Johnstone, C., Higgins, J., & Fedorchak, G. (2019). Assessment in an era of accessibility: Evaluating rules for scripting audio representation of test items . British Journal of Educational Technology , 50 (2), 806–818. https://doi.org/10.1111/bjet.12603

First published online (1/4/18)

Audio format—oral delivery by recorded human voice or via text-to-speech software—was reviewed in this summative research review of a series of three studies (Johnstone et al., 2012; Higgins & Katz, 2013; Higgins et al., 2015). The "A3" model (advocacy, accommodations, accessibility) was presented as a frame for understanding "technological innovations for assessments" (p. 807); the common issue of scripting rules underlying the audio representation of tests was examined.

This summative review of three separate but related studies consisted of three sets of participants: (a) 93 high school students including 31 students with disabilities; (b) a maximum of 89 students including 52 students with disabilities; (c) 281 students in grades 3–12 including 65 students with disabilities. Data sources included test scores with systematic comparison conditions, descriptive survey data, and qualitative data from cognitive labs and other interviews with small number case examples. Disability categories of participants included math-related learning disabilities or visual impairments. All studies were completed in U.S. public education contexts.

Across all three studies, participants' perceptions of and performance on academic test items in science, mathematics, or language arts were measured when presented with different audio formats. These studies removed decoding and reading prerequisites, allowing students to better demonstrate what they know in the content areas.

This summative review highlighted technology-based innovations that have bolstered accessibility for assessments while decreasing the need for individual accommodations. The three studies included in this paper provided insights into how assessments can be presented in audio formats, minimizing exclusion. A set of three complementary findings for designing, developing, and using technology-based assessments emerged: (a) The only scripting rule that was statistically significant in its impact on student performance was the representation of math equations using parenthetical expressions (e.g., "open parenthesis, value, close parentheses"); (b) students with learning challenges performed better under all audio representation conditions than without them; (c) students with disabilities preferred graphs that were described in terms of key symbols.