Johnstone, C., Higgins, J., & Fedorchak, G. (2019). Assessment in an era of accessibility: Evaluating rules for scripting audio representation of test items . British Journal of Educational Technology , 50 (2), 806–818. https://doi.org/10.1111/bjet.12603

Journal Article

Johnstone, C., Higgins, J., & Fedorchak, G. (2019). Assessment in an era of accessibility: Evaluating rules for scripting audio representation of test items. British Journal of Educational Technology, 50(2), 806–818. https://doi.org/10.1111/bjet.12603

Notes

First published online (1/4/18)

Tags

Elementary; High school; K-12; Language arts; Learning disabilities; Math; Middle school; Multiple ages; No disability; Oral delivery; Recorded delivery (audio or video); Science; Text-to-speech device/software; U.S. context; Visual impairment (including blindness)

URL

https://bera-journals.onlinelibrary.wiley.com/journal/14678535

Summary

Accommodation

Audio format—oral delivery by recorded human voice or via text-to-speech software—was reviewed in this summative research review of a series of three studies (Johnstone et al., 2012; Higgins & Katz, 2013; Higgins et al., 2015). The "A3" model (advocacy, accommodations, accessibility) was presented as a frame for understanding "technological innovations for assessments" (p. 807); the common issue of scripting rules underlying the audio representation of tests was examined.

Participants

This summative review of three separate but related studies consisted of three sets of participants: (a) 93 high school students including 31 students with disabilities; (b) a maximum of 89 students including 52 students with disabilities; (c) 281 students in grades 3–12 including 65 students with disabilities. Data sources included test scores with systematic comparison conditions, descriptive survey data, and qualitative data from cognitive labs and other interviews with small number case examples. Disability categories of participants included math-related learning disabilities or visual impairments. All studies were completed in U.S. public education contexts.

Dependent Variable

Across all three studies, participants' perceptions of and performance on academic test items in science, mathematics, or language arts were measured when presented with different audio formats. These studies removed decoding and reading prerequisites, allowing students to better demonstrate what they know in the content areas.

Findings

This summative review highlighted technology-based innovations that have bolstered accessibility for assessments while decreasing the need for individual accommodations. The three studies included in this paper provided insights into how assessments can be presented in audio formats, minimizing exclusion. A set of three complementary findings for designing, developing, and using technology-based assessments emerged: (a) The only scripting rule that was statistically significant in its impact on student performance was the representation of math equations using parenthetical expressions (e.g., "open parenthesis, value, close parentheses"); (b) students with learning challenges performed better under all audio representation conditions than without them; (c) students with disabilities preferred graphs that were described in terms of key symbols.