Lee, D., Buzick, H., Sireci, S. G., Lee, M., & Laitusis, C. (2021). Embedded accommodation and accessibility support usage on a computer-based statewide achievement test . Practical Assessment, Research, and Evaluation , 26 (1), article 25. https://doi.org/10.7275/21926674

Also located on journal webpage https://openpublishing.library.umass.edu/pare/article/id/1574/

Accessibility supports examined individually included highlighter by student, line reader, masking, oral delivery via text-to-speech software, and paper-and-pencil format (also called "print") in an otherwise digital testing environment. Text-to-speech was provided and used either throughout the entire tests, or partially as needed for specific words or parts of tests. These supports were each categorized as universal tools, designated supports, or accommodations as part of the findings.

A statewide extant data set incorporating grade 6 students completing an unspecified state's (U.S.) mathematics assessment (N=481,282) and English language arts (ELA) assessment (N=475,060) that was administered in spring 2018. [Note: The larger number, 481,282 students, was designated as the total participants in this study; it is is implicit that many (possibly all) of the math testers also completed the ELA. The two tests' participant numbers were not added together because doing so would indicate that there were many more students in the statewide sample than there were.]

Smarter Balanced extant statewide datasets for English language arts (ELA) and mathematics (Smarter Balanced Assessment Consortium, 2016, 2017) were analyzed and described. Rather than actual test performance scores, other student data were reported. Statewide data for student groups' eligibility for accessibility supports and accommodations, and their recorded incidence of use, were reported separately for ELA and for math. Further, use frequency was measured in three ways: (a) total counts of use by item for each student, (b) numbers of students (by group) using accessibility supports and accommodations at least once for each test, and (c) calculation of average number of items that each accessibility support and accommodation was used in each content area. In addition, these data were reported at the district level to demonstrate the range of frequencies across the state.

State policies indicated that line reader and highlighter were universal tools, masking was a designated support requiring pre-approval for any student's use, and paper-and-pencil testing instead of digital format was an accommodation. Text-to-speech software was considered an accommodation when provided during ELA tests for reading text segments to students with disabilities, and considered a designated support for all other test content and conditions. [The eligibility rates, and incidence of use, for accommodations by students with disabilities and English learners with disabilities is emphasized in this statement, rather than detailing these patterns for all accessibility supports for students without disabilities.] Eligibility for using designated supports and accommodations differed by student group, yet was similar across statewide ELA and mathematics assessments. About 5% of general education students and 16% of English learners were eligible to use text-to-speech software as a designated support; in contrast, about 43% of students with disabilities, and 65% of English learners with disabilities were eligible for text to speech as a designated support. About 30% of students with disabilities and 46% of English learners with disabilities were eligible for text-to-speech as an accommodation—for oral delivery of ELA text passages. Less than 1% of students with disabilities, and about 1% of English learners with disabilities, were eligible for paper-and-pencil testing. Incidence of use for students with disabilities and English learners with disabilities was comparatively lower than eligibility to use accommodations, yet ranged among individual accommodations and between test content. About 83% of eligible grade 6 students with disabilities and about 86% of eligible grade 6 English learners with disabilities who were provided text to speech on ELA assessments were recorded as using that accommodation throughout the entire assessment; in contrast, actual use of partial text-to-speech—such as for selected words or items—was actually used by about 15% of students with disabilities and 17% of English learners with disabilities who were provided that version of the accommodation. For the relatively rarely-provided paper-and-pencil format, about 29% of eligible students with disabilities and 25% of English learners with disabilities actually used it on ELA tests, and 10% of eligible students with disabilities and 15% of eligible English learners with disabilities actually used it on math tests. The frequency of actual use of universal tools and designated supports by eligible students was also reported. The phenomenon of lowered rates of eligible students actually using an accessibility feature or accommodation was termed "underuse" when analyzing district-level data across the state; "overuse" was also identified when access to a tool was provided to ineligible students. The researchers demonstrated the procedures for identifying districts with overuse and underuse rates, noting the importance of addressing these implementation concerns. Implications for practice were offered, limitations of the study were reported, and future research possibilities were suggested.