Lin, P. Y., Childs, R. A., & Lin, Y. C. (2016). Untangling complex effects of disabilities and accommodations within a multilevel IRT framework . Quality & Quantity , 50 (6), 2767–2788. https://doi.org/10.1007/s11135-015-0288-8

[First published online (11/30/15)]

For this application of multilevel measurement modeling, Rasch item response modeling and differential item functioning (DIF) analysis, the researchers investigated the impact of the setting accommodation -- typically referred to as separate setting, individual setting, or quiet setting.

Grade 6 students who took the Ontario provincial assessments were sampled to include 3,956 students with learning disabilities (LD) and 4,875 students without disabilities. The researchers noted that the scores of students using the setting accommodation who did not have disabilities also were individually determined to need the setting accommodation, and the sample included students who were English learners, had temporary conditions, or were new to the school and had not been identified with disabilities. Commenting that the sampled groups of students may not be matched and "may not possess the same learning characteristics," (p. 2772), the researchers stated, "Nonetheless, it is worth noticing that this study used the DIF detection method to match all test takers on the basis of their math or reading ability while comparing the group differences in item difficulty" (p. 2772). [Note: At this time, the identification of learning disabilities followed the IQ-achievement discrepancy model approach, established in 2001 by the Ontario Ministry of Education.]

The extant data set from 2005-2006 for the Junior (grade 6) Ontario provincial assessment (in Canada) was sampled for this secondary analysis and non-experimental study. The math test had 28 multiple choice items and the reading (comprehension) test had 32 multiple choice items. The specific content strands for each of the tests were detailed.

Students using the setting accommodation had lower test scores on average than students not using the setting accommodation. Students without disabilities had significantly higher scores in both math and reading than students with learning disabilities (LD). Students without disabilities using the setting accommodation were the lowest-scoring group in reading, below both groups of students with LD. In contrast, the accommodated students without disabilities scored higher in math than both groups of students with disabilities, yet lower than students without disabilities not using accommodations. Students with LD using the setting accommodations scored lower than students with LD not using the setting accommodations, in both reading and math. Non-accommodated students with LD evidenced lower item difficulty than students with LD using the setting accommodation. Individual item functioning varied for a few items between the two groups of students without disabilities in math and reading; however, there were no individual item effects for accommodated and non-accommodated students with LD in either math or reading. Overall, the researchers stated that the setting accommodation "does not appear substantially helpful for these test takers in allowing them to bypass their attention or learning difficulties in the current study" (p. 2784). They remarked that decisions about providing setting accommodations need to be made on an individual basis, and in combination with other accommodations as appropriate. The researchers concluded that the multilevel measurement modeling approach was useful in examining accommodations' effects, because the data could be analyzed for multiple predictors and interactions at the person-level and item-level. Limitations of the study were reported, and future research directions were suggested.