NEA Recommends New Approach to Assessing Students With Disabilities
Dr. Patricia K. Ralabate
National Education Association
February 4, 2009
Before the National Assessment Governing Board
Good morning. My name is Dr. Patricia Ralabate. I am here today representing the National Education Association (NEA), the nation's largest professional employee organization. Our 3.2 million members work at every level of education-from pre-school to university graduate programs. Our membership includes general and special educators, university and college educators, specialized instructional personnel and education support professionals in every state and in more than 14,000 communities across the United States.
As the voice of education professionals, we are committed to advancing the cause of public education. We believe every child deserves a great public school that prepares him or her to succeed in a diverse and interdependent world. NEA fully supports the inclusion of students with disabilities in large-scale assessment and accountability systems. The challenge is how to do this in a way that validly and fairly represents what they have learned.
This brings me to today’s issue, the policy options regarding the participation of students with disabilities in NAEP being considered by the National Assessment Governing Board. We appreciate the opportunity to offer comments on this important topic.
As you know, the Board has been wrestling with this issue for nearly a decade. As we all know, there is no simple, easy fix. However, since NAEP is currently used as a key measure of state performance, it is critical that official reporting of NAEP scores represents the full range of our diverse student population, including the performance of students with disabilities. How to do this in an inclusive and reasonable way is a longstanding dilemma.
On behalf of the NEA, I submit the following for your consideration:
(1) Sampling of the students with disabilities by categories is not a reasonable option.
It’s important to keep in mind that the “students with disabilities” subgroup is not a homogeneous one. There are significant differences in students’ means of expression, levels of performance, and ways of engaging in learning and assessment situations. In addition, how students with disabilities are identified differs across state and even district lines. Therefore, it is inappropriate to simply use a sampling of the 13 disability “categories” outlined in the Individuals with Disabilities Education Act (IDEA) as a means of representing students with disabilities.
(2) A decision-making framework based on rules for exclusion is unacceptable.
Approaching this dilemma by focusing on whom to exclude will lead to discriminatory decision-making. For example, the use of the “decision tree” -- designed by the Board in 2004 as a positive effort to allow state and local officials to determine which students should participate with or without accommodations -- has instead led to unacceptable disparities between state and local exclusion rates. These wide variations undermine the credibility of NAEP results and make it extremely difficult to make valid cross-state comparisons.
Given the heterogeneous nature of students with disabilities, defining a “model exclusion rate” may be impossible. What is a reasonable exclusion rate? Who should be excluded? Estimations and calculated formula can never truly predict how children will perform if they are provided with appropriate supports. Rather than concentrating on “who” is excluded, we need to focus on “how” students with disabilities participate.
(3) Public reporting should treat all scores and all populations in the same way.
Disaggregated reporting of assessment results is an accepted practice because it helps to focus positive attention on those groups of students who may need more assistance. However, weighted or calculated estimations for specific populations, such as students with disabilities, can be discriminatory practices that lead to negative stereotyping and unfair characterizations. We’ve all seen articles in the media about how a certain school missed AYP because the students with disabilities or English language learner subgroup didn’t meet proficiency under the ESEA/NCLB accountability requirements. Certainly, we don’t want public reporting of NAEP scores to be handled in the same way.
In addition, publicizing which states do or do not meet a “model exclusion rate” is a negative approach. It requires an extensive discussion about why differences in populations occur and the effect of exclusion rates on potential state performance. The meat of these statistical effects is likely to get lost in translation and all that will remain are hyped headlines focused on the concept of exclusion or poor performance by certain groups that affect the ranking of other groups.
(4) Increasing the number of approved accommodations will improve participation.
According to a recent report, many students with disabilities were excluded from participating in NAEP because the accommodations recommended in their Individualized Education Programs (IEPs) were not available (Stancavage, Makris & Rice, 2007).
As you know, local and state assessment and accommodation policies vary substantially. An accommodation that is permitted in one state may not permitted in another. A recent study by the National Center on Education Outcomes found that changes in timing or scheduling (e.g., extended time, multiple testing sessions, separately timed test sections) is the most common type of accommodation used in state standardized assessments (Thurlow, Quenemoen, Thompson & Lehr, 2001). NAEP allows for most of these timing/scheduling accommodations except for separating sections of the test into different timed sessions administered over a series of days.
Adaptations in how the assessment is presented (e.g., computerized administration, reading sections aloud, computerized text-to-speech, using audiocassettes) were the second most frequent type of accommodation used for state assessments. Frankly, few of these presentation accommodations are allowable under NAEP policies.
Because there is no set of nationally approved accommodations and state-approved accommodations vary significantly, we recommend that a stakeholder group be convened to develop a consensus around what accommodations should be allowed for NAEP assessments. This group should include general and special education classroom teachers, state assessment policymakers, assessment experts who are knowledgeable about accommodations for students with disabilities, and representatives of disability advocacy organizations. By expanding the availability of approved accommodations, more students with disabilities will be able to participate.
(5) Use principles of universal design to develop future modifications to NAEP.
Universal design for learning (UDL) is a framework for making curriculum, instruction, teaching materials, and assessments accessible to all learners, including students with disabilities and English language learners. According to ESEA/NCLB regulations, universally designed assessments should "be designed to be valid and accessible with respect to the widest possible range of students, including students with disabilities and students with limited English proficiency" (NCLB Regulation (July 5, 2002), Section 200.2(b)(2)).
We recommend that the Board utilize the principles of universal design to develop future modifications to NAEP. Universal design principles call on test designers to develop test items that minimize the effects of extraneous factors (e.g., avoid unnecessary use of graphics that cannot be presented in Braille, use font size and white space appropriate for clarity and focus, avoid unnecessary linguistic complexity when it is not being assessed) that can interfere in the ability of the student to comprehend the question or respond with a correct answer. The Center for Applied Special Technology (CAST) (http://www.cast.org/) is an excellent resource on this topic.
(6) Develop an alternate assessment form or modified NAEP that uses computer-adaptive techniques.
To comply with ESEA/NCLB, most states now offer an alternate assessment based on alternate achievement standards. Some states also offer an alternate assessment based on grade-level achievement standards for students with disabilities. Yet no such assessment is available to students who participate in NAEP. Developing an alternate assessment that uses computer-adaptive techniques can provide students with sensory disabilities, such as vision or hearing impairments or students with moderate to severe disabilities access to the assessment that the paper/pencil format does not allow.
Computer-based testing allows for accommodations to be standardized and built into the administration protocol for the assessment. For instance, computer-based testing permits choice of presentation mode, such as using larger print size or language translation options. And, it eliminates the possibility of human fatigue or human error which can occur when directions or test sections are read aloud by on site test examiners.
In summary, we encourage the Board to explore new options for NAEP that will allow the greatest number of students, including students with disabilities, the opportunity to participate in this important measure of state academic achievement. Please be inclusive in your thinking.
Thank you again for your time and consideration.
Contact information:Dr. Patricia K. Ralabate, Senior Policy Analyst
Education Policy and Practice Department
National Education Association
1201 16th Street, NW
Washington, DC 20036
Stancavage, F., Makris, F., & Rice, M. (2007). SD/LEP inclusions/exclusions in NAEP: An investigation of factors affecting SD/LEP inclusions/exclusions in NAEP. Washington, DC: American Institutes for Research. Retrieved February 3, 2007 from http://www.air.org/publications/documents/NAEP_inclusion.pdf.
Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for everyone! (Number #14). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved February 3, 2009 from http://cehd.umn.edu/NCEO/OnlinePubs/Policy14.htm.
Thompson, S. J., Thurlow, M. L., Quenemoen, R. F., & Lehr, C. A. (2002). Access to computer-based testing for students with disabilities. (Synthesis report #45). Minneapolis, MN: University of Minneapolis, National Center on Education Outcomes. Retrieved February 3, 2009 from http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis45.html.
Thurlow, M., Quenemoen, R., Thompson, S., & Lehr, C. (2001). Principles and characteristics of inclusive assessment and accountability systems. (Synthesis report #40). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved February 3, 2009 from http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis40.html.
Zenisky, A.L., & Sireci, S.G. (2007). A summary of the research on the effects of test accommodations: 2005-2006. (Technical report #47). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved February 3, 2009 from http://cehd.umn.edu/NCEO/OnlinePubs/Tech47/default.html.