Skip to Content

Letter on Multiple Measures from the Expert Panel on Assessment Convened by the Forum on Educational Accountability

Letter on Multiple Measures from the Expert Panel on Assessment Convened by the Forum on Educational Accountability


August 2, 2007

The Honorable George Miller
Chair, Committee on Education
2205 Rayburn House Office Building
Washington, DC 20515

The Honorable Howard P. McKeon
Ranking Member, Committee on Education
2351 Rayburn House Office Building
Washington, DC 20515

The Honorable Edward Kennedy
Chair, Senate Committee on Health, Education, Labor and Pensions
317 Russell Senate Office Building
Washington, DC 20510

The Honorable Michael Enzi
Ranking Member, Senate HELP Committee
379A Russell Senate Office Building
Washington, DC 20510

Dear Sirs:

As members of the Expert Panel on Assessment convened by the Forum on Educational Accountability, and authors of Assessment and Accountability for Improving Schools and Learning, we are writing to strongly affirm our support for the use of multiple sources of evidence of student learning and school attainments for accountability purposes. We hope that you will carefully consider the recommendations contained in our report as you reauthorize the Elementary and Secondary Education Act (ESEA)/No Child Left Behind (NCLB) Act. These recommendations represent solid research evidence and policy that are consistent with the body of work on educational assessment and accountability.

The use of multiple measures and sources of evidence accords with the principles espoused in the American Educational Research Association, American Psychological Association, and National Council on Measurement's Standards for Educational and Psychological Testing; as well as the reports on assessment issued by the National Academy of Sciences and the National Research Council, including High Stakes: Testing for Tracking, Promotion, and Graduation. Those who advocate use of a single measure or source of evidence do so in the face of compelling evidence and arguments against the validity of engaging in such a practice. ESEA/NCLB should not directly or inadvertently support a practice that violates appropriate uses of assessment data, most especially when it comes to high stakes accountability.

All assessments used for accountability purposes within a multiple measures approach should meet rigorous criteria so that inferences based on multiple assessments' results are valid for making accountability decisions. Therefore, the assessments would be based on state-approved standards. All students would be included in this system, groups of students would not be held to different standards (except for the instances already accepted under NCLB), and all students would have the opportunity to show their learning in a variety of ways over time.

Below, we sum up some of the key points we make regarding multiple forms of evidence, excerpted from the fuller set of principles.

Principle I addresses general equity and capacity building for student learning, framing the ensuing principles.  Our second principle calls for the inclusion of local assessments in comprehensive state assessment systems in order to better serve the learning needs of all students, and to provide richer evidence for accountability decisions:

Principle II: Comprehensive State and Local Assessment Systems

Construct comprehensive and coherent systems of state and local assessments of student learning that work together to support instruction, educational improvement and accountability.

  1. Provide incentives for states and districts to develop comprehensive and coherent assessment systems that inform instruction and decision-making in ways that state tests alone cannot and do not. Coherent and comprehensive assessment systems provide evidence of student and school performance in relation to rich and challenging educational goals, using multiple indicators of student learning from a variety of sources at multiple points in time.
  2. Provide states incentives and supports to include high quality local assessment systems in meeting ESEA's accountability requirements, alone or by augmenting state assessments. Fund pilot projects in which interested states demonstrate how they can meet ESEA's accountability requirements through standards-based, locally-developed assessments of students' learning or by integrating local assessments with state assessments.

Principle III addresses problems resulting from the lack of valid and reliable assessments for students with disabilities and for English Language Learners. Given the diversity of these subgroups (e.g., in the nature and severity of disabilities, the variety of languages spoken by English Language Learners, differences in students' proficiency in their native language and English), it is unrealistic and impractical to develop many different standardized assessments each tailored for a small number of students. Given that, Principle III reaffirms support for the use of local measures within comprehensive systems and the use of multiple measures for a more comprehensive picture of what these students know and are able to do.

We also explicitly support the use of multiple forms of assessment to evaluate schools and districts:

Principle IV: Fair Appraisal of Academic Performance

Use multiple sources of evidence to describe and interpret school and district performance fairly, based on a balance of progress toward and success in meeting student academic learning targets.

  1. Encourage states and districts to use multiple sources of evidence drawn from their comprehensive and coherent systems of classroom-, school- and district-based assessments to summarize and appraise student performance.

Multiple sources of evidence include various types of valid and reliable assessment of student learning collected over time. These can be local or a mix of state and local assessments, in line with Principle II.

In making accountability decisions, we recommend the use of multiple indicators, not limited to state and local assessments in reading and mathematics:

Principle V: Fair Accountability Decisions

Improve the validity and reliability of criteria used to classify the performance of schools and districts to ensure fair evaluations and to minimize bias in accountability decisions.

  1. Encourage states to include all subjects - not just reading, math and science - in their comprehensive assessment systems, but use compensatory processes to ensure that the inclusion of more subjects does not become another means for schools and districts to fail accountability requirements.
  2. Encourage states and districts to use multiple sources of evidence drawn from their comprehensive and coherent assessment systems to make accountability decisions about the quality of school and district performance and determine which schools and districts need what forms of assistance.
  3. Retain the ESEA requirement for gathering and reporting disaggregated information by subgroups based on the comprehensive assessment system.

When multiple measures are used for accountability, the various indicators should be given sufficient weight as to ensure they are meaningful. In the text, we also support the use of other indicators, such as graduation rates, in addition to assessment information.

And Principle VI, on interventions, adds:

1. Encourage states and districts to use multiple sources of evidence from state and local assessments and other forms of evidence to inform actions such as interventions and technical assistance.

In short, we believe that the use of multiple forms of evidence, including varying forms of assessment and other indicators of school quality and attainments, should be actively supported in the next law. This will require not only explicit language, but also a significant financial commitment.

We are aware that the use of multiple sources of evidence has been challenged by those who advocate against altering the current assessment and accountability requirements. We disagree for two basic reasons.

First, the current requirements are causing serious damage in too many schools as educators are pressured to narrow the curriculum to the tested subjects, then narrow instruction in those subjects, in an effort to avoid sanctions under the law. While we believe the Adequate Yearly Progress (AYP) structure and the sanctions approach should be changed, we conclude that a significant portion of the solution to these problems is to use multiple measures.

Second, multiple sources of evidence neither leads to different sorts of assessments for different students nor to ignoring a priority commitment to reading and math. All students should have access to multiple means to demonstrate their learning. Only a very limited number of students, as already recognized in law, should be held to different expectations. A weighted system can retain the priority focus on reading and math while giving enough emphasis to other factors and subjects so as to avoid unfairly limiting curricular opportunities or undermining relations among teachers and students or damaging the school climate.

Our report contains recommendations and discussion on many components of assessment and accountability that are relevant to the reauthorization. We hope that you, other members of the Committees, and your staffs, will use our recommendations as you craft legislation.

Thank you for your consideration. We would be pleased to further discuss this issue with you. You can reach us through Monty Neill, Chair of the Forum on Educational Accountability.

Sincerely,

Jamal Abedi, Professor, Graduate School of Education of University of California, Davis; research partner, Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Pete Goldschmidt, Assistant Professor, College of Education, California State University Northridge; Senior Researcher, Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Brian Gong, Executive Director, National Center for the Improvement of Educational Assessment, Inc.

Margo Gottlieb, Lead Developer for World-Class Instructional Design and Assessment (WIDA), Wisconsin Center for Education Research, Madison; Director, Assessment and Evaluation, Illinois Resource Center.

Alba A. Ortiz, Professor, Department of Special Education, and Director, Office of Bilingual Education, College of Education, University of Texas at Austin.

Pedro Pedraza, educational researcher, Centro de Estudios Puertorriquenos, Hunter College, City Univerity of New York; Director, National Latino/Education Research Agenda.

James Pellegrino, Distinguished Professor of Psychology and Education, and Co-Director, Learning Sciences Research Institute, University of Illinois at Chicago.

Pat Roschewski, Director of Statewide Assessment in Nebraska

Jim Stack, former Director of Achievement Assessments, San Francisco Unified School District.
 
*Institutions listed for identification purposes only.
cc:  Members, House Committee on Education and Labor and Senate Committee on Health, Education, Labor and Pensions


RELATED LINKS