News: Interview
Lorrie Shepard
How to Fight A 'Death Star'

A leader among testing experts speaks out on high-stakes tests and what to do about them.

Lorrie Shepard, professor of research methodology at the University of Colorado at Boulder, is a national authority on standardized tests and other forms of assessment. Last year, she was president of the American Educational Research Associa-tion, the leading school research organization. She spoke recently with NEA Today's Alain Jehlen.

Why are standardized tests getting so much attention these days?
Politicians are paying attention to education and relying on testing to make things change. The word "accountability" has gotten a different meaning. In the past, it could mean "honest reporting." Today, the idea seems to be that you can force change using rewards and punishments for test scores.

Why do passing rates vary so much state to state?
Some states have set standards well above what was the norm, but others have not.

Texas has reading standards at the 25th percentile, so, no wonder, the first year the test was used, 75 percent passed.

Here in Colorado, there were not going to be any individual student consequences, so teacher groups set ambitious standards. Some of these teachers feel burned now that the schools are being bashed with the test results.

Some people say that what was the average score should be the goal for all children. That's admirable, but it's a tall order. You can't just announce that everyone will be above average and start beating up on schools for not reaching that goal.

Even within states, there are differences in the standards. Colorado's reading standard was set right at the median, while the writing standard was set at the 70th percentile.

When half of the students tested proficient in reading, but only 30 percent in writing, people said, "We're not teaching writing." But that was predictable because the writing standard was far above the reading standard. These two passing rates don't have any comparative meaning, but the media thinks they do because they're both called "proficiency."

How should these tests be used?
I have always been strongly in favor of large-scale assessment programs like the National Assessment of Educational Progress to track trends over time. For example, during desegregation in the South, you could see that the gap between Blacks and whites was really closing.

But they shouldn't be used to test every student, every year, in a program that's driven from the statehouse.

What problems do you see with the way the tests are now being used?
Politicians believe test scores are a measure of the quality of education, rather than of resources and the students' socio-economic characteristics. You see them rewarding the Beverly Hills school districts, even though, when you mention it, they say, "Oh, yes, we know that these kids started out ahead."

Another problem is that an external test can't be used to guide instruction because it's not there when you need it.

A test that's useful for instruction has to be right in the middle of the learning process. Good teachers who open a standardized test booklet would know in advance which problems their students can do and which they can't.

The most important thing we should be doing is working on forms of classroom assessment--that's what I really care about. That means not just picking right answers, but open-ended tasks where students show what they understand.

But external, high-stakes tests are driving out quality classroom assessment. I've used Darth Vader and the Death Star to convey the idea that these tests are looming over and threatening our classroom practice.

What are test experts doing to be heard on this?
Last July, the American Educational Research Association adopted a position paper setting out 12 conditions for sound and fair high-stakes testing programs.

The first is, "Decisions that affect individual students' life chances or educational opportunities should not be made on the basis of test scores alone." And the second, paraphrasing, is that penalties shouldn't be imposed on students until there is assurance of an opportunity to learn what is required.

In the past, courts have turned to AERA standards. When there are challenges to high-stakes tests, these standards may have some legal weight.

What would you do if you were a teacher and your state imposed a test for your students?
First, I would make sure my kids were learning the content. Then, I might spend one day with them looking at the test format, so they wouldn't be caught off guard or surprised.

But I would not teach to the test in a narrow way. Our research shows that if you focus on teaching to one test format, and then you give a different test--not a harder test, just a different format--kids lose about 20 to 30 percent.

What if you felt the test was unfairly difficult?
The school can help parents understand the test is unfairly difficult by showing samples of student work for each proficiency level. If you can't release the actual student response to a test item, you can still show other samples of what students at each level can do. So I would show that work.

And not just to parents. We also need to connect with the broader community. That may mean putting sample student work in newspapers--not just prize-winning work, but essays and sample problems at several levels of proficiency.

I did something like that with a school board near Boulder. Board members were upset by reports of students failing, so we gave them student papers at each level. The level called "partially proficient" counted as failing, but those kids wrote wonderful stories! The board members certainly didn't think those kids were illiterate, and they were up in arms.

Another idea--the school boards association sponsored this with a newspaper--was "Take The Test Day," to let people see what it was like. Our governor refused to take the test.

We need to speak to the public about these things.

Resources

• The AERA Position Statement Concerning High-Stakes Testing in Pre-K-12 Education, www.aera.net/about/policy/stakes.htm.

• Standards for Educational and Psychological Testing, jointly developed by the AERA, the American Psychological Association, and the National Council on Measurement in Education, which are the three major American organizations of experts in testing. Order forms are available at www.aera.net/products/standards.htm. Or call (800) 628-4094. $31.95. (The AERA's new position paper is based on this document.)

• The Role of Assessment in a Learning Culture, by Lorrie Shepard, in ER Online. Go to http://www.aera.net/pubs/er/index.htm and look in the October 2000 archive.

• E-mail Lorrie Shepard: Lorrie. Shepard@Colorado.edu.