Skip to Content

NCLB: Is it Working?


The ‘scientifically-based’ answer


 

By Alain Jehlen


No Child Left Behind, the biggest social engineering project of our time, put 50 million school children and their 3 million teachers under the gun. The law passed mainly because many people were convinced that low-income, minority students learn less than middle-class, White children because their teachers don’t try hard enough.

 

“The impetus for change built into NCLB was to effectively ‘shame’ schools into improvement,” wrote Susan Neuman, a former top Bush education official, after she left the government.

Now, seven years later, NCLB is up for renewal or change. Is it working? Or as President George W. Bush once asked, “Is our children learning?” We hear a lot about test-stressed students, curricula stripped down to make way for teaching to the test, and exasperated teachers leaving the profession.

But NCLB supporters say if students do better in reading and math, and if low-income, minority students close the achievement gaps, that’s worth the agony.

And we do hear that in many schools, teachers are getting out of their silos and working together to help all children achieve.

What’s more, scores on state tests are definitely climbing.

So—is that proof of success?

No, it isn’t, according to leaders in the science of testing. Scores always rise when you put high stakes on a particular test, whether or not students actually know more. This phenomenon even has a name: Campbell’s Law.

Harvard University Professor Daniel Koretz, a leading test researcher, explains it with an analogy to polling before an election. Pollsters can’t call every voter. Instead, they choose a small sample. Let’s say a campaign polled 1,000 likely voters and poured all their energy into winning over just those voters, ignoring everyone else. They would probably see encouraging gains among the 1,000 voters—and then lose the election by a landslide.

Koretz says a math test works the same way: No test can cover all the skills from every angle that students should master, so the test is just a small sample. If you focus on teaching kids to correctly answer problems that use a particular question format and only cover a narrow range of skills, students will do better and better—that is, until someone asks them questions in a different way, or measures a different set of skills from the larger curriculum.

Koretz carried out an ingenious demonstration of this phenomenon in the 1980s in a school district he had to agree not to name. The stakes on test scores in that district were “laughably low compared with today’s,” he says, but teachers did feel pressure to get scores up.

When the district switched to a new test, Koretz says, “scores dropped like a rock.” But over the next four years, they rose steadily.

Now comes the clever part: Koretz gave students the old test, the one that no longer carried high stakes so teachers didn’t prep students to take it. Their scores plummeted. His conclusion: Four years of rising scores did not reflect real achievement, just teaching to a new test.

Research on scores on high-stakes tests in Kentucky and Texas also showed Campbell’s Law in action.

So to see whether NCLB is really boosting achievement, we can’t rely on high-stakes state tests. We need to look at scores on a test for which students don’t get prepped.

Luckily, there is one: the National Assessment of Educational Progress (NAEP). It’s given to large, random samples of students periodically, but there are no scores for individual schools so nobody’s career is at stake.

Let’s look at two sets of NAEP scores, one for the era of NCLB (signed in 2002), and one for the period between 1971 and 1988. We’re printing just the eighth grade reading scores for Black and White students, but our Web site also links to math scores, other groups of students, and grades four and 12.

Success in the 70s and 80s

The chart for the 1970s and 80s shows the achievement gap cut in two. The gaps for other grade levels, for math, and for Hispanic students also shrank, most by about a third to a half.

 

What happened to make that possible? The War on Poverty? School desegregation? Shrinking class size? Head Start and other new school programs? Maybe all of the above.

One thing’s clear: This is what success looks like.

Now look at the chart showing recent trends. If you didn’t know about NCLB, you would never guess that something bold and drastic was taking place right now in our schools. Reading scores are fairly level. The achievement gap has shrunk a little. The news is slightly better in fourth grade, slightly worse in 12th.

Math scores are trending upward (as they did during the 70s) but they’re not rising faster than they were before NCLB, and math achievement gaps show little progress.

This is not what success looks like.

The bottom line: this strategy doesn’t work.

“Vilifying teachers and saying we are going to shame them was not the right approach,” says Susan Neuman, the former Bush official. She’s now joined a group supporting a broad effort to counter the devastating impact of poverty on children.

The No Child Left Behind law uses the phrase “scientifically-based” 115 times to emphasize that schools should use proven methods—not hunches or educated guesses or ideological beliefs.

A good idea. How about applying it to the federal law itself?


 

All the test scores in this article are from the web site of the National Assessment of Educational Progress, run by the U. S. Department of Education, which carries out periodic assessments for reading, math, and other subjects.

Here is more NAEP information on reading and math scores in recent years and in the period during the 1970s and 80s when America made significant progress in shrinking achievement gaps.

 

Recent period:

Students in grades four and eight were tested in 2007, and results are available as a pdf file at http://nces.ed.gov/nationsreportcard/pdf/main2007/2007496.pdf

We've broken out the relevant pages in the document for you below:

Fourth grade:

Overall scores: page 8
Racial and ethnic group scores: page 10
Black-White and Hispanic-White achievement gaps: page 11
Higher and lower-income achievement gaps: page 13

Eighth grade:

Overall scores: page 26
Racial and ethnic group scores: 28
Black-White and Hispanic-White achievement gaps: page 29
Higher and lower-income achievement gaps: page 31

 

The 12th grade age group was tested in 2005. The results are at http://nces.ed.gov/nationsreportcard/pdf/main2005/2007468.pdf

Relevant pages below: 

Overall scores: page 5
Black-White and Hispanic-White achievement gaps: page 6
There was no analysis by income level.

Earlier period:

NAEP runs a “long-term trend” series

nine, 13, and 17-year-olds:

The reading scores are at http://nces.ed.gov/nationsreportcard/pdf/main2005/2005464.pdf

Overall scores: page 10
Black-White and Hispanic-White achievement gaps: page 32-35

(No data on higher and lower-income group scores for the earlier period.)

 

MATH

Recent period:

Again the fourth and eighth grade students were tested in 2007. The results are at: http://nces.ed.gov/nationsreportcard/pdf/main2007/2007494.pdf

Fourth grade:

Overall scores: page 8
Racial and ethnic groups: page 10
Black-White and Hispanic-White achievement gaps: page 11
Higher and lower-income achievement gaps: page 13

Eighth grade:

overall scores: page 24
Racial and ethnic group scores: 26
Black-White and Hispanic-White achievement gaps: page 27
Higher and lower-income achievement gaps: page 29

12th grade:

A new 12th grade math test was introduced in 2005 with questions covering topics in math that are different from the range covered in earlier tests, so these NAEP scores can’t be used to judge recent changes in 12th grade math achievement scores.

Earlier period:

Math scores for nine, 13, and 17-year-olds are at http://nces.ed.gov/nationsreportcard/pdf/main2005/2005464.pdf

Overall scores: page 17
Black-White and Hispanic-White achievement gaps: pages 41-44
(No data on higher and lower-income group scores for the earlier period.)

Published in:

Published In

January, 2009


  • anc_dyn_linksOctober | November 2009
  • anc_dyn_linksAugust | September 2009
  • anc_dyn_linksMay 2009
  • anc_dyn_linksMarch | April 2009
  • anc_dyn_linksJanuary | February 2009

Advertisement

Advertisement