Study Finds Test Scores Up Since NCLB, But Cause Remains Unclear
It is being widely reported that a new study shows there have been improvements in math and reading test results since passage of the No Child Left Behind Act five years ago, but the report itself says the cause of the gains is not clear.
The independent nonprofit Center on Education Policy (CEP) released the report, Answering the Question that Matters Most: Has Student Achievement Increased Since No Child Left Behind?, which CEP described as the "most comprehensive and thorough study to date of the results of state tests administered as part of the landmark federal education law."
In an official statement, NEA President Reg Weaver said, "The report clearly indicates that given the current available data, 'an accurate and complete picture of NCLB is a moving target.' Essentially, the report reinforces that NCLB has done very little to improve accountability and not nearly enough to close the achievement gaps."
Weaver added, "The CEP report provides further evidence that measuring individual student progress through a single, one-size-fits-all, high-stakes test is not the answer and that NCLB is not living up to its original promises."
'There Should Be No Rush to Judgment…'
The CEP points out in its own news release that "the report notes that the gains cannot be attributed directly to No Child Left Behind, as considerable federal, state, and local reform efforts have all been underway prior to and since 2002."
"American educators and students were asked to raise academic achievement, and they have done so," said Jack Jennings, president & CEO of the independent, private nonprofit Center on Education Policy. "The weight of evidence indicates that state test scores in reading and mathematics have increased overall since No Child Left Behind was enacted. However, there should be no rush to judgment as there may be many factors contributing to the increased achievement."
The difficulty of analyzing student achievement trends was underscored by the release within days of the CEP report of a government report confirming that there are substantial differences from state to state in the tests used for NCLB purposes. An Associated Press report said the Department of Education study revealed "huge differences in where states set their benchmarks."
The CEP report says the number of states in which achievement gaps among groups of students have narrowed far exceeds the number of states in which gaps widened since 2002.
Analysis Limited to 'Comparable Data'
CEP said the study uses verified data from all 50 states and explores student achievement trends both before and after passage of NCLB, but also "limits its analysis to testing data that is comparable from year-to-year, eliminating data in grades and subjects where states have made significant changes to their assessment systems."
According to CEP, the report also uses two methods for evaluating achievement, including the percentage of students considered proficient -- the primary measure of adequate yearly progress under NCLB -- and effect sizes, a measure based on average test scores that addresses some of the limitations of the percentage proficient measure. Using either method, the report finds that the number of states showing achievement gains since 2002 is far greater than the number showing a decline. In addition, yearly gains are generally greater after 2002 in states where comparable data prior to 2002 was available.
The CEP researchers found that test score gains were greater in math than in reading and greater at the elementary level than at the middle and high school levels. They also reported that achievement gaps between white and minority students appears to be narrowing somewhat, but that the gaps "remain substantial."
Critics Find Fault with Study's Methodology
The limitations on usable data led some experts to criticize the report. The Washington Post reported:
Some scholars criticized the report's methodology. Bruce Fuller, a professor of education and public policy at the University of California at Berkeley, said it made little sense to draw conclusions when so few states have adequate data. He also said the researchers overstated small gains and did not adequately address states that he said have been dumbing down standards.
"These big-hearted analysts, to amend an adage, look at a glass that's nine-tenths' empty and celebrate that it's one-tenth full," Fuller wrote in an e-mail.
Fuller's criticism is based in part on the small number of states the CEP researchers were able to study in some categories. For example, the CEP report finds that average yearly gains in test scores were greater after 2002 than before, but the finding applies to just nine of 13 states that have enough good data to determine trends both before and after NCLB.
The report admits that " because such a limited number of states had both pre-and post-NCLB data, it is difficult to say whether this trend is truly representative of the national picture."
The CEP's news release and the full report can be accessed here at the CEP Web site .