How the Los Angeles Times used junk science to malign an entire city of teachers.
By Michael D. Simpson, NEA Office of General Counsel
L.A. By now, most NEA members have heard about the “study” of teacher effectiveness that the L.A. Times published last August. The once-respected paper obtained the tests scores of Los Angeles elementary students over a seven-year period and asked an “expert” economist from the RAND Corp. to assess teacher effectiveness using the “value-added model” or VAM.
VAM is based on the logical-sounding theory that one can judge how good a teacher is by measuring how much progress his/her students make on standardized tests over the course of a school year. The Times then published an on-line, searchable database of the names of some 6,000 LA teachers accompanied by their ratings from “least effective” to “most effective” in math and English.
There’s only one problem: Every respected, independent testing expert in the country agrees that VAM is not a valid or reliable measure for making high-stakes decisions about teacher effectiveness. It is junk science. A sampling:
- In October 2009, the prestigious Board on Testing Assessment wrote an open letter to ED Secretary Arne Duncan stating: “Too little research has been done on [VAM’s] validity to base high stakes decisions about teachers on them. A student’s scores may be affected by many factors other than a teacher….”
- In July 2010, ED’s own Institute of Education Sciences concluded in a 36-page analysis of VAM research data that “more than 90 percent of the variation in student gain scores is due to the variation in student-level factors that are not under the control of the teacher.”
- In August 2010, a panel of experts assembled by the Economic Policy Institute (EPI) warned in a report that it would be “unwise” to give substantial weight to VAM scores in measuring teacher effectiveness.
- And researchers for RAND concluded, “The research base is currently insufficient to support the use of VAM for high-stakes decisions about individual teachers.”
In a sharp rebuke to the Times, Rutgers Professor Bruce Baker analyzed the “study” and concluded that its ratings of teachers are racially biased. Calling the flawed study “offensive,” Baker wrote that black teachers have the lowest VAM scores and that Asian teachers scored higher than white teachers.
Which begs the question: would it be legal for school districts to fire teachers based on student test scores using VAM?
That question hasn’t reached the courts…yet. But there’s a good chance the teachers might win. First, for tenured teachers, the district has to prove “just cause” for dismissal. And if the termination decision is based on junk science—a disproven and discredited theory about VAM—then that ain’t just cause.
Second, if as Baker suggests, the use of VAM has a racially-disparate impact on teachers, then the district will have to show that the use of VAM has been “validated” for the purpose for which it is used. Again, no such study exists.
In its report, the EPI experts raised two serious concerns about the use of VAM. First, it will “likely” lead to “expensive … litigation in which experts will be called to testify, making the district unlikely to prevail.” You can take that to the bank.
Second, it’s likely to “demoralize teachers,” which tragically has been proven true in Los Angeles.
Less than a month after the Times published its database, fifth-grade teacher and CTA/NEA member Rigoberto Ruelas Jr. committed suicide. The 14-year veteran educator jumped off a bridge in a nearby national park.
The Times “study” had labeled him “less effective.” Did the Times cause his death? We’ll never know; he left no note. But according to numerous media reports, friends, family, and co-workers described him as “upset” and “very concerned” over the rating, and he had recently lost a lot of weight.
On the day of his memorial service, the school district confirmed that Ruelas had received a “great performance rating” the previous July.
In a press release, the Times expressed its “sympathy” to the family. But, at this writing, his name and “less effective” rating still appear in the Times on-line database.