State education officials broke their own rules in scoring California's new achievement tests, counting so few examinations that the scores for hundreds of schools may be wildly inaccurate and the results in half the cases are less precise than was promised, a Times computer analysis shows.
Among the most extreme examples found in The Times study are a San Bernardino County middle school where only 1% of the math tests were counted, and two schools--Vicentia Elementary in Corona and Columbia Elementary in Tuolumne County--where results were based on exactly one student's work.
At Roosevelt Elementary in Indio, 169 students took the tests, but the state graded only four papers, then reported that everyone there scored at the lowest possible level, showing "little or no mathematical ability."
"I kept reading (the results) over and over again. . . . I kept trying to turn the bar graph upside-down. It really had me panicked," Principal Kennedy Rocker said. "It's not even a snapshot. It's a very blurry Polaroid with some Vaseline over the lens."
Short of time and money in the first year of the revolutionary California Learning Assessment System, which 1 million students took last spring at a cost of $15 million, education officials decided to score only a fraction of the tests at each school, promising that results would remain statistically sound.
But when told of The Times analysis, several independent statisticians agreed that although the scores in reading, writing and math may offer a general picture of California students as a whole, sampling error makes them too imprecise to be used for judging most districts or individual schools.
"It appears they lost control," said Lee Crombach, a retired Stanford professor of education who has studied testing for 60 years. "(Sampling) is certainly something that can be managed, but in the first run it can go wrong, and it apparently went wrong here."
State officials acknowledged dozens of sampling problems with the scoring system, and said last week that reporting results on schools where the most extreme snafus occurred was a serious error. They said corrections and apologies would go out to some schools this month, while a new group of students takes the tests with a state promise that this time many more exams will be scored.
Still, education officials strongly defended CLAS, a new approach to testing that evaluates students' thought processes, as well as their ability to derive correct answers, and measures performance against tough statewide standards.