The temptation is great to look at high test scores as the ultimate indicator of a good school. Test scores are one of the few pieces of concrete evidence available to parents, and they appear to offer a quick and irrefutable summary of a school's academic track record.
But there are some important things test scores don't tell you.
They don't tell you what kind of students go to the school. Are many of them gifted? In special education? Still learning English? How many of those students are included in the testing?
The answers to those questions can make a significant difference in a school's test scores.
Test scores also don't tell you about the home environment of students. That's important, because test scores tend to reflect the socioeconomic status of students more than the abilities of a school's teachers.
Research shows that students from wealthier homes tend to score better than those from poorer homes. Experts say this is because better-off households tend to be headed by better-educated adults, who are more likely to stress the importance of education and to offer their children opportunities that give them a head start on academics.
So it's possible that a big-city school where half of the students are poor has great teachers or an academic program every bit as good as a wealthy suburban school. But that's unlikely to be reflected in test scores.
In an attempt to even the playing field, The Times used a common statistical technique called a regression analysis to take into account the impact of students' socioeconomic status on test scores.
The premise: A school with a large number of low-income students would have a much lower predicted score than a well-off school. If the poorer school's actual scores significantly exceed the predicted scores, this suggests something else may be at work: perhaps a strong principal, a rigorous academic program or teachers who are doing something special.
The analysis: Each school's third-grade test scores on the state-required Iowa Tests of Basic Skills (ITBS) were examined to see if they were higher or lower than could be predicted by sizing up the socioeconomic background of students at the school. The Times used a 2-year average of test scores from 2001 and 2002 to account for fluctuations in student populations that occur year to year.
The demographic information comes from looking at the number of students receiving free or reduced-price lunches at the school (a measure of student poverty). The Times also looked at how many students at each school have a computer at home, whether English is spoken at home and other questions students answer on a state questionnaire that accompanies the test. (Student names are deleted from records provided to The Times.)
The results: Of the 343 local schools examined, most came out close to where the statistical model predicted they would. A group of 87 schools showed significant differences. Their performance is denoted as "much higher," "higher," "lower" or "much lower" than would have been predicted.
Twenty-two area schools were not included in this analysis because they don't have a third grade, because there wasn't enough student data to make a comparative rating or because they offer a specialized program for gifted students.
This method has its own limitations. Some educational experts question the accuracy of the answers students provide on the state's questionnaire. They caution that the analysis may give a false appearance of precision. And the analysis can't account for the impact of high numbers of gifted, special-education or English-as-a-second-language students on a school's test scores.
Still, the analysis adds one more piece of the complex puzzle parents must assemble as they study schools.