With the recent unveiling of The Times' teacher and school "effectiveness" database, teachers and parents have asked us what to make of this information. Here's our advice.
I try my best to be an excellent teacher, and I'm always trying to improve. I really thought I was doing a good job. But The Times gave me an "average" value-added rating. Should I change how I teach now that I know I'm just "average"?
No. You might actually be an excellent teacher — even just based on this limited measure of improving students' math and reading California Standards Test (CST) scores. A recent study by the U.S. Department of Education indicates that value-added scores using three years of data (the typical amount used in practice) can be so unreliable that there may be about a 25% chance that your score put you in the wrong category. (Teachers end up in The Times database when they've taught at least 60 students, which at the elementary level represents about three years of data.) In other words, if The Times rated you as average, you could be great or you could be terrible.
My principal assigns me to teach some of the most challenging kids at my school. I think that's why I got rated as less effective than other teachers. But The Times says that my value-added score generally won't be affected "by low-performing students, English-language students or other students with challenges." Who's right?
You're right. The kinds of students in your class do affect your score. See one relevant study here. The Times' own technical report shows it too — just look at the estimates of the effects of "averaged lagged test score" (a measure of how well the students in your class were doing before they got to you) in Table 6 and the proportion of gifted, English-language learners or African American students in each classroom in Table 7. Each effect looks small, but each could affect your score. And as you know, none of these variables completely captures the unique mix of students (and potential obstacles to good teaching) that you get each year.
I teach in a school in a well-off neighborhood where parents send their kids to expensive summer camps and hire tutors. The Times labeled me as "most effective," but I suspect that some of that effectiveness results from kids' outside activities. Am I wrong?
Not necessarily. One problem with The Times' value-added scores is that the time between the tests includes summer vacation. It's well known that kids from disadvantaged backgrounds fall behind their more advantaged counterparts in reading during the summer. So poor kids who had the same scores as rich kids in the spring will probably be further behind when they go back to school in the fall. The poor kids' teachers will need to produce larger gains than the rich kids' teachers to earn the same value-added score. And, yes, anything that parents do during the school year to improve kids' reading and math skills (hiring tutors, helping with homework and so on) gives teachers more value-added credit than they deserve.
Students in my classroom go to other teachers for their math instruction. How do I know that The Times is calculating my math value-added score based on the students I actually teach? What if my scores are based on misinformation?
The data set undoubtedly contains some errors. The Times analyst also made specific decisions about which teachers to exclude and how to attribute kids' learning gains to teachers when students switch classes for particular subjects. Teachers have a right to know more about those decisions and how they affect the scores. Now that The Times has publicly labeled teachers with these statistical constructions in the interest of transparency, it should release the underlying data and programming code so teachers and researchers can explore the sensitivity of the results to different data management and analytical decisions. Both fairness and science demand that this research be replicated. You should too.
I always try to do what's best for my children. I see from The Times' analysis that a couple of the teachers in my child's grade are "most effective." Should I try to get my child into one of those classes?