Skip to content

Why It’s a Bad Idea to Release Teacher Ratings

October 22, 2010

by Norm Fruchter

The United Federation of Teachers has gone to court to block the NYC Department of Education’s (DOE) release of teacher performance ratings based on value-added measures of their students’ test scores. The DOE agreed to hold off on publishing the ratings until a court hearing on November 24th.

The New York City Department of Education (DOE) has a contract with the Wisconsin Center for Education Research to produce reports on value-added results based on the student test scores of more than 12,000 NYC teachers who teach ELA and Math in the fourth through eighth grades. The union, a partner in this research, had negotiated an agreement with the DOE not to make the results public. Instead, the results were to be shared with school principals and used in teacher evaluations and tenure decisions. When the DOE violated their agreement by deciding to make the results public and release them to the news media, the union decided to go to court to block that release. (When the Los Angeles Times published the value-added ratings of thousands of teachers in the Los Angeles Unified School District, one L.A. teacher committed suicide.)

Should such value-added teacher ratings be publicly released? I’d argue absolutely not, for several reasons. First, in NYC, these measures are based on the results of state standardized testing. As the New York State Education Department has recently demonstrated, that testing is so unreliable that the state had to completely rescale the exams correct deficiencies. Second, many teachers do not receive any value-added score because they do not teach students within the grade ranges and subject areas tested. Why single out only some teachers for public praise or condemnation?

Third, current value-added measures have an enormous range of uncertainty. Using the NYC model, NYU’s Sean Corcoran found that using several years of available data, a teacher’s performance could range from the 46th to the 80th percentile. Using only one year of data, a teacher’s average performance range extends from the 30th to the 91st percentile. This enormous variation is clearly of little use in rating and more importantly, in helping teachers improve their teaching.

Fourth, value-added measures have a very high degree of year-to-year variability – this year’s highly rated teacher performance may plummet on next year’s ratings. That variability suggests the fifth problem – students’ testing outcomes are responsive to many more factors than just a teacher’s instructional capacity. To take one current example, sudden family homelessness, an increasing result of our national recession, can cause students to change school, miss days of schooling, or come to school suffering from severe psychological stress. All these factors can easily contribute to severe reductions in student test performance.

Finally, if all these significant problems with current value-added testing could be resolved (and they may in the future), should we publish individual teachers’ ratings? I think not. Why transform what might someday become a useful tool for improving teachers’ practice into a ritual of public shaming?

Norm Fruchter is a senior policy analyst at the Annenberg Institute for School Reform

 

6 Comments leave one →
  1. Jessica permalink
    October 24, 2010 2:28 pm

    Another problem with value added scores is that teachers don’t always have control of their curriculum. If you were to give teachers the freedom to teach students as they saw fit, then you could hold them fully accountable for their results. But if you mandate certain programs, part of what the value-added scores reflect is the effectiveness of that program, not the effectiveness of the teacher.

    • October 25, 2010 1:02 pm

      Good comment; I agree. Another complexity I should have noted is that student performance is never the result of one teacher’s work, but of the ensemble of teachers in a school. If that ensemble works consistently at improving instruction and analyzing and responding to student academic needs, individual teachers’ classroom scores will benefit. The reverse, alas, is also true.

      – Norm F.

  2. October 25, 2010 11:28 pm

    The question of whether to publish even if the estimates are perfect is essential, as virtually all of the back-and-forth has focused on accuracy. Aside from the valid points about fairness and shaming, there is an additional, more practical reason to not publish even perfect estimates: because doing so will promote greater non-random assignment of students to teachers, and therefore increase the inaccuracy of the measures in the future.

    If you’re interested, we discussed this here:
    http://shankerblog.org/?p=1093

    Either way, thanks for the post.

    • October 26, 2010 2:58 am

      I agree. I think there is a fundamental differences between using value-added estimates, once they are far more accurate, as indicators of teacher effectiveness, to be shared with principal and teacher. I see far less reason to make them public, and lots of harm from that enterprise.

      Norm

  3. ronnie permalink
    October 27, 2010 12:12 pm

    OK, so let me get this straight. The UFT says that the value added rating are realiable enough to use for teacher evaulations and tenure decisions, but not realiable enough for parents to view. What ever happened to transparency? If the ratings are so unrealiable, then why use them for such important decisions? I think that if we are asking everyone who is apart our educational system for transparency and accountablity, I don’t understand why we want to exclude teachers. As a parent, I fell that if teachers are being paid with public dollars, than their evaulations should be made public.

Trackbacks

  1. Remainders: What Bill Gates learned from home-schooling | GothamSchools

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: