"I found that some of the Los Angeles County Fair winners that I tried were just not that good," he said.
But Metzger's wife, Barbara, has had good luck buying wines that have won awards at competitions, he said.
Advertisement
Wine judges are rather unsteady, study finds
Only 10% in a four-year study of California State Fair judging were able to consistently give the same rating, or something close, to the same wine sampled multiple times in a large blind tasting.
Advertisement
"The medal winners tend to be less expensive than the $40, $60, $80 wines you see in Wine Spectator," Metzger said.
Hodgson said he doesn't put any more trust in the 100-point-scale ratings of wines from magazines and newsletters than he does in medal winners.
"Consumers need to gain more self-confidence in their own opinions and tastes rather than listen to what other people think wine should be like," Hodgson said.
Last year, 649 wineries entered 2,917 California wines into the annual state fair contest, the oldest wine competition in America. More than half, or 1,587 wines, won awards.
Hodgson, an oceanography professor who also taught statistics at the university, owns the small Fieldbrook Winery just north of Eureka, in Humboldt County. He said he designed the study because he didn't understand why "we would have wines that we sent off and would get gold medals in some competitions and in others would get poop. It seemed like a gold medal was just a matter of luck."
Hodgson said he learned that the judging business is characterized by inconsistent decision-making by judges and wide variability among competitions.
"Wine judges in the setting of a competition must make about a hundred decisions a day. It is in this environment where I think their ability is taxed beyond a reasonable level," Hodgson said.
He also discovered that a "super judge" who is consistent in his or her ratings one year does not maintain that superiority the next year.
"This emphasizes the chance argument in placing awards," Hodgson said.
The findings of the study create a conundrum for Hodgson and his small winery.
"I use gold medals to sell my wine," he said, "and now I have written this paper saying the wine competition system that awards those medals isn't perfect."
--
jerry.hirsch@latimes.com
