Judges at the California State Fair wine competition scored poorly at giving the same wine an identical rating when they tasted it multiple times in a blind tasting.
That was the conclusion of a four-year study of judging decisions at the California State Fair Wine Competition by retired Humboldt State professor Robert Hodgson.
"Consumers should have a healthy skepticism about the medals awarded to wines from the various competitions," he said.
Hodgson's findings have prompted state fair officials to consider making changes in the way they operate future wine competitions.
In a study published Wednesday by the Journal of Wine Economics, Hodgson wrote that only 10% of the judges were able to consistently give the same rating, or something very close, to the identical wine sampled multiple times in a large blind tasting.
At the opposite end, another 10% of the judges gave the same wine far different ratings, ranging from worthy of a gold medal to deserving of no medal at all on successive tastings. The remaining 80% of the judges also varied in their ratings, but by a narrower range.
State fair officials, who cooperated with the study, said Hodgson's findings provided valuable information that could be used to improve wine competitions. The next state wine judging is set for June 10-12.
"We want to do whatever we can to smell and taste wines to get the best results," said G.M. "Pooch" Pucilowski, a professional wine educator and chief judge for the state fair.
The competition plans to reduce the number of wines sampled per day -- which can approach 150 or more -- to as few as 75 to help judges avoid the sensory fatigue that can cloud their ratings.
The fair is also using the data "to weed out" judges who year after year are unable to rate different samples of an identical wine consistently, Pucilowski said.
Finding ways to evaluate the skills and consistency of judges is an important issue for wine competitions, which often draw from the same small pool of industry members and aficionados for their rating panels. It's not unusual for judges to work as many as six different competitions annually.
Wine shoppers such as Glendora urologist Dr. Charles Metzger rarely look for wines that are medal winners when deciding what to purchase.
"I mostly look to what other people recommend, and then I look at the 100-point ratings from publications like the Wine Spectator," Metzger said.
