Shoppers at a supermarket in Carlsbad line up to stock up on ice during a massive… (Don Bartletti / Los Angeles…)
SAN DIEGO — Poor planning, failure to share critical information and a series of human errors led to the massive blackout in September that plunged a swath of Southern California, Arizona and Baja California into darkness, according to a report issued Tuesday by two energy agencies.
The 150-page report, produced after an eight-month study by the Federal Energy Regulatory Commission and the North American Electric Reliability Corp., portrays the complex electrical grid as vulnerable to a single small-sized failure — in this case, a transmission line in Arizona that went down and triggered a "cascading and uncontrolled" blackout that left 2.7 million customers in the dark.
The report calls for sweeping improvement in how utility companies, regulators and coordinating agencies share real-time information and how they plan for emergencies so that a "single point of failure" cannot result in massive failure.
"There is no limit to how much improvement in communication and coordination should be made," Dave Nevius, senior vice president of the North American Electric Reliability Corp., said at a Washington news conference.
The outage hit on the afternoon of Sept. 8 and lasted in many areas until early morning Sept. 9. In Southern California, customers in Imperial, San Diego and parts of Orange and Riverside counties were affected.
When the Hassayampa-North Gila transmission line, operated by Arizona Public Service, failed during maintenance and a day of heavy usage, it overloaded adjacent systems, starting with those operated by the Imperial Irrigation District, over the California border. A worker at Arizona Public Service skipped a critical step in the safety procedure that would have stopped the spread.
Instead, the overload spread west within minutes, including to the San Onofre nuclear plant, which stopped providing power to more than 1 million customers in San Diego County. Without information from neighboring agencies about what was happening, some utility companies were clueless as residential and business customers called in to register surprise, panic and anger.
The companies lacked "situational awareness" as events unfolded, according to the report.
"I think FERC is on the right track, in terms of forcing communication," Mike Niggli, president and chief operating officer of San Diego Gas & Electric Co., told reporters in San Diego. "You can never guarantee that a system will work perfectly, but I think the FERC recommendations will go a long way."
Niggli said the California Independent Systems Operator, which manages the flow of electricity along high-voltage and long-distance lines, has agreed to coordinate the changes suggested by the report. Cal-ISO is also reviewing the blackout and ways to prevent future outages.
Heather Polzin, lead staff investigator on the report for FERC, declined to say whether there is an investigation into possible fines or other punishments, possibly for violating federal regulations requiring that emergency systems be kept up to date. Such probes are confidential, she said.
The failure of the Arizona line should not have caused a massive blackout, investigators said. "The bulk electric system is required to be operated so that the loss of a single line, or the occurrence of any other single contingency, such as loss of a generator or a transformer, does not result in instability, uncontrolled separation or cascading," Polzin said.
Some of the harshest criticism in the report was aimed at the Salt Lake City-based Western Electricity Coordinating Council, whose role is to coordinate the Western electrical grid among various states and utilities.
When the Arizona line went down, the council's reliability coordinator was quickly notified but did not make sure protection systems already in place were put into effect, according to the report.
A spokeswoman for the council, which works for the North American Electric Reliability Corp., said the council would have no comment on the criticism. "We are uncomfortable commenting until we can review the report," Kathi VanderZanden said.
As a matter of practice, some agencies did not consider the situation of neighboring agencies in making their "day ahead" planning, according to the report. Among other things, some did not realize that transmission lines in northern Mexico were under repair and could affect power availability in the United States.