In some offices at FirstEnergy, it is referred to as “the event.”
It has been 10 years since a massive electricity blackout began in FirstEnergy Corp.’s territory and shut down power to eight states and a part of Canada.
Today, Akron company officials say many upgrades have been made and industry standards have been put in place to reduce the risk of another system failure.
A joint U.S.-Canadian government task force including the U.S. secretary of energy in 2004 concluded that errors by FirstEnergy, the parent company of Ohio Edison, involving trees that tripped transmission lines were the cause of the blackout Aug. 14, 2003. Some people as far away from Akron as New York City had no electricity for two days.
The task force concluded in an eventual 238-page report that, “Computer failures leading to the loss of situational awareness in FirstEnergy’s control room and the loss of key FirstEnergy transmission lines due to contacts with trees were the most important causes.”
Carl Bridenbaugh, FirstEnergy vice president of transmission who was the director of transmission, planning and system protections at the time, said “you can never say never” about the chances of such a power failure again. Bridenbaugh is one of those who describes the crisis as “the event.”
However, Bridenbaugh said there are many more safeguards built into the Akron company’s systems as well as the industry as a whole.
“We do not dispute the final blackout report ... which concluded that there were issues on the transmission system that day involving many companies and organizations, including FirstEnergy. Since then, the industry and FirstEnergy have addressed and implemented all of the report’s recommendations, and collectively continue to further improve various aspects of management of the transmission system,” Bridenbaugh said.
The task force said an incident that began at 12:15 p.m. began the timeline of what would eventually result in a blackout, occurring about 4:05 p.m. and leading to a domino effect of failure for another eight minutes. The problems were later discovered to involve computer issues and trees interfering with transmission lines.
It was later calculated that there were 800 separate incidents that resulted in a cascade of power failures that led to more than 100 plants shutting down, cutting off power to as many as 50 million people.
“The vulnerability created by inadequate system planning and understanding was exacerbated because the FirstEnergy operators were not adequately trained or prepared to recognize and deal with emergency situations,” the report said.
The blackout thrust FirstEnergy into an unwanted national spotlight in a way that buying naming rights to sports stadiums does not. FirstEnergy this year became the name of the Cleveland facility where the Browns play.
Less than a month after the blackout, U.S. Rep. Edward Markey, D-Mass., accused FirstEnergy of engaging in a “pattern of neglect” going back several years. He said the Akron company “should not have a license to drive a car, let alone operate a nuclear power plant.” Markey, a longtime critic of the nuclear industry, made the remarks during a hearing of the House Energy and Commerce Committee.
Looking back, FirstEnergy spokesman Mark Durbin said Markey’s comments were “political grandstanding.”
“A lot of employees knew that’s not our company [Markey’s statements] and we do what we can do every day to get even better,” Durbin said. “We lived through that [the blackout] once; we don’t want to do it again.”
For its part, FirstEnergy outlined some of the improvements made in the past 10 years:
• Pieces of equipment called “capacitor banks” have been installed at substations throughout service areas to provide additional voltage support to the transmission system. The utility said these automatic devices have been utilized on numerous occasions to help maintain grid stability.
• Other equipment has been installed at locations throughout the FirstEnergy service territory in Northeast Ohio and is designed to cut power automatically in targeted areas as a safety net to help prevent a more widespread power outage. These processes are called “load shedding.”
• A new $45 million transmission control center is being built in West Akron with completion by the end of the year. It will replace the company’s existing center in Wadsworth, which will be used as a backup and training facility.
• FirstEnergy is converting two retired generating units at its Eastlake Plant in suburban Cleveland into what are called “synchronous condensers,” equipment designed to adjust conditions on the transmission grid by stabilizing grid voltage.
• The utility is using an enhanced transmission system operator training program, including a state-of-the-art Dispatcher Training Simulator module that allows operators to train on real-time system data.
• The company said it has strengthened its tree-trimming or vegetation-management program for transmission rights of way. The company said it has “zero tolerance” for tree contacts. Before the Energy Policy Act of 2005, which directed the North American Electric Reliability Corp. (NERC) to develop mandatory and enforceable vegetation management standards, companies had their own policies, but there was no industry standard. Helicopters are used annually by FirstEnergy to inspect and trim areas, in addition to traditional methods.
The standards developed by the industry and now enforceable by NERC have been “major strides” for the industry and “required everybody to step up their game,” said David Cook, NERC senior legal counsel, who 10 years ago was general counsel for NERC. Cook said he remembers sitting in his office — then in Princeton, N.J. — and watching the lights dim off and on a few times before eventually remaining on. Eventually, he and his colleagues found out what was occurring to the electric grid in eight states.
Never say never
Cook also said “we will never say never” when asked about a repeat. He noted that standardization of system operator education, tree-trimming, better communication among utilities, more precise measurements of “relays” on equipment and advancements in technologies could reduce the scale of a future outage.
“Events happen on the system that don’t result in cascading outages,” he said.
An analysis completed for the Associated Press shows that the industry has responded directly to the blackout. The analysis of spending on maintenance and transmission equipment by more than 200 utilities nationwide by Ventyx, a software and data services firm that works with electric utilities, shows that spending rose sharply in the years after the blackout.
Maintenance spending for overhead lines increased an average of 8.2 percent per year from 2003 to 2012. In the previous 10 years, it grew 3 percent on average per year.
Spending on transmission equipment also increased, according to the AP analysis. From 2003 to 2012, utilities spent an average of $21,514 per year on devices and station equipment per mile of transmission line, compared to an average of $7,185 per year the previous 10 years.
The number of miles of transmission line remained roughly the same, suggesting new money was mostly spent on equipment to make the existing system stronger and more responsive, Ventyx anaylst Chris Tornow told the AP.
Charles E. Jones, president of FirstEnergy Utilities, who was vice president of regional operations at FirstEnergy 10 years ago, said: “The electric industry has focused on making improvements to the reliability of the transmission grid. Our company has successfully implemented numerous enhancements, including new facilities, equipment, new operational standards, and expanded operator training, all designed to make our transmission system more robust.”
At the new Transmission Control Center being built in West Akron, FirstEnergy will have advanced computer systems, large-screen visualization and specialized software to monitor electric grid reliability in FirstEnergy territory, Bridenbaugh said.
At the time of the blackout, the company was upgrading its systems, but operators only had terminals in front of them to see what was happening, Bridenbaugh said. Since the upgrades, there have been large walls of mounted screens for operators to see, he said.
However, not everyone is pleased with FirstEnergy’s investments.
Michael Langford, national president of the Utility Workers Union of America, which represents some FirstEnergy workers, in a news release said: “Ten years later, FirstEnergy and other utility companies are still ignoring the lessons of that failure. Instead of investing in the physical and human infrastructure necessary to maintain a reliable, 21st century electric power grid, companies such as FirstEnergy have slashed utility worker staffing levels to the point that these companies are incapable of meeting the essential needs of the public during routine emergencies — let alone during disasters such as the Hurricane Sandy.
“Utility workers know first-hand that the existing ‘mutual aid’ staffing system relied upon by utility companies to restore essential services following power outages is grossly inadequate to protect the public safety. We call upon regulators to make certain that public utilities are required to invest in an adequate, skilled workforce and the physical infrastructure necessary to maintain a reliable, safe, and modern electric power grid.”
Two financial analysts who follow FirstEnergy, a public company, said the blackout didn’t seem to affect the company’s long-term finances.
Paul Fremont, an electric utility analyst with Jefferies & Co. in New York, said there was a lot of discussion at the time of the blackout of the possibility of lawsuits against FirstEnergy for damages incurred during the blackout. But none came to fruition, Fremont said.
“While the longer-term value of the company was essentially not impacted by the event, you could argue that at the time ... part of the weakness in the shares was directly attributed to the blackout,” but were temporary, Fremont said.
Shares of FirstEnergy stock fell from $31.01 on Aug. 14, 2003, to what is now its 10-year low of $27.75 on Aug. 18, 2003. The stock was back up to its pre-blackout price within a month and steadily rose until the company suffered the effects of the recession of 2007-09. The 10-year-high was $83.21 on July 11, 2008. FirstEnergy stock closed Friday at $37.90.
Phil Adams, a senior bond analyst with GimmeCredit, said “you can be critical of FirstEnergy not keeping their foliage trimmed, but the bottom line is the New York megalopolis was reliant on power from Ohio and the Ohio Valley being delivered on a transmission grid not really designed for that purpose. It wasn’t robust enough to be reliable and the system wasn’t designed to take three lines in Ohio tripping without a cascading failure,” Adams said.
Said Bridenbaugh: “The industry as a whole is in a much better spot than 10 years ago.”