News Section
Stories from Climate Central's Science Journalists and Content Partners

Experts Say Sandy Showed Limits of an Accurate Forecast

Thanks to a vast network of data-gathering instruments on the ground, attached to balloons launched twice daily across the U.S., and orbiting satellites higher in space, along with the sophisticated computer models that run on supercomputers of ever-increasing speed, weather forecasts are more accurate than they have ever been, as was demonstrated with Hurricane Sandy.

A week before the “superstorm” devastated the East Coast, the models available to the National Weather Service showed that Hurricane Sandy was going to do something that no storm had ever done — slam into the New Jersey coastline at a nearly 90-degree angle, going from east to west. That track would bring a massive storm surge into New York City, New Jersey, and Long Island, a nearly perfect storm or worst-case scenario.

In this 3-D map of potential temperature, relatively cool air wraps around Sandy's core near the surface (purple and blue colors), while air parcels gain heat from moisture condensing into clouds and precipitation as they ascend through the storm’s core.
Credit: UCAR.

Yet Sandy also demonstrated the frustrating limitations of an accurate forecast. Despite the advanced notice and the huge threat it posed everywhere from Massachusetts to Maryland, the storm still killed 159 people, including 44 within New York City alone. Most of the New York victims drowned from coastal flooding.

Even now, nearly a year after Sandy helped reshape the weather enterprise in the U.S., a gap is growing between the capabilities of weather forecasters and the state of risk communications and emergency preparedness in the country.

The growing disconnect between what forecasters can do — predict a storm’s path and intensity well in advance — and how the public and officials use that information was driven home by Gary Szatkowski, a meteorologist-in-charge of the NWS’ forecast office outside of Philadelphia, which handles forecasting for the Jersey Shore.

More than any other forecaster, Szatkowski sounded the alarm for New Jersey residents in particular, gaining national recognition for his clear, consistent warnings and use of social media to reach the public. At one point prior to the storm, Szatkowski took the extraordinary step of sharing his cell phone number in a public briefing, inviting people to call him directly to complain if Sandy failed to live up to its billing (as Tropical Storm Irene had a year earlier) and they had evacuated their homes, rather than ignoring evacuation orders and risking their lives.

"If you think the storm is overhyped and exaggerated, please err on the side of caution," he wrote. "You can call me on Friday and yell at me all you want . . . I will be very happy that you are alive & well, no matter how much you yell at me."

Szatkowski said Sandy proved that an accurate forecast is not enough to avoid loss of life and massive damage from an extreme weather event.

“I do think Sandy raised the bar regarding weather forecasts and warnings, and more importantly, opened the door to some new questions,” Szatkowski said in an email to Climate Central. “I’ve been drawing the contrast between how accurate the forecast was and how the information was utilized (in some cases, not so well).”

In two well-documented gaffes, New Jersey Transit parked dozens of train cars and expensive engines in a low-lying lot in Hoboken that was forecast to flood, ultimately causing more than $120 million in damage. And in New York, officials had to hastily evacuate patients from Coney Island Hospital, Bellevue Hospital, and NYU Langone Medical Center, among other facilities, including patients in critical condition and more than 15 infants in neo-natal intensive care. Bellevue Hospital and NYU Langone Medical Center lost power when floodwaters from the East River breached an electrical substation.

“The best technical forecast won’t achieve the desired results if the threat is not understood or not prioritized correctly,” Szatkowski said. “Everywhere it flooded during Sandy was forecast to flood.”

“I find our risk assessment as a society is quite interesting,” Szatkowski said. “As a society, we are comfortable with recalling large quantities of food if something inappropriate is found in a bottle or jar. As a society, we are comfortable with closing down a bus or train terminal if an unattended bag or parcel is found. Yet as a society, with a killer hurricane coming up the East Coast, we struggle with evacuation decisions, and protection of expensive infrastructure.”

Post-Sandy Focus Is On New Warnings, Communications

Hurricane Sandy prompted a number of changes throughout the different corners of the weather community, including the public and private sectors. New emphasis has been placed on improving how severe weather warnings are communicated, and rolling out new forecast products to help the public better understand threats. In addition, more funding has been secured to help shore up weaknesses in U.S. computer modeling and satellite development.

If ever there were a weather event designed to be a forecasting success but a communications failure, it was Hurricane Sandy. The storm was complex, owing part of its DNA to a hurricane and part to an extra-tropical nor’easter, and its track had no previous analog that could be compared. It produced up to 3 feet of snow in West Virginia at the same time as it was flooding Lower Manhattan. Plus, it struck just 1 year after the underwhelming Tropical Storm Irene passed over Manhattan, leading people to be more skeptical than usual about the dire forecasts.

New York City Mayor Michael Bloomberg briefs the press about Hurricane Sandy on Oct. 27, 2012. On that day, he downplayed the threat posed by the potential storm surge, and did not order large-scale mandatory evacuations.
Credit: NYC Mayor's Office.

Take the storm surge forecasts, for example. They were accurate, yet they were delivered in a text-based format that made it nearly impossible for most people to understand their implications. The format of the surge forecasts is suspected as being one of the reasons why so many New Yorkers and Jersey Shore residents were caught off guard by the height and force of the storm surge.

At The Battery in Lower Manhattan, Hurricane Sandy's storm surge of 9.4 feet contributed to a 13.88 foot storm tide above the normal low tide level. That resulted in an inundation of between 4 to 9 feet above ground level, enough to flood all of the subway tunnels between Manhattan, Brooklyn, and Queens, wipe out homes in Staten Island, and flood neighborhoods like Red Hook in Brooklyn and the Rockaways in Queens.

Although the NWS accurately forecast the storm surge magnitude and timing, the agency provided little in the way of graphical guidance for people to envision the impact of storm surge on their neighborhoods. Sans that visual aid, most coastal residents had no way to understand what a particular storm surge height would mean for their homes.

“There are ways of getting this mapped more explicitly,” Louis Uccellini, director of the NWS, said in an interview. “We’ve got to get it down to the street level.” 

Despite the communications shortcomings, Uccellini said the agency's forecasts and warnings saved countless lives. “We also feel that our forecasts and our messaging saved lives. That there were evacuations in areas that if they weren’t evacuated… there would have been many more lives lost.”

A post-Sandy federal review of the NWS’ performance found that the forecasts the agency issued, including the surge forecasts, were remarkably accurate, but they were not communicated in ways that made it easy for officials and the public to understand. The confusing surge forecasts may have been partly to blame for why New York Mayor Michael Bloomberg downplayed the storm’s threat on Oct. 27, with less than 48 hours to go until landfall.

"It is not expected to be a tropical storm or hurricane-type surge," Bloomberg said at a press conference. "With this storm we're likely to see a slow pileup of water rather than a sudden surge, which is what you would expect from a hurricane and which we saw with Irene 14 months ago, so it will be less dangerous, but make no mistake about it there will be a lot of water and low-lying areas will experience flooding."

Spurred by Sandy as well as Hurricane Isaac, which struck Louisiana in 2012, the NWS is developing storm surge warnings as well as mapping tools that will allow forecasters to bring the threat closer to home for the public.

A prototype of NOAA's new storm surge depth maps, which will be in use starting in 2014.
Credit: NOAA/NWS.

Jamie Rhome, the storm surge lead for the NWS, said that Congress appropriated funds for storm surge forecasting improvements in the wake of Sandy, but that much of the work had already begun before the storm. The $10 million in post-Sandy funds devoted to storm surge modeling will accelerate the development of these warnings and enable the NWS to deliver this nationally earlier than expected.

The NWS is also working to roll out experimental inundation graphics in 2014 and storm surge warnings in 2015. These warnings, and the maps used to convey them, are being designed collaboratively between forecasters and risk communication experts.

While the NWS and weather community more broadly work to develop better computer models and satellite systems, along with more effective storm warnings, the central lesson from Sandy may not concern any one particular technology or forecast product, or even a single satellite.

Rather, as David Bernard, chief meteorologist at CBS’s Miami affiliate, told Climate Central, the problem that forecasters face in future storms is that more progress needs to be made in improving risk communication and threat response.

“The meteorology is moving along but our ability to react and prepare is not,” Bernard said.

In other words, forecasters are already bumping up against diminishing returns, with improvements failing to translate directly into lives and property saved.

Before Sandy, the NWS was already beginning to work with social scientists to help redesign its forecast products and warnings. Post-Sandy, the agency has accelerated such efforts. “(It’s) not just physical science anymore,” Uccellini said of the job of severe weather forecasting at the NWS. “It’s physical science plus social science, if in fact we want people to take the response that we expect when we make the forecast.”

“Sandy has gotten academia, the private sector, and federal government to move beyond ‘lip service’ on social science, communications, and related research,” Marshall Shepherd, a professor at the University of Georgia and the president of the American Meteorological Society, said.

So far, though, the NWS has not hired any social scientists to work full-time alongside its meteorologists. Instead, the agency is training certain field experts, known as warning coordination meteorologists, to make them aware of the insights that social science research can provide regarding the weather warnings process. “This is going to become an increasingly important part of their job description,” Uccellini said.

In addition, storm surge specialists like Rhome are working closely with communications experts to design the storm surge warning products.

Bryan Norcross, the senior hurricane specialist at The Weather Channel who was on air throughout Hurricane Sandy, told Climate Central that the meteorological community needs to come to grips with what he says is the sorry state of risk management in the U.S.

“It has become frighteningly predictable what the research is going to say after every big storm: most people did not understand the threat. They knew the storm was coming, but they could not translate the avalanche of bulletins, advisories, watches, warnings, and opinions into a clear understanding of the threat to their home and their family,” he said.

Shoring Up Critical Weather Infrastructure Before the Next Sandy

By ripping apart pieces of the East Coast and causing billions of dollars of damage, Sandy moved the needle on the National Oceanic and Atmospheric Administration (NOAA) funding requests for both computer modeling improvements and satellite development.

The storm hit the U.S. at a critical time for the public and private weather sector. A successful weather forecast relies on products from observational systems, like satellites and weather balloons, as well as tools that scoop up that data and make sense of it, feeding it into sophisticated computer models that assist human forecasters in making a prediction.

Life cycle of Hurricane Sandy: Surface Wind Speeds. Credit: UCAR.

If any one of these three critical areas — observations, data assimilation, and computing capacity — are lagging, then forecasts could suffer, Uccellini said. Before Sandy hit, NOAA was growing increasingly concerned about its declining computing capacity relative to other nations, as well as its troubled satellite development program.

The agency faced a political climate that was toxic to any requests for funding increases, especially for programs that could be spun as “climate-related,” which earned the suspicion of many Members of Congress who are skeptical of the existence of manmade global warming.

Sandy provided NOAA with a powerful argument to use on Capitol Hill, where lawmakers had been harshly critical of the agency’s mismanagement of the satellite program, in particular, and largely unaware of the supercomputing lead other nations were opening over the U.S.

As they had done after previous major storms, NOAA officials went before lawmakers and showed them how much less accurate the forecast would have been had the computer models been deprived of satellite data. Kathryn Sullivan, the NOAA administrator, touted a study from meteorologists at the European Center for Medium-Range Weather Forecasts in Reading, England, in which they re-ran their computer model after depriving it of the data that comes from the current fleet of polar-orbiting satellites, and found the forecast would have been hundreds of miles off course.

The Hurricane Sandy forecast from ECMWF with and without the polar-orbiting satellite data.
Click to enlarge the image. Credit: NOAA

The test was done because cost overruns, launch setbacks, and bureaucratic delays in the U.S. satellite program threaten to cause a gap of a year or more between when one of the current polar-orbiting satellites — known as the Suomi NPP — reaches the end of its design lifetime in 2016, and when the next polar-orbiting satellite is ready for launch in 2017, at the earliest.

Partly as a result of that lobbying push, the continuing resolutions that provided funding for reopening the government following the 16-day shutdown included a provision for keeping satellite funding at levels that would prevent any launch dates from slipping further behind, and there was a similar provision in a bill that funded the government through Fiscal Year 2013.

The storm also publicly demonstrated the shortcomings of the premier U.S. forecasting model, known as the Global Forecast System, or GFS, compared to the European’s medium-range model. Meteorologists had known for years that the Euro model often outperformed the GFS, particularly several days in advance of a weather event, but lawmakers and the public were largely unaware.

The Euro model picked up the scent of Sandy’s track, including its bizarre left hook into New Jersey, sooner than the GFS model did, providing earlier warning to public officials.

It is no accident that Congress has since provided critical support for boosting the agency’s computing capabilities. The NWS is using $25.2 of the $48 million allocated to the NWS in the Sandy relief legislation, along with funds called for in President Obama’s fiscal year 2014 budget proposal, to bring about unprecedented computing upgrades.

“We just tripled the size of our operational computer,” Uccellini said, adding that in a year and a half, the agency will add another 10 times the current amount of capacity. “The Sandy supplemental allowed us to break through the glass ceiling,” Uccellini said of the NWS’ computing capabilities. “I thank the Hill every time I get the chance for supporting that.”

The new money will allow the NWS to go from an operational computing capacity of 213 peak teraflops at the end of the current fiscal year, to 1,950 peak teraflops by the end of fiscal year 2015. A teraflop is a measure of how many calculations a computer can make per second, and indicates that a computer can make one trillion “floating point calculations” in just one second.

But as Sandy sadly showed, all that computing power doesn’t mean anything if the message isn’t communicated well and heeded by public officials and the people in harms’ way. 

Related Content
Hurricane Sandy: Coverage of the Storm and its Aftermath
National Weather Service Adding Major Might to Computing Power 
Recent Storms Highlight Flaws in Top U.S. Weather Model 
Sandy a Warning Rising Seas Threaten Nuclear Plants
Heeding Sandy's Lessons, Before the Next Big Storm
The Most Detailed Visuals of Hurricane Sandy, Revealed
Research Spawns Stunning Hurricane Sandy Animations
Hurricane Isaac Spurs Design of Storm Surge Warnings
Panel Finds Flaws With NWS Guidance on Sandy's Surge

Comments

By Susan Jasko (Canonsburg, PA, 15317)
on October 28th, 2013

As a social scientist who has worked closely with the NWS including serving on the Hurricane Irene Service Assessment Team, I want to thank the author for this detailed and focused article. The issues raised within this writing are both salient and urgent for both the east coast and the nation as a whole. We have for too long rested in reactivity and need to reset our sensibilities to proactivity in regard to natural disasters of all kinds.

Importantly, this article touches on the issues of communication, perception, risk, and responsiveness. Appropriately, much attention has been directed toward the communication issues inherent in both message design, delivery, and consumption. Much more needs to be done in the area of conveying to emergency professionals, politicians, and local publics a visceral sense of the likely and the potential weather impacts where each person stands.

But also important is attention to the culture of the experts involved in the creation of weather knowledge, especially the organizational culture of the NWS. Crucially, monies have been made available for the technical demands of this effort, but at great risk no funding, or no sufficient funding, has been made available for the pursuit of arriving at a better understanding of the human factors involved both among the expert communities and local publics served by those experts. Although Dr. Shepherd notes interest beyond lip service, the support, access, and serious attention to social science is still lagging in the NWS - even while the operations end of the organization makes strong and inventive efforts to build better warnings and other messages while they embrace the brave new world of social media and the ever-changing digital landscape.

Bryan Norcross notes accurately that we already know that the products and mechanisms for powerfully and compelling conveying weather impacts do not yet exist. This is not due to a lack of effort, interest, or motivation, but rather the result of micromanagement by Congress, a lack of personnel in key positions who might know how to assess and redirect agency efforts towards innovative, comprehensive, and communicatively astute reconfiguration, and under funding.

There are other problems within and across the weather enterprise. But, I have rarely seen a community so dedicated to the welfare of others and so eager to improve and willing to embrace change. It has been a great privilege of my professional life to work among the staff of the NWS and all of the other members of the weather enterprise. As an agency, the NWS is crucial to our national security. And the partnership among the NWS, broadcast meteorologists, journalists, private enterprise, and the academy is essential.

It will take “all hands on deck” to address the impeding climate and weather challenges before us as a nation and before the global community. Many of us from a range of disciplines - mine is human communication - stand ready, willing, and able to participate in this endeavor. We need a bit of funding, open doors, and venues for sharing our efforts, ideas, and insights. It is past time for the human portion of the weather/climate resiliency equation to come up to speed.

Reply to this comment

By Jo (NY, NY 10014)
on October 28th, 2013

And yet the NWS stopped calling it a “Hurricane” and even stopped all Hurricane Warnings after the NC/VA border so that residents of NY, LI, NJ didnt think it was going to be so bad.  Why did they do this?  Rumor has it so that insurance companies would pay out if it damage was not due to a “hurricane”.  Even the media came up with that stupid “super storm” BS marketing lingo.  It was NOT a SUPER storm.  It was a strong Category 1 HURRICANE when it made landfall in Southern NJ with 90+ MPH gusts all up and down the NJ and LI coasts.  What part of “accurate” does all of that fit into?

Reply to this comment

Name (required):
Email (required):
City/State/Zip:
Enter the word "climate" in the box below:

[+] View our comment guidelines.

Please note: Comment moderation is enabled. Your comment will not appear until reviewed by Climate Central staff. Thank you for your patience.