Superstorm Sandy: Triumph of the Forecasting Models

Introduction

All in all, Sunday, Oct. 21, 2012, was an average day for forecasters in the United States.

Meteorologists at The Weather Channel's Global Forecast Center had a few strong thunderstorms to watch in West Texas, but otherwise there wasn't much going on stateside. 

There was an area of potential tropical development circled in the Caribbean, but that's typically a favored area for development anyway in October. And while we were concerned for Haiti and other Caribbean nations, most computer forecast models were taking this disturbance out to sea without any direct effect on the U.S. mainland.

But late that night, the most reliable computer forecast model in the business showed a powerful low pressure system coming up from the tropics and slamming into the New Jersey coast eight days later.

When that model data landed on forecasters' computer screens early on the morning of the 22nd, meteorologists were jolted awake.

Senior meteorologist Stu Ostro recalls it well. "I had out-of-town travel planned, and so I was keenly aware of the models both professionally and personally," he said.

This ominous forecast, made by the European Centre for Medium-Range Weather Forecasts – the ECMWF computer forecast model – was a standout at the time. It was predicting a weather event virtually unprecedented in modern weather observation.

Naturally, forecasters had a healthy dose of skepticism, especially after this European forecast model botched forecasts of Tropical Storm Debby earlier in the season.

Early days: Long-range models and the "MJO"

The first signs that trouble might be brewing actually came weeks before Sandy brought catastrophe.

Long-range computer forecasting models were tracking a feature called the Madden-Julian Oscillation. The MJO, one of many three-letter acronyms meteorologists bandy about when discussing global-scale features of the atmosphere, is essentially a slow-moving pattern of rising and sinking air that travels around the globe in the tropics.

Meteorologists at The Weather Channel spotted the rising-air cycle of this pattern moving over the Caribbean and Eastern Pacific in mid-October, tipping them off to the possibility that the relatively quiet late September and early October in the tropics might be about to end.

On Oct. 16, Ostro wrote in his tropical outlook: "Various signals are showing up in the longer-range models that are suggestive of the western Caribbean being an area to monitor for potential tropical cyclone formation next week."

By Oct. 19, the National Hurricane Center was tracking an area of disturbed weather in the Caribbean.

First signs: Models go out on a limb

The Canadian "GEM" forecast model was the first model to suggest something big would happen in the Northeast. On the night of Oct. 19, two and a half days before Sandy was even a tropical depression, the GEM model had a hurricane whisking north through New England in a manner similar to 1938's Long Island Express or 1991's Hurricane Bob.

The Canadian model forecast attracted little notice at the time. And while it was the first to hint at the general idea of a hurricane threatening the Northeast, it was off on both track and timing, suggesting an Oct. 27 landfall, two days too fast.

The European model first suggested trouble for the Mid-Atlantic and Northeast with its morning run on Oct. 21: A large hurricane, though not as intense as what would later come to fruition, making landfall on the Delmarva Peninsula with a wide area of rain, wind, and storm surge extending out from there.

Twelve hours later, that same model made an even more serious prediction: a hurricane with a much lower pressure – an extremely low pressure, signifying an exceptionally powerful cyclone. And the center? Just offshore of southern New Jersey.

Computer forecasts rarely get the details right eight days in advance, and the European model had failed miserably against its main competitor, the American "GFS" model, during Tropical Storm Debby in June.

But Ostro was still concerned. "Debby notwithstanding, if the ECMWF is showing something like that, you have to take it seriously."

Sandy officially becamed a named storm on the afternoon of Oct. 22, when Tropical Depression Eighteen strengthened to tropical storm status.

Consensus: The chorus grows louder

At the same time that the European model made its uncanny forecast of a New Jersey landfall, the GFS model was still showing a weaker storm curving northeast and east out to sea.

Usually, that's a more plausible forecast. Never before had a strong tropical cyclone made a left hook into New Jersey, and in late October there's usually a strong jet stream over the U.S., ready to sweep hurricanes quickly out to sea.

Indeed, as Sandy developed, there was a strong jet stream over the Lower 48, but there was also a blocking pattern to the east, with strong high pressure over Greenland and a giant low-pressure gyre swirling over the North Atlantic.

Ostro remarked, "If ever a hurricane was going to take a track like that at this time of year, this was the pattern that could do it."

To get a better feel for the range of possible weather outcomes, forecasters look at sets of computer models called "ensembles."  The operational American, European, and Canadian models – the ones you often see depicted on television – all have a corresponding ensemble, a set of 20 to 50 simulations of the parent model using slightly different initial conditions to account for uncertainties and limitations in our ability to measure the current state of the atmosphere.

At the time that the operational European model was forecasting a massive storm slamming into New Jersey, the American model's ensemble was nearly unanimous in forecasting the future Sandy to go out to sea, though one or two of its 20 simulations hinted at a storm curving left.

"Some of the ECMWF ensemble members [still] had it taking the escape route out to sea," Ostro recalls. "So that also gave us reason to be careful about appearing to be 100 percent sure and end up crying wolf."

But gradually, over the next two or three days, more and more of those American ensemble simulations peeled away from the out-to-sea forecast, while run after run, the European model remained steadfast in forecasting the left hook into the coast.

More ominously, almost all of the model simulations that predicted the left hook predicted a very strong low-pressure system. Virtually none of the left-hook scenarios were calling for a weak storm to come ashore.

And once that happened, there was no turning back. No subsequent run of the GFS operational model had Sandy going out to sea. Four and a half days before landfall, the computer models had reached a remarkable consensus: a historic storm was on a collision course with America's most densely-populated corridor.

Perspective: Lucky breaks and breakthroughs

Looking back, the fact that the European computer model's forecast was so incredibly accurate so early is striking.

Ostro said he's never seen a model make such an accurate forecast of such an unusual storm eight days in advance.

"We've seen times when one or more of the models has correctly predicted the development of a very significant storm, sometimes even a hurricane, far in advance," Ostro said. "But how precise and accurate a 204-hour model forecast was in combination with how extraordinary a storm Sandy was is stunning. We've never seen anything like that before.

"It's certainly a remarkable success story."

Ostro and others at The Weather Channel have remarked that the model forecasts of Sandy are emblematic of the increasing success of computer models, known in the weather business as "numerical weather prediction."

For many of those seasoned experts, the success reminded them of another epiphany in the annals of computer-assisted meteorology.

In early March 1993, computer forecast models started to predict that a powerful storm would spin up along the East Coast, bringing heavy precipitation and powerful winds.

And indeed, six days later, much of the East was digging out and cleaning up from one of the worst non-tropical storms in memory – the first "Superstorm."

"In the three-to-five-day range our eyes were bugging out of our heads," Ostro remembers.

Ostro says it was the biggest breakthrough in the history of numerical weather prediction up until that time. The "ensemble" models, then a fairly new invention, participated in the success of the 1993 event. Papers were written analyzing the success of the models, and anticipating what lessons would be applied to future advances in the science of forecasting.

But in the days leading up to the 1993 storm, forecasters were skeptical of forecasting such an extreme event because of recent model failures. Computer forecasts at the time were known for predicting phantom storms that never materialized, or botching the details even at short lead times before storms hit.

And while computer models are considerably more sophisticated today, they still occasionally commit major gaffes.

As Tropical Storm Debby stalled over the north-central Gulf of Mexico in late June, the European model was routinely forecasting it to lurch west into Texas. Two days later, Debby made landfall in northern Florida.

The European model: Superior? or lucky?

By most objective, mathematical measures, the European model is the best-performing computer forecasting system in the world today.

But as Debby showed, that doesn't mean it is always superior.

Ostro noted that the range of model scenarios showed wildly divergent, fork-in-the-road forecasts for both Debby and Sandy, albeit within much different weather environments.

Debby was a smallish storm caught in a weak steering environment in the Gulf of Mexico in June, with subtle atmospheric features ultimately determining its path.

Sandy, on the other hand, was a giant storm that moved much farther north into a mid-autumn environment of strong jet streams and a cold air mass plunging south from Canada.

Since the European model seems to often forecast sharper patterns of high and low pressure features, it would logically have the advantage in a more "amplified" steering pattern scenario like the one interacting with Sandy.

Even considering the Debby debacle, Ostro noted: "Since the ECMWF is such a consistently good model, one should not assume it's going to have an epic fail every time."

What will we learn from this experience?

Above: an interactive map of the official National Hurricane Center forecast cones for Hurricane Sandy from the first advisory on Oct. 22 through the final advisory on Oct. 29. Mouse over the blue-shaded regions for details on each individual cone. Each cone describes the most likely future path for the center of the storm. For more details, click here.

Ostro said that, like the models' success with the 1993 Superstorm, scientists will be mining the computer forecasts of Sandy for clues on how to improve forecasting in general.

He expects scientists will investigate "exactly why the ECMWF latched onto [Sandy] so much earlier. We need to understand that, as well as why it failed in Debby and why the GFS was better."

Ostro suggested another area of study should be whether the European model's spectacular eight-day forecast was related to its widely-praised "initialization" scheme - the methods it uses for analyzing the current state of the atmosphere before running its simulations – or whether there was something about the atmosphere's patterns that gave it a leg up on the American models.

But the fact that all of the model guidance came into lock-step on a high-impact landfalling storm more than four days in advance is reason for hope.

"This is cause for optimism that at least the forecast models can help foretell these massive cyclones far in advance and help us alert people long before they hit," he concluded.

But he cautioned that the science of computerized weather forecasting still has some hurdles to overcome.

He noted that eight-day model forecasts aren't always this accurate, and there are other smaller-scale, shorter-range phenomena such as tornadoes, smaller tropical cyclones, and winter phenomena such as freezing drizzle that can be difficult to forecast well even a few hours in advance.

"There are now higher-resolution models that have demonstrated value in, for example, depicting exactly where and when severe thunderstorms will develop. Those two extremes of time scales – weeks and hours/minutes – are where the greatest strides are being made and where the greatest challenges are.