ForecastAdvisor Weather Forecast Accuracy Blog

A Blog Documenting the Intersection of Computers, Weather Forecasts, and Curiosity
 Subscribe in a reader    Subscribe by email:   Delivered by FeedBurner


Friday, March 6, 2009

How Good Are Week Out Weather Forecasts?

I am often asked if forecast for a week or more out is worth anything. Worth is a hard thing to measure, but skill is easier. And if there is no skill, then likely the forecast would not be worth much.

At ForecastWatch we use two measures of skill. How much better is a weather forecast than climatology, and how much better is a weather forecast than persistence. Those are both unskilled forecasts, in that it doesn't take any skill to create them. The climate weather forecast says that the weather will be exactly like the 1971-2000 average. A persistence forecast says the weather however many days in the future will be exactly like today.

The easiest way to measure skill, at least for temperature forecasts, is to compare average absolute error. The difference between the temperature forecast, and the actual temperature that occurred, is the error. Take the positive difference, average it over a lot of forecasts, and that is a measure of how good a forecast is at predicting the temperature. An average absolute error of 3 degrees is generally better than an average absolute error of 6 degrees.

The chart below shows the average absolute high temperature forecast error for all of 2008, for all stations, and all providers. Lower is better. Red is the unskilled persistence forecast. Yellow is the climate average forecast. The green line is the average of all weather forecast providers. Lower is better. The climate average forecast is a straight line because it doesn't matter how many days you forecast out, the climate average forecast for a given date is always the same.

2008 High Temperature Error Summary

What sticks out on this graph is the intersection of the yellow climate average line with the green forecasters' line at nine days out. What that means is that the weather forecasters predicted high temperatures nine days out just as well (or just as badly) as using the 1971-2000 climate average high temperatures as the forecast prediction for each location.

What is even more interesting is that for high temperature forecasts greater than nine days out, weather forecasters (at least the ones ForecastWatch tracks) do WORSE than the climate average forecast. This means that you would do better just looking at historical average temperatures when determining the temperature more than nine days out.

The American Meteorological Society recently said as much. Every ten years, they release an information statement describing the current state of weather science. The statement released August 2007, made this observation:

"The current skill in forecasting daily weather conditions beyond eight days is relatively low."

The AMS is very wise. And in fact, for this sample, composed of nearly 18 million forecasts over the year from multiple national providers, there is no skill in daily weather forecasts beyond nine days.

On the positive side, meteorologists always did better than the persistence (tomorrow is like today) forecast. There are many other interesting things about this graph that I'll talk about later.



December 2005   January 2006   March 2006   June 2006   July 2006   August 2006   September 2006   October 2006   November 2006   December 2006   January 2007   February 2007   March 2009   September 2009   March 2010   April 2010   February 2011   April 2011   June 2011   February 2012   September 2012   June 2013   October 2013   February 2014   June 2016   Current Posts