Wednesday, January 6, 2016

Temperature forecast skill sinks in Seattle

Today I was surprised to see that the NWS had forecast a high of 41 Fahrenheit this afternoon here in Seattle, but the temperature had gotten up to 51F!


They aren't the only ones who have been struggling with the temperature forecasts recently.  Let's look at some of the forecast performance.

Here is an image showing the last 30 days of high temperature forecasts for Sea-Tac Airport (KSEA) from the GFS Model Output Statistics (MOS) forecast. In the top panel, the blue line shows the 1-day forecast high temperature and the black line shows what actually occurred.  The gray dashed line in the background shows the climatological normal high temperature for each day. The bottom bar chart shows the error in the forecast each day.



You can see that Seattle's climatology (the gray dashed line) "bottomed out" around December 21st.  Our coldest high temperatures of the year are now behind us, on average.  You'll note that before then (or rather, before about Dec. 15), the GFS forecasts were actually doing fairly well.  There were several days with the high temperature forecast perfect or only within 1 degree of what actually occurred.

However, since then (over the past two or three weeks), the temperature skill has gone crazy.  There's no net bias in the forecast (the GFS hasn't been consistently too cold or too warm) but there have been a lot of days with errors greater than five degrees.  The day-to-day differences in high temperature have also been quite large, and it seems like the model just isn't capturing these swings well.

The NAM model hasn't been a whole lot better:

There have been some misses on the low temperatures, but not as bad.  Here's the same plot, but for the GFS low temperature forecast.

Still a few big misses, but not as bad as the high temperatures.

We can compare different model forecasts by looking at their skill scores, which compares the performance of the model against some standard baseline forecast.  Two common baselines used are climatology and persistence.  A climatology forecast basically follows that dashed grey line in the plots above...it just assumes you forecast the average value for that day, every day.  A persistence forecast assumes that whatever happens today will happen again tomorrow, and that's where you get your forecast.  So if today's high temperature was 51 degrees, our forecast for tomorrow would be 51 degrees again.  We would expect that a good weather forecasting system or service would be able to beat either of these baselines, otherwise it's not adding any value to the forecast.

So skill scores compare the errors from forecasts against these baselines.  A negative skill score means that the forecast system does worse than the baseline overall (bad!).  A zero skill score means the forecast system does exactly the same as the baseline, and a positive score means it does better.  A skill score of 1 means it produces a (near) perfect forecast.

Below are the skill scores for the GFS, NAM, and several other weather forecast producers like Accuweather (ACUWX), the Weather Service (NWS) and Weather Underground/The Weather Channel (WUTWC) in their high and low temperature forecasts at KSEA over the last 15 days. Blue is for low temperature, red is for high temperature.  The lighter bars are skill measured against a persistence forecast and the darker bars are skill measured against a climatology forecast. 

You can see that these scores are all positive...so we're doing better than our baselines.  But not by much!  In fact, the GFS has been almost exactly the same as persistence, and only marginally better than climatology.  The other forecast sources also have skill scores generally below 0.5, and typically here in Seattle these scores are closer to the 0.7-0.8 range.

So why have we been struggling so much with our high temperature forecasts?  For one, we've been stuck in a "split-flow" regime for the last few weeks, with storms either being directed to our north or down south into Oregon and California.  It's actually times like these when we don't have a strong synoptic-scale weather signal that our models struggle with the most, for it's on those days where local idiosyncrasies really start impacting our weather.

In particular, we've had a number of clear nights which have set up low-level cold inversions and fog overnight.  Our models tend to have a difficult time representing these low-layer inversions, particularly when it comes to mixing them out during the day.  As a result, sometimes they keep the cold and fog around for too long and sometimes they too quickly erode it away.  This has huge implications for the high temperature forecasts.

In addition, on January 3-4, we had strong easterly flow, which dried out our air but didn't actually raise the temperatures that much.  The GFS bit on the idea that downslope warming with these easterly winds coming off the Cascades would bring up our high temperatures, but that ended up not happening.

So there's a lot of little things going on that have contributed to lower skill in our forecasts.  With more ridging and dry weather expected in Seattle over the next few days, we may have to live with low predictability for a little while...