Climate Science Glossary

Term Lookup

Enter a term in the search box to find its definition.

Settings

Use the controls in the far right panel to increase or decrease the number of terms automatically displayed (or to completely turn that feature off).

Term Lookup

Settings


All IPCC definitions taken from Climate Change 2007: The Physical Science Basis. Working Group I Contribution to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Annex I, Glossary, pp. 941-954. Cambridge University Press.

Home Arguments Software Resources Comments The Consensus Project Translations About Support

Twitter Facebook YouTube Mastodon MeWe

RSS Posts RSS Comments Email Subscribe


Climate's changed before
It's the sun
It's not bad
There is no consensus
It's cooling
Models are unreliable
Temp record is unreliable
Animals and plants can adapt
It hasn't warmed since 1998
Antarctica is gaining ice
View All Arguments...



Username
Password
New? Register here
Forgot your password?

Latest Posts

Archives

Lessons from Past Climate Predictions: Arctic Sea Ice Extent 2012 Update

Posted on 10 October 2012 by dana1981

Last year we examined a few predictions of the September annual Arctic sea ice extent in 2010 and 2011.  We now have another year of predictions in the books, and a new record sea ice minimum.  The Study of Environmental Arctic Change (SEARCH) has also collected sea ice extent predictions since 2008.  In this post we will examine how these predictions have fared year-by-year, and Figure 1 below shows the average accuracy of individuals or groups who have made at least two annual Arctic sea ice extent predictions since 2008. 

In all of the graphics in this post, orange bars represent model-based predictions, blue bars represent statistics-based predictions, purple bars represent heuristic predictions (experience-based, which essentially means any predictions not based on statistics or models), and green bars represent a combination of methodologies.  Observational data from the National Snow and Ice Data Center (NSIDC) are provided to evaluate the accuracy of the predictions.  Sea ice extent is defined as the portion of the Arctic covered by at least 15 percent ice.

overall

Figure 1: Percent difference between predictions and actual annual Arctic sea ice minima for individuals and groups which have made a minimum of two predictions.

Bear in mind that Figure 1 is not an entirely fair comparison, because as we will see below, the sea ice minimum in some years have been easier to predict than others; years in which the minimum extent falls near the long-term trend are easier, while extreme years like 2012 are much more of a challenge.  Thus a group or individual which made two predictions in easier years has an advantage over those who made two predictions in more anomalous years.

Note that in this post we are only considering initial predictions, usually made in May or June each year.  SEARCH also allows for revised submissions in July, August, and sometimes September, but those revised predictions are much less interesting.

2008 Predictions

2008 was the first year that SEARCH solicited Arctic sea ice predictions. It was a challenging year because the previous year had shattered the minimum sea ice extent record, and it was therefore difficult to know if ice conditions had changed to allow even further declines, or if the ice would "rebound" and regress toward less extreme values.  As the Arctic Sea Ice Escalator in Figure 2 shows, the latter happened.

down escalator

Figure 2: NSIDC September Arctic sea ice extent (blue diamonds) with "recovery" years highlighted in red, vs. the long-term sea ice decline fit with a second order polynomial, also in red.

14 groups submitted their predictions to SEARCH in 2008, with most being heuristic (no model-based or statistical methodology noted).  Interestingly, the two model-based predictions of Lindsay and Zhang performed the best, predicting the ensuing sea ice extent almost perfectly (Figure 3).  Note that in Figure 1 above, Lindsay (who has made predictions every year from 2008 through 2012 using varying methodologies) has had the most overall predictive success, and Zhang's model has also performed well through the years.  The statistically-based estimates were the least successful in 2008, on average.

2008

Figure 3: 2008 SEARCH September Arctic sea ice extent predictions vs. observation (4.52 million square kilometers).

2009 Predictions

In 2009 there was a second consecutive regression after the 2007 record minimum.  In fact, there was a larger increase from 2008 to 2009 than there was from 2007 to 2008 (as is evident in Figure 2).  As a result, all 15 SEARCH submissions under-predicted the 2009 extent (Figure 4).  The statistical and combined approaches performed the best in 2009, with the models having less success.

2009

Figure 4: 2009 SEARCH September Arctic sea ice extent predictions vs. observation (5.36 million square kilometers).

2010 Predictions

The climate contrarians were emboldened by the sea ice "recoveries" in 2008 and 2009, and made a few unofficial predictions of their own.  Anthony Watts and Steve Goddard predicted that there would be another 500,000 square kilometer rebound from 2009, thus predicting a 5.75 million square kilometer extent in 2010.  There were also 18 submissions to SEARCH in 2010, and amongst the climate realist bloggers, tamino weighed in with his own prediction by fitting a quadratic trend line to the annual September sea ice extent data, as did Skeptical Science's own Gavin Cawley (Dikran Marsupial) using a Gaussian process model, as did Robert Grumbine, who also submitted a prediction to SEARCH in 2009.

There was no continued "rebound" in 2010 - instead Arctic sea ice extent declined to a level closer to the long-term downward trend (Figure 2).  Overall, the statistical and model-based predictions fared equally well in 2010, and the heuristic predictions were not far behind, with Watts and Goddard dragging their accuracy down.  There was also a rather strange combined heuristic and statistical prediction by Wilson of just 1 million square kilometers, which was very far off (Figure 5).

2010

Figure 5: 2010 SEARCH, Anthony Watts and Steve Goddard, tamino, and Cawley September Arctic sea ice extent predictions vs. observation (4.9 million square kilometers).

2011 Predictions

Watts and Goddard may have been deterred by their inaccurate 2010 prediction, and did not record any predictions for 2011.  However, in 2011 Watts submitted a prediction from his WattsUpWithThat (WUWT) blog readers to SEARCH.  Another climate contrarian, Joe Bastardi weighed in with his own prediction.  Coincidentally, the WUWT and Bastardi predictions were almost identical, predicting another large "rebound" from 2010.  Cawley and tamino also weighed in with their predictions once again (Figure 6), and there were also 18 total SEARCH submissions.

predictions

Figure 6: JAXA Arctic sea ice extent data, with an approximate re-creation of Joe Bastardi's 2011 prediction (red), WUWT readers' prediction (blue dot), tamino's prediction (green dot), Dikran Marsupial's prediction (orange dot), and the actual 2011 data (black).  Source.

2011 saw yet another sea ice minimum decline, again close to the long-term downward trend (Figure 2 and Figure 6).  As a result, the statistical approaches performed the best, with the models performing fairly well, and the heuristic predictions doing very poorly (Figure 7).

2011

Figure 7: 2011 SEARCH, Bastardi, tamino, and Cawley September Arctic sea ice extent predictions vs. observation (4.6 million square kilometers).

2012 Predictions

The climate contrarians were perhaps discouraged by their poor prediction performances in 2010 and 2011, so only WUWT made a prediction in 2012 with a submission to SEARCH.  Cawley also submitted his prediction to SEARCH, as did 18 other groups, and tamino submitted his own prediction on his blog.

As we now know, 2012 demolished the previous minimum Arctic sea ice extent record by approximately 760,000 square kilometers.  As a result, all submissions over-predicted this year's extent.  Once again the statistical predictions performed the best, with the heuristic predictions doing reasonably well thanks to Morison and Lukovich (despite WUWT dragging their average accuracy down), while the models were not very accurate (Figure 8).

2012

Figure 8: 2012 SEARCH, and tamino September Arctic sea ice extent predictions vs. observation (3.6 million square kilometers).

Overall Accuracy

Between 2008 and 2012, the model-based and statistical predictions have had the most accuracy, with an average difference from the observational data of 13%.  However, in recent years the statistically-based predictions have been the most accurate.  The heuristic predictions have been slightly less accurate, on average 15% off from the observations between 2008 and 2012.

The lower accuracy of the heuristic submissions is primarily due to the inaccurate predictions of the climate contrarians, which one Skeptical Science contributor has described as "hubristic," since they are based more on irrational optimism, unwise focus on short-term noise, and hubris than logic or experience.  Ignatius Rigor's heuristic predictions have also not fared well (Figure 1), which are based on the belief that the Arctic Oscillation plays a major role in sea ice extent - a hypothesis which appears not to be borne out by the data thus far.

Climate contrarian bloggers and blog readers (Watts, Goddard, Bastardi, and WUWT readers) averaged a 23% miss with their hubristic sea ice extent predictions, while climate realist bloggers and blog contributors (tamino, Cawley, and Grumbine) averaged just a 9% miss with their statistically-based predictions.  The take-home lesson here is that statistical predictions based on the long-term Arctic sea ice death spiral are clearly more realistic than denial-based optimistic predictions that sea ice will somehow magically recover to previous levels.

0 0

Printable Version  |  Link to this page

Comments

Comments 1 to 29:

  1. I've been waiting for this article for a while. Well done thanks SkS team :) PS, unless you are a time traveler or more east than I am in Sydney, is the posting date of the article supposed to be the 25th of September 2012?
    0 0
  2. Thanks yocta. I play a little fast and loose with the posting date/time, which is entered manually. We generally publish one post per day, and this one is intended as the post for the 26th, Aussie time.
    0 0
  3. Eli Rabett has a "Your Joe Bastardi Entertainment Moment" video clip up on his blog. Bastardi makes public his prediction for 2013. He draws this yellow line as his prediction while he talks: "...I think that's where we're gonna be next year. We're gonna recover dramatically here, with the cold that is coming over the next 9 to 12 months. I think next year you're gonna see the latest start to the sea ice melt that we've seen in a long time...." One might ask, why, Joe? "that is simply because of (hesitates)... you know... my studies show. And we'll see if I'm right". Then, he claims he predicted correctly what happened this year. First, he said ""I was right about this year. It did not get down below here": "If people want to say, and they keep finding some metric to make you think its lower than it actually is... And I'm the guy who said this is where its going to wind up" (Confidently and proudly draws this yellow line): You heard it from Joe. That's where it wound up this year. Then, he issues his prediction for the next 30 years. "...It's not going to get back to normal. That shouldn't happen for another twenty or thirty years. But we should be back up here next year. The higher you climb away from your low point the harder it is to get back, and I fully expect twenty, thirty, some of those years beyond, to be a little bit above normal...." (Draws the upper yellow squiggle line well above all other lines): This is just like war propaganda. (snip)
    0 0
    Moderator Response: [RH] Inflammatory snipped.
  4. I missed the fact that this entertainment was issued in 2010. Sorry. I find it hard to take these clowns seriously. It would be funnier if he had done this a few days ago.
    0 0
  5. David - yes if you click the first link in the above post, you'll see a screenshot of that Bastardi prediction, and then how it compared to the observations (the latter is also shown in Figure 6 above).
    0 0
  6. Bastardi would do better by divining goat entrails.
    0 0
  7. I've never seen this data before. Ramez Naam has a "guest blog" posted on Scientific American's site. The post contains a chart Naam cooked up starting with data from the 2011 Kinnard et.al. study Reconstructed changes in Arctic sea ice over the past 1,450 years which combines historical and proxy data to provide an educated guess as to what the record was from around 600 AD to 2006. Naam has added data up to the present, to produce this: Although Naam says he had to ask Kinnard for the chart, and the Kinnard et.al. study is behind a paywall at Nature, the Supplementary Information in pdf form isn't. Page 10 in the supplement has what looks a lot like the data Kinnard gave Naam on a chart containing a lot of other data.
    0 0
  8. David, click the last link in this post. It covers Kinnard.
    0 0
  9. I seem to remember that this Burt Rutan (of aerospace fame)fellow sometime this spring claimed that the Arctic sea ice was recovering. Appears to me that his claim was inaccurate.
    0 0
  10. Joe Bastardi said: "...I think that's where we're gonna be next year. We're gonna recover dramatically here, with the cold that is coming over the next 9 to 12 months. I think next year you're gonna see the latest start to the sea ice melt that we've seen in a long time...." This is the same guy who can't read a map.
    0 0
  11. RE #2 No Problem :) RE#3 I think you mean he is referring to his 2011 Prediction not his 2013. The video the Eli rabbet has posted refers to Bastardi's 2011 prediction (and indeed your screenshots show this) As far as I can tell, Bastardi hasn't put out a 2013 prediction although you can find one tweet about it here to which he tried to predict the melt was over this year on Aug 26
    0 0
  12. DB, I had always assumed that's how he did it. I recall a study about goats disappearing near the WeatherBell compound . . .
    0 0
  13. That graph constructed by Ramez Naam doesn't show a hockey stick so much as it shows Arctic sea ice has fallen off a cliff.
    0 0
  14. [7] I don't think that Kinnard et al is an accurate representation of the variability in sea ice over the past 1400 years. In fact I think you will all be surprised with the results that the next Kaufmann et al paper will show - ie much greater Arctic climate variability.
    0 0
  15. Robert, why do you say that about Kinnard? And what will be the difference between the methodologies? Do you have a link to a pre-print for Kaufmann?
    0 0
  16. Esop @9, I believe that Rutan's comment was about a sea ice decline that had "stabilized." I'm trying to find the original quote.
    0 0
  17. [14] how do you know what Kaufmann is about to tell us, and why not just tell us what he's going to say instead of telling us we'll be surprised? Polyak et.al. 2010 looked at the data from 300 past and ongoing studies before concluding that "the ice loss we see today... appears to be unmatched over at least the last few thousand years and unexplainable by any of the known natural variabilities". Science Daily news report on the study is here. Kaufmann et.al. published this in 2010, which revised their chart from their 2009 paper, on long term temperatures in the Arctic. Anomaly is relative to the average during 1961 to 1990:
    0 0
  18. Why such concern about the Arctic sea ice extent? What is the consequence if the Arctic sea ice retreats? In fact, why so much concern about global warming? What is the consequence(s)? Most importantly, what is appropriate policy response? Uncertainty about the problem (AGW) is a given; uncertainty about the chosen solution is inexcusable. This is to say, we should be confident that our solutions are going to be effective, and the more expensive the solution the more confident we should be.
    0 0
  19. Peter @18 - The Arctic plays a big role in global weather patterns, for starters. There are a lot of reasons to be concerned about the sea ice death spiral. However, this is not the place to talk about climate solutions. If you want to discuss that subject, I recommend the rebuttal to the myth 'CO2 limits will harm the economy'. We'll be updating that rebuttal with a new blog post next week, coincidentally.
    0 0
  20. Peter @ 18: possibly a troll but let's give you a pointer: if you have a heart attack you will accept any solution that's likely to be effective and worry about cost later. If you have sore toe, you may think twice about extensive surgery. I have a couple of posts on one of my blogs about this year’s massive drop: the minimum was about 3.4sq-km (e.g., this one, with a graph showing the minimum). If you update the graphs here and linked articles with the new minimum it looks pretty dramatic. Anyone remember the 2006 paper that was widely attacked for being “alarmist” for apparently predicting a near ice-free Arctic as early as 2040? We are well ahead of any trend modeled in that paper. Here's the reference for anyone who wants to compare it with the new reality as in this graph showing one of their runs vs. today: Marika M. Holland; Cecilia M. Bitz; Bruno Tremblay. Future abrupt reductions in the summer Arctic sea ice. Geophysical Research Letters. 2006, 33(23): 217. 25
    0 0
  21. philipm, I'm affraid Peter@18 may be incensitive to the medical emergency analogy. I know deniers who even in the wake of such emergency (hart attack) ignore any wisdom and seek help with e.g. homeopathy. Peter's complete misunderstanding of uncertainty (or denial of the meaning of uncertainty), as indicated by his bold text, suggests that he may also misunderstand/deny the graph from Holland 2006. So maybe this newer article from the same site debunking the true intentions of those who (like Peter) want to delay the policy response, will open his eyes. But, back to the topic, the article in question makes big claim that fossil fuel companies do go with mainstream science & predict the arctic ice is going to melt soon when it works to their advantage but there is no citation to back up such claim. I'm interested if the proof of such claim exists, or if the claim is unverified rumours, giving deniers an argument that this article is "another conspiracy theory by warmists"...
    0 0
  22. Peter Lang@18: Read this... ..the ask the question again, "why the concern about arctic Ice retreat?" Decreasing our carbon footprint *will demonstrably* be a good first, if tiny step: all need to do as much as they are capable of doing. The biggest bugaboo will be getting governments in line with that thought, and passing stricter controls on carbon, such as the cap and dividend scheme. If we *all* focus on that, and NOW, there is some chance to avoid a really bad outcome. BAU, and the game's up. It's really quite that simple.
    0 0
  23. woops..I meant to state "Decreasing our individual carbon footprint."
    0 0
  24. Looks like some goalpost shifting is underway. It's the Antarctic that 'skeptics' have moved their attention too.
    0 0
  25. Peter Lang. You mention the issue of uncertainty. Having the ice cap melt faster than climate models have predicted is surely a cause for concern when you consider the other implications of the same models?
    0 0
  26. Correct me if I'm mistaken, but the 2012 minimum of 3.4 is the daily extent minimum. The SEARCH predictions are of the average extent for the month of September. We won't know what that is until early October. It sticks out even more because of the word 'Actual' appended to the value. Can this be amended until the real figure comes in? As it happens, there was a late entry in this year's SEARCH predictions, which did consider the absolute (daily minimum). Late Summer Update
    0 0
  27. barry @26 - thanks. SEARCH is kind of confusing in that in some places it talks about predicting the minimum, and in others it talks about predicting the September average. It seems you are correct - this won't change the general results of the post, but will change a few of the numbers slightly (Figure 8 in particular). I've added a note to the top that the post will be revised when the September monthly data are out.
    0 0
  28. Predicting the future behaviour of any non-linear system (such as Arctic sea ice) is difficult, very difficult. But with that said, it is telling how when the correct approach is used and when one is guided by the data rather than ideology or dogma, then the forecasts made by the true skeptics tend to be more realistic and accurate overall. The fake skeptics (and fake skeptic bloggers) have been trying to reassure themselves (and the gullible and/or extremists who frequent their blogs) that we are due for a recovery in Arctic sea ice any day, or year, now. Yet the long-term trend is undeniably and statistically significantly down. These are the same folks who reassure themselves and anyone who is willing to listen that global cooling is imminent, any month, year, decade now....yet the long-term trend is that of warming, especially over the northern high latitudes.
    0 0
  29. They could certainly improve on modelling, they got the trends so I think they're on the right track. We must not however rely solely on this kind of data, there's a lot of errors.
    0 0

You need to be logged in to post a comment. Login via the left margin or if you're new, register here.



The Consensus Project Website

THE ESCALATOR

(free to republish)


© Copyright 2024 John Cook
Home | Translations | About Us | Privacy | Contact Us