Arguments
Nate Silver's Climate Chapter and What We Can Learn From It
Posted on 5 October 2012 by dana1981
In the interest of full disclosure, many Skeptical Science team members are big fans of Nate Silver's FiveThirtyEight blog at The New York Times. Silver runs a model which uses polling results and various other input factors (such as economic indicators) to predict election outcomes in the USA, with an impressive track record of accuracy.
Thus we were intrigued to hear that Silver had included a chapter on climate change in his newly-published book The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, particularly since we at Skeptical Science are often forced to explain the difference between signal and noise. Having great respect for the work and climate-related opinions of Michael Mann (who Silver consulted in writing the book), we were also concerned to see his criticisms of Nate Silver's climate chapter.
Nevertheless, Mann recommended that people read the book for themselves, praising much of the content. So I did just that, and overall I believe that if we take Silver's analysis a step further, we can learn a lot about the accuracy of climate models. It's also important to remember that, as Silver himself notes in the chapter, our basic understanding of how the climate works and how much it will warm in response to our greenhouse gas emissions is not just dependent on models.
Correlation is not Causation without Physical Connection
Silver's climate chapter starts out very well, noting that correlation does not necessarily imply causation, and that determining climate change causation requires a physical understanding of the climate system.
"...predictions are potentially much stronger when backed up by a sound understanding of the root causes behind a phenomenon. We do have a good understanding of the cause of global warming: it is the greenhouse effect."
Failing to consider physics in trying to determine the cause of global warming has been the pitfall for many a climate contrarian, for example Roy Spencer, Craig Loehle, Nicola Scafetta, Syun-Ichi Akasofu, and many others, so Silver's point is an important and relevant one. It is easy to fall into the curve fitting trap.
Silver goes on to explain some of that fundamental physics as discussed in the IPCC report - that atmospheric CO2 has increased steadily and rapidly, that this CO2 increase will in turn increase the greenhouse effect and cause global surface warming (which we've known for well over a century), and that water vapor will amplify that global warming as a feedback effect, ultimately noting "The greenhouse effect isn't rocket science."
Healthy Skepticism or Noise?
After this good start, the chapter then proceeds to discuss what Silver considers the healthy form of scientific skepticism, noting that
"In climate science, this healthy skepticism is generally directed at the reliability of computer models used to forecast the climate's course."
Silver then discusses J. Scott Armstrong as an example of this type of healthy skeptic of science who is concerned about the accuracy of climate model predictions. Armstrong is basically used to establish the 'skeptic' criticisms of climate models, though his arguments are very weak, basically boiling down to 'climate models are too complex to be accurate.' Armstrong also tends to focus on short-term noise rather than long-term trends, which Silver does eventually point out toward the end of the chapter. After establishing Armstrong's criticisms, Silver moves on to the more interesting part of the chapter, evaluating the accuracy of past climate models.
Testing Hansen's 1988 Model Accuracy
Sliver attempts to evaluate the accuracy of climate models by examining the model projections made by James Hansen in 1988 and the IPCC in 1990 and 1995. We should note here that Skeptical Science has evaluated many other temperature projections going back as far as Wallace Broeker's 1975 paper in the Lessons from Past Predictions series, with the results summarized in Figure 1 (though not all of these are based on climate models). Note that most of the accurate predictions have come from mainsream climate scientists and models, while the least accurate predictions have come from various 'skeptics'.
Figure 1: Various best estimate global temperature predictions evaluated in the 'Lessons from Past Climate Predictions' series vs. GISTEMP (red). The warmer colors are generally mainstream climate science predictions, while the cooler colors are generally "skeptic" predictions. The Hansen projection in pink is from Hansen et al. 1988.
Silver first examines James Hansen's 1988 projections, but not in great detail, simply noting that they are difficult to evaluate because they rely on various emissions and radiative forcing (global energy imbalance) assumptions, concluding
"Even the most conservative scenario somewhat overestimated the warming experienced through 2011."
Silver is right that Hansen's 1988 model projected more warming than has been observed. But what can we learn from this?
The overall climate sensitivity (the total amount of climate warming in response to a given greenhouse gas increase, including feedbacks) in Hansen's model was 4.2°C for a doubling of atmospheric CO2 levels. This is significantly higher than most of today's climate models, which put the value around 3°C for doubled CO2. In order to accurately project the ensuing warming, Hansen's model sensitivity would have had to be close to that in today's climate models (Figure 2).
Figure 2: A rough adjustment of the Hansen 1988 Scenario B temperature projection to reflect a 3°C rather than 4.2°C climate sensitivity (red) vs. GISTEMP observations (black)
Thus we can be confident that today's climate models are, not surprisingly, more accurate than James Hansen's 1988 model.
Testing IPCC Model Accuracy
The chapter proceeds to evaluate the 1990 IPCC report's temperature projections. Silver notes that under the various scenarios, the models projected between approximately 2°C and 5°C global surface warming from 2000 to 2100 (Figure 3).
Figure 3: 1990 IPCC projected global warming in the BAU emissions scenario using climate models with equilibrium climate sensitivities of 1.5°C (low), 2.5°C (best), and 4.5°C (high) for double atmospheric CO2
Silver then compares this rate of warming to the rate of warming from 1990 through 2011 and concludes that the 1990 IPCC report somewhat over-predicted the ensuing warming. We can take this analysis further and address the question why their warming projections were a bit high.
While Silver's discussion compares observations to all 3 IPCC climate sensitivity scenarios (low = 1.5°C for doubled CO2, best = 2.5°C, high = 4.5°C), he has not yet considered the emissions scenario (BAU = business as usual). The 1990 BAU scenario considered a 3.5 Watts per square meter (W/m2) greenhouse gas forcing in 2011, whereas the actual greenhouse gas radiative forcing was approximately 2.8 W/m2 in 2011. The IPCC BAU forcing was too high for two reasons
1) In 1990, the radiative forcing caused by a doubling of CO2 was believed to be about 4.4 W/m2. We now know it's closer to 3.7 W/m2.
2) Greenhouse gas emissions have not risen quite as fast as the IPCC BAU scenario.
This resulting lower real-world radiative forcing (an input, not output of the model) accounts for most of the model-data discrepancy Silver observes (Figure 4).
Figure 4: 1990 IPCC FAR BAU "best" global warming projection reflecting the observed GHG forcing changes (blue) vs. observed average global surface temperature change from GISTEMP (red) since 1990.
When adjusting the BAU scenario projections to reflect the actual greenhouse gas changes since 1990, the model would expect to see roughly 0.2°C per decade warming, which is slightly more than has been observed, but within the 95% uncertainty range in all temperature data sets. That the observed rate of warming has most likely been a bit lower than the IPCC projection is also not surprising considering all the short-term cooling influences over the past decade. In fact, later in the chapter Silver discusses one of these recent cooling effects - increased aerosol emissions from Chinese coal plants have likely dampened the observed warming over the past decade.
Silver does note that the 1990 IPCC BAU scenario "was somewhat too pessimistic," (point #2 above) but goes on to claim that
"Nevertheless, the IPCC later acknowledged their predictions had been too aggressive. When they issued their next forecast, in 1995, the range attached to their business-as-usual case had been revised considerably lower: warming at a rate of about 1.8°C per century. This version of the forecasts has done quite well relative to the actual temperature trend. Still, that represents a fairly dramatic shift."
We should point out that the 1995 IPCC report considered a number of different emissions scenarios, with a corresponding average global surface warming ranging from about 1.6 to 2.5°C between 1990 and 2100 (Figure 5). Describing it as simply projecting about 1.8°C per century warming does not capture the full spread of warming projections.
Figure 5: 1995 IPCC report projected global mean surface temperature changes from 1990 to 2100 for the full set of IS92 emission scenarios. A climate sensitivity of 2.5°C is assumed.
Again we can take this analysis a step further and get into the model nuts and bolts to understand the reason behind the lower rate of projected warming in the 1995 IPCC report as compared to the 1990 report. As it turns out, the difference was mainly due to the revised emissions scenarios - model inputs, not outputs. The 1995 IPCC report still used the too-high CO2 radiative forcing value (point #1 above), but used what has turned out to be a more accurate range of greenhouse gas emissions scenarios than the 1990 BAU (addressing point #2 above).
When we adjust for the actual greenhouse gas radiative forcing, the 1995 IPCC report projects a very similar amount of warming as the similarly corrected 1990 report 'best estimate', which we would expect, since both used climate sensitivities of 2.5°C.
Ultimately the difference between the projected warming in the two scenarios mostly boils down to using to different emissions and radiative forcing scenarios - model inputs, not outputs. As illustrated in Figure 1 above, the 1990 and 1995 IPCC temperature projections performed very similarly when adjusted for actual greenhouse gas and radiative forcing changes.
The 'dramatic shift' Silver refers to simply reflects a change in emissions scenarios - a model input. Both the 1990 and 1995 IPCC model projections have been quite accurate when we adjust for those inputs. If we are just evaluating model accuracy here, the IPCC did very well in both 1990 and 1995. If we are evaluating the IPCC's ability to predict CO2 emissions, well, that's the rub. We don't know how human CO2 emissions will change in the future, but if they continue on their current path, the models project a whole lot of warming.
The IPCC, Al Gore, and Polar Bears
There are a few relatively minor errors in the chapter worth noting. For example, Silver states:
"And however many models there are, the IPCC settles on just one forecast that is endorsed by the entire group."
This is not quite correct. While the IPCC report does publish a graphic illustrating the multi-model mean temperature projection for each emissions scenario, it also shows the envelope of individual model projections on the same figure (Figure 6).
Figure 6: Temperature Projections from the 2007 IPCC Report. Solid lines are multi-model global averages of surface warming (relative to 1980–1999) for the emissions scenarios A2, A1B, and B1, shown as continuations of the 20th century simulations. Shading denotes the ±1 standard deviation range of individual model annual averages. The orange line is for the experiment where concentrations were held constant at year 2000 values. The grey bars at right indicate the best estimate (solid line within each bar) and the likely range assessed for the six emissions marker scenarios.
Silver also criticizes Al Gore's film An Inconvenient Truth as
"...sometimes [being] less cautious, portraying a polar bear clinging to life in the Arctic, or South Florida and Lower Manhattan flooding over. Films like these are not necessarily a good representation of the scientific consensus."
However, Arctic sea ice is actually declining significantly faster than the climate models used in the IPCC report predicted, and it's certainly true that south Florida and Manhattan could eventually become flooded as a result of sea level rise. Additionally, while Gore's film did get a few details wrong, as Michael Mann noted, it got the basic science right.
Good Points in the Chapter
Silver's climate chapter also makes many good points for which he deserves credit.
- It debunks the myth that most climate scientists were predicting global cooling and/or an impending ice age in the 1970s, noting that this was primarily a media construct.
- As noted above, Silver discusses the cooling effect of human aerosol emissions over the past decade and the fact that correlation is not causation.
- Silver points out that climate models simply cannot replicate the current climate without accounting for greenhouse gas increases.
- The chapter contains a graphic similar to The Escalator to show that there are often short-term changes in the opposite direction of the long-term trend, but that this just represents noise in the system.
- Silver references William Nordhaus in noting that uncertainty is actually a reason to reduce greenhouse gas emissions, because the worst climate scenarios cannot be ruled out.
- The chapter debunks myths associated with Climategate.
So aside from the issues noted above, the chapter does contain a lot of good information.
Discouraging Conclusion
The chapter ends on a bit of a discouraging note, with Silver suggesting that climate scientists should not try to directly influence climate policy.
"It is precisely because the debate may continue for decades that climate scientists might do better to withdraw from the street fight and avoid crossing the Rubicon from science into politics."
Note that by "politics" Silver appears to refer to "policy" in this context. However, the problem with Silver's suggestion is that we simply don't have decades to waste. The evidence that humans are causing dangerous climate change has been building for many decades, with climate scientists advising the importance of emissions cuts, and yet policymakers have continually failed to achieve the necessary action to steer us away from this path. Climate scientists have essentially been forced into pressing harder and harder for serious climate policy - to put it simply, we are quickly running out of time to avoid potentially catastrophic climate change.
Silver's discouraging conclusion may result from not taking that extra step to evaluate why the past warming projections discussed here were too high, as we have done. Silver may have drawn the conclusion that the climate is not warming as fast as we expect, which would suggest that we have more time than climate scientists believe to solve the problem.
However, when correcting the model projections to account for the actual greenhouse gas emissions and forcing changes, we see that their temperature projections have been very accurate. Thus climate scientists are correct to worry that unless we quickly enact climate policies to reduce human greenhouse gas emissions, we will suffer some very painful consequences. In fact, as Silver himself notes in the chapter, we don't even need climate models to realize that we're in for a lot of global warming this century.
Silver is right that we are making too little progress in terms of climate policy, and he is is of course correct to note that climate scientists must be careful to ensure that their predictions and warnings are scientifically accurate and defensible. However, remaining silent in the face of a potential catastrophe is simply not an option. Effective policy depends on active engagement between policymakers and the scientific community. It's important for scientists to know the limits of science in shaping policy, but it's also vital to ensure that policy grapples with scientific realities. If climate scientists say nothing, this creates a vacuum to be filled by people like Armstrong and other climate contrarians Silver discusses, who frankly are way out of their depth when it comes to climate science and climate models, but don't know enough to realize it.
Note that I have not yet read the rest of Silver's book, which looks like it may be very interesting. Overall the climate chapter does give us a good start in evaluating the accuracy of climate models, as long as we take the analysis a few steps further.
[...] sometimes [being] less cautious, portraying a polar bear clinging to life in the Arctic, or South Florida and Lower Manhattan flooding over. Films like these are not necessarily a good representation of the scientific consensus."
As far as I know, as long humans continue to emit massive amounts of CO2 and cause further warming, the flooding examples will occur of necessity. Not having seen An Inconvenient Truth I am not in a position to say whether it exaggerates the timeframe in which, say, South Florida will be swallowed up by the ocean. But I am very confident that, failing decisive action by humans to cut down our emissions and sequester carbon, it will eventually happen.
It's worth reiterating that climate models are not the only means of making projections of future climate. Past climate tells us at least as much, and gives the same kind of answers. The Knutti and Hegerl 2008 graphic (below) provides a (now dated) illustration of this:
There is a nice collection of empirical climate sensitivity estimates on this page. For a more recent work see Padilla et al 2011.So even if the models are complete junk (a claim which goes well beyond Silver's claims), we can still be confident that the short term temperature projections based on TCR and long term projections based on EQS are in the right ballpark.
You mentioned Chinese aerosols as a cooling influence. I would also mention the huge amount of ice that's melted this past decade. Your Figure 5 in this discussion
http://www.skepticalscience.com/GRACE-and-glaciers.html
illustrates how dramatic this effect is.
I have an impression that Silver simply does not make enough effort to understand the physics of climate science deeply enough to make accurate judgement. Therefore he falls into the traps of "equal balance" and the "debate", wheareas there is no debate.
That impression is reinforced after reading Mike Mann's review you pointed above. Mike's comments are important not only because he's an expert but also because he is an insider to some extent: Silver consulted him before writing the book. Mike is disapointed that Silver misrepresented a few point from that consultation. For example "the uncetainty about the influence of GW on ElNino" was spelled out in the book as "we don't know much about ElNino phenomenon". To me, that's an indication that Silver tried to repeat something from that consultation but misunderstood it and consequently distort it. Mike also points a problem with Silver's view of uncertainty:
That is at odds with one of your good points that Silver accepts William Nordhaus' view. So I don't know what Silver wanted to opine here:
- that scientist are "exaggerating" so we have some more time to address the problem, or
- that we should be addressing the problem now because that's the correct response to uncertainty that works both ways.
To me, those two opinions are mutually exclusive and cannot possibly cohexist in this book.
It remains entirely obvious that inaction is not an option.
He may be right about that, but you can't just quit trying to solve the biggest threat to humanity because you might fail.
I was thinking it may be because he cut his teeth at the Booth School of Business at Chicago, where Milton Friedman pioneered free market libertarianism. Although Friedman was in favor of Pigovian taxes (applied to negative externalities, ie a carbon tax), so who knows.
Or it could be as simple as he cares about science and thinks it will lose credibility, which can be possible when analysis and advocacy mix. I completely disagree with this; I think it's possible to advocate using sceintific analysis, like Hansen does. I also think scientists are worried, as they should be, that nothing is getting done, which has forced them to be more vocal. But some "purists" find this argument irrelevant and feel that scientists should stay quiet.
There are a number of typos, one amusing one (from the Kindle edition) quoting somebody at NASA who supposedly said (with my emphasis): “At NASA, I finally realized that the definition of rocket science is using relatively simple psychics to solve complex problems”. No wonder the O-rings failed.
There are other strange statements, such as one claiming that the IPCC Arctic Ice shrinkage predictions "have done quite well", when they have, as is well known, greatly underestimated the rate of shrinkage. The book could have benefited from a critical reading by an expert.
As for the implication that scientists who stray into advocacy risk losing credibility, I would argue that the opposite is true: a scientist who has credible information that has implications for human welfare has a professional and moral obligation to speak out on matters of public policy.
Rachel and here collegues were called "big alarmists" in late 50s by chemical companies who had vested interest in producing and selling as much DDT as possible. It's sad that she did not live up to her vindication that came in 70s.
If Rachel lived today, she would just laugh how precisely the "deja vue" history repeated itself with tobacco companies denying cancer from smoking, CFC refrigerant producers denying ozone hole, and now FF lobby denying AGW. It's just a human nature to deny the problems caused by addictions, especially the addictions that are hard to ferret out. And FF addictions are the hardest type of addiction humanity ever dealt with.
Rachel Carson was seriously attacked during her lifetime - and you'll still see some people sneer at her in conversations about environmental issues. There was one gift she could bring to the table that very few people, let alone scientists, could do. Read "The Sea Around Us" to get the idea.
She wrote like an angel.
It's also a duty for any human being to alert others in potential danger. I really don't see any reason why someone should ask people not to ring the alarm bell if they have good reasons for concern. It would be kind of immoral.
As long as scientists remain inactive:
1) People will figure that climate change is not that big of a deal (otherwise the scientists would be acting out, right?);
2) The active voices will be filled by denialists and lukewarmers;
3) Business as usual will prevail and the needed solutions will not be reached.
Rachel Carson is a great example of an astute scientist who sought to change the dominant harmful paradigm - and did. Jim Hansen is another such hero. The time for complicity is long gone.
By way of analogy, in many states of Australia teachers and other professionals with similar responsibility are required by legal mandate to report suspected cases of child abuse. Now, it's not the responsibility of teachers and other professions to explicitly go out and police child welfare, but they have a professional duty to report any possibility of harm when they become aware of such.
If the same principle applied to scientists there would be no antipathy to them speaking out on matters in which they have expert understanding - indeed, governments and the community should be listening with full attention, and tabloid current affairs reporters should be chasing fossil fuel executives in the streets and over fences.
It just goes to show what we want to hear, and what we don't.
"Commoner insisted that scientists had an obligation to make scientific information accessible to the general public, so that citizens could participate in public debates that involved scientific questions."
Thomas Jefferson would surely agree with that sentiment. That statement and a good summary of Commoner's life, work, and contribution to humanity are in the article Barry Commoner, Pioneering Environmental Scientist and Activist, Dies at 95.