Climate Science Glossary

Term Lookup

Enter a term in the search box to find its definition.


Use the controls in the far right panel to increase or decrease the number of terms automatically displayed (or to completely turn that feature off).

Term Lookup


All IPCC definitions taken from Climate Change 2007: The Physical Science Basis. Working Group I Contribution to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Annex I, Glossary, pp. 941-954. Cambridge University Press.

Home Arguments Software Resources Comments The Consensus Project Translations About Donate

Twitter Facebook YouTube Pinterest

RSS Posts RSS Comments Email Subscribe

Climate's changed before
It's the sun
It's not bad
There is no consensus
It's cooling
Models are unreliable
Temp record is unreliable
Animals and plants can adapt
It hasn't warmed since 1998
Antarctica is gaining ice
View All Arguments...

Keep me logged in
New? Register here
Forgot your password?

Latest Posts


Climate Hustle

The Cook et al. (2013) 97% consensus result is robust

What the science says...

The 97% consensus has been independently confirmed by a number of different approaches and lines of evidence.

Climate Myth...

97% consensus on human-caused global warming has been disproven
Cooks ’97% consensus’ disproven by a new peer reviewed paper showing major math errors (Anthony Watts)

Communicating the expert consensus is very important in terms of increasing public awareness of human-caused climate change and support for climate solutions.  Thus it's perhaps not surprising that Cook et al. (2013) and its 97% consensus result have been the subject of extensive denial among the usual climate contrarian suspects.  After all, the fossil fuel industry, right-wing think tanks, and climate contrarians have been engaged in a disinformation campaign regarding the expert climate consensus for over two decades.  For example, Western Fuels Association conducted a half-million dollar campaign in 1991 designed to ‘reposition global warming as theory (not fact).’

The 97% Consensus is a Robust Result

Nevertheless, the existence of the expert consensus on human-caused global warming is a reality, as is clear from an examination of the full body of evidence.  For example, Naomi Oreskes found no rejections of the consensus in a survey of 928 abstracts performed in 2004Doran & Zimmerman (2009) found a 97% consensus among scientists actively publishing climate research.  Anderegg et al. (2010) reviewed publicly signed declarations supporting or rejecting human-caused global warming, and again found over 97% consensus among climate experts.  Cook et al. (2013) found the same 97% result through a survey of over 12,000 climate abstracts from peer-reviewed journals, as well as from over 2,000 scientist author self-ratings, among abstracts and papers taking a position on the causes of global warming.

In addition to these studies, we have the National Academies of Science from 33 different countries all endorsing the consensus.  Dozens of scientific organizations have endorsed the consensus on human-caused global warming.  Only one has ever rejected the consensus - the American Association of Petroleum Geologists - and even they shifted to a neutral position when members threatened to not renew their memberships due to its position of climate denial.

In short, the 97% consensus on human-caused global warming is a robust result, found using several different methods in various studies over the past decade.  It really shouldn't be a surprise at this point, and denying it is, well, denial.

Quantifying the Human Global Warming Contribution

There have also been various studies quantifying the human contribution to global warming, as we have previously documented.

attribution 50 yr

Figure 1: Net human and natural percent contributions to the observed global surface warming over the past 50-65 years according to Tett et al. 2000 (T00, dark blue), Meehl et al. 2004 (M04, red), Stone et al. 2007 (S07, light green), Lean and Rind 2008 (LR08, purple), Huber and Knutti 2011 (HK11, light blue), Gillett et al. 2012 (G12, orange), Wigley and Santer 2012 (WS12, dark green), and Jones et al. 2013 (J12, pink).

Again, there's very little controversy here.  The scientific literature is quite clear that humans have caused most of the global surface warming over the past half century, as the 2013 IPCC report is expected to state with 95% confidence.

In Cook et al. (2013), we broadened the focus beyond definitions that quantify the human contribution, because there's a consensus gap on the mere question of whether humans are causing global warming.  Nevertheless, we used the 2007 IPCC position as one of our consensus position definitions:

"We examined a large sample of the scientific literature on global [climate change], published over a 21 year period, in order to determine the level of scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)."

The IPCC position (humans causing most global warming) was represented in our categories 1 and 7, which include papers that explicitly endorse or reject/minimize human-caused global warming, and also quantify the human contribution.  Among the relatively few abstracts (75 in total) falling in these two categories, 65 (87%) endorsed the consensus view.  Among the larger sample size of author self-rated papers in categories 1 and 7 (237 in total), 228 (96%) endorsed the consensus view that humans are causing most of the current global warming.

The self-ratings offer a larger sample size on this quantification question because of the limited real estate in a paper's abstract.  Most journals have strict word limits on their abstracts, so authors have to focus on the specifics of their research.  On the other hand, the author self-ratings are based on the full papers, which have much more real estate and are thus more likely to both take a position on the cause of global warming, and quantify the human contribution.

Confused Contrarians Think they are Included in the 97%

There have been a number of contrarians claiming that they are part of the 97% consensus, which they believe is limited to the position that humans are causing some global warming.  The first error in this argument is in ignoring the fact that the data collected in Cook et al. (2013) included categories that quantify the human contribution, as Andrew Montford and the GWPF recently did, for example.

The second error has been made by individuals claiming they're in the 97%, but failing to actually check the data.  For example, Roy Spencer claimed in testimony to US Congress that he is included in the 97% consensus.  Since we made all of our data available to the public, you can see our ratings of Spencer's abstracts here. Five of his papers were captured in our literature search; we categorized four as 'no opinion' on the cause of global warming, and one as implicitly minimizing the human influence.  Thus Spencer's research was included in the fewer than 3 percent of papers that either rejected or minimized the human contribution to global warming.  Bjorn Lomborg made a similar error, claiming:

"Virtually everyone I know in the debate would automatically be included in the 97% (including me, but also many, much more skeptical)."

In reality Lomborg is included neither in the 97+% nor the less than 3% because as far as we can tell, he has not published any peer-reviewed climate research, and thus none of his writings were captured in our literature search.  The 97% is a consensus of climate science experts, and that, Lomborg is not.

Nir Shaviv took the opposite approach, claiming he was wrongly included in the 97%.  Though Shaviv also admitted that Cook et al. correctly classified his abstracts based on their content, but claimed that he worded the text in a way to slip it past the journal reviewers and editors.

"I couldn’t write these things more explicitly in the paper because of the refereeing, however, you don’t have to be a genius to reach these conclusions from the paper."

However, Shaviv, Spencer, and all other authors were invited to participate in the self-ratings process that resulted in the sae 97% consensus conclusion.

Tol's Rejected Comment

Richard Tol has also advanced various criticisms of Cook et al. (2013).  It's worth noting that Tol does not dispute the existence of the consensus, writing:

"There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans. I have very little reason to doubt that the consensus is indeed correct."

Tol has nevertheless criticized the methods applied during the Cook et al. survey.  For example, he has argued that the literature search should have been conducted with Scopus rather than the Web of Science in order to capture more papers, but also that fewer papers should have been included in the survey in order to focus on those specifically researching the causes of global warming.  Tol has also applied various statistical tests comparing the abstract ratings to the author self-ratings, but these tests are invalid because the two phases of the survey considered different information (abstracts only vs. full papers) and are thus not comparable.

In fact, when we released the self-rating data, we explicitly discussed the difference between the two datasets and how the difference was actually instructive.  As John Cook wrote,

"That's not to say our ratings of abstracts exactly matched the self-ratings by the papers' authors. On the contrary, the two sets measure different things and not only are differences expected, they're instructive."

Ultimately Tol submitted his criticisms to Environmental Research Letters as a comment, but the submission was summarily rejected by the editor who described it as a speculative opinion piece that does not identify any clear errors that would call the paper's conclusions into question. 

In short, the 97% consensus has passed peer-review, while Tol's criticisms have not.  Moreover, all of Tol's criticisms only apply to the abstract ratings, while the self-ratings also found the same 97% consensus result, completely independent from the abstract ratings.

Taking Consensus Denial to the Extreme

One critique of the consensus has been published in a paper in the journal Science & Education.  The argument made in the paper was first published by Christopher Monckton on a climate contrarian blog.  Monckton has also suggested the conspiracy theory that the journal Environmental Research Letters was created (in 2006) specifically for the purpose of publishing Cook et al. (2013).

The Monckton paper takes the point about quantification above to the extreme.  It focuses exclusively on the papers that quantified human-caused global warming, and takes these as a percentage of all 12,000 abstracts captured in the literature search, thus claiming the consensus is not 97%, but rather 0.3%.  The logical flaws in this argument should be obvious, and thus should not have passed through the peer-review process. 

Approximately two-thirds of abstracts did not take a position on the causes of global warming, for various reasons (e.g. the causes were simply not relevant to or a key component of their specific research paper).  Thus in order to estimate the consensus on human-caused global warming, it's necessary to focus on the abstracts that actually stated a position on human-caused global warming.

When addressing the consensus regarding humans being responsible for the majority of recent global warming, the same argument holds true for abstracts that do not quantify the human contribution.  We simply can't know their position on the issue - that doesn't mean they endorse or reject the consensus position; they simply don't provide that information, and thus must first be removed before estimating the quantified consensus.

As noted above, when we perform this calculation, the consensus position that humans are the main cause of global warming is endorsed in 87% of abstracts and 96% of full papers.  Monckton's argument is very similar to the myth that CO2 can't cause significant global warming because it only comprises 0.04% of the atmosphere.  99% of the atmosphere is comprised of non-greenhouse gases, but these other gases are irrelevant to the question of the CO2 greenhouse effect.  The percentage of CO2 as a fraction of all gases in the atmosphere is an irrelevant figure, as is the percentage of abstracts quantifying human-caused global warming as a percentage of all abstracts captured in our literature search.

It's also worth noting that based on Monckton's logic, only 0.08% of abstracts reject human-caused global warming.

Climate Consensus Denialism

Overall, the critiques of Cook et al. (2013) have all exhibited the characteristics of scientific denialism.  Given the long history of consensus denial campaigns by fossil fuel interests and climate contrarians, continued resistance to the consensus is an expected result.  Nevertheless, the 97% consensus is a robust result from several different studies taking a variety of approaches, including two independent methods used by Cook et al. (abstract ratings and author self-ratings).  The criticisms of the paper have all exhibited the same few logical flaws, some more extreme than others, but all erroneous.

Intermediate rebuttal written by dana1981

Update July 2015:

Here is a related lecture-video from Denial101x - Making Sense of Climate Science Denial

Last updated on 9 July 2015 by pattimer. View Archives

Printable Version  |  Offline PDF Version  |  Link to this page


Comments 1 to 38:

  1. Guys this is really bad. It would be better if you just focused on the facts.

    This does not do anything to address the issue: "Thus it's perhaps not surprising that Cook et al. (2013) and its 97% consensus result have been the subject of extensive denial among the usual climate contrarian suspects. After all, the fossil fuel industry, right-wing think tanks, and climate contrarians have been engaged in a disinformation campaign regarding the expert climate consensus for over two decades. For example, Western Fuels Association conducted a half-million dollar campaign in 1991 designed to ‘reposition global warming as theory (not fact).’"

    And under "The 97% Consensus Is a Robust Result", you start by citing a study that produce a 75% consensus level -- but you don't tell us that. You cite that study without even revealing the figure, which does not support the claim in your headline. Also, it was a study of abstracts, not scientists, so it's unclear why you think we can mix the two.

    The other 97% finding you cite is based on 77 people, which you don't mention.

    There is no survey of scientists that gives us 97%. The Cook study was about papers, and you make no mention that most of the papers included in the consensus were coded by human raters as implicitly endorsing AGW. Nor do you mention the substantial disagreement between the raters, or the fact that the disagreed upon observations were still included in the results.

    Since the debate is mostly about severity and confidence levels, studies that use broad or ambiguous litmus tests of simple agreement with human caused warming are not very useful. Actual surveys are better, especially if they ask more useful, finer grained questions, like this:

    Try to find your 97% there. Severe outcomes are endorsed by less than half of the sample.



    [DB] Tone-trolling snipped.

  2. Phronesis, the 2004 Oreskes study found 75% of papers explicitly or implicitly supporting the consensus, 25% taking no position, and 0% contradicting the consensus. Thus, amongst papers expressing any view that study found 100% support for the consensus. Your claim that it found only "a 75% consensus level" is a logic error on your part. If you count papers which don't address the issue at all you could bring in hundreds of thousands of papers on quantum physics, stellar cartography, economics, sociiology, et cetera and claim that since none of these take any position on global warming there is less than 1% support for it... or any other subject you want to dismiss via blatantly flawed logic.

  3. Phronesis seems to have a problem with things not mentioned, but is carefull not to mention a few things himself.  I'll come to that, but first lets look at Doran and Zimmerman (2009).  Phronesis dismisses that as a survey of just 75 people, but that is false.  It was a survey of 3,146 scientists.  Among those scientists, overall 82% answered yes to question 2:

    "Do you think human activity is a significant factor in changing mean global temperatures?"

    That question is closest to the proposition tested by Cook et al (2013).  That low figure is because most scientists asked were not specialists in the relevant field, and consequently had little more knowledge on the topic than any non-scientist.  Of the small portion of the those surveyed who were specialists in climate science, and actively publishing in the field so that they had up to date information, 97.4% agreed with question 2.  There were only 77 of those, but they were part of a much larger sample.

    Turning now to Farnsworth and Lichter (2012), we find that it is a survey of scientists, 50% of whom were members of the AMS, and 50% of whom were members of the AGU, and all of whom were listed in the American Men and Women of Science.  Their 489 respondents are therefore comparable to Doran and Zimmerman's 3,146 respondents, not to the 77 publishing specialists in climatology.  Further (the relevant fact Phronesis did not reveal), only 41% of those 489 actively research in any aspect of global climate science.  

    It is not clear how those 200 scientists compare to Doran and Zimmerman's classification.  For Doran and Zimmerman, and active publisher must have published at least 50% "... of their recent peer-reviewed papers on the subject of climate change".  That is, by a reasonable measure, at least 50% of their research must be on the topic.  In contrast, the 200 scientists from Farnsworth and Lichter need only have actively researched on any aspect of climate science, ie, greater than 0% and need not have brought the research to publication.  Nor need they be specialists in climatology. They are probably best equated with Doran and Zimmerman's active publishers.  For the other 289, however, the closest category would be Doran and Zimmerman's non-publishers/non-climatologists.

    From all of Farnsworth and Lichter's respondents, 84% responded yes to the question:

    "In your opinion, is humanly induced global warming now occuring?"

    Assuming Farnsworth and Lichter's non-researchers resonded at the same rate as Doran and Zimmerman's non-publishers/non-climatologists (ie,76.6% affirmative), that represents 221 affirmative responses.  That leaves only 190 affirmative responses to come from Farnsworth and Lichter's researchers, meaning that 95% of them responded affirmatively. 

    Given the low bar on expertise set by Farnsworth and Lichter, and the strong correlation between expertise and current topic knowledge and acceptance of AGW found by Doran and Zimmerman, that 95% is surprisingly high.  It shows, however, that the results of Farnsworth and Lichter is entirely consistent with those of Doran and Zimmerman, and also of Cook et al, (2013).

    Phronesis may rightly reject the assumption that Farnsworth and Lichter's non-researchers affirmed AGW at the same rate as Doran and Zimmerman's  as speculative.  The implication of that, however, is that no comparison can reasonably be made between their results and those of Doran and Zimmerman because they do not survey groups with the same demographics.  Farnsworth and Lichter would then provide information about general scientific acceptance, but not specifically about acceptance by those with specialist knowledge in the field.

    Finally, Phronesis misrepresents the results of Farnworth and Lichter on expected outcomes.  The question put, and results were:

    "Overall, if present climate trends continue, do you regard the likely effects of global climate change in the next 50 to 100 years as:
    Trivial to Catastrophic
    1–3 (NET) 13
    4–7 (NET) 44
    8–10 (NET) 41
    Don’t Know 2
    Mean 6.6 "

    Thus, less than half (but only just less than half) think the results of current climate trends will be catastrophic, or near catastrophic.  Slightly more, but still less than half think the results will be moderate, while only 13% think the results will be trivial or near trivial.

  4. Hi guys. One thing at a time. We have no idea what the Oreskes study did. There is no methods section in the paper. The paper is only a page long, and the part that actually deals with the study is only 126 words long.

    We're told that papers that were classified as explicitly endorsing, evaluating impacts, or mitigation proposals were lumped together as endorsing the consensus. That's all we know. Note that the fact that a paper evaluates impacts does not imply endorsement or might report small impacts. We just don't know. We can't do anything with a paper when we don't know how it was done.

    We also can't be converting 75% to 100% based on a paper that includes unspecified implicit criteria for inclusion into that bucket. Your logic is sound except for the implicit part. I'm not going to infer unanimity via an implicit classification scheme, much less a complete unknown classification scheme. Is this controversial?

    Also, maybe your logic isn't sound after all. Let's linger on the fact that these figures are converted into "75% of climate scientist agree", or as you would have it, "100% of climate scientists agree".

    How did we get there? Papers. And we ignored papers -- and scientists -- who expressed no position (by the completely unknown filtering and coding protocol Oreskes used). The sample was much smaller than Cook -- I'm not sure why. There would technical papers, heads-down so to speak, that would address narrow issues -- not really at the level of abstraction of human caused climate change, which is pretty high level. Am I correct in assuming there'd be all manner papers related to climate mechanism that would not ever have occasion to use the relevant phrases? We don't know anything about what those scientists think. We only know about the set that were included and rated by methods we have not been apprised of.

    That makes me even more reluctant to say anything definite, certainly not that 100% of climate scientist think x, y, or z.

    Let me know if I've missed supplemental materials on Oreskes. I didn't see anything.

    Lastly for now -- we need detailed questions, nuanced views. Endorsement of the proposition that humans impact the climate or contribute to warming or are the primary cause of warming is not useful. Not if we want to do something with this knowledge, like pass laws and so forth. We need severities, probability and confidence, scientists' views on the likely roles of different forcings and feedbacks, interpretations of the pause in mean global surface temps, estimates of consequences and benefits of warming, etc. Obviously a 1 F warming by 2100 probably doesn't justify major increases in the costs of energy, for most people. I know the lower bound of IPCC 5 is higher than that, but I'd like to know what scientists think. I'd also like to see studies they have the opportunity to just express their judgments in their own words.

  5. And is it true that the raters in Cook et al were not independent? I saw it here:

    Rater independence is crucial in a study based on rater data.

  6. Phronesis @4 & 5, it is more than a little hypocritical to not respond to one issue on the basis of limited time "one thing at a time", and then to yourself introduce additional side topics.  It strongly suggests your intention is a gish gallop, where you introduce topics that are rhetorically convenient, but plead time constraints to avoid having to answer on issues where you have been shown to have been both hypocritical (leaving out relevant information, while complaining about what you consider to have been relevant information having been left out) and to misrepresent the study you quote.

  7. Phronesis,

    Here I linked two very recent reports (both less than one month old) from the National Academy of Science (and the Royal Society of England) and the American Association for the Advancement of Science that state clearly what they think about AGW.  Both of these reports state clearly that AGW is likely to be extremely severe and that strong actions need to be taken immediately.  For your convienence I have linked them again below.  These reports state exactly what you claim above you need to make decisions.

    What is your issue with these clearly worded reports from the top science organizations in the USA and England?  All other Academies of Science in the World agree with these three organizations.  The problem is that you have not bothered to try to find out what scientists clearly say.  

    AAAS report

    NAS/Royal Society report

  8. Phronesis #1: " start by citing a [Oreskes] study that produce a 75% consensus level"

    Phronesis #4: "We have no idea what the Oreskes study did."

    So you've gone from claiming that the Oreskes study did something it provably did not, to claiming that we just can't know... even though you originally claimed we did know.


  9. Phronesis:

    With respect to your desire for information regarding the severity of global warming impacts, etc. - read the IPCC Assessment Reports. That is their purpose.

  10. Phronesis... I don't want to pile on here but would just like to offer one suggestion.

    When a scientist gets his hackles up over some published finding, usually the response is to test the results him/herself. This is something you can easily do yourself. You don't need to review 12,000 papers. Honestly, Cook et al was way overkill in regards to what is statistically necessary to establish the point. 

    There are two easy ways to do this. Here on the SkS site, you can rate abstracts yourself! If for some reason you don't trust the system that SkS has set up, that's fine. Just go to google scholar and start pulling up papers. Set up your own spreadsheet to check the results.

    You can get a rational statistical sampling with a couple hundred papers. That might take a few hours to process. But then you know for yourself with data created by someone you trust. You!

  11. Guys, has anyone responded substantively to anything I said in my last round? I don't see anything.

    Tom Curtis offers an insight about my intentions, that it is a "gish gallop". I don't know what that means, but I assume it's not charitable.

    Michael Sweet cites the AAAS report. My earlier posts undercut one of the three sources used in that report, so I'm not sure why the report would be a counterargument. (Or why any of these reports should be interesting, given that they don't use standard scientific methods for aggregating knowledge, e.g. meta-analyses, at least not the AAAS.)

    Dunkerson says that I mentioned Oreskes' 75% figure and that I later mentioned how we have no idea what was done in that study (because the paper doesn't tell us). I take the argument to be that the number of problems I've discovered with Oreskes, or the order I in which I've presented them, somehow refutes me?

    Honeycutt regrets piling on another irrelevant, non-substantive reply, and talks about Cook. I had asked a question about whether the Cook et al study used independent rates, as stated in the paper. Honeycutt doesn't answer that question, but invites me to start rating abstracts.


    ( -snip-.)


    [DB] Sloganeering snipped.

  12. Phronesis @5&11:

    With regard to Schollenberger's criticism of Cook et al, Victor Venema had this to say:

    "if you scan the manuscript, you find that much of what you “discovered” in the stolen forum posts was already written into the article manuscript:

    “Each abstract was categorized by two independent, anonymized raters. … Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party.”

    It would have been fair, if you would have told your readers that this was written in the article."

    To that Schollenberger admited:

    "I never claimed to have “discovered” anything in the SKS forums. I didn’t claim anything I said was a secret or new. Quoting one source doesn’t indicate other sources were silent on those topics."

    So, right from the start much of Schollenberger's criticism consists of describing processes described in the paper, carefully not mentioning that fact, and then going "tut-tut".

    There is a small measure of validity in his criticism, however.  Some (5-10) abstracts were discussed in the forum, and should not have been in order to preserve independence.  To calculate the effect of this, I shall assume that 50 abstracts were discussed, and that all of those were rated as endorsing the consensus.  Both of these assumptions are known to be false.  However, they are conservative.  That is, they will massively overstate the impact of the breakdown in proceedure on the paper.

    To estimate that effect, I simply reduce the number of endorsing papers by 50.  The result is that the level of endorsement from abstract ratings falls from 97.1% to 97%.  That is, this breakdown in procedure at most overstated endorsements by 0.1%, and did not change the headline result.  It is very probable that the impact was negligible, because I have significantly overstated the number of papers involved, because not all papers involved were rated as endorsing the concensus, and because even of those endorsing the concensus, there is a high probability that the rating would not have changed to neutral, or not endorsing the consensus without the discussion.

    Further, this breakdown in proceedure cannot effect the author ratings which also show a 97% endorsement rate.

    Now, the question for you is - did you realize these facts before raising the issue?  If not, why did you raise it, given as you claim to be pressed for time? 

  13. Phronesis...  My point is substantive if you care to entertain the idea. What I'm telling you is that you can cast stones all day long, never hitting a target and convince no one, including yourself, of anything.

    If you have genuine questions about what the reality is regarding the consensus of the published research, test the results yourself. Just be ready to adjust your position on this issue when you do.

    My guess is, much like everyone I've ever argued with who rejects AGW, you're not really interested in the reality of the situation. You're merely interested in testing your throwing arm. 

  14. Sloganeering, DB?  More like Hubris ad Absurdum. 

  15. Anyone who thinks that the consensus is significantly less than 97% can demonstrate this relatively easily. All they have to do is to find, let's say,  5-10% or more papers in a given random sample that implicitly or explicitly reject humans as the main cause of  global warming. They don't have to do a fine sub-classification like we did, or sort the no-position papers from the rest, just find the thumbs-down papers. (Hint, some doubters of AGW already have some long--and dubious--lists of papers that supposedly reject AGW, so you could just search for them in any given sample.)

    A few hours of careful work requiring no special equipment and Cook et al could, in principle, be falsified. Several people who question our work seem to have put lots of effort into writing blogposts, examining our statistics, submitting formal rebuttals, making FOI requests and writing angry letters to all and sundry. Yet, if they were right, a convincing knock-out punch would  be easy to deliver, just by looking through a random sample of scientific papers on climate change. But this work has not even been tried, or if it has, it hasn't been reported.

  16. Phronesis,

    It is interesting to learn that you know more about scientific opinion on AGW than the American Association for the Advancement of Science.  I am sure other readers will make their own choice about whether they should believe the AAAS or a nameless voice on the internet.  

    This is a scientific board.  People are expected to provide evidence to support their claims.  Peer-reviewed evidence is best.  You do not appear to care about evidence.   You might find that your method of argument is better received at one of the skeptic sites where they do not care about evidence either.

  17. Phronesis:

    To guide the What We Know initiative, AAAS convened a group of prominent experts in climate science.

    Mario Molina (Chair), U of California, San Diego and Scripps Institution of Oceanography

    James McCarthy (Co-chair), Harvard University

    Diana Wall (Co-chair), Colorado State University

    Richard Alley, Pennsylvania State University

    Kim Cobb, Georgia Institute of Technology

    Julia Cole, University of Arizona

    Sarah Das, Woods Hole Oceanographic Institution

    Noah Diffenbaugh, Stanford University

    Kerry Emanuel, Massachusetts Institute of Technology

    Howard Frumkin, University of Washington

    Katharine Hayhoe, Texas Tech University

    Camille Parmesan, U of Texas, Austin and University of Plymouth, UK

    Marshall Shepherd, U of Georgia

  18. Though Shaviv also admitted that Cook et al. correctly classified his abstracts based on their content, but claimed that he worded the text in a way to slip it past the journal reviewers and editors.

    I went through that link in that statement and don't see anywhere Shaviv agreeing that Cook et al. correctly classified his abstract of his paper "On climate response to changes in the cosmic ray flux and radiative budget" . And neither, as far as I can see, does the following point Shaviv makes there regarding the wording of the abstract imply he agrees :

    "I couldn’t write these things more explicitly in the paper because of the refereeing, however, you don’t have to be a genius to reach these conclusions from the paper.”

    Is there some other statement by Shaviv I may have missed where he clearly says he agrees with the Cook et al rating of his abstract?

    The main reason I ask is because I recently checked out the Schulte consensus paper (that overlapped with Cook et al), and saw that it had categorised the Shaviv paper as rejecting the consensus. I then checked its Cook et al rating and was surprised to see it was instead classified as endorsing the consensus.

    I would have agreed witht the Schulte rating and although the abstract is rejecting the consensus. Or at least not supporting it. This last past part of the Shaviv absrract seems to put it outside.

    Subject to the above caveats and those described in the text, the CRF/climate link therefore implies that the increased solar luminosity and reduced CRF over the previous century should have contributed a warming of 0.47 ± 0.19°K, while the rest should be mainly attributed to anthropogenic causes. Without any effect of cosmic rays, the increase in solar luminosity would correspond to an increased temperature of 0.16 ± 0.04°K.

    Doesn't the above key passage clearly indicate the paper holds a thesis that there is a greater proportion (>50%) increase in recent temperature rise that is not explained by anthropogenic causes and so puts it clearly outside the consensus?

  19. Uncomfortable as it makes me, I have to agree with tlitb1 on this one.  In partuclar, in stating that he understated his claims in the paper to get past peer review, Shaviv did not state that the claims in their understated form did not reject AGW.  Further, in the abstract, Shaviv clearly asserts a climate sensitivity of 0.35 K/W/M^2 (equivalent to 1.3 C per doubling of CO2), and asserts that solar effects including indirect effects through changes in cosmic ray intensity are responsible for 0.47 K of the increase in temperature over the twentieth century.  Using AR4 figures, that is 0.47 out of 0.7 K.  Both of these should have been sufficient to classify the abstract as rejecting AGW.

    The Shaviv paper just happens to be one of the demonstrably small number of mistakes in classification in Cook et al.  It is silly to think that a project like the Consensus Project would be mistake free.  What is remarkable about the project is how few the mistakes were, and how little impact (if any) they have on the headline result. 

  20. @Tom Curtis at 16:39 PM on 15 May, 2014

    I'm glad you agree with me about the Shaviv assessement. My attention was only drawn to Shaviv because I had just for the first time read the Schulte paper, which itemises 6 papers (including Shaviv) as examples rejecting the consensus, and I picked Shaviv to compare first.

    I've now had a look at the remaining 5 papers picked out by Schulte as rejecting the consensus - with an aim to see how they stack up against Cook et al. - and found: 2 dont appear in Cook et al, 2 agree, and I think have found one other paper, Lai et al. (2005), which Cook et al rates as supporting the consensus but seems actually to be rejecting it. IMO the abstract certainly is not clearly stating a position on anthropogenic contribution being >50%

    Global warming and the mining of oceanic methane hydrate Chung-Chieng A. Lai, David E. Dietrich, Malcolm J. Bowman

    If you look at the abstract I think you will see there is nothing that puts it in the consensus. If I am wrong I would love to know what I missed. In fact I think these two statements alone

    "However, the extent to which anthropogenic factors are the main cause of global warming is still being debated."


    "we propose a new hypothesis for global warming." 

    by rights should be enough to define this abstract as *not* being in the consensus.

    I think the statement "we propose a new hypothesis for global warming" is as near enough to saying 'this paper is not in the consensus' as you can get! ;)

    The Shaviv paper just happens to be one of the demonstrably small number of mistakes in classification in Cook et al. It is silly to think that a project like the Consensus Project would be mistake free.

    I certainly wouldn't expect Cook et al to be mistake free but I have to say finding a disagreement between 2 of the 4 overlapping examples here in the Schulte paper, a paper which Cook et al itself draws attention to, is quite interesting; interesting because I think the two examples also indicate an insight as to why they were mis-rated, and why more papers could be too.

     I think the mis-rating of Lai et al. (2005) could be explained if the raters were slavishly only looking for quantification, and not really parsing any deeper, then at a push I could see it being put in the neutral category, however its being rated in the endorse category still seems wrong.

    And again with Shaviv, there is some dispute about the 'wording' indicated in the article above, as if it wasn't playing by the rules, i.e. as Dana Nuccitelli implies above, "he worded the text in a way to slip it past the journal reviewers and editors" . But surely if Shaviv did not lie in his abstract, no matter how it was 'worded' , Cook et al's methodology should have been robust to correctly rate it?

    It clearly seems not in this case, could that have happened in more cases?



  21. tlitb1 @20, sorry, my computer ate my long response, so you will have to make do with the short.

    1)  I agree with you about Lai et al (2005), which I would rate as (6) "explicitly rejects but does not quantify".

    2)  From the full rating data, which has been released by John Cook, it can be determined that there is an initial rating error rate between "endorsement levels" of 12.9% on average, which results in an expected error rate between levels after dispute resolution of 4.7%.  Given that distinctions between "implicit" and "explicit" are graduated rather than binary (things can be more, or less explicit) such an error rate is unsurprising.  It is also overstated in that it treats a large number of ratings of "0" (= uncertain) as errors, which is not the case.

    3)  The error rate between endorsing or rejecting AGW falls to a low 0.04%.  That is because the vast majority of errors are between categories just one level apart, and because nearly all errors (98%) for initial ratings of (4) involve mistakenly identifying an implicitly endorsing abstract as being neutral.  Overall, adjusting for errors based on internal data appears to slightly increase the endorsement percentage.  (That is a provisional result using simplifying assumptions.  More accurate results may change teh sing of the result but will not significantly change the magnitude which is less than 1%.  Note that Richard Tol's claim that this adjustment makes a large increase in the "dissensus rate" depends on the false assumption that the error rate and distribution for all rating values is the same.) 

    4)  Despite the low overall error rates, the sheer number of endorsing abstracts means a small fraction will have been incorrectly rated, ie, should have been rated as rejecting the consensus.  On the figures above, we would in fact expect two such abstracts.  Those figures show only internally detectable errors, however, so the number may by slightly larger.  

    5)  Because we expect some such errors as a matter of course, no amount of highlighting single abstracts being correctly rated will show the Cook et al results to be false.  That is because such anecdotal evidence does not provide a basis for statistical analysis.  At most the response required is to adjust the values reported for endorsement and rejection in Cook et al by the number of individual abstracts found to be in error.  Thus, you have found two abstracts in error.  Therefore we would adjust the Cook et al figures of 3896 endorsing and 78 rejecting to 3894 endorsing and 80 rejecting, which changes the percentages (as a percentage of papers endorsing or rejecting) to 97.99% endorsing, down from 98.04%.  Even that ignores the probable existence of papers with errors in the other direction, and makes no difference worth mentioning.  Even if you were to find a net error of 100 abstracts in favour of endorsement, after correction you still have an endorsement rate of 95.52%, which again is not a challenge to the Cook et al result.

    6)  Because of this, the only valid method to challenge Cook et al is to do another survey yourself, of at least 2000 abstracts (and prefferably more).  Make sure you state your classification criterion clearly.  You will find either a result within 5% of the Cook et al value, or your classification criteria will be transparently tendentious.

  22. likeithot...  Then Mike Hulme is welcome to replicate the research and show everyone how he believes it should be done. As it is, Cook13 confirms all the previous research on this topic, including Oreskes04, Doran/Zimmerman09 and Anderegg11.

  23. He actually gives several examples of how the "study" could have been done. 

    1.For starters, hire some independent people to actually review the abstracts.

    2. create a definition of AGW that is actally meaningful, like for a paper to count as supporting AGW it would need to actually state that, and state the degree to which they think humans are responsible, more than 50%, or less, for example.  This is obviously crucial, as most scientists don't deny warming or that CO2 has some effect.  The debate is about how much of an effect, as they undoubtedly knew and avoided in order to be able to forward their political adgenda.

    Incidentally, where else in science is a number like 97% so religiously adhered to/miraculously arrived at in "independent" studies?  <Snip>


    [PS] Read the comments policy. Compliance is not optional. Future violations are likely to result in the comment deleted rather than snipped.

  24. likeithot said... "He actually gives several examples of how the 'study' could have been done."

    How about we change the scare quotes to " the study 'could' have been done."

    To my point once again, if Hulme has ideas on how the study could be done better, then by all means, let him do what scientists actually do in these situations where they disagree. Replicate the study. If it can be done better, then do it better! 

    I would suggest that the vast majority of those critiquing Cook13 don't want to replicate because they know they're going to come up with the same result. 

    In fact, you too can replicate the study, likeithot. Right here on this website. We've created a tab at the top where you can rate abstracts yourself. So, spend a few hours doing it. Maybe you'll come up with different results. If so, you can let us all know.

    Anything short of that is, as they say, just belly-aching.

  25. You didn't respond to any of the points I made regarding how the study should have been done.   In short, it is poorly done because it avoids the critical question of the whole debate:  HOW MUCH of the warming that we have witnessed is becasue of human activity.  Almost no one argues that humans have had no impact, so disproving that is just knocking down a straw man.

    As to your suggestion that I re-do the "study", I find the whole point of the study to be political in nature, trying to prove a scientific point by some kind of opinion poll and then bragging all over the press that your opinions are right because you claim to have lots & lots of people on your side.  I have no interest in popularity contests.   It is a sign that your arguments are weak, and a distraction from the fact that most models are very inaccurate and didn't predict the pause in warming of the past 17 years.


    (Rob P) - Readers will note that the strike-through text relates to other climate myths dealt with here (Climate Models Show Remarkable Agreement with Recent Surface Warming) & here (Nuccitelli et al. (2012) Show that Global Warming Continues), for instance.

  26. likeithot if you want a different question answered, then perform a survey of your own.  However, if you want to ask how much of the warming we have witnessed is because of human activity, then you are basically only looking at papers that specifically address the question of attribution.  Fortunately, there is a report that already summarises th litterature on this topic (chapter 10 looks like a good place to start).

    "As to your suggestion that I re-do the "study", I find the whole point of the study to be political in nature, trying to prove a scientific point by some kind of opinion poll"

    nobody is claiming that the existence of a consensus on a scientific question is in any way proof that the mainstream position is correct, that is a straw man.  The point is to provide evidence of what the mainstream position on the science actually is.  Why is this point worth addressing?  Because there is a gap in the public perception of the mainstream position and the reality.  This is explicitly mentioned in the article, so it is hard to understand how you have misunderstood the purpose of the study.

    If you want evidence that the science is right, the IPCC AR5 WG1 report is a good place to start.

  27. likeithot - While the paper could have been more explicit with each individual endorsement level description (at the risk of some repetition), reading the set of exclusive choices as a whole and the guidelines used for evaluations clarifies that endorsement levels of 1-3 are for a majority anthropogenic influence, while endorsement levels 4-7 are for minority or negligible anthropogenic influence. The levels are in fact quite clear on that. 

    You can only claim a lack of attribution levels in the Cook et al paper by ignoring the context of the multiple exclusive choices presented - in essence by taking things out of context. That's an error on your part, and on the part of many who have criticized the paper. 

    As to trying some ratings yourself, that's a suggestion made because you (and any critic) have some options available regarding this paper and the consensus.

    1. If you think the Cook et al raters were biased in their work, do some ratings yourself - a few hundred from the evaluation set should be managable in a weekend day, enough to see if the 97% estimate is supported by the abstracts evaluated. Otherwise you're criticizing w/o evidence, hand-waving. That's one of Tols errors. 
    2. If you agree with the general consensus level, but are just criticizing the methods used in Cook et al, then you are by your very agreement with results not able to argue rating bias. This is another core error in Tols comment.
    3. If you disagree with the level of consensus entirely, with Doran, Anderegg, Oreskes, and Cook et al, you need to provide some independent evidence, i.e. do the work. Or you're again engaging in unsupported hand-waving.

    There really aren't any other choices wrt this paper. 

    In regards to consensus vs. science - the consensus is not the science, but rather is driven by the science, by the available evidence and data. As in any public policy issue, no one person is expert on everything (although some people act like they are, oddly enough) and we therefore rely on expert opinions. And as noted in the Cook et al paper, the gap between the existing scientific consensus and the public view of that consensus (due in large part to signficant ongoing efforts at obfuscation and misinformation) means that our public policies will be misinformed as well - unless and until that misperception is corrected. 

    Claims that a scientific consensus on climate doesn't exist despite multiple studies or even cursory looks at the literature, or that expert opinions are meaningless, are really just efforts in denial, and attempts to halt reality-driven public policies. I consider such claims to be wholly ideological rather than evidential. And (IMO, mind you) I regard your claims in this thread in like manner. 

    If you wish to argue public policy, great, do so, although I suspect SkS is perhaps the wrong place for purely political discussions. But policy discussions need to be based on accurate information, including the reality that the vast majority of people studying climate agree on the basics of AGW. 

  28. A clarification on my previous comment:

    For those unhappy with Cook et al., there are some issues that need to be addressed. 

    • If you agree with the overall level of consensus, you cannot claim the Cook et al. raters were biased. 
    • If you feel the raters were biased while ignoring independent results, the Cook et al. data (the abstracts) are publicly available - do the work and support your claim, or drop it as opinion and not fact. 
    • If you disagree with the overall level of consensus, provide some evidence. Or again, hand-waving opinion unsupported by facts. 

    And if you are arguing that expert consensus is unimportant, why is consensus one of the most frequent primary 'skeptic' claims presented to argue against policy changes? Perhaps, just perhaps, because expert consensus is actually critical to policy decisions...

    Again I (IMO) view arguments against the consensus, made without evidence, to just be efforts to influence public policy - trying to persuade the public to ignore evidence, to ignore reality. 

  29. likeithot...  Every single paper or piece of research ever done, and every one that ever will be done, can be done better. That's just a fact of life. 

    I will take from your combativeness and unwillingness to test the results of Cook13 as an admission that the results are likely to be correct.

  30. R. Honeyutt:

    Lindzen’s JP&S article "Science in the Public Square: Global Climate Alarmism and Historical Precedents" sums up pretty well the problem with group think, “look how many people are on our side” type arguments:

  31. likeithot - If you are in agreement with Lindzen, are you arguing that the consensus is unimportant, that attribution is impossible, or that scientists are wholly motivated by money and status rather than facts? Because all of those contentions are quite false. [And as a side note, the comparisons to eugenics, Nazis and Lysenko in Lindzens article indicate to me that he isn't resenting a fact-based argument, rather just rhetoric...]

    In the interests of remaining on topic, I'll just note (as many others have) that public policy decisions are driven by the information available, that on complex subjects we depend upon expert opinion, and that due to some rather serious efforts by 'skeptics' there is a gap between the expert opinion and the public perception of the same. Consensus is very important in informing policy. 

    I will also note that your "look, squirrel" changes of subject mean that you have not supported any of your claims against Cook et al's methods or results. 

  32. How do you think policy should be guided? Personally I think that in highly technical areas, policy should be guided by the consensus of experts. It might be wrong, but that is still the best way for policy makers to proceed.

    Of course you need to know what the consensus is and interestingly, several ways of examining this have arrived at the same conclusion. If you dont think that this is the consensus opinion, then where is your evidence to the contrary?

    I think the idea of group-think in science is total joke. "Science is a contact sport" is more realistic - and utter applies to Lindzen's example which was clearly not group-think. Actually it is example of politically-motivated abuse of science much like the anti-AGW stance which is rather ironic.

    Otherwise, the piece consists of misinformation that Lindzen would certainly not say to his peers and the usual political argumentation method of selective historical presentation to support an argument. How about showing a little skepticism for this sort of stuff and checking it against the actual science yourself?

  33. What Lindzen publishes in journals gets noted. What Lindzen presents to the naive is rather different and depressing.

    Eugenics was not the scientific consensus opinion by a very large margin. Democracy would work okay if the population was actually accurately informed but as you are aptly demonstrating, people prefer to get their "information" from sources which confirm preconceived beliefs.

    Spare us the strawman arguements please. You obviously read a pack of pseudoskeptic sites, why not read what the science is actually saying and predicting instead of the nonsense that would appear to fit your prejudices? Have you ever look at the IPCC WG1 report? It would appear not from those statements.

  34. As a post-script, I have a pretty clear of set of criteria for data that would falsify climate theory based on what climate theory actually predicts.


    - OHC flat or declining while known net forcing same or increasing.

    - 30 year surface temperature trends flat or declining

    - Ratio of insurance costs for weather events/geophysical events on a 10 year average flat or declining

    - sealevel rise declining over a 10 year interval.

    Do have a set of criteria for changing your mind?

  35. Scaddenp,

    There are some soi-disant skeptics who don't respect the IPCC but do respect the US National Academy of Sciences.  Those people might learn something from the 36-page booklet Climate Change: Evidence and Causes, jointly published by the NAS and the Royal Society of the UK.  It should be considered to represent, for what it covers, the scientific consensus on AGW.

  36. likeithot said... "...Lindzen is a highly recognized scientist with a lot more published science than all of those who work on this site combined."

    So, let me get this straight. One scientist presents a position you like, so that trumps 97 other equally qualified scientists who disagree with him?

    Look, no one says that the 97% are absolutely right and the 3% are absolutely wrong. That's not how science works either. But, when it comes to making critical policy decisions you have to have a reference for how to make those decisions. What are the chances the 3% are right and the 97% are wrong? 

    I would suggest that, even if the figures were 50/50, that would be more than enough reason to take aggressive action on climate change. 

    If there were a 50/50 chance my house was going to burn down if I didn't get the wiring replaced, you'd be darned sure I'd be making that investment asap.

  37. likeithot - Your most recent reply contains several logical fallacies, namely Argument from Authority (while Lindzen is a climate scientist, he is in a distinct and tiny minority on his views, with multiple papers debunked), Red Herrings (eugenics), the Argument from Uncertainty regarding the maturity of the field. You also have Gish Galloped false claims about extreme weather, sea level rise rates,  and the temperature record

    From your posts it appears you, in fact, are the person who has been spending too much time in the echo chamber - deep in denial myths. 

    At this point it's clear to me that you are simply trolling, and are not interested in a rational discussion of the data and science. I would suggest that the correct response is DNFTT (Do Not Feed The Troll). 

  38. I would also point out the accusations of " sycophant research" is simply empty rhetoric - a dismissive to hide that fact that there is no counter-theory, and at odds with the mass of hard data collection that is the core of climate research. 

    FF companies have the scientists, the money, computer power and the motiviation to find alternative theories. Instead they invest in PR. Why do you think that is? Perhaps because of what their own scientists tell them? I work in petroleum science - denialists are rare among scientists.

Post a Comment

Political, off-topic or ad hominem comments will be deleted. Comments Policy...

You need to be logged in to post a comment. Login via the left margin or if you're new, register here.

Link to this page

COP21 LiveBlog

The Consensus Project Website



(free to republish)



The Scientific Guide to
Global Warming Skepticism

Smartphone Apps


© Copyright 2016 John Cook
Home | Links | Translations | About Us | Contact Us