Climate Science Glossary

Term Lookup

Enter a term in the search box to find its definition.

Settings

Use the controls in the far right panel to increase or decrease the number of terms automatically displayed (or to completely turn that feature off).

Term Lookup

Settings


All IPCC definitions taken from Climate Change 2007: The Physical Science Basis. Working Group I Contribution to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Annex I, Glossary, pp. 941-954. Cambridge University Press.

Home Arguments Software Resources Comments The Consensus Project Translations About Support

Twitter Facebook YouTube Mastodon MeWe

RSS Posts RSS Comments Email Subscribe


Climate's changed before
It's the sun
It's not bad
There is no consensus
It's cooling
Models are unreliable
Temp record is unreliable
Animals and plants can adapt
It hasn't warmed since 1998
Antarctica is gaining ice
View All Arguments...



Username
Password
New? Register here
Forgot your password?

Latest Posts

Archives

Pal Review - the True Story and the Fairy Tale

Posted on 6 June 2012 by dana1981, JohnMashey

We often hear claims from climate contrarians that climate scientists are guilty of what they describe as "pal review."  The conspiracy theory goes something like this - climate scientists conduct biased research with the goal of confirming the human-caused global warming theory.  They then submit their biased results to a peer-reviewed journal with friendly editors ("pals") who pass their paper along to friendly reviewers (other "pals") who give their fraudulent work the green light for publication.  Thus, the contrarians argue, the preponderance of peer-reviewed literature supporting human-caused global warming is really just a sign of corruption amongst climate scientists.

However, while climate contrarians are never able to produce any evidence to support their conspiracy theory, John Mashey has thoroughly documented a real world example of true pal review.  Contrary to the standard conspiracy theory, the pal review did not involve mainstream climate scientists, but instead the climate contrarians themselves.

The True Story of Climate Research Pal Review

Mashey has done an excellent job documenting a real life case of pal review, which happened at the journal Climate Research between 1997 and 2003.  That particular journal was once again brought to the forefront in the recent second Climategate stolen email release.

In those emails, various climate scientists had expressed concern that Climate Research was publishing shoddy papers by a small group of climate contrarians, and discussed what they could do about it.  The most infamous of these papers was one by Soon and Baliunas (2003) which concluded that current global temperatures are not anomalous compared the past 1,000 years.  After publishing this paper, Soon was invited by Senator James Inhofe to testify before US Congress, and the Soon and Baliunas paper was used by Congressional Republicans to justify opposition to climate legislation.

However, the paper contained numerous major fundamental flaws, such as equating dryness with hotness, and was subsequently roundly refuted by an article in the American Geophysical Union journal Eos written by a number of prominent climate scientists.  This paper, and Climate Research's refusal to revise or retract it, led to the resignation of five of the journal's editors, including recently-appointed editor-in-chief Hans von Storch, who explained the reason for his resignation:

"..the reason was that I as newly appointed Editor-in-Chief wanted to make public that the publication of the Soon & Baliunas article was an error, and that the review process at Climate Research would be changed in order to avoid similar failures. The review process had utterly failed; important questions have not been asked....It was not the first time that the process had failed, but it was the most severe case....I withdrew also als editor because I learned during the conflict that [Climate Research] editors used different scales for judging the validity of an article. Some editors considered the problem of the Soon & Baliunas paper as merely a problem of "opinion", while it was really a problem of severe methodological flaws. Thus, I decided that I had to disconnect from that journal, which I had served proudly for about 10 years."

In short, the journal's chief editor voiced the exact same concerns as the climate scientists in the Climategate 2 emails - that certain Climate Research editors were systematically publishing methodologically flawed papers in their journal.  Soon and Baliunas were far from the only climate contrarians to benefit from the journal's friendly editorial policy.  In fact, the biggest pal review beneficiary bears a very familiar name.

Patrick Michaels and Pals

Mashey has examined the publications in Climate Research in great detail, and has produced a spreadsheet of its publications and a report summarizing his findings.

Prior to Hans von Storch's promotion to Climate Research editor-in-chief in 2003, the journal did not have a chief editor, and so authors sent their manuscripts to an Associate Editor of their choice.  One particular Associate Editor, Chris de Freitas, published 14 separate papers from a select group of 14 climate contrarians during the 6 year period of 1997 to 2003:

Sallie Baliunas, Robert Balling, John Christy, Robert Davis (both Climate Research author and editor), David Douglass, Vincent Gray, Sherwood Idso, PJ "Chip" Knappenberger, Ross McKitrick, Pat Michaels, Eric Posmentier, Arthur Robinson, Willie Soon, and Gerd-Rainer Weber.

As Mashey shows, from 1990 to 1996, Climate Research published zero papers from this group.  From 1997 to 2003, the journal published 17 papers from this group, 14 with de Freitas as the Associate Editor.  Serial data deleter Patrick Michaels was an author on 7 of the 14 pal reviewed papers, which also accounted for half of his total peer-reviewed publications during this timeframe.  During this period, 14 of the 24 (58%) papers accepted by de Freitas came from this group of contrarians.  After von Storch's resignation in 2003, de Freitas published 3 more papers from authors outside this group before leaving the journal in 2006.

Another on the list of 'pals', Robert Davis, was another Associate Editor at Climate Research who accepted 36 papers during his tenure, two of which were co-authored by another pal, Robert Balling.  The journal also published 5 other papers from this group by non-pal editors.  However, in total, at least 16 of the 21 (76%) of the papers published by Climate Research which were authored by this group of climate contrarians had pal review editors, mostly de Freitas (67%) during this six year window.

After von Storch's resignation, Mashey documents that the pals' Climate Research publications dried up.  Davis accepted one of Balling's papers submitted in 2004, and papers co-authored by Balling and by de Freitas were published by the journal in 2008 (Table 1).  18 of the 21 (86%) of the 15 pals' Climate Research publications were submitted in the 1997 to 2003 timeframe.

Table 1: Climate Research publications grouped by Associate Editor.  Grey bars show approximate editor tenure as derived from received dates of papers.  The "pals" papers are shown in red capitals, 14 accepted by de Freitas (bold), and 7 handled by others (red, underlined italics). De Freitas also accepted 13 seemingly normal papers from other authors (lowercase black).

pal review

Mashey also finds that the 15 'pals' were closely connected in climate contrarian activities outside of Climate Research as well, for example working for various anti-climate think tanks, most being connected with either Fred Singer or Patrick Michaels.

"all have shown persistent involvement with organizations that do climate anti-science, most of which also have tobacco connections."

There is also substantial overlap with the pals joining together to author these papers (Figure 1).

pal overlaps

Figure 1: Overlap between pal authors of the 14 de Freitas Climate Research pal review publications between 1997 and 2003.  The node numbering represents the Climate Research volume and page number of the pal publications, while the node connections represent papers written by the same pal authors (i.e. 9.3p14 and 23.1p15 were both authored by Michaels and Knappenberger).  Image by jg and Kevin C.

The Purpose of the Mainstream Pal Review Myth

For those who oppose the prudent path forward with regards to climate change, which involves major global greenhouse gas emissions reductions, the scientific consensus on human-caused global warming is a very inconvenient thing.  Despite the public relations damage resulting from Climategate, people still trust climate scientists' opinions about climate science (although political conservatives' trust in scientists in general has declined).  However, much of the public (at least the American public) doesn't realize that there is a scientific consensus on human-caused climate change.  Polls in October 2010 and September 2011 found that 44% and 37% of the American public believes that scientists are divided regarding the cause of global warming, respectively.

According to the March 2012 George Mason Center for Climate Change Communication (CCCC) national poll, climate scientists are the most trusted source for climate science information, with 74% of public trust (Figure 2).  However, a large segment of the population believes there is a major scientific debate on the subject, no doubt thanks to the false media balance which gives the ~3% minority of experts who think humans aren't the dominant cause of the current climate change (and their non-expert surrogates) ~50% of the media attention.  Therefore, many people  don't believe that humans are the primary cause of global warming (approximately 41% of Americans).

george mason poll

Figure 2: Responses to the George Mason CCCC poll question "How much do you trust or distrust the following as a source of information about global warming?"

The numbers reveal a stark picture: 76% of Americans trust climate scientists, but 41% think scientists are divided on the causes of the warming, and 41% think the observed warming is mostly natural.

Thus as Ding et al. (2011) concluded, if a larger percentage of people realized that there is a scientific consensus on the issue amongst the group they trust most on the subject (and rightly so), more people would believe that humans are causing global warming, and more people would demand that we do something about it. The lack of public awareness of the scientific consensus on human-caused climate change is one of the biggest obstacle to taking climate mitigation action.

For this reason, climate contrarians have attacked the scientific consensus from many different angles.  Some have tried to attack the credibility of the many different surveys and studies documenting the consensus.  Others simply ignore this documentation and deny the consensus exists at all. 

The third group, discussed in this post, attacks the credibility of the consensus itself, claiming it's all part of a massive fraudulent conspiracy of thousands of corrupt climate scientists (note that conspiracy theories are one of the five characteristics of scientific denialism).  Ironically, this conspiracy theory has been most recently voiced by pal review beneficiary Patrick Michaels.

"Peer review has become ”pal review.”  Send a paper to one of the very many journals published by the American Geophysical Union–the world’s largest publisher of academic climate science–and you can suggest five reviewers.  The editor doesn’t have to take your advice, but he’s more likely to if you bought him dinner at the last AGU meeting, isn’t he? That is, of course, unless journal editors are somehow different than government officials, congressmen, or you."

Michaels of course provides no evidence whatsoever to support this conspiracy theory of peer-review corruption.  He expects us to swallow his tale of "pal review" - the conspiracy theory that thousands of climate scientists are publishing thousands of biased papers every year in order to keep the human-caused global warming theory propped up - based on nothing more than his say-so.

While Michaels is indeed something of an expert on the subject, his expertise comes from himself being one of the individuals most guilty of engaging in climate research pal review.

Pal Review Summary

While Patrick Michaels has accused mainstream climate scientists of a vast conspiracy involving pal review (and exposed his own characteristic of scientific denialism in the process) without any substantiation or supporting evidence, in reality Patrick Michaels himself was the biggest beneficiary in the one actual demonstrated case of climate science pal review, as documented by Mashey.

A group of 14 climate contrarians found a sympathetic journal editor who proceeded to publish a large number of papers from this group over a very short timeframe, many of which were scientifically flawed, some of which were subsequently used by politicians to oppose climate legislation.

Ironically, the climate scientists who tried to do something about this problem have themselves been accused of trying to "hijack" or "subvert" the peer-review process.  And of course the guiltiest party of all, Patrick Michaels has accused thousands of climate scientists of the sort of pal review he himself engaged in.

Our tale is one of irony, hypocrisy, and projection.  The next time you see a complaint about the fairy tale of rampant climate science "pal review", direct the accuser to John Mashey's documentation of a pal review true story.

Note: this post has been used as the Intermediate rebuttal to the myth Climate science peer review is pal review.

0 0

Printable Version  |  Link to this page

Comments

Prev  1  2  

Comments 51 to 66 out of 66:

  1. Rufus9 There is an important point that you don't appear to understand. In science, all that matters is whether an argument is valid and supported by the evidence. It isn't about trust. It especially isn't about gaining the trust of the general public. It is fine to be skeptical about some finding, but the solution is to take the time to find out whether it is supportable or not. Peer review is only to be relied on as a basic sanity check. Expecting it to be proof of validity (trustworthyness) is completely unralistic. That is why we have scientists who have expertise to discern whether some new idea is good and worth investigating further or not. Peer review is only the start of the test of a paper, not the end. Ultimately whether a paper is any good or not is indicated by whether other scientists are convinced by it, take it up and work on it further. So if you want a better indication of trustworthyness, look at the number of citations.
    0 0
  2. Dikran As the cancer issue showed, some of those papers were cited >200 times, so perhaps that is not a good metric.
    0 0
    Moderator Response:

    [DB] As noted by others earlier, the field of medicine is not a good comp to that of climate science, for a variety of reasons.

    Please stick to the OP of the thread, offer links to the supportive (climate science) literature to support your points, discuss the material raised by others to counter your previous assertions (which you have avoided doing) or concede them.

  3. Rufus9 I give up, the point I am making is that at the end of the day you if you are unwilling to accept the findings of mainstream science (on pretty much any area of science) then need to make an effort to investigate the science and find out for yourself. This is something you appear to be completely unwilling to do and want someone to create an automatic system for ensuring that all scientific findings are solid and trustworthy. However this is completely unrealistic. If it were possible, we would have put it into practice already. The skeptics like to talk about "trust" and "pal review" because often they don't want to talk about the validity of the science itself. Sadly this is the impression that you are giving (intended as helpful advice on the assumption that this is not what you intend). (snip)
    0 0
    Moderator Response:

    [DB] As noted, the interjection of the field of medical research by Rufus9 constitutes a strawman argument, as it is not a valid comp. As such, responses to it are a waste of time.

    OT snipped per request.

  4. Rufus9 I have written a peer-reviewed journal paper on the cause of the observed increase in atmospheric CO2 (as you will know from reading the SkS post I mentioned). Do you trust me when it comes to this particular issue? If not, why not, and what specifically would it take for you to trust my paper (above the fact that it was peer reviewed)?
    0 0
  5. Rufus9 - Your 'trust' issues are red herrings, really. Science is about results, repeatable results. Peer review is simply a first-pass filter which, as discussed in the opening post, can be subverted - and interestingly enough, in the only case on this scale I am aware of, was subverted by folks who decry 'pal review' most loudly. Once published, the more stringent review begins - does the work contain anything new (if not, it will likely be ignored)? Are the results repeatable (not by 'audit', or code inspection, but by others investigating along the same lines)? Are the results consistent with other evidence? Do they open up new lines of investigation? The various 'pal review' papers discussed in the OP all failed one or more of these criteria, and hence are not good science. Not because of 'trust', but because they failed in the real world of objective criteria. That does not mean that they are ignored - they have instead been used as argumentative 'talking points' by those either not aware of or not caring about their shortcomings. Science is about objective results. Those, like the well-connected group discussed above, who appear to be sliding bad work through as rhetorical talking points, are not helping matters. From your post:
    "I put this article as: Imitation is the ultimate flattery. Essentially, this influence network has been described on both sides of this debate..."
    You have provided zero support for this assertion, no evidence whatsoever. You have therefore (IMO) not demonstrated any regard for objective truth.
    0 0
  6. (snip) I have seen a similar influence network as figure 1 along with similar allegations in Montford's book. That may be viewed as low trust in this group for legitimate reasons. KR - you have provided some of the trigger criteria. The equation could say Trust or Accuracy. It depends on what exactly a paper is measuring and whether it is possible to reach a definitive answer. The effect of AGW on hurricanes is likely to require more latitude. As such, there could be differences in whether answering "no" to one of your questions is fatal or not. For example, it is possible to have results not consistent with other evidence. Separately, I have looked at the spreadsheet. It appears there is a slight error in the calculation of the average days to review and differences between the summary at the bottom Row 750. I believe the intention was to calculate the average for the bolded rows, not select the ranges that are in the formula. As such, if you do so, these are the results: Other = 266 days Pals = 186 days If we look at the median review, it is 146 days across the Freitas population. If you take all papers and sort numerically by Days, you visually see no obvious pattern. As such, the Significant? question appears to be "no".
    0 0
    Moderator Response: [DB] Inflammatory tone snipped.
  7. Rufus9 - The set of "pal-review" papers discussed in the OP are scientific failures based on those objective criteria, at a level approaching 100%. Mann et al and other papers discussed in Montford's book are not - the work in those publications has been replicated, confirmed, and extended, leading to further investigation and information - again, at a level approaching 100%. Mann's early work regarding paleoclimate temperature reconstruction (the core of the Montford book) has been validated over and over in multiple studies. That's a clear distinction - one group of works is reasonable science, the other, as sheparded through publication, is not. Montford's book is merely polemic, and inaccurate at that. Your arguments otherwise, and your apparent lack of concern regarding objective criteria, are simply not supportable.
    0 0
  8. Rufus9 - To clarify my point a bit: Your posts here have been in essence a set of Tu Quoque (You too) fallacies. There is no evidence whatsoever that any significant amounts of bad science is being published supporting AGW - you have certainly presented none at all. Your initial statements here are quite simply bogus.
    0 0
  9. Here is one - "on hold" http://retractionwatch.wordpress.com/2012/06/11/paper-claiming-hottest-60-year-span-in-1000-years-put-on-hold-after-being-published-online/
    0 0
    Moderator Response: [DB] Note: this has been discussed over at RealClimate for several weeks now. This is a normal part of the peer-review/publication process.
  10. Rufus9 - Real scientists will, upon notice of raised issues, pull a paper while working it out. And that is what Gergis et al have done. This particular paper and the issues involved are in discussion at RealClimate, incidentally. The "pal-review" papers discussed in the OP have not, on the other hand, been pulled, retracted, or updated with any information that withstands scrutiny against observations, despite the multiple issues that have been raised. I would (IMO) consider that poor behavior on the authors part. At this point, Rufus9, I would have to consider your comments on this thread simply trolling. You've presented zero support for your Tu Quoque fallacies, zero response to other comments, and continue to try to minimize the faults of the de Freitas papers. No amount of Tu Quoque fallacies will excuse the documented excesses of that set of papers. Enough. You are just making pointless noise.
    0 0
  11. All participants, please note: Rufus9 has requested that his SkS account be terminated. I have formally passed that request on to the proper authority. Please respect Rufus9's wishes in this matter by letting this interlude pass without further comment. Thanks in advance.
    0 0
  12. Well, it is official now. Having absolutely no grounds on which to refute the peer reviewed science, the 'skeptics' have now switched to 'refuting' peer review itself. Bob Carter claims peer review is corrupt.
    0 0
  13. Carter writes "Scientific knowledge, then, is always in a state of flux. Much though bureaucrats and politicians may dislike the thought, there is simply no such thing as “settled science,” peer-reviewed or otherwise. Gavin Schmidt writes "Unusually, I’m in complete agreement with a recent headline on the Wall Street Journal op-ed page: “The Climate Science Isn’t Settled” ... The phrase “the science is settled” is associated almost 100% with contrarian comments on climate and is usually a paraphrase of what ‘some scientists’ are supposed to have said. The reality is that it depends very much on what you are talking about and I have never heard any scientist say this in any general context – at a recent meeting I was at, someone claimed that this had been said by the participants and he was roundly shouted down by the assembled experts.
    0 0
  14. I can only say that if Dr Carter asserts that money can corrupt science [snip] It looks like this, combined with Dr Carter's previous FP article (debunked by Skeptical Science) is another instalment of the National Post/Financial Post "war on science".
    0 0
    Moderator Response: TC: Accusations of dishonesty snipped.
  15. Interesting, attacks on the peer-review process seem to be becoming more frequent, presumably as attacks on the science itself become more difficult.
    0 0
  16. re: 65 yes It was not accidental that I though the de Freitas / Michaeles / CR history worth writing up, starting a year ago.
    0 0

Prev  1  2  

You need to be logged in to post a comment. Login via the left margin or if you're new, register here.



The Consensus Project Website

THE ESCALATOR

(free to republish)


© Copyright 2024 John Cook
Home | Translations | About Us | Privacy | Contact Us