Climate Science Glossary

Term Lookup

Enter a term in the search box to find its definition.

Settings

Use the controls in the far right panel to increase or decrease the number of terms automatically displayed (or to completely turn that feature off).

Term Lookup

Settings


All IPCC definitions taken from Climate Change 2007: The Physical Science Basis. Working Group I Contribution to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Annex I, Glossary, pp. 941-954. Cambridge University Press.

Home Arguments Software Resources Comments The Consensus Project Translations About Donate

Twitter Facebook YouTube Pinterest

RSS Posts RSS Comments Email Subscribe


Climate's changed before
It's the sun
It's not bad
There is no consensus
It's cooling
Models are unreliable
Temp record is unreliable
Animals and plants can adapt
It hasn't warmed since 1998
Antarctica is gaining ice
View All Arguments...



Username
Password
Keep me logged in
New? Register here
Forgot your password?

Latest Posts

Archives

Climategate: Perverting Peer Review?

Posted on 24 November 2010 by James Wight

This is the fourth part in a series on the fake scandal of Climategate (start here).

A third set of allegations examined by the Muir Russell Review is the potential corruption of the peer review process. Contrarians claim that a small group of scientists, including those at University of East Anglia Climatic Research Unit (CRU), attempted to hijack the peer review process, pressuring journals to reject papers whose conclusions contradicted their own. There are three main instances in which this is alleged to have happened.

The first involved a paper by Soon and Baliunas published in Climate Research in 2003, reviewing the literature on temperature change during the recent millennium. It concluded that late 20th century Northern Hemisphere temperatures were not unprecedented, contradicting the majority of the other analyses which came before and after it. The paper was approved by four reviewers and one of the journal’s ten review editors, Chris de Freitas, but received a hostile reception from the climate science community, as is reflected in the CRU emails. For example, Jones wrote in an email dated 11/3/2003:

I think the skeptics will use this paper to their own ends and it will set paleo back a number of years if it goes unchallenged. I will be emailing the journal to tell them I’m having nothing more to do with it until they rid themselves of this troublesome editor, a well-known skeptic in NZ. A CRU person is on the board but papers get dealt with by the editor assigned by Hans von Storch.

Michael Mann replied:

This was the danger of always criticising the skeptics for not publishing in the “peer-reviewed literature”. Obviously, they found a solution to that — take over a journal! So what do we do about this? I think we have to stop considering “Climate Research” as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal. We would also need to consider what we tell or request of our more reasonable colleagues who currently sit on the editorial board…

Contrarians have used these email quotes to argue that a group of scientists including Jones and Mann deliberately hijacked the peer review process to promote a favoured conclusion.

The second incident involved the editor of Energy and Environment, Dr Boehmer-Christiansen, who claims “[t]he hacked emails revealed attempts to manipulate peer review to E&E’s disadvantage, and showed that libel threats were considered against its editorial team. Dr Jones even tried to put pressure on my university department.”

The third involved Briffa’s actions as the editor of Holocene. In an email dated 4/6/2003, Briffa wrote:

I am really sorry but I have to nag about that review — Confidentially I now need a hard and if required extensive case for rejecting — to support Dave Stahle’s and really as soon as you can. Please

Based on this email, Briffa was accused of “perverting the purpose of peer review by asking a colleague for help in rejecting a paper that contained research findings contradictory to his own views”.

As well as investigating these individual cases, the Review also commissioned Dr Richard Horton, editor of distinguished medical journal The Lancet, to write an essay about the context of peer review, published as an appendix to the inquiry report.

Dr Horton told the Review that some of the questions raised by the CRU emails “may be based on a misinformed view of the peer review process”. Peer review is quality control, not censorship. Although it is obviously impossible for reviewers to be purely objective, the decision to accept or reject is the editor’s responsibility alone; what an editor seeks from a reviewer is “a powerful critique of the manuscript”. Peer review has an important role to play: it prevents over-interpretation and ensures discussion of uncertainty and context — things which contrarians claim to be in favour of. However peer review is not infallible: “Many well-founded concepts are rejected and many erroneous ideas accepted.”

Horton wrote:

Authors and reviewers are frequently passionate in their intellectual combat over a piece of research. The tone of their exchanges and communications with editors can be attacking, accusatory, aggressive, and even personal. If a research paper is especially controversial and word of it is circulating in a particular scientific community, third-party scientists or critics with an interest in the work may get to hear of it and decide to contact the journal. They might wish to warn or encourage editors. This kind of intervention is entirely normal. It is the task of editors to weigh up the passionate opinions of authors and reviewers, and to reflect on the comments (and motivations) of third parties. To an onlooker, these debates may appear as if improper pressure is being exerted on an editor. In fact, this is the ordinary to and fro of scientific debate going on behind the public screen of science. Occasionally, a line might be crossed. [Appendix 5]

So the question becomes: did the CRU scientists cross that line? It turns out the answer is probably not. Let’s look at the three individual cases named above.

In the case of Soon & Baliunas 2003, it was not only CRU which reacted strongly to the paper. The Review recounts:

A number of review editors resigned as a reaction against the publication of what they regarded as a seriously flawed paper. The journal’s publisher admitted that the journal should have requested appropriate revisions of the manuscript prior to publication. The Editor in Chief resigned on being refused permission by the publisher to write an editorial about what he regarded as a failure of the peer review system. [8.3]

Although de Freitas described this reaction as “a mix of a witch-hunt and the Spanish Inquisition”, the Review pointed out that there were scientific grounds given (namely, the paper “conflated qualitative data on temperature and precipitation from many sources that could not be combined into a consistent proxy record”). These counter-arguments “are strongly put, and suggest that the reaction was based on a belief, for which evidence was adduced, that the science was poor. In light of the reaction of the Journal’s publisher, we do not believe that any criticism of Jones can be justified in this regard.” [8.3]

Considering this in the context provided by Richard Horton’s paper, the Review concluded that “this scale of reaction is not unusual in contested areas […] The Review makes no judgement or otherwise about the correctness or otherwise of the Soon and Baliunas paper, but we conclude that the strong reaction to it was understandable, and did not amount to undue pressure on Climate Research.” [8.3]

In the case of Energy and Environment, the Review Team “see nothing [in] Boehmer-Christiansen’s evidence that supports any allegation that CRU has directly and improperly attempted to influence the journal that she edits.” Furthermore, the emails actually show that Boehmer-Christiansen had been accusing CRU of scientific fraud, and “Jones’ response to her accusation of scientific fraud was appropriate, measured and restrained.” [8.4]

In the case of Briffa’s actions, when the Review examined the full email exchange they found nothing to support the interpretation of collusion in rejecting contradictory ideas:

It appears to reflect an Editor with a strongly negative review in hand, and who presumably had read the paper, asking for confirmation that the paper should be rejected, possibly to reduce one of the many complications that assail an editor; and in view of the delay in communicating to authors, hoping for  a strong decision from the referee. On receiving a second, more equivocal review, he offers the authors the opportunity to re-submit. [8.5]

The Review’s conclusion on the peer review allegations was as follows (its emphasis):

On the allegations that there was subversion of the peer review or editorial process we find no evidence to substantiate this in the three instances examined in detail. On the basis of the independent work we commissioned (see Appendix 5) on the nature of peer review, we conclude that it is not uncommon for strongly opposed and robustly expressed positions to be taken up in heavily contested areas of science. We take the view that such behaviour does not in general threaten the integrity of peer review or publication. [1.3.3]

Next: Keeping Skeptics Out of the IPCC?

0 0

Bookmark and Share Printable Version  |  Link to this page | Repost this Article Repost This

Comments

Comments 1 to 8:

  1. Your interpretation is much too lenient and rather misses the point. These quotes look like an attempt to lean on the editorial process of scientific journals. Professor Jones for example appears to have been wanting to change the policy of a peer-reviewed journal so that it no longer published papers which contradicted his perspective.

    I should say I am not an onlooker (Richard Horton's phrase), having published in peer reviewed journals, including The Lancet, and undertaken peer-review. You might want to have a look at Richard Smith's critique of academic medical journals, in which he asserts that most peer-reviewed medical research is of poor quality. I suspect this applies to climate research also, be it of a warmist or skeptical perspective.
    0 0
  2. Meerkat, which case are you referring to? Phil Jones wasn't the one who made the comment about Climate Research, that was Mike Mann. And it was very obvious to all involved that they were upset about the poor standard of peer review. Or, and this is getting more and more obvious, deliberate rigging of the peer review process by one editor, Chris de Freitas. Note that three editors of the same journal complained about de Freitas, noting he published studies that anyone should be able to see were flawed.

    In many cases one may claim ignorance when poor science is published. In this case...ignorance would be to believe de Freitas did NOT rig the process.
    0 0
  3. What I keep getting from this is that skeptics are pointing the finger of blame at Jones, Mann and Briffa for attempting to influence the peer-review process, but what is getting totally ignored is how the skeptics had distorted the peer-review process which incited Jones and others to respond in the ways shown in the stolen emails.

    What is the story of the staff at Climate Research? From what I gather there was a concerted effort on the part of skeptics to stage an editorial take over of that journal in order to get their papers published.

    Why is no one in the MSM digging into this story?
    0 0
  4. The interpretation is not lenient and does not miss the point at all. The Soon and Baliunas paper should never have been published anywhere because it lacks the quality of publishable material.

    The real point is that so-called skeptics like De Freitas and Legates have no scruples at all in circumventing the peer-review process when trying to apply the peer-review label to a paper favoring their views, regardless how worthless it is. That is the true scandal here. Kinne did not stand behind that paper, it has been since repudiated by the journal. I don't see why it is bad thing that scientists be up in arms against a piece so bad that the journal feels the need to later recant on its publication.

    I note also that the change to the journal's process was not to bar papers with a "dissenting" view, it was to keep out trash. Von Storch was the one pushing that change and would not have it any other way.

    As for De Freitas ludicrous comment on inquisition and what not, it would be laughable if there weren't so many gullible people to believe it.
    0 0
  5. Rob @3,

    Yes, frustrating isn't it. My post here (well parts of it t least) may be relevant to this discussion, follow the links.
    0 0
  6. Lets see ... meerkat claims that most of published science is of poor quality ... and then complains when people actively seek to improve the quality of published science.

    That's just weird.
    0 0
  7. Some notable comments from von Storch below...Climate Research was certainly a case of perverting the peer review process, but the polar opposite of what contrarians are claiming.

    “The review process had utterly failed; important questions have not been asked, as was documented by a comment in EOS by Mann and several coauthors.”

    “I withdrew also as editor because I learned during the conflict that CR editors used different scales for judging the validity of an article. Some editors considered the problem of the Soon & Baliunas paper as merely a problem of “opinion”, while it was really a problem of severe methodological flaws. Thus, I decided that I had to disconnect from that journal, which I had served proudly for about 10 years. ”

    http://coast.gkss.de/staff/storch/CR-problem/cr.2003.htm

    http://www.csicop.org/specialarticles/show/deja_vu_all_over_again/
    0 0
  8. The professional deniers are whiners as part of their act. The main item here was and is Soon & Baliunas 2003. It was so bad (and the journal publisher refused to print a rebuttal) that Hans von Storch (more or less a lukewarmer) resigned even though he had just become chief editor. Before long four other editors also resigned.

    If the deniers has a scientific case to make they could just make and would not have to complain.
    0 0

You need to be logged in to post a comment. Login via the left margin or if you're new, register here.



The Consensus Project Website

TEXTBOOK

THE ESCALATOR

(free to republish)

THE DEBUNKING HANDBOOK

BOOK NOW AVAILABLE

The Scientific Guide to
Global Warming Skepticism

Smartphone Apps

iPhone
Android
Nokia

© Copyright 2014 John Cook
Home | Links | Translations | About Us | Contact Us