Arguments
How reliable are climate models?
The skeptic argument...
Models are unreliable
"[Models] are full of fudge factors that are fitted to the existing climate, so the models more or less agree with the observed data. But there is no reason to believe that the same fudge factors would give the right behaviour in a world with different chemistry, for example in a world with increased CO2 in the atmosphere." (Freeman Dyson)
What the science says...
| Select a level... |
Basic
|
Intermediate
| |||
|
Models successfully reproduce temperatures since 1900 globally, by land, in the air and the ocean. |
|||||
Climate models are mathematical representations of the interactions between the atmosphere, oceans, land surface, ice – and the sun. This is clearly a very complex task, so models are built to estimate trends rather than events. For example, a climate model can tell you it will be cold in winter, but it can’t tell you what the temperature will be on a specific day – that’s weather forecasting. Climate trends are weather, averaged out over time - usually 30 years. Trends are important because they eliminate - or "smooth out" - single events that may be extreme, but quite rare.
Climate models have to be tested to find out if they work. We can’t wait for 30 years to see if a model is any good or not; models are tested against the past, against what we know happened. If a model can correctly predict trends from a starting point somewhere in the past, we could expect it to predict with reasonable certainty what might happen in the future.
So all models are first tested in a process called Hindcasting. The models used to predict future global warming can accurately map past climate changes. If they get the past right, there is no reason to think their predictions would be wrong. Testing models against the existing instrumental record suggested CO2 must cause global warming, because the models could not simulate what had already happened unless the extra CO2 was added to the model. All other known forcings are adequate in explaining temperature variations prior to the rise in temperature over the last thirty years, while none of them are capable of explaining the rise in the past thirty years. CO2 does explain that rise, and explains it completely without any need for additional, as yet unknown forcings.
Where models have been running for sufficient time, they have also been proved to make accurate predictions. For example, the eruption of Mt. Pinatubo allowed modellers to test the accuracy of models by feeding in the data about the eruption. The models successfully predicted the climatic response after the eruption. Models also correctly predicted other effects subsequently confirmed by observation, including greater warming in the Arctic and over land, greater warming at night, and stratospheric cooling.
The climate models, far from being melodramatic, may be conservative in the predictions they produce. For example, here’s a graph of sea level rise:
Sea level change. Tide gauge data are indicated in red and satellite data in blue. The grey band shows the projections of the IPCC Third Assessment report (Copenhagen Diagnosis 2009).
Here, the models have understated the problem. In reality the events are all within the upper range of the model’s predictions. There are other examples of models being too conservative, rather than alarmist as some portray them. All models have limits - uncertainties - for they are modelling chaotic systems. However, all models improve over time, and with increasing sources of real-world information such as satellites, the output of climate models can be constantly refined to increase their power and usefulness.
Climate models have already predicted many of the phenomena for which we now have empirical evidence. Climate models form a reliable guide to potential climate change.
Last updated on 26 October 2011 by Sphaerica. View Archives
If you don't put your money on the horse (i.e. you don't trust the models), you are unaffected whether it wins or loses. Your life goes on as normal.
If don't "put money" on AGW (i.e. disbelieve the experts) and it turns out to be true - as the body of science strongly suggests - then you'll be affected.
A better analogy would be if someone kidnapped a loved one, told you to pick the next Kentucky Derby winner, and warned you they'll kill the hostage if you pick the loser. Which horse would you pick then? The favorite (i.e. the one the experts say has a better chance of winning than the others according to odds calculations), or a long shot that experts say is unlikely to win?
"based on" is a meaningless phrase here.
How could the first models have peeked at the previous data ?
You're right about one thing, though, they aren't just constructed using the laws of physics : there's a lot of maths in there too, as well as some chemistry.
That would be akin to predicting the next hurricane's landfall location from a study of prior landfalls. A more reasonably-posed analogy might be to conclude from a study of prior races that there are factors that categorizes the field of entrants into 'likely' winners and 'likely' losers (forgive my alarmist language). For example (from wikipedia): "No horse since Apollo in 1882 has won the Derby without racing at age two." That would make a 'very unlikely' outcome.
So from a study of climate, it is perhaps unreasonable to predict a specific heat wave, but not at all unreasonable to build a model that says: "would these [extreme weather] events have occurred if atmospheric carbon dioxide had remained at its pre-industrial level of 280 ppm?", an appropriate answer in that case is "almost certainly not."
But what is all this about models not having a peek at prior data? On another thread, there have been comments to the effect that models and evidence somehow pollute each other; that makes no sense to me. What use is a model if it is not built on prior data and tested by subsequent data?
My problem is that when I look on this site, or anywhere, for good evidence that manmade CO2 is going to cause significant harm, the only evidence of any significance is that the models match the data, or the data matches the models.
Since the models are developed to match the data, what do they expect?
Don't quote it as evidence, that's all I'm saying.
But we have long experience of successful building.
We have zilch of successful climate forcasting.
So I think I'm right to be sceptical of the models' ability to get it right at this stage.
so we have a phenomenon, we build a theory (a model) and compare it to the observed phenomenon. If they agree I throw the model away because it is trivial, if it does not I throw it anyway. I'm puzzled.
More seriously, the first model dates back to 1896. Not enough subsequent data to test it? Yes, of course. It has then be refined and tested again, and so on for many decades. Apparently you're seing just last generation of models, as if they came out of nowhere.
Predicting the past is still a prediction (see retrodiction). If you build a simulation of a physical system, it is appropriate to test that simulation by comparing it to past performance of the real system. This is true of any physical model, including models of bridges and space shuttles.
The fact that you are comparing to past data does not mean that the simulation has past data "programmed in" as you are implying. What you are thinking of is a statistical model, where the inputs are directly mapped to outputs via a mathematical relationship derived directly from historical data. This is not how physical climate models are derived.
Since the model is built on physical laws and not on direct statistics, there is no reason to assume that a particular model could ever recreate past climate behavior, unless that model has some basis in reality. If the basic physics underlying the model are significantly off, then no amount of tweaking would ever result in an accurate recreation of past performance.
The fact that it can recreate past performance is therefore evidence that the model is correct, since the likelihood is very slim that the model would be able to accurately recreate real performance if it was significantly wrong in its recreation of physics.
You're forgetting a significant cause of such unpleasant events: Google search 'operator error accidents'.
Such is not the case in a climate model, where there is no one to push the wrong button, run past a red signal or close a valve that should be left open.
Since the model is built on physical laws and not on direct statistics, there is no reason to assume that a particular model could ever recreate past climate behavior, unless that model has some basis in reality
The situation is not so nice as you paint it. Sub-grid processes, aerosols and the like are always parametrized in models, that is, these are not derived from first principles, but are chosen to reproduce the past. And as von Neumann said "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk."
Computational climate models tend to agree on past trends but diverge considerably in their predictions. It is a sure sign they do use the leeway provided by the parametrization process and use it disparately.
Do you have a better procedure in mind?
On the same page, von Neumann also said "I think that it is a relatively good approximation to truth — which is much too complicated to allow anything but approximations — that mathematical ideas originate in empirics."
"Another solution is to bring trained computer scientists into research groups, either permanently or as part of temporary alliances. Software developer Nick Barnes has set up the Climate Code Foundation, based in Sheffield, UK, to help climate researchers. He was motivated by problems with NASA's Surface Temperature Analysis software, which was released to the public in 2007. Critics complained that the program, written in the scientific programming language Fortran, would not work on their machines and they could therefore not trust what it said about global warming. In consultation with NASA researchers, Barnes rewrote the code in a newer, more transparent programming language —Python — reducing its length and making it easier for people who aren't software experts to understand how it functions. "Because of the immense public interest and the important policy issues at stake, it was worth taking the time to do that," says Barnes. His new code shows the same general warming trend as the original program."
Seriously, poptech, how could consistent trends among dozens of climate models and several global temperature averaging algorithms, each coded by separate groups, result from random coding errors?
You are applying several general statements in the article to climate modelers specifically when 1) none of the most damning examples provided by that article relate to climate modeling and 2) you haven't even bothered to find out what procedures and cross checks climate modellers have in place. This is the definition of quote mining.
Poptech,
This is what you should have said, which might have been somewhat closer to the truth, albeit still highly misleading:
"Nature Admits
ClimateScientists are Computer Illiterate"My retort would be (as exemplified by your post and as noted by Stephen @270 above):
"Climate "skeptics" illiterate on the science and fact checking"
Anyhow,
1) Nature did not admit that "climate scientists are computer illiterate" as you would so dearly love to believe. The title you elected to use is clearly your spin of the article's content.
2) The example from the University of Edinburgh that Wilson discusses (and which you bolded) does not seem to apply to climate scientists, but scientists in general. Yet, you oddly chose to conclude that he was referring to all climate scientists.
Also, From the very same Nature article:
"Science administrators also need to value programming skills more highly, says David Gavaghan, a computational biologist at the University of Oxford, UK."
"The mangled coding of these monsters can sometimes make it difficult to check for errors. One example is a piece of code written to analyse the products of high-energy collisions at the Large Hadron Collider particle accelerator at CERN, Europe's particle-physics laboratory near Geneva, Switzerland."
So using your (flawed) logic, are all CERN scientists, and by extension all physicists, computer illiterate Poptech? No, of course not.
"Aaron Darling, a computational biologist at the University of California, Davis, unwittingly caused such a mistake with his own computer code for comparing genomes to reconstruct evolutionary relationships."
So using your (flawed) logic are all computational biologists, and by extension all biologists, computer illiterate Poptech? No, of course not.
"The CRU e-mail affair was a warning to scientists to get their houses in order, he says. "To all scientists out there, ask yourselves what you would do if, tomorrow, some Republican senator trains the spotlight on you and decides to turn you into a political football. Could your code stand up to attack?"
So your post at #270 was seriously flawed, and a perfect example of confirmation bias and cherry picking which is often used by "skeptics" to distort and misinform. You would have been better of citing the example of Harry at CRU that they discussed...oh, but that has already been done months ago, and is not sufficient evidence to make sweeping generalizations. Please, do not insult us here with your misinformation.
There's a crust of value in this observation, as it must apply equally to both sides. Hence, analysis of surface temperature measurements, all of the 'no, its not' repeated ad nauseum here, etc. must be subject to the same risk of error. If you can't trust a climate scientist, why should you trust a climate skeptic?
Oh now it is "natural scientists"...need I remind you of your original headline?
Anyhow, you demonstrated above that you not only cherry picked your cut and pasts, but you distorted the content of the article to suite your means. Yes, that is poor form. EOS.
Do all scientists (with the obvious exception of computer science) need to hone and improve their computer skills? Yes, I can certainly agree with that.
You have that backwards. The fact that the coding differs and the same results are acheived is actually a good thing -- an indication that the science is settled with respect to effects of CO2 on climate.
You know as well as I that the code would differ even if the models worked at the same spatial resolution, represented ocean-atmosphere coupling in the same way and had identical degrees of detail the terrestrial carbon cycle (not to mention other things). Different software languages are used, there are different limitations on computing resources, and scientists have to make innumerable little decisions about how to handle input data.
The fact that all the models can only produce the increase in temp over the last century only if the effect of CO2 is included indicates that those coding decisions have no bearing on the issue of whether anthropogenic CO2 has an effect on climate.
I also agree with the sentiment of the Nature article
- scientists (and biologists in particular) need more expert training in coding. But, I think climate scientists are probably the ones on the cusp of the effort to code better and more openly.
Do these new models also include the negative feedback from the ocean equivalents of green plants, ie - photosynthetic creatures like algae, diatoms, coral, etc? The huge biomass in the oceans would seem to be more important than terrestrial plants?
The Science Daily report on the NASA study
http://www.sciencedaily.com/releases/2010/12/1012080
Here is the abstract from the paper
http://europa.agu.org/?view=article&uri=/journals/gl/gl1023/2010GL045338/2010GL045338.xml&t=gl,bounoua
"Several climate models indicate that in a 2 × CO2 environment, temperature and precipitation would increase and runoff would increase faster than precipitation. These models, however, did not allow the vegetation to increase its leaf density as a response to the physiological effects of increased CO2 and consequent changes in climate. Other assessments included these interactions but did not account for the vegetation down‐regulation to reduce plant's photosynthetic activity and as such resulted in a weak vegetation negative response. When we combine these interactions in climate simulations with 2 × CO2, the associated increase in precipitation contributes primarily to increase evapotranspiration rather than surface runoff, consistent with observations, and results in an additional cooling effect not fully accounted for in previous simulations with elevated CO2. By accelerating the water cycle, this feedback slows but does not alleviate the projected warming, reducing the land surface warming by 0.6°C. Compared to previous studies, these results imply that long term negative feedback from CO2‐induced increases in vegetation density could reduce temperature following a stabilization of CO2 concentration."
Chris Shaker
http://www.sciencedaily.com/releases/2007/12/071211233441.htm
"Interestingly, warming temperatures in response to rising carbon dioxide levels could make more nitrogen available, said Xiaojuan Yang, a doctoral student in Jain’s lab. This factor must also be weighed in any calculation of net carbon dioxide load, she said.
“Previous modeling studies show that due to warming, the soil releases more carbon dioxide through increased decomposition,” she said. “But they are not considering the nitrogen effect. When the soil is releasing more CO2, at the same time more nitrogen is mineralized. This means that more nitrogen becomes available for plants to use.”"
Chris Shaker
http://en.wikipedia.org/wiki/Primary_production
"Using satellite-derived estimates of the Normalized Difference Vegetation Index (NDVI) for terrestrial habitats and sea-surface chlorophyll for the oceans, it is estimated that the total (photoautotrophic) primary production for the Earth was 104.9 Gt C yr−1.[12] Of this, 56.4 Gt C yr−1 (53.8%), was the product of terrestrial organisms, while the remaining 48.5 Gt C yr−1, was accounted for by oceanic production."
So, I hope to read that these negative CO2 feedback effects are also being modeled for algae as well.
Chris Shaker
I highly recommend the book, A Random Walk Down Wall Street (http://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street) for a very approachable explanation of some of the techniques that modelers use. The author's arguments that it is impossible to "beat" the stock market through this analysis isn't relevant to weather modeling. However, I think he does a good job of addressing the question of identifying success based on past data.
This is not a problem for climate science alone. It is a problem for any time-based models. I have been involved in environmental predictions based on using multivariate regression analysis of GIS data correlated with soil types. This falls into the same problem, but it can be amended by later sampling of soils at predicted locations, and correlating the observed soil type with the predicted soil type and running a t-test to establish the reliability of the prediction. It would be useless to sample the same site that the model was based on. The only way that the same calibration could be carried out in time-based climate models is by comparing forecasts with what happens in the future and not the past. Are the models therefore proper science. Without reference to the future they are unfalsifiable. The absence of controls is another issue.
I understand the practical problem of testing the accuracy of long-term models in this way. The damage may have been done before the data is in. Can you post a link to papers which articulate the assumptions behind these models?
Models are not just based on hindcasting - that's a required check, but the assumptions going into the models are based on physics, not just mathematical modeling of previous behavior.
Part II of that post then provides more details on parameterizations, including specifics on clouds.
http://epw.senate.gov/public/index.cfm?FuseAction=Minority.Blogs&ContentRecord_id=84E9E44A-802A-23AD-493A-B35D0842FED8
The reason I am posting this link is to show there is considerable controversy and evidence of GCM's not being reliable or valid from 1900 to 2011.
Controversy is one thing, but controversy supported by data and published in peer-reviewed science is another.
OklahomaPetrodollars. This isn't 'controversy,' its junk.For example:
SURVEY: LESS THAN HALF OF ALL PUBLISHED SCIENTISTS ENDORSE GLOBAL WARMING THEORY - Excerpt: "Of 539 total papers on climate change, only 38 (7%) gave an explicit endorsement of the consensus.
Even for 2007, that was blatantly false.
and here: "comparing results of global climate models to records for individual stations is missing the point, and shows a fundamental misunderstanding of what those models tells us."
The paper in question compares GCM output to data at selected stations using point-by-point, year-by-year observation to computed model output.
The authors note the following regarding their prior work:
criticism appeared in science blogs (e.g. Schmidt, 2008). Similar criticism has been received by two reviewers of the first draft of this paper, hereinafter referred to as critics. In both cases, it was only our methodology that was challenged and not our results.
It would seem that if the methodology is challenged, the results are implicitly challenged. The comparison of yearly data to model output is clearly one of apples and oranges: It is little more than comparison of weather to climate. In addition, there's a deeper flaw in the methods used to process individual station data:
In order to produce an areal time series we used the method of Thiessen polygons (also known as Voronoi cells), which assigns weights to each point measurement that are proportional to the area of influence
This was an issue in the early days of computer-aided mapping. Rather than spatially average a local cluster of stations first, this process allows any error or anomaly at a single station to propagate into the database. Why not filter for a noisy (or bad) datapoint before gridding?
So it's not at all clear the original criticisms were answered by the newer paper.
I noticed also out of the 26 references cited in the paper that the authors cited themselves 10 times and NASA was only mentioned once in the entire paper. I also find it rather less than scholarly to have a blog cited as one of the references, albeit Gavin Schmidt at realclimate.org., of which you have already described.
http://iopscience.iop.org/1748-9326/6/1/014008:
"
Abstract
This study assesses the accuracy of state-of-the-art regional climate models for agriculture applications in West Africa. A set of nine regional configurations with eight regional models from the ENSEMBLES project is evaluated. Although they are all based on similar large-scale conditions, the performances of regional models in reproducing the most crucial variables for crop production are extremely variable. This therefore leads to a large dispersion in crop yield prediction when using regional models in a climate/crop modelling system. This dispersion comes from the different physics in each regional model and also the choice of parametrizations for a single regional model. Indeed, two configurations of the same regional model are sometimes more distinct than two different regional models. Promising results are obtained when applying a bias correction technique to climate model outputs. Simulated yields with bias corrected climate variables show much more realistic means and standard deviations. However, such a bias correction technique is not able to improve the reproduction of the year-to-year variations of simulated yields.
This study confirms the importance of the multi-model approach for quantifying uncertainties for impact studies and also stresses the benefits of combining both regional and statistical downscaling techniques. Finally, it indicates the urgent need to address the main uncertainties in atmospheric processes controlling the monsoon system and to contribute to the evaluation and improvement of climate and weather forecasting models in that respect."
http://onlinelibrary.wiley.com/doi/10.1002/wcc.60/full:
"Abstract
Computer models are powerful tools that allow us to analyze problems in unprecedented detail and to conduct experiments impossible with the real system. Reliance on computer models in science and policy decisions has been challenged by philosophers of science on methodological and epistemological grounds. This challenge is examined for the case of climate models by reviewing what they are and what climate scientists do with them, followed by an analysis of how they can be used to construct new trustworthy knowledge. A climate model is an executable computer code that solves a set of mathematical equations assumed to represent the climate system. Climate modelers use these models to simulate present and past climates and forecast likely and plausible future evolutions. Model uncertainties and model calibration are identified as the two major concerns. Climate models of different complexity address different question. Their interplay helps to weed out model errors, identify robust features, understand the climate system, and build confidence in the models, but is no guard against flaws in the underlying physics. Copyright © 2010 John Wiley & Sons, Ltd."
These two last links with Abstracts included are not skeptical arguments against using GCM's or climate change. Yet they highlight regional issues, weather events, patterns,microclimate, and making projections based upon these and finding trends in climate. Not that these papers conclude nothing can be understood better or analyzed but that models still contain lots of unreliability, which is the topic of this thread.
GEP Box said "all models are wrong, but some are useful". You have not established that the models are unreliable, nor have you demonstrated that the stated uncertainty of the projections is so high that they are not useful.
It is well known that GCMs don't work well at smaller spatial scales, but that doesn't mean they are not accurate in projections of global climate variables.
First
link:http://www.nature.com/climate/2010/1003/full/climate.2010.19.html
First Quote:
Initial studies of how the rivers will respond to ice loss show modest changes in stream flow — far from the IPCC report's dire scenario of rivers running dry. Even if the glaciers were lost completely, flows down the Indus would drop about 15 per cent overall, with little or no change in the dry-season flow, one recent study found9. Lall cautions, however, that climate models are poor at simulating rain and snowfall, especially for the Asian monsoons. “I wouldn't hold these models to be very accurate,” he says.
In the absence of clear predictions of what's to come, close monitoring of changes in the mountains is all the more important, as rising temperatures will probably affect the whole water cycle, says Eriksson of ICIMOD. “There has been too much focus on the glaciers as such,” he says. “It's urgent to understand the whole [impact] of climate change on snow, ice and rainfall, and that is not happening.”
Second Quote back a few sentences prior to the above quote:
Cogley. Upmanu Lall, director of the Columbia Water Center at Columbia University in New York, agrees. Lall says the idea that the rivers could run dry because of shrinking glaciers seems to stem from a confusion about how much glaciers contribute to river flows, compared with the contribution from melting of the seasonal snowpack."
Link 2, I am introducing because it because you can download it and and place it into GOOGLE Earth and look at data and so called trends yourselves as individuals. It also illustrates how short any decent temperature record keeping has been around and how few stations there have been. The download is free.
http://www.climateapplications.com/kmlfiles.asp
The above link can assist everyone in analysis
Concerning your first link please note that it is a news article in Nature, not a peer reviewed study. Secondly, the error about the Himalayan glaciers in the IPCC report has long since corrected "by the IPCC". However, the Himalayan glaciers are receding, just not as fast as initially reported. There are many discussions concerning the Himalayan glaciers on this site. Please review Return to the Himalayas.
That would be unsupported opinion, at odds with models reliability to date. What's your idea of "short term"? At decadal level, models are hopeless. At longer scales they do exceedingly well.
Cutting to the chase - how much accurate prediction by models can you take before you change your mind?
Your second linked abstract "Computer models are powerful tools ... " provides no actual scientific criticism of models.
Their interplay helps to weed out model errors, identify robust features, understand the climate system, and build confidence in the models, but is no guard against flaws in the underlying physics.
This is merely a caveat; the author suggests that models could have 'flawed' physics. Do you automatically assume that because an author says something could happen, it necessarily does?
A far more robust criticism would be 'here is a model study with flawed physics.' I assume you did not use this because you have no such example at hand?
Large uncertainty does not imply unreliability. In fact it means that models are more likely to be reliable as the model projections cover a wider range of possibilities. If you ignore the stated uncertainty of the projections, that is your error, not a shortcoming of the models.
As it happens, the error bars on projections for large time and spatial scales are not broad compared to the variation in the ensemble mean. See e.g. IPCC report chapter on attribution.
If you don't take on board points that are made, or at least counter them, don't be surprised if contributors stop responding to your posts.