STATS ARTICLES 2008
Can We Trust Climate Models?
Julie J. Rehmeyer, April 24, 2008
Sometimes, the press echoes skeptics who say that climate models are unreliable. But the next moment, the news is filled with reports of the results of studies that use them. So what are we to believe?
Computer models are one of the key tools climatologists use to understand how climate works and to predict how it will change. At the same time, models are a favorite whipping boy for skeptics of global warming, who argue that the models are so crude that they shouldn’t be relied on to guide policy decisions.
Sometimes, even the modelers themselves seem to agree with the skeptics. Climate models are nowhere close to being able to fully simulate the physical processes of climate, according to David Stainforth, a climate modeler at the University of Exeter in England who is an author of the Intergovernmental Panel on Climate Change reports. As a result, he says, "there's no compulsion to hold that the projections of even the best models can provide useful probabilities for the future of the real world.”
Hearing a statement like that, we might find ourselves sympathizing with climate change skeptic Steven Milloy, who asked August 9 on Fox News:
“If existing climate models are so prone to error, then why would Congress want to rely on them as a basis for enacting energy price-hiking and economy-harming laws and regulations?”
The press tends not to give enough information to allow us to answer these questions, at one moment giving voice to the skeptics and at another reporting the latest scientific findings using climate models as authoritative. So what is the public to make of it?
The short answer is that the models are very reliable about some things and not very reliable about others. Here are the things they’re good for: They’ve shown with great certainty that the increase in average global temperature is caused by increased levels of greenhouse gases. They’ve also shown beyond doubt that even if all further greenhouse gas emission were stopped, the world would still continue to warm up for some time, and that greater levels of continued greenhouse gas emissions will lead to great warming.
But that reliability begins to break down when the models are asked to predict just how much warming will occur, and where. As policymakers have come to accept the reality of global warming, they’ve begun to clamor for just such predictions, sometimes pushing the models beyond what they can reliably do. But the good news is that even here, scientists are learning how to squeeze solid information out of uncertain models.
“The models are useful tools,” Stainforth says. “Very useful tools. We shouldn’t just throw them out, but we shouldn’t just believe their results either.”
Models as Climate Laboratory
Climate scientists have a problem: They can’t do experiments. To perform the experiments they’d like, scientists would need a few million Earths, billions of years, and omnipotence. Then they could pump extra greenhouse gases into the atmosphere of one Earth, prod volcanoes into mad eruptions on another, summon up sunspots to stream extra radiation to the third. They could stop the oceans from circulating, cover the sky with clouds, melt the polar ice. Then they’d sit back and watch what happened, deducing from the consequences how climate works.
But they have only one Earth and whatever powers their brains and computers can summon. And instead of a few billion years, scientists have policymakers peering over their shoulders, needing to know what’s going to happen in the next few years and decades.
So during the 1960s, climatologists started building computer models, creating an artificial Earth inside a computer upon which they could wreak havoc. The first models were crude reconstructions of the most basic climate dynamics. The slow computers at the time limited how much complexity the scientists could code in.
The original goal wasn’t to predict climate change, since at that point, there was little reason to think that climate was undergoing major changes. The goal instead was to understand how the different aspects of climate interrelate. How does temperature affect precipitation? How do changes in ocean currents impact storms? Modelers hoped that understanding these dynamics would also help them predict large-scale climate events like El Niños, which occur every few years and affect weather around the world.
Over the last several decades, the models have grown into fantastically complex creations, built by hundreds of scientists working in parallel. By the mid-1990s, scientists were able to produce climate simulations that looked similar to the climate we actually experience, and they’ve continued to improve rapidly since then.
Nevertheless, none of the models is able to accurately reproduce all aspects of past and current climate. They can now generate climate simulations with temperature shifts over large areas that are quite close to reality. But each model falls down in various ways, sometimes badly. Absolute temperature is off by several degrees in most models. They can also be terrible at predicting cloudiness and rainfall. Rain falls at the wrong time of day in much of the tropics.
Some scientists believe that climate is so complex that no single model will ever be able to realistically reproduce climate as we experience it. The discrepancies between the models “is to be understood as an inherent limitation of models of this class on a question of this type, rather than a measure of the immaturity or inaccuracy of the models,” says James McWilliams, a climate modeler at the University of California, Los Angeles. “Climate forecasts are not likely to come to greater mutual agreement as we go forward into an era of climate change.”
Why is the world warming up?
The project of modeling took on much greater urgency as scientists observed that the world was warming up, dramatically. Glaciers were melting, sea levels were rising, heat waves were getting hotter and longer.Why was it happening? This is exactly the kind of question that climate models are designed for – showing one aspect of climate affects others.
In particular, modelers aimed to identify the phenomenon driving global warming by finding its “fingerprint” in the data. A climate fingerprint is a pattern of warming that results from a particular change to the climate system. For example, when modelers turned up the radiation from the sun in their computerized Earths, the upper atmosphere heated up along with the lower atmosphere.
In reality, though, only the lower atmosphere has warmed while the upper atmosphere has cooled. This is just one of the pattern mismatches that showed that solar activity couldn’t be the primary cause of current warming.
But when it came to increasing greenhouse gases instead of radiation, the models all showed that the lower atmosphere warmed up and the upper atmosphere cooled. In every other significant way the scientists found that the models all produced the right pattern of warming when they were run with increased levels of greenhouse gases. And conversely, if the modelers left the increased greenhouse gases out of the model, they couldn’t produce the observed warming, no matter how much they emphasized other factors that might be contributing to it.
Even though the models disagree with one another when they try to predict future climate, they are unanimous when it comes to identifying these patterns.
The models are also agree that the world will continue to warm to some extent, even if we were to stop all further greenhouse gas emissions. The greenhouse gases that have already been released will continue to trap heat.
Predicting the future
So we’re going to have to adapt to a hotter world. To plan for that, policymakers need to know more details about what’s likely to happen. How warm will the world get? Will hurricanes get worse? What about heat waves, or cold spells?
The models provide a seductive way of trying to answer that question. One might imagine that if you fed the current conditions into the model as the initial conditions, then the climate simulation the model produces would be pretty close to the climate we’ll actually experience.
But remember that the models were not designed to be crystal balls; they were designed to illuminate how the different aspects of climate affect one another. If the models were a perfect reconstruction of the physical world, their predictions could be expected to be accurate, but they’re not anywhere close to that and will probably never be. At the moment, they can’t even reproduce current climate accurately – a far easier problem than predicting a future climate that mankind has never experienced. So the projections of climate models about future climate may not have any clear relation to what will actually happen.
When scientists tried running the models forward from current data, they ran into a big problem: The different models produced substantially different results. All agreed that the world would warm, but they disagreed over just how much and how the changes would affect other aspects of climate like rainfall, cloudiness, and storms.
One reason for the differences is that the models can’t possibly simulate everything that impacts climate, and the small omissions can add up to big inaccuracies. One example of this is how models simulate clouds. The smallest scale the models can look at is a square that is about 50 miles across – far larger than a single cloud, and absurdly larger than the molecules whose interaction creates the cloud. So the models use numbers called parameters to simplify the problems.
One parameter, for example, gives the rate at which moisture turns to ice and falls out of a cloud. In reality, that rate depends on a variety of physical factors, such as the amount of dust in the cloud, which varies from one spot to another. But to make the problem manageable, the models treat the rate as constant. Because these parameters are simplifications of complex processes, there’s no single “best” value for them to have, and different models make slightly different choices. These different choices lead to different projections.
Another difficulty is uncertainty about initial conditions. Scientists can’t possibly tell the models the location and temperature of every molecule on Earth, so the information they feed in about initial conditions is, by necessity, approximate. For weather, this approximation dooms even the best possible model to inaccuracy more than a couple weeks out; small uncertainties about the current conditions magnify into large uncertainties about the future weather over time. Climate is a bit less susceptible to this problem than weather, because climate is average weather, which tends to be far more regular than the particular weather on a given day. Nevertheless, the uncertainties about the exact conditions do make climate forecasts worse.
Predicting future climate also has the fundamental difficulty of any kind of prediction: unexpected things can happen. Climate scientists run into this problem as they see evidence of more and more feedback loops. In a feedback loop, rising temperatures bring changes that may produce even more warming. One such example is that as polar ice melts, more of the sea becomes exposed. The water is much darker than ice and hence absorbs far more of the sun’s rays, leading to more warming, which melts more ice, which exposes more sea, which creates more warming.... Because of these feedback loops, warming might take off like a runaway horse in response to small additional amounts of warming, in ways that are hard to predict.
Fortune-telling without a crystal ball
Despite all these obstacles, scientists are learning to wring useful information out of the models. But they’re not doing it by simply running the models forward and believing the predictions. Instead, they’re using the models to do what they do best: illuminate how the different aspects of climate interact.
Let’s take as an example the most fundamental climate prediction question – how much will the world warm up? This graph from the Fourth Assessment Report of the Intergovernmental Panel on Climate Change shows the scientists’ current best guess. The colored lines show projections under four different emission scenarios, from ceasing all additional greenhouse gas emissions (shown in orange) to continuing on a similar path to the one we’re on (shown in red).
The colored shaded areas around the lines show the range of model projections. Thus, under the highest emission scenario shown here, temperatures are projected to rise 3 to 4 degrees Celsius (5½ to 7 degrees Fahrenheit) by 2100.
A prediction like this is a nice thing, but it doesn’t do much good unless you know how likely it is to actually come true. And remarkably enough, the scientists have found a way to say just how likely their predictions are.
The critical thing to notice here is the big grey bars to the side. What scientists can actually predict with confidence (to be precise, with 90 percent confidence) is that the temperature rise for each scenario will fall within those grey bars.
So for the highest emission scenario, the scientists say that there’s a 90 percent chance the Earth will warm up at least 2 degrees but no more than 5½ degrees Celsius (3½ to 10 degrees Fahrenheit). That range is more than three times as big as the range of the models’ projections. Note, though, that in any of these cases, the world is going to warm up significantly.
Those grey bars are awfully big, which may suggest that we should trust the scientists’ projections less. But in fact, those grey bars should be enormously reassuring. They’re telling us that the scientists aren’t just taking their best shot at a prediction. They’re able to tell us just how likely that best guess is. And within that (rather large) range, the prediction is pretty darn likely.
So where do those big grey bars come from? How do scientists know how likely their predictions are?
Getting Certain About Uncertainty
Even though the forecasts of the climate models cannot be directly used to make predictions with confidence, the models have nevertheless been the key tool that scientists have used to find those grey bars that quantify the range of certainty. They just had to use them in a different way. This time, they used them to illuminate how different aspects of climate impact one another.
The models have established that over the coming decades, carbon dioxide will cause far greater warming than any other factor, from methane to sunspots to volcanoes. So the key question is, how much does a certain increase of carbon dioxide warm up the Earth?
Fundamental physics gives a mathematical formula that answers that very question. (The formula, to be precise, is that exponential increases in greenhouse gases lead to linear increases in temperature.) But the formula has one missing value, which gives the exact rate of the warming. That can only be supplied by analyzing past climate data to see how much the world has warmed in response to past increases in carbon dioxide.
Scientists know pretty well both how much the world has warmed up over the past several decades and how much carbon dioxide has increased. But there’s a complication: the globe has warmed up and cooled down in response to lots of changes, not just carbon dioxide levels, from volcanoes to methane to sunspots. So the scientists have to separate out the effect of carbon dioxide from everything else.
This is where the models really shine. They have shown that the different factors affecting climate create different patterns of warming and cooling – and critically, they all agree closely about what those patterns are. Using these patterns, they are able to figure out how much the overall warming has been caused by carbon dioxide increases rather than the other factors. And this gives the missing value in the formula.
Armed with this final piece of information, scientists can start predicting temperatures. They take a certain scenario, where carbon is emitted at some given rate, plug that into the formula, and get a pretty good estimate of what the temperature will be under that emission scenario.
Even more importantly, the formula allows the scientists to figure out both how much and how little the world might possibly warm. If the world warms up a different amount from their prediction, it’s because something about their reasoning wasn’t quite right. They know the physics behind the formula is precisely accurate, because that’s been verified by experiment. But there was that one missing number which the models helped them fill in. That came from analyzing the data, and data is never perfectly accurate. If that number wasn’t quite right, it would skew their answers.
So how could that number be wrong? That number came ultimately from data about past warming in response to increases in carbon dioxide. So they looked for all the limitations in their equipment and methods and figured out the lowest and highest possible numbers that might be correct. Then they ran those two ends of the range through the formula, and they got a range of possible amounts of warming that may result. That’s where the big grey bars come from.
Of course, there are still a lot of other ways the scientists could be wrong. They still don’t know, for example, how much feedback loops will impact warming. And in fact, they’ve adjusted their estimates in other ways to allow for that uncertainty (which is why the top section of the gray bar, the region above the actual projection, is larger than the bottom section). But they also know that the most uncertain feedback loops are unlikely to affect temperatures enormously over the next hundred years – though over several centuries, those feedback loops may have a big effect.
The important thing to notice, though, is that the disagreements between the amount of warming the models directly predict don’t affect the size of those gray bars. The models provided a way to recognize the important factors are likely to contribute to warming, and they helped the scientists to derive the missing number in the formula by separating out the effect of carbon from the effects of other things. On those kinds of questions, all the models produce the same results.
There’s a lot more to climate than average global temperature, and other aspects of climate are substantially harder to predict. Wrangling solid information out of models is correspondingly more difficult. But the principles of this example apply to more complex climate questions as well. Despite models’ inability to provide predictions that match one other about future climate, scientists are able to use the models to generate solid information about how climate is likely to change.
But as policymakers ask for more detailed information about smaller areas, the scientists’ ability to provide answers is strained. So the engineer who needs to know the average windspeed off the coast of Scotland in 2050 is going to have to wait a long time before modeling can provide possible answers.
All of this means that climate skeptics are right, in a narrow sense: The models aren’t capable of serving as crystal balls, telling us our climate future; nevertheless, scientists are able to use the models as a tool to help them get a reasonable sense of how climate is likely to change, and how big a difference action now may make in the future.