Shades of "Gray Literature": How Much IPCC Reform Is Needed?

A better way to compile and review climate science starts with making sure the organization charged with it has an adequate and accountable full-time staff















Share on Tumblr

russian-glaciers-from-space

GLACIAL MELTDOWN?: Glaciers across Asia, such as those on Russian Arctic islands pictured here via the Landsat satellite, are melting down rapidly. Image: Courtesy of Julian Dowdeswell, Scott Polar Research Institute, Cambridge, UK / USGS

The Intergovernmental Panel on Climate Change's 2007 report from the group working on global warming's impacts contained at least one error. "Glaciers in the Himalaya are receding faster than in any other part of the world (see Table 10.9) and, if the present rate continues, the likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the Earth keeps warming at the current rate," the report notes.

A detailed analysis by the InterAcademy Council (IAC)—a composite board of many of the world's national scientific bodies, such as the National Academy of Sciences in the U.S.—of the history behind this statement finds that at least three reviewers, including the government of India, challenged it as the 2007 report was being drafted. And it was undercut in the report's very next sentence: "Its total area will likely shrink from the present 500,000 to 100,000 km2 [square kilometers] by the year 2035 (WWF, 2005)."

"100,000? You just said it will disappear," ecologist David Saltz of the Jacob Blaustein Institute for Desert Research Institute at Ben-Gurion University of the Negev noted during the IPCC's review process, according to records kept by the IPCC that were among the data reviewed by the IAC.

Nevertheless, both sentences made it into the final report—an example of potential problems in the IPCC process that the IAC was charged in March to review.

The Himalayan glaciers mistake "came from just not paying close enough attention to what review editors commented," economist Harold Shapiro of Princeton University noted, calling for review editors to identify the most important points that come up and focus lead authors on addressing those criticisms. "It may have been they were overwhelmed by the sheer number of comments. That's just my inference." Shapiro chaired the IAC effort, the results of which were released August 30 at a news conference at the United Nations in New York City. The review drew from data ranging from IPCC participants' testimony to an online survey open to the public.

The IAC concluded that "the overall structure of the IPCC assessment process appears to be sound, although significant improvements are both possible and necessary," the report's authors wrote.

Calling for "fundamental reforms" in the IPCC process, the IAC noted that the undertaking is becoming enormous—in addition to wrangling 194 government representatives and more than 3,000 scientists, the IPCC authors had to deal with some 90,000 comments on their most recent report. So, for example, the IAC report recommends that IPCC lead authors not be required to respond to comments that are "purely editorial," in Shapiro's words.

Other reforms include: forcing governments to comment before plenary sessions (government changes consistently slow the release of IPCC summaries); a clear and codified conflict of interest policy; and a permanent management structure consisting of a "small" permanent executive committee that includes members from "outside IPCC or even outside the climate science community" as well as a full-time chief executive in the form of an executive director who is a senior scientist.

"We need a management structure at the top," IPCC chair Rajendra Pachauri agreed at a press conference at the U.N. following the IAC's report release. "It's still a very informal structure. I'm very happy that the IAC has recommended that this structure, in somewhat different shape and form, be formalized."

Fundamental flaws?
The Himalayan glaciers error was drawn from non-peer-reviewed findings, known as "gray literature," which nonetheless has a place in the IPCC reports, the IAC found. "We found that such material, which can include technical reports, conference proceedings, observational data or model results, often is relevant and appropriate for inclusion in the assessment reports," said Shapiro at the press conference. The key to their fair use is making guidelines for such use both clearer and more strictly enforced "to ensure that unpublished and non-peer-reviewed literature is adequately evaluated and appropriately flagged in the reports."

The IAC report is at least the sixth review of IPCC processes, procedures and outcomes in the past year. None, including the IAC, have found cause to question the underlying conclusions based on the best available science on climate change from the most recent assessment. "Man-made drivers are responsible for most of the climate change that we have experienced in the past 50 years," explained climatologist Thomas Stocker of the University of Bern, co-chair of the working group on the physical science of climate change for the next IPCC assessment, at the press conference following the IAC's report release. "This result has never been challenged."

The IAC also calls for leaders of the IPCC, including the chairs of the various working groups, to serve for only one assessment, although this did not reflect any judgment on current leadership, according to Shapiro. Pachauri is now overseeing his second such assessment.

How to handle uncertainty
Most importantly, perhaps, the IAC concludes that the IPCC must be strict in its handling of uncertainty in climate science. Shapiro and his colleagues recommend that the IPCC adopt a "level of understanding" assessment of the likelihood of any given outcome or scientific statement as well as a clear description of the basis for that assessment. Such a model is employed by the IPCC working group on mitigating climate change and delineates whether there is high or low agreement among experts on the finding as well as a large or small amount of evidence for that finding. Such an assessment could then be backed by a more quantitative assessment of uncertainty when appropriate, such as in the case of directly observable measurements like CO2 concentrations in the atmosphere.

"I have been involved with a group that is reworking the uncertainty guidance for [the next assessment report], and we are trying to fix those issues," says economist Gary Yohe of Wesleyan University, who was interviewed by the IAC for its report and served as a lead author for the IPCC. "It is difficult, and one size does not fit all…but we think we will have something better to offer authors soon."

The IAC-recommended approach would have prevented some of the other errors that found their way into the overall report. "The authors reported high confidence in statements for which there is little evidence, such as the widely quoted statement that agricultural yields in Africa might decline by 50 percent," the report's authors note, one of three explicit errors that have been identified in the more than 4,000 pages of the combined IPCC reports, including one caused by an inaccurate submission of data by the government of the Netherlands. The latest peer-reviewed science suggests the IPCC also erred at least two more times in underestimating the speed and scale of melting glaciers and sea ice.

Once such reforms are in place, such as a clearly defined conflict of interest policy, the report's authors also suggest further opening the IPCC to other viewpoints—ranging from providing travel grants for developing world experts to participate in the IPCC as well as opening the doors to private companies. "Their research and support of the process could significantly expand the available knowledge base concerning adaptation and mitigation options," the report notes.

That is a conclusion backed by the current IPCC chair Pachauri, who recently was criticized for potential conflicts of interest deriving from consulting work undertaken for private companies, although a formal apology was published by the U.K.'s Sunday Telegraph, which first revealed such criticisms in an article it later retracted. "Why should we not provide business advice?" Pachauri asked at the press conference. "They have to be part of the change if there is a change. I functioned purely in an advisory capacity."

The IPCC report was "a success and served society well"
The report's authors also went beyond their specific review of the IPCC processes and procedures to suggest further reforms that might strengthen future outcomes, including setting up a Web-based system for including and synthesizing the latest research or further staggering the release of the various working group's reports to allow those on impacts and mitigation to make full use of the report from the working group on the physical science of climate change.

"Errors did dent the credibility of the process. Trust is something you have to earn every year," Shapiro said. "We think what we recommend will help restore some of this trust."

But, overall, the "IPCC's assessment process has been a success and served society well," Shapiro noted. "The assessments have put IPCC on the world stage, raised public awareness of climate change, and driven policymakers to consider options for responding to climate change." That's a conclusion backed by entities ranging from the Dutch and U.S. environmental agencies (pdf) to independent academic researchers, who have all completed reviews of the IPCC's scientific claims in the past year.

The increasing scrutiny—and expanding workload—does not seem to have diminished scientists' and other experts' interest in participating in the next IPCC report, due in 2013. "The number of nominations to work on [the fifth IPCC assessment] increased almost 50 percent to 3,000," Pachauri noted. "From those we selected 831 experts compared with 559 in 2004 for [the fourth IPCC assessment]. What a tremendous show of support."

And, in the end the IPCC's errors, in this case, were errors of degree rather than fundamentals, although the IPCC retains the goal of "eliminating every questionable statement or error that it's humanly possible to prevent," said biologist Chris Field of the Carnegie Institution for Science's Department of Global Ecology at Stanford University and co-chair of the working group on impacts for the next IPCC assessment. Himalayan glaciers may not disappear by 2035 but the U.S. Geological Survey 2010 report on glaciers throughout Asia found that most of them are retreating as a result of climate change.

"The IPCC sits at the intersection of science and policy and, in many ways, it represents a significant social innovation," Shapiro said. "Most of our key recommendations are aimed at helping IPCC manage this increasingly complex process and doing so under the gaze of a public microscope."
 
As for what happens now that the report is public, Pachauri said that "it is for the IPCC and all the governments to decide when they want to implement the recommendations and which ones they want to implement." The IPCC will next meet from October 11 to October 14 in Busan, South Korea.



Rights & Permissions

24 Comments

Add Comment
View
  1. 1. Global Yawn 08:42 AM 8/31/10

    So where is the problem? People not accepting change wherever it comes from. The answer is contingency planning. An Atlantic Tsunami will knock out the Atlantic conveyor and we get an ice-age.
    Yellowstone Park erupts, so?
    San Francisco / LA destroyed by earthquake.
    Hurricane hits eastern sea-board of US. So?
    A perfect storm of all these factors at the same time that a meteor hits the planet and so........?
    Who's worried about global warming? All of those who have made a fortune out of green energy.
    Global W.......YAWN !

    Reply | Report Abuse | Link to this
  2. 2. gunslingor 10:41 AM 8/31/10

    Global Yawn,

    Ironic you claim environmentalists aren't willing to accept change, while it's you sir who are the luddite. GW is a real danger, so are asteroids, nuclear war, etc.. The point is, all we have to do to mitigate the dangers from GW is to stop burning dead fermented million year old bio matter. If modern fossil equivalents weren't far superior to traditional fossil, your agruement of why not to change MAY be slightly more valid, but modern equivalents are far superior if nothing more than to drastically reduce cancer rates and provide cheaper fuel. Mitigation of any chance of climate change, in your mind, should just be considered an added bonus.

    So stop crying over your antiquated fossil burning technology and get with the freaking program, better technology is ready for implementation now.

    Reply | Report Abuse | Link to this
  3. 3. John_Toradze 08:27 PM 8/31/10

    Nuclear power. Had we gone nuclear in the 1960's, we would not be having this problem now.

    Reply | Report Abuse | Link to this
  4. 4. MarkLovendale 09:39 PM 8/31/10

    The WSJ article today (2010-08-31) on its front page stated there was call for 'Fundamental Reform' of the IPCC. This statement is not accurate. And the "few errors" did not invalidate IPCC's 2007 conclusions.

    In science there are usually some errors. Einstein made a few errors, but few people who understand the laws of physics doubt E=MC2.

    When the facts are intentionally misrepresented to support a belief, the belief is usually fatally flawed.

    Human-caused climate change is a very real and a very serious problem that all caring people need to help solve.

    If Joseph Gobbles, the minister of propaganda for Hitler, were still alive he would be smiling his approval at the words used in the WSJ article.

    Mark Lovendale

    From the WSJ:
    Investigation Calls for 'Fundamental Reform' at U.N. Group on Global Warming
    By JEFFREY BALL
    AUGUST 31, 2010
    http://online.wsj.com/article/SB10001424052748703369704575461452636059886.html?mod=ITP_pageone_0#printMode

    "An independent investigation called for "fundamental reform" at the United Nations' Intergovernmental Panel on Climate Change, saying the organization's 2007 report played down uncertainty about some aspects of global warming.

    "The probe of the IPCC, a preeminent climate-science body that won the Nobel Peace Price three years ago, was conducted by the InterAcademy Council, a consortium of national scientific academies. Leaders of the IPCC asked the council to conduct the probe following the disclosure of a few errors in its 2007 climate-science report, which concluded, among other things, that climate change is "unequivocal" and is "very likely" caused by human activity."

    Reply | Report Abuse | Link to this
  5. 5. rjones137 09:40 PM 8/31/10

    There are several glaring errors in the report. The one that has gotten the most press was about the Himalayan Glaciers. India was quite upset over that snafu. There are others. Those who do not agree with the politics of "Global Warming" are finding lots of ammunition in it. The UN would be better served to have the report peer reviewed by some without a vested interest in the results.

    That would mean, of course that the report might not support the policies of the UN organization that funded the report, so I don't expect that to happen. So far, I haven't been surprised. The review groups were carefully vetted to exclude anyone with a possibly different view. If they want to allay suspicion from those who doubt the claims, they are going about it wrong. Openness and honesty with both methods and data would be much better. I don't see that from either side.

    Reply | Report Abuse | Link to this
  6. 6. Ultracrepidarian 04:20 AM 9/1/10

    So the glaring error was that one statement said the glaciers would be gone in 25 years, and the other stated they would be 80% gone in 25 years. Wow! That really smacks of conspiracy.

    Reply | Report Abuse | Link to this
  7. 7. Izegrim in reply to Ultracrepidarian 07:32 AM 9/1/10

    Scientists make mistakes & that's ok with me; they're human after all. However, when each and every one of those mistakes favour their beloved theory they resemble the barkeeper who, when making up the bill, sometimes makes mistakes — but always in his own advantage.

    Reply | Report Abuse | Link to this
  8. 8. vendicar9 in reply to rjones137 12:37 PM 9/1/10

    "There are several glaring errors in the report." - Little Sambo

    Correction: There were a couple of errors found in the summary reports.

    Woo Hoo.

    Reply | Report Abuse | Link to this
  9. 9. dling 06:37 PM 9/2/10

    I agree nuclear is part of a temporary solution, but had we gone nuclear in the 60's, we'd probably be out of fuel by now. Estimates I've read (I'm a geologist) are that we have 30-50 years worth of useable fuel if we're at maximum nuclear build-out. People forget it's a very finite resource.

    "The UN would be better served to have the report peer reviewed by some without a vested interest in the results." If you think the National Academies of Science have a "vested interest," you are totally, utterly, completely anti-science. That begs the question: if in your view the world's most prestigious, reputable and arguably important scientific body is now just a "vested interest," then what IS a valid source of science? You are aware that in this country, we used to be proud of what science has achieved to better humanity and to not just strengthen, but largely create our economic growth. Now politics will destroy all of it apparently. Make no mistake, strong, uninformed opinions from the scientifically ignorant - that they know more about how to do science than our best scientists - is very destructive to America.

    "The review groups were carefully vetted to exclude anyone with a possibly different view." The review groups were the national academies of the various developed nations - the most important, reliable, and presitigious science bodies in the world. I don't know where you're getting your information (probably Rush Limbaugh or Fox News) but it shows extreme lack of knowledge of the real world of science.

    "However, when each and every one of those mistakes favour their beloved theory they resemble the barkeeper who, when making up the bill, sometimes makes mistakes  but always in his own advantage." Did you even read the report?

    "The latest peer-reviewed science suggests the IPCC also erred at least two more times in underestimating [UNDERESTIMATING - sic] the speed and scale of melting glaciers and sea ice."

    "The above is exactly the problem with the warm mongers. "So what if we are wrong? We are still driving policy the right way and that's the main thing." NO IT ISN'T. This is science, not religion." Actually, the IPCC doesn't do research. Their job is to do metastudies of existing studies, in order to report out to governments what is happening. Raising public awareness and informing policymakers IS EXACTLY THE JOB OF THE IPCC!

    So, once again, the people who are against science show their ignorance of it.

    Dan, geologist

    Reply | Report Abuse | Link to this
  10. 10. ssm1959 06:56 PM 9/2/10

    The question regarding motivation will always be present as long as the IPPC remains a political body. As a political agency, opinion holds too much sway and often trumps good research if it does not fall in line with the crisis du jour. At same time it is too willing to blindly accept supporting data that lacks rigorous validation. The result is alienation of the non-scientific public which will ultimately damage the ability implement changes need to mitigate human contributions to climate change. Ultimately trust is the only route to credibility the science community has with public at large. If we compromise that trust it will not matter if we are right or wrong.

    Reply | Report Abuse | Link to this
  11. 11. Quinn the Eskimo 09:58 PM 9/2/10

    the IPCC is a political creation. From a scientific position--it cannot be saved.

    Like UNICEF -- its sole goal is to de-fang the evil U.S.

    The UN wanted algore to be prez for life. They lost.

    Now, it's trash day.

    Reply | Report Abuse | Link to this
  12. 12. Dr. Strangelove 10:26 PM 9/2/10

    "The IPCC report was a success and served society well."
    "... in the end the IPCC's errors... were errors of degree rather than fundamentals,"

    Wrong. Climate change is all about degree. If doubling atmospheric CO2 would increase global temp. by 0.3C, is that catastrophic? But its a different story if it's 2C or 4C. The problem is the uncertainty, hence the error of IPCC, is not only in degree but also in fundamentals. The Assessment Report conveniently forgets that some climate models are showing cooling rather than warming. The uncertainty is not just how much warming but is it warming or cooling?

    The climate uncertainty is due to the fact that the most significant greenhouse gas is not CO2 but water vapor, whose concentration is 30 times more than CO2. The effect of clouds is quite unpredictable because they can be positive or negative feedback. They can enhance the greenhouse effect and they can also increase albedo since they reflect incoming solar radiation.

    Climate models cannot even predict the direction of change in temp. (warming or cooling) unless the model is artificially adjusted  tweaking the model to show the result that you want (warming). This is the fundamental issue that IPCC fails to mention and the general public does not know.

    Reply | Report Abuse | Link to this
  13. 13. dling 07:42 AM 9/3/10

    Well said ssm1959 . But the IPCC is not entirely political. Doesn't anyone remember when all the scientists walked out after the politicians watered down the conclusions? We need to make the IPCC more of a science body, with members nominated by their peers and the academies, and weed out the policial appointments. To say it cannot be saved is to say there is no hope, which is a cop out. It must be saved. There is no scientific body whose job is to do met-analysis of existing studies and report out to decision-makers, except the IPCC. We need that role to be performed. But it can, and must, be improved.

    The truth is, the baseline data that's been coming in this decade shows that the IPCCs assessments were wrong in that they were too conservative (as the scientists had earlier complained). We now know the Earth is warming faster, and the ice is melting faster, and the sea is acidifying faster than we had predicted, in line with the IPCC's worst-case scenario.

    It appears to me that Dr. Strangelove is informed (he's correct about water vapor and clouds), but chooses to take a political position which differs from the concensus of climatologists. Why would he do that unless he were himself a climatologist (in the 2% minority), or were anti-science? I don't think you can bash climate change any more than evolution without taking the position that you are better informed and better qualified and more experienced than the best scientists. Dr. Strangelove, have you looked at the hundreds and hundreds of studies yourself in order to assess the "degree" to which the entire climatology community is wrong? If not, what is the scientific source of your opinion? Have you read this in a climate journal?

    Reply | Report Abuse | Link to this
  14. 14. Dr. Strangelove in reply to dling 03:18 AM 9/5/10

    @dling
    I suggest you read this article by Dr. Patrick Frank.

    http://www.skeptic.com/the_magazine/featured_articles/v14n01resources/climate_of_belief.pdf

    I sent a manuscript to Journal of Geophysical Research for publication. The editor rejected it. By the way, the editor is a climate scientist of IPCC and my paper criticized the climate model that he developed.

    Reply | Report Abuse | Link to this
  15. 15. vendicar9 in reply to John_Toradze 07:59 AM 9/7/10

    "Had we gone nuclear in the 1960's, we would not be having this problem now." - Insect

    Iran is saying the same thing.

    Reply | Report Abuse | Link to this
  16. 16. vendicar9 in reply to MarkLovendale 08:08 AM 9/7/10

    "There are some publications that are pursuing their own objectives and distorting the message. We made some comments to the effect that [the IPCC] ought to be more careful in using non-peer-reviewed literature, and one headline came out in Europe saying we said: “the data is terrible.” Of course, we said nothing of the kind. " - Harold Shapiro IPCC Review

    Reply | Report Abuse | Link to this
  17. 17. vendicar9 08:11 AM 9/7/10

    "Oh yeah, and the report is chock full of errors and lies" - Texas Fatman

    You must be referring to a Faux news report.

    Reply | Report Abuse | Link to this
  18. 18. vendicar9 08:15 AM 9/7/10

    "Assessment Report conveniently forgets that some climate models are showing cooling rather than warming." - Conservative Liar

    No climate models show cooling. Since no climate models violate the fundamental laws of physics.

    Reply | Report Abuse | Link to this
  19. 19. Dr. Strangelove in reply to vendicar9 08:59 PM 9/7/10

    "No climate models show cooling." Read Dr. Patrick Frank

    http://www.skeptic.com/the_magazine/featured_articles/v14n01resources/climate_of_belief.pdf

    "Since no climate models violate the fundamental laws of physics." Read Dr. Gerhard Gerlich and Dr. Ralf Tscheuschner.

    http://arxiv.org/PS_cache/arxiv/pdf/0707/0707.1161v4.pdf

    Reply | Report Abuse | Link to this
  20. 20. supertexan 03:17 AM 9/12/10

    I love it when I see somebody has replied to me on these things when my original comment was deleted, as usual. Would whichever one of you warm mongers that does that at least have the decency to just say so? A simple 'this comment deleted because of x" would suffice. You wont though because x in this case is that you are a weak fake science magazine pushing a weak political agenda and you want to keep these "denier" posts down to minimum lest folks get the idea that there is still a debate to be had on global warming.

    Reply | Report Abuse | Link to this
  21. 21. rtaylortitle 05:11 PM 10/31/10

    The IPCC has lost all of its credibility (if it ever had the quality to begin with). There is still not ONE scintilla of evidence that mankind 'contributes' to earth's natural weather cycles.

    Reply | Report Abuse | Link to this
  22. 22. rtaylortitle in reply to ssm1959 11:54 PM 10/31/10

    Bravo, excellent summation.

    Reply | Report Abuse | Link to this
  23. 23. rtaylortitle in reply to gunslingor 12:03 AM 11/1/10

    No Luddite here, sir, but I am an adult who is keenly aware of political power plays adorned with well-sounding altruisms. I still see no "hard" evidence as to mankind's supposed contribution to a natural cycle of weather and solar radiations. It all sounds to "convenient" to those who would want to aggrandize their personal wealth as Gore is doing. And, as to the science behind it, computer models, it can easily be manipulated as we have seen from the emails we are all familiar with. Remember the Club of Rome back in the '60's predicting an upcoming ice age? Disaster predictions are becoming stale and obvious to the rest of us.

    Reply | Report Abuse | Link to this
  24. 24. rtaylortitle in reply to MarkLovendale 12:05 AM 11/1/10

    And what are your "real facts" about humanity's contribution to natural global warming? Please be very, very, very specific.

    Reply | Report Abuse | Link to this
Leave this field empty

Add a Comment

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Click one of the buttons below to register using an existing Social Account.

More from Scientific American

See what we're tweeting about

Scientific American Editors

Tweets could not be retrieved at this time

Free Newsletters


Get the best from Scientific American in your inbox

Solve Innovation Challenges

Powered By: Innocentive

  SA Digital
  SA Digital

Email this Article

Shades of "Gray Literature": How Much IPCC Reform Is Needed?

X
Scientific American MIND iPad

Tap into your MIND

Get Both Print & Tablet Editions for one low price!

Subscribe Now >>

X

Please Log In

Forgot: Password

X

Account Linking

Welcome, . Do you have an existing ScientificAmerican.com account?

Yes, please link my existing account with for quick, secure access.



Forgot Password?

No, I would like to create a new account with my profile information.

Create Account
X

Report Abuse

Are you sure?

X

Institutional Access

It has been identified that the institution you are trying to access this article from has institutional site license access to Scientific American on nature.com. To access this article in its entirety through site license access, click below.

Site license access
X

Error

X

Share this Article

X