In a self-publicized
Environmental Research Letters paper, Cook
et al claim to have performed a climate consensus-hunting literature survey they call the most comprehensive of its kind using the largest sample. The comprehensive nature of the survey is important to the author and his group. For instance, close associate Lewandowsky
writes:
There has been evidence in the peer-reviewed literature already that more than 95 out of 100 climate scientists agree on the basic premise that human greenhouse gas emissions are warming the planet. [...]
But until now, tools for the visualization of that evidence have been limited.
This is where the new study by Cook et alplays such a particularly important role: Going beyond previous surveys of climate scientists, Cook et al. performed a systematic review of the massive literature on climate change.
In a nutshell, they used a scientific search engine (ISI Web of Knowledge) to gather allpapers published on global climate change or global warming between 1991 and 2011. This search returned a mind-boggling 12,000 papers
Several folks have looked at the Cook paper, assuming its basic soundness. But does the paper do what it says it does?
The climate change literature comprises well over hundred thousand articles and books. Cook
et als strategy was to focus on papers directly related to global warming or global climate change in
Web of Science. Heres how they describe it:
In March 2012, we searched the ISI Web of Science for papers published from 19912011 using topic searches for global warming or global climate change. Article type was restricted to article, excluding books, discussions, proceedings papers and other document types.
A Web of Science search performed following the authors description to the letter actually returns 30,940 entries, not 12,464. Excluding the Arts and Humanities Citation Index (A&HCI), this becomes 30,876. This is when search phrases are not enclosed in double-quotes (i.e., global warming instead of global warming).
Scopus is an academic database covering technical, medical, and social science disciplines. Surprisingly, when Scopus is searched using the correct search phrases, a total of 19,417 entries are retrieved. A
Web of Knowledge search returns ~21,488 records. These figures are 7473 records (Scopus) and ~9544 records (Web of Knowledge) greater than what Cook
et al eventually analysed.
These results make plain that a large body of relevant literature has been excluded by the authors in their study.
Conclusion
The Cook
et al numbers are somewhat replicable, only if search is limited to the Science Citation Index and Social Sciences Citation Index databases in Web of Science. Presumably, this made the job of classification easier. Contrary to claims however, this makes their literature search incomplete. It is neither comprehensive nor produces the largest possible data set. The finding of incomplete search has further implications as it affects all conclusions drawn in the paper.
