President Barack Obama and Chinese President Xi Jinping’s deal [1] to dramatically reduce greenhouse gas emissions in both countries couldn’t have come at a more ironic time.
Their Nov. 11, announcement came just days before the fifth anniversary of ClimateGate, which cast a pall over the credibility of major climate research institutions. On Nov. 17, 2009, it was discovered that hundreds of emails and files were either hacked or leaked from the Climatic Research Unit (CRU) of the University of East Anglia (UEA). According to Roger Pielke, Jr., months before ClimateGate, CRU admitted it did not have the raw data its climate science is based on.
“We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.,” CRU said on its website, according to Pielke [2] [Emphasis his]. Pielke’s response was, “Say what?! CRU has lost track of the original data that it uses to create its global temperature record!? Can this be serious?”
The ClimateGate files shed further light into the problems with CRU data, but media coverage of the scandal didn’t. The three broadcast networks ignored the breaking news for a full 13 days in November 2009. Since then, the networks have mentioned it just nine stories (the most recent was in May 2010) and never mentioned the Harry Read Me file, according to Nexis searches. Later the networks [3] sought to “exonerate” accused scientists.
One of the most disturbing files was a more than 200-page document called HARRY_READ_ME.txt [4]. Darrel Ince, professor of computing at the Open University, wrote in The Guardian [5] (UK) that it is imperative that scientists relying on computer programs release their programs given that “a slip of a keyboard could create an error in programs that will be used to make financial decisions which involve billions of pounds and, moreover, that the probability of such errors is quite high.”
PJ Media [6]and others said it appears to have been written by Ian “Harry” Harris who worked tirelessly to fix CRU’s convoluted data. While working on the project, he catalogued his many challenges between 2006 and 2009 in the file [4], the most extensive single document released.
His frustrations did not inspire confidence in CRUs data given his many exclamations like, “This whole project is SUCH A MESS. No wonder I needed therapy!!” “Harry” complains that every time he seemed to make progress “it ends worse than before.” At another point, he exclaims, “Just another thing I cannot understand, and another reason why this should all have been rewritten from scratch a year ago!”
The HARRY_READ_ME file was a long-running commentary, describing the disconcerting breath of problems he encountered while attempting to edit and organize his climate data sets at CRU.
These challenges included CRU loss of data, predecessors accidentally deleting important calculations, receiving poorly organized data from various parts of the world, and having to work with a glitchy array of customized computer programs.
In circumstances such as these, the writer was forced to make many judgment calls on how to proceed, despite admitting, “I haven't had any training in stats in my entire life.”
Despite these revelations, CRU’s climate models have been defended by the U.S. government’s Environmental Protection Agency (EPA) [7] and the United Nation’s Intergovernmental Panel on Climate Change (IPCC) [8].
BBC reported [9] on Nov. 20, 2009, that researchers at CRU “played a key role” in IPCC findings. The IPCC’s conclusions on climate change in turn heavily influenced the EPA’s ruling that it has the authority to regulate carbon dioxide in order to prevent global warming, according [10] to NewsMax on Dec. 9, 2009.
Yet to critics, the file demonstrated the muddled state of CRU’s climate data and illustrated the vast complexity required to generate climate models. Christopher Booker, a columnist with The Telegraph (UK), wrote, “What is tragically evident from the Harry Read Me file is the picture it gives of the CRU scientists hopelessly at sea with the complex computer programmes [sic] they had devised” to model purported global warming.
In February 2010, the minority staff of the U.S. Senate Committee on Environment and Public Works released a report [11] on the ClimateGate scandal. Sen. James Inhofe, R-Okla., was the committee’s ranking minority member at the time. Minority staff said in the report that the Harry Read Me File “raises several serious questions as to the reliability and integrity of CRU‘s data compilation and quality assurance protocols.”
That report also said, “Moreover, we believe the emails and accompanying documents seriously compromise the IPCC-based consensus and its central conclusion that anthropogenic emissions are inexorably leading to environmental catastrophes.”
Here are just the are 10 most stunning quotes from the HARRY_READ_ME file. WARNING: EXPLICIT LANGUAGE:
“OH FUCK THIS. It's Sunday evening, I've worked all weekend, and just when I thought it was done I'm hitting yet another problem that's based on the hopeless state of our databases. There is no uniform data integrity, it's just a catalogue of issues that continues to grow as they're found.”
“So, once again I don't understand statistics. Quel surprise, given that I haven't had any training in stats in my entire life, unless you count A-level maths.”
“I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more.”
“Bear in mind that there is no working synthetic method for cloud, because Mark New lost the coefficients file and never found it again (despite searching on tape archives at UEA) and never recreated it.”
“So.. should I really go to town (again) and allow the Master database to be ‘fixed’ by this program? Quite honestly I don't have time - but it just shows the state our data holdings have drifted into. Who added those two series together? When? Why? Untraceable, except anecdotally. It's the same story for many other Russian stations, unfortunately - meaning that (probably) there was a full Russian update that did no data integrity checking at all. I just hope it's restricted to Russia!!”
“Had a hunt and found an identically-named temperature database file which did include normals lines at the start of every station. How handy – naming two different files with exactly the same name and relying on their location to differentiate! Aaarrgghh!!”
"Here, the expected 1990-2003 period is MISSING - so the correlations aren't so hot! Yet the WMO codes and station names /locations are identical (or close). What the hell is supposed to happen here? Oh yeah - there is no 'supposed', I can make it up. So I have :-)"
“Now looking at the dates.. something bad has happened, hasn't it. COBAR AIRPORT AWS cannot star[t] in 1962, it didn't open until 1993!”
"So with a somewhat cynical shrug, I added the nuclear option - to match every WMO possible, and turn the rest into new stations (er, CLIMAT excepted). In other words, what CRU usually do. It will allow bad databases to pass unnoticed, and good databases to become bad, but I really don't think people care enough to fix 'em, and it's the main reason the project is nearly a year late."
- “Because although I'm thrilled at the high match rate (87%!), it does seem worse when you realise that you lost the rest..”