date: Tue, 28 Oct 2003 08:45:38 -0500
from: "Michael E. Mann" <mann@virginia.edu>
subject: Fwd: Re: Fw: New Study Questions Kyoto Global Warming Data
to: asocci@cox.net, Gabi Hegerl <hegerl@duke.edu>, Jonathan Overpeck <jto@u.arizona.edu>, Keith Briffa <k.briffa@uea.ac.uk>, Malcolm Hughes <mhughes@ltrr.arizona.edu>, Mike MacCracken <mmaccrac@comcast.net>, Phil Jones <p.jones@uea.ac.uk>, Ray Bradley <rbradley@geo.umass.edu>, Ben Santer <santer1@llnl.gov>, Steve Schneider <shs@stanford.edu>, "Lonnie G. Thompson" <thompson.3@osu.edu>, Ellen Mosley-Thompson <thompson.4@osu.edu>, Tim_Profeta@lieberman.senate.gov, tom crowley <tom@ocean.tamu.edu>, Kevin Trenberth <trenbert@cgd.ucar.edu>, Tim Osborn <t.osborn@uea.ac.uk>, Tom Wigley <wigley@meeker.UCAR.EDU>, apetsonk@environmentaldefense.org, "Jim Salinger" <j.salinger@niwa.co.nz>

   Dear All,
   Hopefully, the last time. After considerable review of what the authors have done, and some
   feedback from others, I've revised and streamlined my "response". The authors convolution
   of incommensurate scaling factors with the posted Mann et al eigenvectors, resulting from
   their use of a different instrumental temperature data set and inconsistent normalization
   convention (standard deviation of detrended series was used in Mann et al while
   non-detrended gridpoint standard deviations have been used by the authors to unnormalize
   the Mann et al eigenvectors--this leads to a spatially-variable biased enhancement of
   variance) appears likely to be the culprit.
   Its clearly an error. There may be several others yet uncovered, but this alone certainly
   invalidates what they have done.
   The attached, hopefully final version of my "response" should suffice.
   Thanks,
   mike

     Date: Mon, 27 Oct 2003 15:50:38 -0500
     To: mann@virginia.edu
     From: "Michael E. Mann" <mann@virginia.edu>
     Subject: Re: Fw: New Study Questions Kyoto Global Warming Data
     Cc: asocci@cox.net, Gabi Hegerl <hegerl@duke.edu>, Jonathan Overpeck
     <jto@u.arizona.edu>, Keith Briffa <k.briffa@uea.ac.uk>, Malcolm Hughes
     <mhughes@ltrr.arizona.edu>, Mike MacCracken <mmaccrac@comcast.net>, Phil Jones
     <p.jones@uea.ac.uk>, Ray Bradley <rbradley@geo.umass.edu>, Ben Santer
     <santer1@llnl.gov>, Steve Schneider <shs@stanford.edu>, "Lonnie G. Thompson"
     <thompson.3@osu.edu>, Ellen Mosley-Thompson <thompson.4@osu.edu>,
     Tim_Profeta@lieberman.senate.gov, tom crowley <tom@ocean.tamu.edu>, Kevin Trenberth
     <trenbert@cgd.ucar.edu>, Tim Osborn <t.osborn@uea.ac.uk>, Tom Wigley
     <wigley@meeker.UCAR.EDU>, apetsonk@environmentaldefense.org, "Jim Salinger"
     <j.salinger@niwa.co.nz>
     Dear All,
     One small correction upon further reading--they did appear to use subsets of
     eigenvectors corresponding to those nominally used by Mann et al (1998) upon further
     reading of their description.
     However, the eigenvectors aren't the same, because the surface temperature dataset is a
     different one from that used by Mann et al! So the eigenvectors #s that they used are
     not the same as the corresponding eigenvectors used by Mann et al (1998). Again, because
     they didn't apply the objective criterion themselves (they appear to have simply used
     the same nominal eigenvectors, but of a different(!) temperature dataset), they didn't
     appear to discover the problem...
     mike
     At 02:38 PM 10/27/2003 -0500, Michael E. Mann wrote:

     Dear All,
     The link below has been passed along to me by a colleague (whose name has been expunged
     to protect his identity):
     Based on the description of data and method provided, which the authors claim is a
     complete (!) description [this is important], I  have confidently concluded that they
     have neither used the same data (instrumental or proxy) but, more problematically, the
     same method as Mann et al (1998). In fact, there is a huge problem in the statistical
     calibration procedure they used, which omits an essential step used by Mann et al (1998)
     to protect against statistical overfitting in the presence of an increasingly sparse
     proxy data network back in time.
     1) They used the wrong instrumental data set.
     Mann et al (1998) used the older version of the CRU instrumental surface temperature
     data set that dates back to 1854; they apparently used the newer version of the data set
     that goes back to 1856 which Phil published and updated to in the mid 90s--the data sets
     are actually significantly different in places. They differ significantly, for example,
     in where they have missing data. Phil can provide people details. Thus, the eigenvectors
     are assured as being different, and the missing data are different. It would have been
     surprising if they had found otherwise!
     2) They used the wrong proxy data.
     The authors apparently used an excel spreadsheet version of the MBH98 data that my
     associate Scott Rutherford had sent them. It appears that the data got shifted and
     scrambled a bit in the process of being converted to an excel spreadsheet or upon being
     downloaded or opened. This would explain the numerous transcription errors the authors
     find in the file. Of course, we used the uncorrupted data in our study.  These ascii
     versions of the data have always been publicly available on our computer "holocene". Had
     the authors used the (correct) ascii series on the data set, they wouldn't have
     encountered transcription errors. From what I But they appear to have had a corrupted
     version of the data. The authors then describe an elaborate effort to download suitable
     approximate versions of the proxy data series they couldn't get ahold of. In many cases,
     these appear to be substantially different versions of the proxy series than the ones
     used by Mann et al (1998) But again, I don't think this matters.
     3) They did not implement the Mann et al approach!!
     From the description of the method provided, it appears that the authors skipped the
     essential step of (1) applying an objective criterion (i.e., Presidendorffer's Rule N as
     used in Mann et al, '98) to determine the optimal size N of the subset of the full (16)
     candidate instrumental principal component series to retain in the calibration of the
     proxy data and (2) optimizing the calibration resolved variance with respect to all
     subsets of the leading PC series of size N. These crucial aspects of the procedure were
     clearly layed out in Mann et al (1998), and is perhaps one of the most essential
     steps---it is only the application of this objective criterion that prevents an obvious
     statistical overfitting problem--the authors *always* appear to use a subset of all 16
     PC series!  However, the criterion used by Mann et al (1998) dictated the retention of
     a maximum of 11 PC series, only a few PC series prior to AD 1600, and only one prior to
     AD 1450. So the authors appear to have tried to fit 16 PC series to the reconstruction
     from AD 1400-AD 1450, when an objective criterion would only dictate 1!
     This is a really basic statistical error, and its likely this massive overfitting that
     is responsible for the wild behavior in their reconstruction prior to about AD 1600.
     Can't beleive they made such a basic error? See for yourself:
     [1]http://www.climate2003.com/computations.html
     Any statistical climatologists worth their salt would have picked this up. But I believe
     the paper wasn't even reviewed!
     4) Their result is, not surprisingly, wacky
     The resulting reconstruction they show, with an enormous warm anomaly during the 15th
     and 16th centuries, looks like nothing ever produced in any objective estimate of past
     Northern Hemisphere temperature trends that I'm familiar with. I guess they could term
     this the "Renaissance Warm Period", but of course no other model or empirical Northern
     Hemisphere temperature reconstruction looks anything like this. Frankly, the ridiculous
     result should have let them know they did something wrong.

     Mike,
     This seems to be the HTML version of McIntyre's paper. (I got this
     unsolicited from Gene Avrett, Soon & Baliunas's boss at Harvard.)

     ---------------------- Original Message -----------------------
      From:    "Eugene H. Avrett"
      To:
      Date:    Mon, 27 Oct 2003 12:09:22 -0500
      Subject: article in Energy and Environment
     ----
     Dear        ,
     You may be interested in the article by McIntyre and McKitrick
     just published in Energy and Environment which questions the
     validity of the Mann et al. (1998) study that provided the basis
     for the claim that 20th century warming is unprecedented.
     See [2]http://www.uoguelph.ca/~rmckitri/research/trc.html.
     Yours sincerely,
     Gene Avrett

     ______________________________________________________________
                         Professor Michael E. Mann
                Department of Environmental Sciences, Clark Hall
                           University of Virginia
                          Charlottesville, VA 22903
     _______________________________________________________________________
     e-mail: mann@virginia.edu   Phone: (434) 924-7770   FAX: (434) 982-2137
              [3]http://www.evsc.virginia.edu/faculty/people/mann.shtml

     ______________________________________________________________
                         Professor Michael E. Mann
                Department of Environmental Sciences, Clark Hall
                           University of Virginia
                          Charlottesville, VA 22903
     _______________________________________________________________________
     e-mail: mann@virginia.edu   Phone: (434) 924-7770   FAX: (434) 982-2137
              [4]http://www.evsc.virginia.edu/faculty/people/mann.shtml

   ______________________________________________________________
                       Professor Michael E. Mann
              Department of Environmental Sciences, Clark Hall
                         University of Virginia
                        Charlottesville, VA 22903
   _______________________________________________________________________
   e-mail: mann@virginia.edu   Phone: (434) 924-7770   FAX: (434) 982-2137
            [5]http://www.evsc.virginia.edu/faculty/people/mann.shtml
   Attachment Converted: "c:\eudora\attach\EandEResponse-final.doc"

