cc: "Tim Osborn" <t.osborn@uea.ac.uk>, "Ross McKitrick" <rmckitri@uoguelph.ca>
date: Wed, 12 Nov 2003 00:36:23 -0500
from: "Michael E. Mann" <mann@virginia.edu>
subject: Re: MBH98
to: "Steve McIntyre" <smcintyre@cgxenergy.com>

   Dear Mr. McIntyre,
   There seems to be some confusion on your part regarding the public posting of the MBH98
   data.
   All of the data used by MBH98 have been available in plain ASCII format on this public ftp
   site
   [1]ftp://holocene.evsc.virginia.edu/pub/MBH98/
   They have been available in the various clearly indicated sub-directories, back through at
   least summer 2002 according to the dates on those directories. This includes all 159
   predictors used by MBH98 back to AD 1400 *and* all of the proxy data that go into those
   indicators.
   When I sent you the below email message on 4/9/2003, it was my expectation that you would
   go to that ftp site to get the individual data series in question.  I was not party to the
   various emails you and Scott Rutherford exchanged regarding alternative versions of the
   dataset  that he prepared, though I am told he offered you all of the proxy data, and you
   instead preferred a dataset of 112 proxy indicators (that is the number of indicators
   available  back to 1820).
   With regard to the latest changes made by Scott on the ftp site, I believe this was to
   replace the incorrect spreadsheet version of the data that had been posted previously with
   a corrected version, so that people do not continue to download an incorrect version of the
   data set.
   To reiterate once last time, the original data that you requested before and now request
   again are all on the indicated ftp site, in the indicated directories, and have been there
   since at least 2002. I therefore trust you should have no problem acquiring the data you
   now seek.
   Mike Mann
   >Dear Mr. McIntyre,
   >
   >These data are available on an anonymous ftp site we have set up. I've forgotten the exact
   >location, but I've asked my Colleague Dr. Scott Rutherford if he can provide you with that
   >information.
   >
   >best regards,
   >
   >Mike Mann
   At 01:47 PM 4/8/2003 -0400, Steve McIntyre wrote:
   Dear Dr. Mann,

   I have been studying MBH98 and 99. I located datasets for the 13 series used in 99 at
   [2]ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/Millennium/DATA/PROXIES/ (the
   convenience of the ftp: location being excellent) and was intereseted in locating similar
   information on the 112 proxies referred to in MBH98, as well as listing (the listing at
   [3]http://www.ngdc.noaa.gov/paleo/ei/data_supp.html is for 390 datasets, and I
   gather/presume that many of these listed datasets have been condensed into PCs, as
   mentioned in the paper itself. Thank you for your attention.

   Yours truly,

   Stephen McIntyre,
   Toronto, Canada
    At 11:39 PM 11/11/2003 -0500, Steve McIntyre wrote:

     <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
     November 11, 2003

     Professor Michael E. Mann
     School of Earth Sciences
     University of Virginia



     Dear Professor Mann,

     We apologize for not sending you a copy of our recent paper (MM) in Energy and
     Environment for comment, as we understood from your email of September 25, 2003 that
     time constraints prevented you from considering our material. We notice that you seem to
     have subsequently changed your mind and hope that you will both be able to clarify some
     points for us and to rectify the public record on other points.

     1) You have claimed that we used the wrong data and the wrong computational methodology.
     We would like to reconcile our results to actual data and methodology used in MBH98. We
     would therefore appreciate copies of the computer programs you actually used to read in
     data (the 159 data series referred to in your recent comments) and construct the
     temperature index shown in Nature (1998) (MBH98), either through email or, preferably
     through public FTP or web posting.

     2) In some recent comments, you are reported as stating that we requested an Excel file
     and that you instead directed us to an FTP site for the MBH98 data. You are also
     reported as saying that despite having pointed us to the FTP site, you and your
     colleague took trouble to prepare an Excel spreadsheet, but inadvertently introduced
     some collation errors at that time. In fact, as you no doubt recall, we did not request
     an Excel spreadsheet, but specifically asked for an FTP location, which you were unable
     or unwilling to provide. Nor was an Excel spreadsheet ever supplied to us; instead we
     were given a text file, pcproxy.txt. Nor was this file created in April 2003. After we
     learned on October 29, 2003 that the pertinent data was reported to be located on your
     FTP site [4]ftp://holocene.evsc.virginia.edu/pub (and that we were being faulted for not
     getting it from there), we examined this site and found it contains the exact same file
     (pcproxy.txt) as the one we received, bearing a date of creation of August 8, 2002. On
     October 29, 2003, your FTP site also contained the file pcproxy.mat, a Matlab file, the
     header to which read: MATLAB 5.0 MAT-file, Platform: SOL2, Created on: Thu Aug  8
     10:18:19 2002. Both files contain identical data to the file pcproxy.txt emailed to one
     of us (McIntyre) in April 2003, including all collation errors, fills and other problems
     identified in MM. It is therefore clear that the file pcproxy.txt as sent to us was not
     prepared in April 2003 in response to our requests, nor was it prepared as an Excel
     spreadsheet, but in fact it was prepared many months earlier with Matlab. It is also
     clear that, had we gone to your FTP site earlier, we would simply have found the same
     data collation as we received from Scott Rutherford. Would you please forthwith issue a
     statement withdrawing and correcting your earlier comments.

     3) In reported comments, you also claimed that we overlooked the collation errors in
     pcproxy.txt and slid the incorrect data into our calculations, a statement which is
     untrue and made without a reasonable basis. In MM, we described numerous errors
     including, but not limited to, the collation errors, indicating quite obviously that we
     noticed the data problems. We then describe how we firewalled our data from the errors
     contained in the data you provided us, by re-collating tree ring proxy data from
     original sources and carrying out fresh principal component calculations. We request
     that you forthwith withdraw the claim that we deliberately used data we knew to be in
     error.

     4) On November 8, 2003, when we re-visited your FTP site, we noticed the following
     changes since October 29, 2003: (1) the file pcproxy.mat had been deleted from your FTP
     site; (2) the file pcproxy.txt no longer was displayed under the /sdr directory, where
     it had previously been located, although it could still be retrieved through an exact
     call if one previously knew the exact file name; (3) without any notice, a new file
     named mbhfilled.mat prepared on November 4, 2003 had been inserted into the directory.
     Obviously, the files pcproxy.mat and pcproxy.txt are pertinent to the comments referred
     to above and we view the deletion of pcproxy.mat from the archival record under the
     current circumstances as unjustifiable. Would you please restore these files to your FTP
     site, together with an annotated text file documenting the dates of their deletion and
     restoration.

     5) We note that the new file mbhfilled.mat is an array of dimension 381x2016. Could you
     state whether this file has any connection to MBH98, and, if so, please explain the
     purpose of this file, why it has been posted now and why it was not previously available
     at the FTP site.

     6) Can you advise us whether the directory MBH98 has been a subdirectory within the
     folder pub since July 30, 2002 or whether it was transferred from another (possibly
     private) directory at a date after July 30, 2002? If the latter, could you advise on the
     date of such transfer.


     We have prepared a 3-part response to your reply to MM. The first, which we have
     released publicly, goes over some of the matters raised in points #2-#5 above. The
     second is undergoing review. It deals with additional issues of data quality and
     disclosure, resulting from inspection of your FTP site since October 29, 2003.  The
     third part will consider the points made in your response, both in terms of data and
     methodology, and will attempt a careful reconciliation of our calculation methods, hence
     the necessity of our request in point #1. Thank you for your attention.


     Yours truly,

     Stephen McIntyre                        Ross McKitrick


     cc: Timothy Osborn

   ______________________________________________________________
                       Professor Michael E. Mann
              Department of Environmental Sciences, Clark Hall
                         University of Virginia
                        Charlottesville, VA 22903
   _______________________________________________________________________
   e-mail: mann@virginia.edu   Phone: (434) 924-7770   FAX: (434) 982-2137
            [5]http://www.evsc.virginia.edu/faculty/people/mann.shtml

