date: Fri Mar 18 11:02:33 2005
from: Phil Jones <p.jones@uea.ac.uk>
subject: RE: HadCRUT2 problems - diagnosis and possible solution
to: "Folland, Chris" <chris.folland@metoffice.gov.uk>

    Chris,
[[[redacted: personal]]]
    Peter is and will be a very good
    scientist - he just needs to be able to write clear English in emails, and also
    to know a little of the history of Had/CRU work. David's email helped there.
    You'll both be a big loss to the HC when you go, but glad to hear it won't be
    until end of next year. This date will be fine for IPCC.
       As for IPCC, I'm sure we can get later submitted work in quite easily. I am
    towing the WGI line though for the time being, which is submit by May05. This is
    a ridiculous date, but I suspect Susan has it to allow a slippage to the end of 05.
    There will still be many relevant papers appearing in 2006, that we won't know about,
    that we should be referring to. Govt. Reviews should enable us to get these in.
    Hopefully DEFRA will have some HC people in on these. Did you get involved in
    these with the TAR - or did you stay totally separate?
       What is your home #?  Is 9am OK on Monday?
    Cheers
    Phil
   At 10:23 18/03/2005, you wrote:

     Phil
     Thanks. I have not seen any HadCRUT3 stuff yet.
     [[[redacted: personal]]]
     It would be a good idea to have a chat about IPCC. I cannot believe IPCC
     will allow itself to be 18 months out of date when it appears. If so,
     long live the TAR! Surely the Government Reviewers like DEFRA wont allow
     that! Can you ring me Monday at home any reasonable time. Working at
     home then. Am would be best.
     Heard no more about Ilarionov. We are ready if he pounces!
     No feedback from the "Sceptic". But at least I did not get misquoted by
     the Wall St Journal - I was not even quoted despite my hour of
     discussion. A good sign probably. So far, touch wood, I have stayed on
     the "right" side of the sceptics over the last decade. Certainly spoken
     to enough of them, often at length, and found out a bit about what makes
     them tick (different in every case). There may come a moment when this
     is useful.
     A long standing problem worth mentioning now is my potential successor.
     A month ago it was formally agreed that I can stay on till 31 Dec 2006,
     so the problem recedes for now. We might want to chat about that. DEP
     goes about the same time on present projections.
     Chris
     Prof. Chris Folland
     Met Office Fellow and Head of Climate Variability Research
     Global climate data sets are available from [1]http://www.hadobs.org
     Met Office, Hadley Centre, Fitzroy Rd, Exeter, Devon, EX1 3PB United
     Kingdom
     Email: chris.folland@metoffice.gov.uk
     Tel: +44 (0)1392 886646
     Fax: (in UK)  0870 900 5050
             (International) +44 (0)113 336 1072) [2]http://www.metoffice.gov.uk
     Hon. Professor of School of Environmental Sciences, University of East
     Anglia, Norwich
     -----Original Message-----
     From: Phil Jones [[3]mailto:p.jones@uea.ac.uk]
     Sent: 18 March 2005 08:58
     To: Folland, Chris
     Subject: RE: HadCRUT2 problems - diagnosis and possible solution
       Chris,
          I'm going to send a brief email to Philip to see where we are with
     HadCRUT3/v.
       There needs to be a submission fairly soon wrt AR4. The Beijing
     meeting
     is May 10-13
       with rewriting the 2-3 months after, before we get a FOD for general
     release to any who
       want to comment. As an aside, I'll be interested to see how WGI plays
     this wrt the FOD.
       I can see bits of text appearing on web sites if Susan isn't very
     careful. I'm sure there
       will be safeguards to stop this.
          As for HadCRUT2/v, I was only suggesting what seemed most practical
     given the
       imminent arrival of HadCRUT3/v. Sticking with the current coverage
     percentages we
       are using seems the best thing to do with 2v. It would have been
     better
     for Peter to
       have just compared where the differences were rather than looking at
     the
     whole world.
       Tasmania is either in the right or wrong position, after all.  Even if
     it
     is wrong it won't
       make much difference.
           Also trying to stop key files (those that will be used in
     HadCRUT3/v)
     being altered
       just seemed common sense to me.
           I've only seen so far one brief outline of the proposed HadCRUT3/v
     paper. It would
       be good to see more of the whole thing rather than bits and bobs and
     potential diagrams
       that Simon keeps sending around.  Although the contract we've done for
     you has ended,
       the person here is still doing some work. We've not sent everything
     back
     yet. We've still
       not sent the new SD file, which is necessary to weed out some typos
     that
     creep in from
       time to time. We have removed some duplicate stations that came in
     last
     time, and
       checked/altered/deleted some of the 1961-90 normals files. Philip
     should
     have these.
          Let me know of any relevant decisions made at the meeting.  The
     next
     time I'll
       be down in Exeter will be for the HC Review meeting (May 17-19), the
     week
     after
       Beijing.
          Any further news on Iliaronov? Also was the person happy with you
     response to the
       question to John Houghton about paleo in the TAR?  This seemed fine
     with
     me. Also,
       I'd be happy to ignore Iliaranov's piece.
          Another aside, NCDC have decided in their new gridding (which might
     not be in
       time for AR4) to forget their first difference method. They say it is
     more trouble than
       it is worth. They are also going to try to use SYNOP-calculated
     CLIMATs,
     which seems
       more trouble than it's worth.
       Cheers
       Phil
     At 22:21 17/03/2005, you wrote:
     >Phil
     >
     >Except for the one hopefully calming email I sent, I have kept out of
     >this so far. I have now asked for a briefing next week (likely
     >Tuesday). If anything silly looks like happening please do not hesitate
     >to contact me!
     >
     >Chris
     >
     >Prof. Chris Folland
     >
     >Met Office Fellow and Head of Climate Variability Research
     >
     >Global climate data sets are available from [4]http://www.hadobs.org
     >
     >Met Office, Hadley Centre, Fitzroy Rd, Exeter, Devon, EX1 3PB United
     >Kingdom
     >Email: chris.folland@metoffice.gov.uk
     >Tel: +44 (0)1392 886646
     >Fax: (in UK)  0870 900 5050
     >         (International) +44 (0)113 336 1072)
     >[5]http://www.metoffice.gov.uk
     >
     >Hon. Professor of School of Environmental Sciences, University of East
     >Anglia, Norwich
     >
     >
     >
     >-----Original Message-----
     >From: Phil Jones [[6]mailto:p.jones@uea.ac.uk]
     >Sent: 17 March 2005 16:59
     >To: Thorne, Peter
     >Cc: Thorne, Peter; i.harris@uea.ac.uk; Chris Folland; Tett, Simon;
     >Parker, David; John Kennedy; Brohan, Philip; Jennifer Hardwick; Jen
     >Teignmouth
     >Subject: Re: HadCRUT2 problems - diagnosis and possible solution
     >
     >
     >
     >   Dear All,
     >       Maybe I've misinterpreted Peter's email, but UEA doesn't want
     >the whole of
     >   HadCRUT2v/HandCRUT2 back.  If the coverage percentage goes back to
     >what it
     >   was then none of HadCRUT2/v will change except for Jan 2005. Unless
     >I've misunderstood,
     >   we've been using the same coverage since 2v started, so there is no
     >need to
     >   send the whole thing back. Also, if the old version goes back, then
     >there
     >shouldn't be
     >   any error messages !!!  Why will there be any if we are doing what
     we
     >were. THERE IS NO
     >   NEED TO CHANGE ANYTHING IF YOU GO BACK TO THE ORIGINAL COVERAGE
     >   FILE. There won't be any grid boxes that are different. I don't want
     >any
     >changes. I keep
     >   saying this and it keeps falling on deaf ears.
     >
     >       As an aside, the coverage percentage file isn't mine. When I
     >started doing HadCRUT2v,
     >   I got this from David Parker, so it's yours - not mine !
     >
     >      Finally, although there are differences they only have an impact
     > if
     >
     >they are between
     >   25 and 75% land/ocean coverage.
     >
     >      The USGS version isn't what you want anyway. Part of the
     >differences come about from
     >   setting ocean squares to land where there is permanent sea ice. This
     >was
     >a simpler
     >   way of allowing some extrapolation over sea ice from land stations
     >without bringing in
     >   any SST and also stopping SST getting into squares where it
     shouldn't.
     >
     >      Merging on errors isn't right either. It still makes no sense.
     >Merging by area, maybe
     >   incorporating errors might be possible. Merging should also consider
     >the
     >number of
     >   samples not just the error. If a box has 75% land with two stations
     >but
     >large error, but the
     >   25% ocean has small error, the merged value should also reflect the
     >greater % land. I
     >   still need to be convinced of this, and the only way I'll be
     convinced
     >is
     >to see the
     >   old and new methods (and little difference between them).
     >
     >   Cheers
     >   Phil
     >
     >
     >   Cheers
     >   Phil
     >
     >
     >At 15:38 17/03/2005, Thorne, Peter wrote:
     > >All,
     > >
     > >attached is an excel spreadsheet (sorry linux people!) showing the
     > >percentage coverage from IGBP
     > >([7]http://lpdaac.usgs.gov/glcc/globdoc2_0.asp) resolved to 5 by 5 and
     > >the
     >
     > >version that Phil has sent of the coverage percentage. Top is Phil's
     > >data which is to nearest whole percent (x100), next is IGBP as a
     > >fractional coverage and then last is the difference field as a
     > >fractional measure (Phil minus IGBP). There are some fairly large
     > >differences throughout. In the poles these are because of different
     > >classification procedures I should think. Differences beyong the 3rd
     > >d.p are probably rounding (obviously). The others look kinda scary so
     > >I'm advocating adopting an Ostrich position ...
     > >
     > >As Simon intimated earlier in this thread in HadCRUT3 the merge is
     > >going to be based upon estimated error rather than fractional
     > >coverage.
     > >
     > >Path of least resistance is install Phil's dataset back in which
     > >we've been using for a long time and run and just ignore any error
     > >messages, send UEA everything (complete rather than solely post-1990
     > >merged hadcrut2 and hadcrut2v files) this month and then keep running
     > >it with a mask we know to be sub-optimal because the whole thing is
     > >going to be
     >
     > >replaced on a timescale of months. We can let you know which
     > >gridboxes have changed in the merged product for the website Phil.
     > >
     > >I haven't got time to chase this further right now so if you do want
     > >it
     >
     > >chased you'll have to scrape together the time from somewhere else
     > >I'm afraid.
     > >
     > >
     > >All those not in favour shout now or forever hold your peace.
     > >
     > >Peter
     > >
     > >--
     > >Peter Thorne     Climate Research Scientist
     > >Hadley Centre for climate prediction and research
     > >Met Office, FitzRoy Road, Exeter, EX1 3PB
     > >Tel:+44 1392 886552 Fax:+44 1392 885681 [8]http://www.hadobs.org
     >
     >Prof. Phil Jones
     >Climatic Research Unit        Telephone +44 (0) 1603 592090
     >School of Environmental Sciences    Fax +44 (0) 1603 507784
     >University of East Anglia
     >Norwich                          Email    p.jones@uea.ac.uk
     >NR4 7TJ
     >UK
     >-----------------------------------------------------------------------
     >-
     >----
     Prof. Phil Jones
     Climatic Research Unit        Telephone +44 (0) 1603 592090
     School of Environmental Sciences    Fax +44 (0) 1603 507784
     University of East Anglia
     Norwich                          Email    p.jones@uea.ac.uk
     NR4 7TJ
     UK
     ------------------------------------------------------------------------
     ----

   Prof. Phil Jones
   Climatic Research Unit        Telephone +44 (0) 1603 592090
   School of Environmental Sciences    Fax +44 (0) 1603 507784
   University of East Anglia
   Norwich                          Email    p.jones@uea.ac.uk
   NR4 7TJ
   UK
   ----------------------------------------------------------------------------

