date: Wed Dec  3 13:09:24 2008
from: Phil Jones <p.jones@uea.ac.uk>
subject: Re: NASA
to: Tom Wigley <wigley@ucar.edu>

    Tom,
       Here is a paragraph I got from Tom Peterson at NCDC overnight
   The story I've gotten about the Oct. CLIMAT is that we received two sets of CLIMAT messages
   from Russia.  The first had September's data in the October message.  The second had
   October's data in October's message.  When I was in charge of GHCN, we had a policy to
   always replace with the latest data as that should be the correct value. However, there
   were some problems with this, for example, when a country would transmit a correction to
   SLP and only transmit that - letting the rest of the fields just be set to missing.
   Apparently the system as it was working last month did not replace with more recent data so
   these September values stayed in the data set.  NASA was blameless as they just use GHCN.
   GHCN has been in serious need of maintenance and improvements (e.g., to incorporate all the
   CLIMAT messages, to account for changing station numbers, etc.) for quite some time. But
   instead of spending time and energy improving GHCN version 2, the plan was to create GHCN
   version 3 instead.  Unfortunately, GHCN version 3 has taken longer than anticipated.
    Cheers
    Phil
    PS Russia is the only country to send its CLIMAT data in large groups (as opposed to
    altogether for the whole country).  This is why it was only part of Russia that was
   affected.
    This also explains why the skeptics think this is all down to GHCN. NCDC must injest
    the climate messages into GHCN. It isn't GHCN that is at fault, but there logic.
    What I do is that if new data come in it replaces old. If it comes in missing then you
   keep
    what you might have had!  Seems fairly trivial to me!

    Tom,
       This is all a storm in a tea cup. GISS shouldn't have got it wrong,
    but far too much is being made of it by skeptics and their journalist
    friends in the US.
       Much of what this report states is garbage. GISS had some of the
    Russian data in wrong for Oct 2008. This comes about from trying to
    do things too soon after the month. GISS pick up the data from NOAA/NCDC
    and it is based on a mixture of CLIMAT (the monthly averages) and SYNOP
    messages off the GTS. Attached is what MOHC produced from these two
    sources for October for the CLIMAT and for the SYNOPS. The latter is
    dodgy as you have to decide how many days are needed to get a complete
    month.  The CLIMAT comes through on the 4/5th of the month and again
    later between the 16-20th (in this case during November).
       GISS try to get this out just after the 5th - as the UAH/RSS get theirs
    out quickly. We wait till the 20th or 21st. People shouldn't be looking at
    individual months, but the skeptics do to keep saying that there has been no
    warming since 1998. To counter this you just have to look at the 1991-2000
    average versus the 2001 to 2008 average.
        GISS blamed NCDC for not doing enough QC on what was coming in, but
    I reckon this isn't right. It is clear that many of the Russian data for October
    were September's data. When they spotted the mistake they reran the
    gridding and put the revision up. The skeptics then said some other parts of
    the world had altered and GISS got worried. What they had forgotten was that
    CLIMAT data keeps coming in during the first 20 days of the month, so the
    second time they ran the gridding they had more data in other parts of the
    world - so that is why other areas appeared to change. What also happens
    is that the SYNOPS come in first (they do each day during October), GISS
    must replace these with the better CLIMATs as they come in during November.
       CRU/MOHC don't use the SYNOPS. MOHC are doing some experiments
    with these data to update daily series for extreme temperature and precip
    analyses - and one of checking these is the comparison on a monthly basis
    with the CLIMATs.
       The real issue is why did NOAA/NCDC pick up the wrong CLIMAT or SYNOPS
    for October (September's values for 90+ stations in Russia).
       According to MOHC, the Russians put the right data out. Countries often
    put the previous months data out by mistake - mainly in Africa and South America.
    None of the GISS or NOAA/NCDC software tests for this, but it's easy to see if
    you look at the precip amounts, which they won't be doing. Gavin says something
    went wrong at NOAA/NCDC, which is reasonable. It is hard for GISS to pick
    up September's data just for 90 stations in Russia, when they get the whole
    month's data from NOAA/NCDC.
       I've made a note to ask Russ Vose, Dave Easterling or Tom Peterson when
    I next email one of them. It is probable it was in DC though and not in Asheville.
      Our numbers are on our web site. CRUTEM3 (land only) for Glob/NH/SH they were
    0.78/0.86/0.71  wrt 61-90.  So October was quite warm. Globally only 2004-2007
    were warmer - since 1900.
      The blogs had a lot about telling NOAA/NCDC and GISS where they can pick
    up more data off NMS web sites. These are only for a few countries though.
    The blogs seem to have no idea of the GTS (never mentioned anywhere). There
    seems to be a belief that NOAA produces the monthly averages - but the
    CLIMATs are produced in the countries. NOAA does from the synops, but
    replaces these as the month progresses.
     It all stems from trying to do things too quickly after the month. Our (MOHC
    and mine) are all automated. MOHC do look at the maps - like those attached.
    Cheers
    Phil
   At 03:35 02/12/2008, you wrote:

     Phil,
     I presume you have seen the attached. Can you comment?
     I/we have never been very keen on what GISS does or
     produces, but their results are still in reasonable
     accord with CRU and NOAA. Is this just luck? Why, I
     wonder do they go off half-cocked like this?
     What do/will CRU/NOAA get for Oct. 2008?
     Tom.

   Prof. Phil Jones
   Climatic Research Unit        Telephone +44 (0) 1603 592090
   School of Environmental Sciences    Fax +44 (0) 1603 507784
   University of East Anglia
   Norwich                          Email    p.jones@uea.ac.uk
   NR4 7TJ
   UK
   ----------------------------------------------------------------------------
