cc: Chris Folland <chris.folland@metoffice.com>, Simon Tett <simon.tett@metoffice.com>
date: Tue Mar  9 16:12:58 2004
from: Phil Jones <p.jones@uea.ac.uk>
subject: Re: 6 month contract - HadCRUT3
to: "Brohan, Philip" <philip.brohan@metoffice.com>

    Philip,
        Here are some comments on the current HadCRUT3 work plan. I've done this as
    comments/suggestions and where CRU's work would fit in.
    1. Land Uncertainties.
          Agree we can neglect measurement error per se, but there is an additional error due
     to transcription/coding mistakes.We could assess this by comparing a subset of stations
    with similar data from GHCN. Suspect differences would be small and random. QC work
    has been done on the input CLIMAT data from the mid-1980s and I've spent loads of
    time in the past checking outliers. Values in excess of 5SDs are currently not used.
          CRU work in this section would be on digitising the corrections applied to land data
    (from work in the mid-1980s) and process these to error fields. Agree that 5 weeks is a
    good estimate, but the error fields need to combine the corrections with the other data
    for each grid box (i.e. a grid box might be made up of 2 stations, yet only one had a
    correction, so the effect of the correction is halved). The influence of the corrections
   would
    be zero after about 1980 as none of this sort of work has been done since the mid-1980s.
          On the thatched sheds and urbanization, I would suggest coming up with best guess
    values for these with some sort of ranges. We could then apply these to the data as
   ranges.
    For example, for urbanization we have some numbers from  a paper in 1991. We could apply
    these, then in a second run double them. It would be good to give readers an idea of what
    effect worst case scenarios (e.g. urban effect of 0.5 over all populated areas) might
   have.
    So for CRU here 6 weeks.
    2. Land -sea Blending.  Would like to be involved here, so suggesting CRU 2 weeks. I
   suspect
    that the exact method isn't that important to large-scale averages, so need to consider a
    method of testing effects - against what etc.
      Aside - I've been working with Adrian Simmons of ECMWF on comparisons of ERA-40
    surface temps with CRUTEM2v. This has spotted lots of problems with the ERA-40 in some
    regions - lack of synops, but it found the Turkish problem in CRUTEM2v in Nov 1981. I
    suspect it is good enough to find others (obviously post-1958), but in many parts of the
   world
    ERA-40 would appear good enough to test the 'growing land'/infilling currently undertaken
    when the land/sea data get blended.
    3. Land Gridder

          The above aside may be the way to assess the effect of growing land/infilling, so
   CRU
    could do some work here - 1 week.   Not sure what reverse engineer means here?
        For data to be used we need normals - see later.
    4/5  Optimal Interpolation/Averaging
           I'll leave this up to you. People want fields not just the one final series, so if
   the
     final OA can be got from simple averaging of OI all the better.  It would be good here to
     test the OI with the simple blending from 3).
    6. Variance Correction
          Key issue here is that this must be done before OI/OA. It doesn't make sense to me
    to do the OI/OA on non-variance corrected fields (at least when this can be done).
    This section is the one that could expand and get out of control. There are a number of
    options/routes to take, so suggest that comparison of the three or combinations of them
    is the way to go. I've always been surprised how relatively small this really is - at
   least
    since the 1950s. Suggest that the various methods need to be compared, then a decision
    on which to use is made. Intuitively, I would like to believe the one that reduces the
   variance
    the most ought to be the best.
         Users want variance reduction because we introduced it. It is vital if you want to
   look
    at changes in extremes - well extremes at the monthly timescale.
     Here CRU 9 weeks.
    7. Generalise to other variables.
         Suspect that this will only work for MSLP and maybe Humidity. It is not going to work
    for precipitation.  For precip, GPCC will have a better product than we can ever produce
    from the 1950s only because they have much more data than we can ever get hold of.
    8. Rationalise Normals
        Given that the SST component won't have this problem, some work on the CRU normals
    might be useful. Work here could expand out of all proportion as we don't have the time,
    nor the resources to check all normals. I suspect though that some work would be
    beneficial. The problem the normals creates in the land component is that the average
    of the 1961-90 period, both for the NH/SH and for individual grid boxes doesn't average to
    zero. You are currently applying an adjustment to account for this.  With some weeks
    work, we could adjust the normals to ensure this. We could us HadCRUT2v to achieve
    this - only changing those stations where the current normals are not based on complete
    1961-90 normals. I know how to do this and suggest it would take about 4 weeks.
        It would solve the need to rezero both at the large scale and at the grid-box scale.
   This
    would omit one later step and reduce the number of versions.
    7. Write-up
         This is the one section that I think is very optimistic. It always takes longer.
   Section 3
    of the detail on new capabilities needs careful thought.  CRU input here 4 weeks, as this
    is the most important part of the work, particularly wrt IPCC AR4.
      So, made the total come to 26, only had to increase the first and sixth by 2 weeks each.
      Here's a budget I've had produced.  As you may know payscales at all of the older
    Universities are still under negotiation, so this may increase a little (3-5%).  I have to
    budget at a level of a typical CRU employee, here one with 3-5 years experience post-PhD.
      Staff  14478
      O/H (46%) 6660
      Travel/subsistence (3 trips to Exeter at 200 each)  600
      Total  21738
      I will be fully paid by UEA during this time and will be afterwards, so the costs are
    just for a 6 month Post-Doc, who will likely be on something else afterwards. So, they
    will be around for discussion etc.
      If this takes off, I need to pass these costs onto UEA, with the outlien of work. There
   is
    no point my doing this until you make a decision at your end.
    Cheers
    Phil
   At 09:31 09/03/2004 +0000, Brohan, Philip wrote:

     Phil.
      We have started putting together a plan for the work we want to do on
     HadCRUT3. I have put the current draft on the web at:
     [1]http://hadobs.metoffice.com/hadcrut3_plan/
      Obviously this is going to require input from you as well as work from
     us, and as we might be able to find 6 months of funding for a post-doc
     it would make sense to identify a subset of the HadCRUT3 work which
     could be done at CRU using that money.
      Please can you have a look at the plan and send me any comments, and
     we'll try to come up with a workpackage that fits into the HadCRUT3 plan
     and meets Simon's constraints below.
     Regards,
      Philip
     On Mon, 2004-03-08 at 21:36, Simon Tett wrote:
     > Phil,
     >       we might be able to fund 6 months of work to you on HadCRUT3 on the
     > land temperature record and its blending with the ocean record. I have
     > been 1/2 asleep (well trying to sort out ozone) and neglected a deadline
     > of last Friday. Can you and Philip Brohan produce something!
     >
     > The constraints are as follows:
     >
     > Please could you confirm that
     > 1. Your suggested project(s) can achieve something useful in 1 year
     > starting April 04 (or soon after).
     > 2. If longer is required that the university will provide funding for
     > the subsequent period.
     > 3. The university/ contractor can provide the right person to do the
     > work on this timescale.
     > 4.  Assuming these issues are addressed, please provide a detailed
     > proposal inclduing outline of work, deliverables and costs.
     >
     > sorry to both of you for the short notice.
     >
     > Simon
     --
     Philip Brohan,  Climate Scientist
     Met Office   Hadley Centre for Climate Prediction and Research
     Tel: +44 (0)1392 884574    Fax: +44 (0)1392 885681
     Email: philip.brohan@metoffice.com  [2]http://www.metoffice.com

   Prof. Phil Jones
   Climatic Research Unit        Telephone +44 (0) 1603 592090
   School of Environmental Sciences    Fax +44 (0) 1603 507784
   University of East Anglia
   Norwich                          Email    p.jones@uea.ac.uk
   NR4 7TJ
   UK
   ----------------------------------------------------------------------------

