date: Thu Feb 21 10:15:59 2002
from: Keith Briffa <k.briffa@uea.ac.uk>
subject: Re: paper in review <fwd>
to: j.burgess@uea.ac.uk

   Sharon
   Thank you for looking at this for us. I am in a difficult situation as Neil
   has (after a long delay) simply come back to me saying he can not do the review. On the
   basis of your review I am not happy to even offer a rewrite , at least without another
   qualified opinion. Can you suggest (in confidence) such a person?
   thanks again
   Keith
   At 09:47 AM 2/21/02 +0000, you wrote:

     Dear Dr Nicholson,
     Thank you for your prompt reply.
     I am very sorry about the incorrectly addressed email.  It was entirely my mix-
     up, which I feel very embarassed about.  Please accept my apologies and
     assurance that the email was intended to go to you, despite having the wrong
     name!
     I do hope you are recovered from your health problems and feeling better now.
     I will pass your reply on to Prof. Briffa, who will be in touch regarding your
     offer to complete the review.
     Yours sincerely,
     Julie Burgess

     Dear Keith,
     Please find enclosed a review of the Brooks/Hulme paper.  I realize now why I initially
     procrastinated on this manuscript.  It is one of those in which you know there are
     serious problems and should not be published, but cannot describe them easily.
     Conceptually, what they did sounded OK, but the method of approach does not appear to be
     well conceived.  My reasons for drawing this conclusion are indicated in the attached
     sheet.
     I will also send a copy of this review and the review form by fax to the number
     indicated at the bottom of your letter.  I really don't want to be identified as a
     reviewer.
     It became apparent that Neil Ward was also reviewer of this paper, when your secretary
     addressed a note to him to my email address.  If I have misjudged the paper, I would
     gladly defer to his review, as he is much more of a statistician than I.  At the very
     least, the authors need to better clarify, explain and justify what they did.
     Best regards,
     Sharon
     Review of "1000 years of rainfall variability in the Sahel: an evaluation of a long-term
     climate model simulation against observational data", by N. Brooks and M. Hulme.
     This paper deals with a clearly defined topic, but has several shortcomings that make it
     unacceptable for publication.  These are 1) inadequacy of the model for studying the
     Sahel, 2) poor validation of the model, and 3) arbitrary and unjustified statistical
     analyses.   I am also uncomfortable with their interpretation of results.  Finally, and
     this is a minor issue, I think better literature could be cited.  When several papers
     were available for citation about a certain point, the choice was generally a minor
     paper, with the most important papers being omitted.  There is a tendency to cite "soft
     science" literature in places where more technical literature is appropriate.
     1. Model and Model Validation: A validation attempt was made via comparison of
     statistics such as the mean, season cycle, and time spectra.  However, in doing so the
     authors compared statistics for this century with those for a 1000-year model run.
     There is no reason to assume that these periods are statistically comparable.  Indeed,
     the results suggest they are not.  However, some aspects of the climate can be
     considered fairly stable, such as the summer rainy season.  The model produces less than
     50% of its rainfall in July, July, August, compared to about 80% in the "real world".
     It also shows many years in which rainfall approaches zero in the rainy season.  The
     proper validation approach would have been to compare a 20th century simulation with the
     observed statistics.  Further, it is important to show that the model can capture the
     mean spatial pattern and the real temporal variability of the observed data.  This was
     not done.
     2. The model results are extensively "massaged", using what appear to be arbitrarily
     chosen filters of 9 years, 25 years, 45 years, 96 years and 101 years.  This is compared
     with unfiltered observational data.  What is the justification of these particular
     filters, how do they affect the results.  Is it appropriate to do statistical analyses,
     such as spectra, on the filtered series?
     3. As a result of all of this statistical manipulation, it is difficult to follow what
     the authors do.  It is even more difficult to judge their results and its statistical
     significance.  This is particular problematic when a major results is correlations for
     thousands of grid points (Fig. 8.).
     If this work were to be revised, much more attention would have to be paid to the
     statistical approach and to validating the results.  At the moment I have no confidence
     in any of the conclusions draw from this simulation.

   --
   Professor Keith Briffa,
   Climatic Research Unit
   University of East Anglia
   Norwich, NR4 7TJ, U.K.

   Phone: +44-1603-593909
   Fax: +44-1603-507784
   [1]http://www.cru.uea.ac.uk/cru/people/briffa[2]/

