cc: nleary@earth.usgcrp.gov, Barrie Pittock <barrie.pittock@dar.csiro.au>, Linda Mearns <lindam@ucar.edu>, "Jones, Roger" <roger.jones@dar.csiro.au>, m.hulme@uea.ac.uk, djgriggs@meto.gov.uk, meehl@ncar.ucar.edu, peter.whetton@dar.csiro.au, Tom Karl <tkarl@ncdc.noaa.gov>, m.manning@niwa.cri.nz
date: Fri, 20 Oct 2000 13:30:06 -0700 (PDT)
from: Stephen H Schneider <shs@stanford.edu>
subject: Re: Table 3-10
to: Timothy Carter <tim.carter@vyh.fi>

Hello all. Thanks TIm for your continued reponsiveness to all points of
view. I am still quite disappointed at the last minute revisionism despite
a phone conference 5 weeks ago and thus we all have last second major
changes to do--to say nothing of the fact that the revisons appear to some
to be about science but are really about their paradigmatic views of
science versus science for policy--I'll attach something I use in classes
to make this point clear (about type 1 and type 2 errors in policy
analysis). This is why we need enough lead time to communicate with
each other and not have paradigm changes at the last moment. Nevertheless,
I appreciate your two alternatives and find the first vastly superior to
the second. The vague and imprecise language in version two, combined with
the lack of training of policymakers will lead them to conclude we know
virtually nothing and thus it doesn't communicate the nuances of
likelihood that the first version does--though new number 1 is much less
nuanced than the old Table 3-10 we all have been using with the guidance
paper scale and the five star system. (And, the closer the final table
3-10 is to the old 3-10 the less CLAs have to do in last second
revisions, and the quicker we can get off final drafts to the TSU--I'm
already set back two days by this flap.) Anyway, I don't like words like
"inconclusive" or "firm" since they violate the clear practice of decision
analysis disciplines , namely, to have a quantitative reference to all
terms so people mean and infer the same degree of likelihood or of confidence
for each term. "Insuffficient" is in that vague category too, as it
begs the reader to ask"insufficient for what" and "what odds constitutes
an insufficient". Also, when you say "insufficient to draw conclusions",
that again begs the questions of what level of evidience is implied for a
conclusion to be drawn? This is simply not the language or paradigm in
which policy analysis is done--relative likelihoods and--dare I say it--
subjective confidences in the underlying science are needed and should b e
as nuanced as the state-of-the art permits. Thus, version 1 with
likelihoods is much better that version two with vague and undefined
terms and references to models that policy makers will be clueless 
to translate into likelihoods--unless you want them to think we think 
all models are equally likely and thus a counting exercise is all
the meta-analysis we need to do to assign confidence. Since it appears
we are to abandon the guidance paper language, at least use possible
rather than insufficient information since the latter gives us virtually
no information. Or say insufficient to make confident judgment about
direction of effect, where confident means greater than, say, two thirds
likelihood. I hope you see the point--not to inflate our understanding by
using a quantitatively linked scale to communicate information that is
really degrees of belief of experts based on studying the state of the
science and the literature--but not to have inconsistent private
interpretations of what we mean by imprecise words like insufficient or
inconclusive. We are talking to policymakers, and shouldn't be worrying
whether it is in the jargon in which one tenth of one percent of the
world's population is familar--namely us. We need to be explicit in what
our terms mean and have back-up in the chapters for this important summary
table. 
  So I'd advise scrapping version 2, and trying to make version 1 as
nuanced as possible since the policy world doesn't mind that we don't have
all the facts yet, but they do mind if we lump all things with some
uncertainty into a catch all like insufficient information, when the
reality is we have often very different priors--Bayesian that is--on these
very different events we've lumped under the catch all umbrella terms. And
again, the closer to the original the more practical for rewriting.
Thanks again for all your hard work. Cheers, Steve


On Fri, 20 Oct 2000, Timothy Carter wrote:

> Dear colleagues,
> 
> I am mailing two alternative versions of Table 3-10 to a small group of you
> to request feedback.
> 
> I am also attaching a copy of the draft/final SPM of WG I which Dave Griggs
> got ready about three hours ago. PLEASE DO NOT PASS THIS ON. It contains
> the new WG I table of extremes, plus some footnotes of importance and some
> other statements that are of relevance to Table 3-10 and to some WG II
> chapters as well. This table is identical to that now included in Chapter 9
> (Table 9.6).
> 
> SPM table (9.6): You will see that there are no longer measures of
> likelihood of changes. Also, many categories have been removed, though they
> are discussed briefly in the footnotes. 
> 
> Note also the statements on the THC and WAIS on Page 7 of the SPM. I have
> already queried the "very unlikely" category assigned a major loss of
> grounded ice (see lines 33-34) which implies from the footnote on page 1 a
> 1-10% chance of occurrence. This seems pretty high for a dramatic event of
> this type when hydrologists are concerned with a 1:10000 event!
> 
> Table 3-10 version 1 (label on header): this takes new Table 9.6 along with
> the old version of Table 9.6 (which was based on Easterling et al. (2000)
> paper in Science and uses the same probability classes) and interprets this
> information in terms of WG I confidence ratings (because they can be traced
> directly to Easterling et al.). Some ratings are WG II ratings, as before.
> All categories of extreme are retained, because they are all of relevance
> for impacts. In fact, TC intensity is now divided into precip and wind
> intensities, as in Table 9.6, and max temperatures and hot day freq is
> combined.
> 
> Table 3-10 version 2 (label on header): this shows the same events as
> Version 1, but uses the descriptive classification shown in Table 9.6. It
> also extends this classification according to categories required for the
> types of events not included in Table 9.6. The description for those events
> is based on my interpretation of the write-ups in WG I chapters, the SPM
> and Easterling et al. (2000). Only WAIS disintegration gets a likelihood
> (as in the SPM), though I think it may need changing to extremely unlikely.
> 
> See what you think. I didn't include a version with our original confidence
> classes because I really think there would be major confusion between WG I
> and WG II, and that is likely to be more serious politically than confusion
> between Chapter 3 and other Chapters in WG II. Version 1 is a "likelihoods
> compromise" which is fairly close to our original, and probably
> transferable to other chapters, but may still be too far from WG I. Version
> 2 is a "WG I/WG II compromise", which adds detail to the WG I tables, but
> requires WG II chapter interpretation to get to likelihoods.
> 
> Perhaps there are other methods of achieving this damage control exercise.
> I am open to suggestions!
> 
> Regards,
> 
> Tim
> 

------
Stephen H. Schneider
Dept. of Biological Sciences
Stanford University
Stanford, CA 94305-5020 U.S.A.

Tel: (650)725-9978
Fax: (650)725-4387
shs@leland.stanford.edu

Attachment Converted: "c:\eudora\attach\Type1&type2 errors.doc"
