cc: Barrie Pittock <pittock@environmental-change.oxford.ac.uk>, Luis Jose Mata <l.mata@uni-bonn.de>, Stephen H Schneider <shs@stanford.edu>, "Kozak, LeeAnn" <LAKOZAK@southernco.com>, tar_cla@usgcrp.gov, tar_la@usgcrp.gov, tar_reved@usgcrp.gov, wgii.bureau@usgcrp.gov, John Houghton <jthoughton@ipccwg1.demon.co.uk>, yhding@public.bta.net.cn, djgriggs@meto.gov.uk, Neil Leary <nleary@usgcrp.gov>, kwhite@usgcrp.gov, ipcc@usgcrp.gov, maureen.joseph@environmental-change.oxford.ac.uk, giorgi@ictp.trieste.it, Jerry Meehl <meehl@meeker.ucar.edu>, cubasch@dkrz.de, ckfolland@meto.gov.uk, stocker@climate.unibe.ch, hewitson@egs.uct.ac.za, Barrie Pittock <barrie.pittock@dar.csiro.au>
date: Thu, 07 Sep 2000 18:29:53 +1200
from: Martin R Manning <m.manning@niwa.cri.nz>
subject: RE: Extremes table
to: tkarl@ncdc.noaa.gov

<x-flowed>
Tom

Thanks for your interesting comments on the uncertainty issue. Some 
responses...

Being a physical scientist I have no problems with the frequentist approach 
and this would be my preference where it can be applied. I also agree that 
it is confusing, and possibly illogical, to associate uncertainties with 
probabilistic statements - I have already argued against doing that in the 
WG II SPM.

Your experience in forecasting must provide a strong basis for considering 
measures of uncertainty or probability for WG I statements - but I would 
argue that you can only apply this approach where the basis for predictions 
is well established (e.g. the models being used are well validated and 
relatively stable).

The TAR also has to deal with some highly policy relevant issues that 
depend on models which are still being developed. In these cases we have no 
guidance from experience with previous forecasts and I do not see how the 
frequentist approach to uncertainty can be used. Your comments about not 
having a large "sample" echo the same concern.

But your response to this problem is....
>"Without a large sample, it seems to me that it is not possible to assess 
>our confidence in our probabilistic statements. Our confidence is already 
>built in --- for low probability events we don't expect them to occur very 
>often.  This is why I believe in working group 1 we changed from 
>confidence statements to likelihood statements."
... and I am afraid I do not follow this argument at all.

If you can not apply a probabilistic approach then it seems to me that you 
have to look for another measure of confidence in model results. To say 
that the confidence is "built in" sounds like giving up rather than 
addressing the problem. It does not recognise inherent limitations in 
making assessments where views on the key driving processes are still 
evolving, model structure still changing, and model validation still 
sparse, as in the case of the THC. Estimates of confidence, based on expert 
judgement as to how comprehensive the present models are, may not meet the 
philosophical rigour of the frequentist approach, but do provide important 
information in these cases.

The Uncertainties Guidance paper gave a number of ways in which you might 
develop such confidence levels from expert judgement on an objective basis. 
Your approach seems to limit the definition of uncertainty to the range of 
results from current models and in doing so may be rejecting an important 
source of information, viz any expert consensus on how good the models 
really are.

Thus my bottom line is that we should be able to make statements about our 
"confidence" in a result even when defining "uncertainty" in that result 
may be problematic. And I really think that WG I need to consider and state 
their confidence in some key statements - such as the THC issue.

There are of course other problems with producing simple summary statements 
about THC changes. It is hard to define exactly what one is talking about - 
most coupled ocean atmosphere models show a weakening of ocean circulation 
patterns in response to GH warming - so the < 1% probability by 2100 
statement being made by WG I presumably refers to some level of major 
collapse in the THC. How do you set that level? How non-linear is the 
dependence of the probability estimate on the chosen degree of THC 
collapse? Then there are the time-scale issues - what is the probability 
that GH warming by 2100 causes significant change in THC after 2100?

For these reasons I would prefer to see WG II deal with singular climate 
change separately from the table being discussed for Ch 3.

Thanks very much for your comments - I am still learning from this 
interesting discussion.
Martin

-----------------------------------------------------------
Martin R Manning
National Institute of Water & Atmospheric Research
301 Evans Bay Parade, Greta Pt
PO Box 14-901, Kilbirnie               Phone: +64 +4 386 0535
Wellington                                    Fax   : +64 +4 386 2153
New Zealand                                 m.manning@niwa.cri.nz
-----------------------------------------------------------

</x-flowed>
