cc: Phil Jones <p.jones@uea.ac.uk>, Ray Bradley <rbradley@geo.umass.edu>, Malcolm Hughes <mhughes@ltrr.arizona.edu>, Steve Schneider <shs@stanford.edu>, tom crowley <tom@ocean.tamu.edu>, Tom Wigley <wigley@meeker.UCAR.EDU>, Jonathan Overpeck <jto@u.arizona.edu>, <asocci@cox.net>, Keith Briffa <k.briffa@uea.ac.uk>, Tim Osborn <t.osborn@uea.ac.uk>, <Tim_Profeta@lieberman.senate.gov>, Ben Santer <santer1@llnl.gov>, Gabi Hegerl <hegerl@duke.edu>, Ellen Mosley-Thompson <thompson.4@osu.edu>, "Lonnie G. Thompson" <thompson.3@osu.edu>, Kevin Trenberth <trenbert@cgd.ucar.edu>
date: Mon, 27 Oct 2003 11:18:31 -0500
from: Mike MacCracken <mmaccrac@comcast.net>
subject: Re: CONFIDENTIAL Fwd:
to: "Michael E. Mann" <mann@virginia.edu>, Michael Oppenheimer <omichael@Princeton.EDU>

   Hi Mike--Back from a couple of weeks away and trying to catch up.
   I thought it might be of interest that OMB has put out draft guidelines on what is supposed
   to constitute peer review for agency reports or reports they rely on about major
   issues--and just to note that there is a sense (in at least one agency and some CCSP
   staff), supproted by some discussions with the author of the guidelines, that most or all
   climate materials would need to live up to the guidelines.
   Two points on what is there:
   a. Frankly, there are a lot of problems with the proposed guidelines in that they seek to
   have all reviewers essentially be so unconflicted that no one who knows anything is likely
   to qualify or be willing to be a reviewer. The whole notion of the content of the review
   comment mattering more than its source is totally lost (much less offering nay guidance on
   how seriously agencies need to take any comments). I imagine those on journal review boards
   or serving as editors (like Steve) might want to check out the proposal and see how their
   guidelines compare--and how they think the OMB guidelines might work (or not work) for
   them.
   b. However, the guidelines do presume that journal peer review provides a challengeable
   qualification to the paper. Interestingly, there is no indication that the journal must be
   of any given quality or follow any approved procedures, so what is sure, if these
   guidelines go through, is that there will be a rash of new journals created, all of little
   stature.
   I have made these and a number of related points to the OMB in response to their
   solicitation of comments. And now, the NRC is going to hold a meeting on them (see email
   notice below--though without form), as apparently I have not been alone in objecting. I'll
   be on travel but did send in my letter to OMB (copy available on request--since it is
   several pages long, I won't burden everyone with the letter). By the way, comments deadline
   has been extended to Dec 15 to accommodate NRC workshop, I presume.
   In any case, this matter of what constitutes "peer review" is coming up for attention by
   this Admin--so perhaps this effort of skeptics to get things into what they call
   peer-reviewed journals is so they can be cited more directly by the Admin.
   Mike MacCracken

   Subject:    PEER REVIEW OF REGULATORY SCIENCE WORKSHOP-November 18, 2003
   Dear Colleague:
   In light of expressions of interest and concern from within the research
   community regarding the newly issued "Proposed OMB Bulletin and Supplemental
   Information Quality Guidelines: Peer Review and Information Quality," and
   with the encouragement of U.S. Office of Management and Budget (OMB), The
   National Academies Science, Technology, and Law (STL) Program intends to hold
   a one-day public workshop on Tuesday, November 18, 2003, in Washington, D.C.,
   at which federal agencies subject to these new standards can share their
   views and hear ideas and concerns from each other and from external
   communities, including academic researchers, about the implications, merits,
   and practicality of the proposed bulletin.  The workshop is intended to
   assist the agencies in developing their agency-specific comments on the
   bulletin and ultimately in developing their peer review procedures.
   Further details on the agenda will be sent out in late October.  Please free
   to forward this announcement to other interested parties.
   If you would like to attend the workshop, please fill out the attached
   registration form and fax to (202-334-2530).
   For more information please contact:
   Contact Name: Stacey Speer
   Email: sspeer@nas.edu
   Phone: 202-334-1713
   Fax: 202-334-2530
   Attachment Converted: "c:\eudora\attach\peer_review_andnfo_quality.pdf"
