[ccp4bb]: Discussion on contouring levels.

[gb10@globalphasing.com]

***  For details on how to be removed from this list visit the  ***
***          CCP4 home page http://www.ccp4.ac.uk         ***

Dear CCP4 BB subscribers,

     I posted this note to the BB last Thursday afternoon, as this topic was 
at its hottest. As my subscription was from an old e-mail address, the note 
was never forwarded. 

     The topic has cooled down somehow, but the remarks below may still be 
of some help in pointing out that basic statistics cannot be ignored, even 
by those who do not love them, in the discussion of this question.

-------------------------------------------------------------------------------

Dear Gerard (K) and Andrew,

     It would seem that the 'central' concept behind this discussion is the 
Central Limit Theorem. If the lack of fit between Fo and Fc is randomly 
distributed without any trends nor correlations, the Fo-Fc map will be made 
up of white noise, i.e. its values will be normally distributed, so that the 
probability of a 5-sigma deviation will be less than 10**(-6). If the number 
of data is so vast that there are of the order of 10**6 independent data 
items or more, then a 5-sigma peak can occur by chance and hence be considered 
as noise. In more commonplace cases, however, the probability of a 5-sigma 
peak occurring by chance would be quite low, and therefore such a peak would 
be highly significant, as stated by Andrew.

     GK's counterexample, with which AL disagreed, does seems rather contrived.
If 10 times the sigma of the Fo-Fc map were to be considered as noise by some 
criterion, then the same criterion should lead one to conclude that the data 
have been grossly overfitted in the first place.


     With best wishes,

         Gerard (B)



According to "Gerard \"DVD\" Kleywegt":
> 
> ***  For details on how to be removed from this list visit the  ***
> ***          CCP4 home page http://www.ccp4.ac.uk         ***
> 
> i seriously disagree with your seriously disagreeing.
> assume that the sigma level at the end of a refinement
> is 0.0001 e/A3. then a 10-sigma feature is noise. my point
> is that the "number of sigmas" of a feature in an fo-fc
> map is meaningless - the absolute sigma level is not, of
> course (but this assumes that you calculate your maps on
> an absolute scale to begin with !).
> 
> --gerard
> 
> >  > - the sigma level of an Fo-Fc is meaningless. In the early
> >  >   stages (poor and incomplete model), a 2-sigma feature
> >  >   may be genuine, whereas near the end of the refinement
> >  >   process (when the difference map is hopefully flat
> >  >   except for noise) even a 5-sigma peak need not be
> > 
> >    I seriously disagree with this remark. The sigma level in an Fo-Fc
> > map does indeed give an indication of the noise level of the map, and
> > one would certainly hope that this noise level, in absolute terms,
> > would go down as refinement proceeds. However, this certainly does NOT
> > mean that a 5 sigma peak in a difference map near the end of
> > refinement is not suggesting an error in the model in that region.
> > Whether or not one can interpret the feature is a different matter, as
> > very often they arise from multiple conformations/disorder that is
> > very difficult if not impossible to model correctly.