Re: [ccp4bb]: Re: structure factors/quality control

[genick <genick@brandeis.edu>]

***  For details on how to be removed from this list visit the  ***
***          CCP4 home page http://www.ccp4.ac.uk         ***



>>
>> Peer review, as frustrating as it can be, has
>> proven to be the only viable quality-control
>> tool in science.
>
> I agree that other crystallographers are probably in the best position 
> to judge, but at least one among us who prefers not to have his or her 
> identity publicized just voiced to me the objection, when I made a 
> similar proposal, that some referees in the past have used anonymous 
> peer review as an opportunity to obtain a competitor's results prior 
> to publication.
I know how bad it feels to get screwed by dishonest reviewers, but
making people sign their names to reviews weakens the peer review system
and will probably lead to even less stringent
review standards and thereby to more wrong structures. (It also does 
not seem
to prevent reviewers from abusing their privileges, just see the NIH 
grant review
process for example where the name of study section members are public.)
So I do not see how your  "names for data"  proposal fixes the problem 
of bad
structures slipping through the review process.
>>
>> And by peer review I do not
>> mean a mob of crystallography vigilantes perusing
>> the PDB to revel in their competitors (maybe honest)
>> mistakes.
>
> Perhaps the best way to prevent this is a trade-off:  I'll submit all 
> of my data along with my paper for peer review if the reviewers sign 
> their names to their reviews.
>
>
>>  I get the impression that many students
>>   do not understand the basic principles
>>   of refinement or handle details of refinement
>>   and structure validation
>>   with a certain degree of laissez-faire.
>>   Worse, apparently many people have
>>   nobody in their own lab who can or is willing
>>   to answer their questions on such very basic
>>   issues.
>>
> That's a great way to keep people from asking important but seemingly 
> basic questions  -- imply they already should know the answer if they 
> are in a reputable research group.
>
> I often see several different answers posted in response to a "basic" 
> question.  Many seemingly simple issues are in fact deceptively 
> complex and deserved to be discussed without inhibition or judgment.   
> For example, should NCS always be applied as a constraint (rather than 
> a weighted restraint where the weight is adjusted as a function of 
> Rfree)?
>
My point is not to discourage people from asking questions,
particularly if they relate to core crystallographic principles.
I am not a crystallography jock so I appreciate a forum where
one can get good and free advice.

I just used the type of questions asked on the CCP4
bulletin board as a barometer for the general level
of competence in the community.

Based on the kinds of questions I see on the bulletin board
I come to two conclusions:

   1)We cannot assume that authors on a
      crystallography paper "probably
      know what they are doing" when we review
     a paper. So we need the data / statistics to double
     check their conclusions. Depending on the
     paper these data may be real-space
     correlation coefficients, ESD's or omit maps.
     It seems unreasonable to me that a reviewer
     would have to re determine a structure from
     scratch, so the Fos themselves are often not
     particularly useful.

   2)We should put some more time into teaching
       students basic principles of crystallography
       and into keeping our own skills honed.


As far as the bulletin board is concerned, I think it is
a great forum to discuss, teach and learn crystallography.
I would just hope that the posted answers would be
more like mini-tutorials rather than of the kind
which I would paraphrase as "Just run this through
programs x y and z, that should be good enough for
  the reviewers".


>