RE: [ccp4bb]: Refmac vs. cns

[Bart Hazes <bhazes@ualberta.ca>]

***  For details on how to be removed from this list visit the  ***
***          CCP4 home page http://www.ccp4.ac.uk         ***


>        In summary, a direct comparison of REFMAC-ARP/wARP and CNS is
> complex, and should include a more careful consideration of the type of
> refinement being performed, the amount of data available, and the
> reliability of the initial model. A collection of anecdotes about how these
> programs compare in a handful of refinements does not add meaningfully to a
> proper assessment of their worth.
>                             Sincerely,
>                            Mark Wilson

I agree in principle with Mark, but if you buy a car you don't just listen to
the salesperson but also ask some friends that bought one about their personal
experiences. I don't want to suggest that the CNS/X-PLOR authors should be
compared to car salespersons, I just want to say that personal experiences do
cary weight. One should also remember that what leads me to prefer one choice
doesn't have to be true for another, as I recently discovered during the
discussion over using dials versus the mouse in molecular graphics. So here is
my personal experience...

Basically it is very similar to that of Dirk. Simulated annealing looks very
attractive from a theoretical as well as practical point of view so I tried it
many times using default protocols and all kinds of variations. I never was
happy about the result. I kept trying because it looked like it should work
but still without any real success stories. These were 2.2 to 2.8 Angstrom
resolution structures. My feeling is that simulated annealing always causes
damage to your model in addition to improvements. The more accurate the model,
the less improvements (simulated annealing) can be made but there is still the
damage. So for a poor starting model I may still go back and give simulated
annealing another chance. For anything half decent I'll stick to wARP if the
data allows it or manual building. Jim's comment that current starting models
are generally better than in the past due to our better model building tools
is another factor here.

The fact that Refmac doesn't have simulated annealing doesn't make it a better
program of course, just the absence of it in Refmac isn't much of an issue to
me. The presence of TLS refinement in Refmac seems to be more of a bonus than
the lack of simulated annealing (although I haven't used it myself yet). My
overnight switch from years of X-PLOR/CNS to Refmac was mainly due to one
reason and a few annoyances.

- I became so lucky to get diffraction data of a quality and resolution that
allowed me to see multiple sidechain conformations. I was using Xfit at that
time and moving between CNS refinement and Xfit was painful. I don't remember
the details but it was what triggered me to give Refmac a try. Refmac and Xfit
speak the same language when it comes to multiple conformations so moving
between the two is trivial. Refmac also immediately gives you Sigmaa weighted
2Fo-Fc and Fo-Fc map coefficients unlike the separate job I had to run in CNS.
This will not be an issue for all projects and combinations of software. Still
there were a few other things I liked about Refmac that kept me using it. Some
of them are more psychological in nature than anything serious, but here they
are.

- The CNS scripting language was improved to the extend that I couldn't use it
anymore. It is very elegant but the level of complexity made the learning
curve too steep. I know the whole point is you don't have to look at that part
of the scripts (just consider it part of the program) but in the X-PLOR days
you could easily do beautiful things with scripting the program. I lost that
feeling of DIY computing the earlier X-PLOR had. Of course you can't do
scripting in Refmac at all, but it just needs a few lines of input which gives
me the feeling I know what's going on.

- In CNS I would run hundreds of cycles with statistics sometimes jumping up
and down, rather then gently refining to the nearest minimum. Did anyone
measure the time to reach a local minimum with Refmac and CNS starting from
the same coordinates? My feeling is that Refmac is a lot faster. I agree that
speed isn't that important but the statistics scrolling over your screen is
kind of hypnotising and I remember trying to use my mental powers to force the
values down.

- CNS log files kept growing in length and complexity. Maybe, like the
scripts, you weren't supposed to look at them but then why do you make them. I
know people wrote software to extract summaries from the log files (I made
some myself) but that only proves they were too unwieldy. Refmac is a lot more
modest and it only runs a few cycles to get the job done.

- For CNS you had to set up your topology files separately, taking care of
disulfide bonds, proper N- and C-termini as well as false ones (at gaps in the
chain). With CNS the topology is generated on the fly.

Good things about CNS are that it can do a lot more than Refmac can, the fact
that it makes a topology file allows you some extra control, it may well be
easier on the novice user, and the people who wrote it are very nice guys (so
are the guys&girls of Refmac). Just judge for yourself if any of my annoyances
annoy you and some may have been fixed by now. Apart from TLS and the smooth
cooperation with Xfit I would think/hope that the final results obtained are
the same with either program.

Bart

===============================================================================

Dept. of Medical Microbiology & Immunology
University of Alberta
1-15 Medical Sciences Building
Edmonton, Alberta, T6G 2H7, Canada
phone:	1-780-492-0042
fax:	1-780-492-7521

===============================================================================