Re: [ccp4bb]: Parallelization of ccp4 programs on linux cluster ?

[Anastassis Perrakis <perrakis@nki.nl>]

***  For details on how to be removed from this list visit the  ***
***          CCP4 home page http://www.ccp4.ac.uk         ***

> Ah, I just remembered that Arp/wARP has incorporated an rsh-based scheme
> that enables you to use different processors.

..... which has died a quite death, since modern sys-mangement hates rsh and the
multi-model
idea is much less powerful when applied to ML refined models (it was designed
for
LS refined models, and basically what it was doing was getting an empirical
estimate
for correct sigmaa).

I do doubt if problems in xtallography are in desperate need for
parallelisation.
I mean, calculating maps, sure, but it takes 10 secs maximum anyway...
Refmac job are fast, even sharp is fast these days, and I like a 30 mins brek
for coffee and discussion ...

My undestanding of parallelisation is mostly to submit easily 5 refmac jobs
with slightly different params and get the best Rfree. Would not mind
submitting
100 Arp/warp jobs with different params either and see whats best
(Frank, any comment ?)

As the number of tunable parameters get more its more difficult to make good
guesses.
The thing we can now change during autotracing go to so many levels
that I am certain we do NOT have the best defaults.
Massive parallelisation can lead to experienced-derived databases
(call them expert systems if you prefer) which can then be queried for new
cases.
If that would be done systematically we can hope to reduce the 'optimal
parameter
space' drammatically. Marie Francoise worked on that with me at Grenoble and
we
came up with an SQL based system that ARP users could submit their parameter
files
and results together with data quality inicators. Then we had a query
interface that
you could give your data qaulity params and the database would return you
parameter
files that had been used and might be optimal. Of course that system relied on
users
optimisng their runs before submitting them - I know quite alot that have
struggles with
arp/warp parameters and defeated the defaults.

This was never released due to lack of time and MF getting a new job.
But it might be an idea to follow ... The code is available.

    Tassos

--
NKI, Department of Molecular Carcinogenesis
Plesmanlaan 121 1066 CX Amsterdam NL
Phone +31-20-512-1951 Fax +31-20-512-1954
SMS: +31-6-28597791 WWW: http://den.nki.nl