Clipper Libraries

Kevin Cowtan, University of York, December 2000. Last update Jan 2013.

Information


Download

Clipper should be obtained as part of the CCP4 suite. It is distributed under the terms of the CCP4 library license, and as such is free software which may be used without obtaining a CCP4 license.

History

Latest changes:
11/05/10:
Allow read access to a few remaining MTZ header fields.
16/04/10:
Minor compatibility fixes. A lot of extra searching and selection functionality added to MiniMol.
15/12/09:
Update anisotropy correction code to handle I's and low resolution data. Update autoconf and bump so version.
14/07/09:
Update mtz write to give correct symbols. Extend csymmatch to handle waters and to do optional origin and hand matching. Provide legacy mtz column label syntax. gcc 4.4 fixes.
22/05/09:
Enable MiniMol to export alternate conformation codes. Correntions to new Clipper2-stype HKL_data handling.
20/05/09:
Clipper2-stype HKL_data handling extended, including CNS, CIF import. Experimental patch to allow clean shutdown in Coot. Screw operator decomposition in RTop_orth (Phil).
10/02/09:
Support for new extended CNS file import, minor warning fixes.
01/12/08:
2.1 release. Support for threading. New MLHL weighting function. Improved anisotropy correction. gcc 4.3 fixes.
18/11/08:
2.1 release. Support for threading. New MLHL weighting function. Improved anisotropy correction. gcc 4.3 fixes.
07/09/07:
Minor bug fix in Rotation::abs_angle. Avoid crash on reading pdb file with missing CRYST1. Compiler warning fix.
25/04/07:
Work around for compiler bug in gcc 3.2/3.3. No other changes.
12/04/07:
MTZ file creation from scratch fixes. Anisotropic scaling improvements. 'caniso' anisotropic correction tool. 'convert2mtz' file conversion tool. Autoconf build system option.
16/06/06:
Rotation fixes. Backport of ccp4 utilities from 2.0 branch.
31/08/05:
Minimol fix. Additional cfft output.
15/07/05:
Fedora Core 4 NaN fix. New mapfilter method to support NCS in pirate.
17/03/05:
Minor portability fixes for CCP4 release. 'Lattman' spelling fix (sorry).
25/02/05:
Minor portability fixes for CCP4 release.
22/11/04:
Minor portability fixes for CCP4 release.
22/10/04:
Added Fortran sfcalc demo code. Fix in SF weighting 'best' coeffs (from RWGK).
12/10/04:
Fixes and testing of SF calc, SF weighting. Additional cctbx methods. cmakereference app for pirate.
16/07/04:
Add some additional spacegroup methods, extend utilities and add 2 new utilties programs.
25/06/04:
Fix for compilation -O3 on gcc 3.2, other portability fixes, add 1 new jiffy programs.
16/06/04:
Add structure factor calculation with bulk solvent method, sigmaa spline method, add 3 new jiffy programs.
25/05/04:
Add error checking on mtz/map i/o, add origin/hand matching method, add map curvature methods, add 4 new jiffy programs.
06/04/04:
Synchronize to fixes in CCP4 libs.
08/03/04:
Bugfix in MiniMol export to MMDB.
18/02/04:
Bugfix in LogPhaseProb conversion to and from HL coeffs.
03/02/04:
Bugfixes in MiniMol, license update, CCP4v5 install.
27/10/03:
Virtual destructors adjusted for gcc 3.2 bug.
23/10/03:
MTZ History now preserved. Created MTZ's now fixed for common spacegroups. Virtual destructors supplied for all virtual bases (I hope).
08/10/03:
Second attempt at update for new CCP4 v5 libraries.
06/10/03:
Update for new CCP4 v5 libraries.
31/07/03:
New classes for phase probability distributions. Lots of portability fixes.
18/07/03:
License now LGPL (or CCP4 part 0). SCONS now build non-shared libraries on alpha and sun. rpath still not implmented.
07/06/03:
Resolution function tidyup, added SCONS build system.
30/05/03:
Bug fix release for xmap crash and gcc 2.95.3.
17/05/03:
API change from previous release reversed.
14/05/03:
Port Xmap and Spacegroup to ObjectCache architecture for improved performance.
24/04/03:
More documentation. Even more documentation. Documentation tidyup. MiniMol tidyup.
17/04/03:
New MMDB lightweight interface added. New MiniMol STL model added.
10/04/03:
Separated out old MMDB interface as separate package. Started work for ObjectCache architecture. Fixes for Windows build using MinGW.
27/03/03:
Support for new CCP4 C-libraries. New fast NaN tests.
12/03/03:
Fix reflection cif build. Fix to mmdb wrapper segfault.
04/03/03:
Reflection cif import added (Paul Emsley). Preparation for new MTZ interface. Minor updates.
14/02/03:
Tiny MMDB 'is_atom()' fix. Minor updates.
07/02/03:
Ported to Sun. This version tested on gcc 3.2, SGI, Alpha, Sun. Hopefully still builds on MacOSX and gcc 2.96 too.
20/01/03:
Implemented AtomShapeFn, with Agarwal coeffs for isotropic atoms.
10/01/03:
Minor cleanups.
08/01/03:
New FFFear methods, MapFilter methods, generic Euler angles.
10/12/02:
+=, -= implemented for all maps, Vec3<>s.
28/11/02:
Heavy spacegroup testing completed: see cctbxtest.
22/11/02:
Sparse FFT map added.
23/10/02:
Ramachandran class, FFFear methods, new FFTmap backend.
10/10/02:
All files and libraries reorganised, mccp4 and cctbx optional.
07/10/02:
Stand-alone spacegroup library.
30/09/02:
MMDB associative behaviour, MMDB unsorted selections, map statistics objects, map sorting object, PHS file i/o.
30/08/02:
Generic rotation class, least squares coordinate fitting, and lots of stress testing on real calculations.
18/07/02:
Function objects for structure factor calculations, skeletonisation. Message passing library added.
25/06/02:
Slight changes for skeletonisation.
24/06/02:
Anisotropic map calculation implemented (see sftest.cpp). Anisotropic basis function added to resolution function evaluator.
22/05/02:
Audit of communication in hierarchies completed. Minor changes to i/o classes.
08/05/02:
Form factor class added. Demo structure factor calculation added, using either FFT or direct summation. Map bug fixed (for spacegroups where symops do not commute, e.g. P422).
26/04/02:
Rewrite of coordinate types. Reflection datalist operators added.
21/03/02:
More complete MMDB wrapper. New classes for Aniso U's.
16/03/02:
First attempt at MMDB wrapper object. Numerous other cleanups.
05/03/02:
NX_operator, NXmap_operator added. Pluggable interpolators added. Code style imporvements. Resolution function evaluator speedups. New mccp4 library.
12/02/02:
More documentation, cleanups.
09/02/02:
Added lots of conversion operators for structure factor arithmetic and manipulation of HL coeffs. A new mini-ccp4 library includes both mmtzlib and cmaplib. Most importantly, the SGI PORT IS WORKING. There is now a 'fake' configure script which detect the platform. You'll need up to date compilers and gmake.
30/01/02:
Cleanups, more documentation.
23/01/02:
Hopefully finished method renaming, updated fft example.
22/01/02:
Major rewrite of the container hierarchy, including new inheritables (resolution, grid sampling), updated i/o classes, more flexibility, more automation. In particular containers can now be stored in STL vectors.
13/01/02:
New classes for grid samplings (consistent with spacegroup, cell, resolution and FFT), and reciprocal space fractional and orthogonal coordinates. Documentation improvements (now 250 pages).
11/01/02:
Bspline FFTmap class added. Classes for orthogonal and fractional gradients and curvatures and their transformations added. Map I/O class now imports and exports NXmap<>s.
07/01/02:
Map I/O class now imports and exports Xmap<>s. NXmap<>s coming soon. The patched CMAP lib has now been packaged, and the auto-install script updated accordingly.
04/01/02:
Map I/O class started. Map import is working, as long as you have a hacked version of the CMAP lib. I'll tidy this up for distribution as soon as I can.
19/12/01:
More tidyups.
15/12/01:
More tidyups.
13/12/01:
More tidyups: easier MTZ import function, lose the last few pointer methods, fill in some default parameters to allow default name generation for container objects.
08/12/01:
Basic interpolation methods added to Xmap. Documentation improvements.
05/12/01:
Smooth bspline resolution basis function implemented. Lots more cleanups.
04/12/01:
'cfft' works again. Major cleanups. 'HKL_data_base' now contains a full list of virtual functions. Naive binner replaced with something rather more sensible.
03/12/01:
Implemented the strict donor/recipient model for inheriting cell and spacegroup information. Some tidying up will probably be needed. I think cfft is broken - I don't know when this happened yet.
01/12/01:
Iterators and reflection classes renamed to something clearer. Updated to use cctbx 1.0. That should make installation rather simpler.
29/11/01:
The mtz i/o layer has been abstracted from the Clipper core, i.e. Clipper is now completely independent of MTZ (and therefore CCP4). This should make it very simple to interface to other file formats, e.g. CNS, and eventually the PHENIX database.
A sigmaa-target has been implemented for the resolution function evaluator. Convergence is not very fast since it doesn't know how to pick sensible starting values, and it produces copious debugging output, but I think it is working.
01/11/01:
Conversion operators to convert between HKL_data types are implented. (Unary only for now, but binary operators are easy). A sample operator to convert phi_fom to abcd is provided. The reciprocal ASU is now mandatory, as this allows rather faster symmetry lookups.
15/10/01:
Resolution function evaluator implemented. This will evaluate an arbitrary function of HKL, by optimising the parameters of a user defined basis function to minimise a user defined target functions. This completely replaces the idea of 'resolution bins', and does a lot of other things besides. A terget for mean |F|2 is provided, a sigmaa taget shouldn't be too hard.
FFTw config has changed again: FFTw should be configured with just --enable-float
01/10/01:
Do type conversion in fft's. FFTw should be configured with --enable-float --enable-type-prefix
25/09/01:
Update to latest version of cctbx.
10/09/01:
Small tidyups and documentation improvements.
04/09/01:
Various small tidyups and documentation improvements. umtzlib extended to handle multi-record MTZ files.
01/09/01:
MTZ interface completely replaced. MTZ i/o is now handled through a new lightweight C library: umtzlib/mmtzlib (Micro/Mini mtzlib). The mtz i/o interface has also been cleaned up.
16/08/01:
Interface redesign completed. The whole package is now (at least from the outside) written in such a way that programs can be written without using new, delete, or pointers. License changed to CCP4 part (i).
09/08/01:
FFTs implemented! You'll need fftw. The interfaces have been largely redesigned to remove most of the pointers and new/delete. My current priorities are: finishing the interface cleanup, relicencing under CCP4 part (i) (to be consistent with fftw), writing a binner/optimiser, and a non-crystal map class. Long term it would be nice to implement mmdb coordinate objects and density generation (including anisotropic atoms).
26/07/01:
Two big jobs completed: Firstly, the optimised map class is working. Secondly, the package now uses the cctbx libraries for all its symmetry information. You'll need to install cctbx to compile clipper. The Makefile needs to be tweaked by hand for now.
29/06/01:
Map class working (but not optimised, and no interpolation yet). More importantly I've done another major re-architecture. Every object now comes in two forms: a non-tree and a tree variant. You can now use the maps and Datalists as stand-alone components, although for Datalists you will have to manage linking a Datalist to an HKL_list yourself. In future the map class will probably be split up in the same way, providing one object which provides organizational and fast indexing services, from which a number of Data objects can be hung.
20/06/01:
Map class started (It doesn't build yet). Spacegroup extended to include ASU calculation.
01/05/01:
MTZ io re-written to allow extension to other formats in future. Other general tidyups.
25/04/01:
Created a simple `cad' utility to combine and manipulate mtz files as clipper datalists. A few extra methods were provided for this purpose. In the light of this work the MTZ import/export framework is going to be rewritten again.
28/03/01:
Updated to latest CMTZ library.
27/03/01:
The mtz i/o has been rewritten to allow any combination of input, input-append, output, output-append. This should make it trivial to write a clipper-cad utility.
05/03/01:
Namespaces have been implemented. The data types have been templatised as `dtype' so that float or double data may be stored, and selected by a single `using' statement. (i/o to mtz must be float of course). All other values are defined as `ftype', defined in clipper_util, which will normally be double. (This could easily be templatised too).
22/02/01:
Efficient handle class for Datalists implemented. MTZ hierarchical name parsing improved. Recursive update fixed. Doxygen documentation implemented. CMTZ cell export bug fixed (my fault - thanks Martyn).
20/02/01:
Project level removed from both CMTZ and Clipper hierarchy. Minor architechture improvements.
12/02/01:
Mark 2 design now exports as well as imports MTZs.
10/02/01:
CMTZ modified to support hierarchical data naming and duplicate names. `reflection_data_mtz' now updated use the new data structure (import works, export still pending).
07/02/01:
Data structure completely rewritten. The data hierarchy is now a completely generic tree object, so any type can be insterted anywhere. Objects are addressed using UNIX-like directory references. MTZ i/o has not yet been ported to the new structure, but MTZ items will arrive in the hierarchy at '/project/crystal/dataset/datalist', and must be similarly positioned for export.

Things to do

Credits

Lots of people have given help and advice in producing this code. (Unless it is rubbish of course, in which case it is all my own work). They include:
  • Ralph Grosse-Kunstleve
  • Eleanor Dodson
  • Paul Emsley
  • Phil Evans (many useful criticisms)
  • Airlie McCoy (data model)
  • Martyn Winn, Jan Pieter Abrahams (CMTZ stuff)
  • Jan Zelinka (C++ advocate)
And many others.
Back to my YSBL page.

Kevin Cowtan: