Re: [ccp4bb]: NFS Performance

[Tim F <fenn@brandeis.edu>]

***  For details on how to be removed from this list visit the  ***
***          CCP4 home page http://www.ccp4.ac.uk         ***


	First off, let me say that I'm also not much of a fortran
coder, so I may be way off base here.  But I have heard this argument
before, so I'll try to shed some light on the problem.

On Wednesday 17 April 2002 11:47, Kay Diederichs wrote:
>
> However, the problem shows when running SOLVE or some components of
> arp/Warp. In both cases, CPU usage drops to below 1% during certain
> I/O situations, and these long-running jobs simply don't advance. It
> appears as if g77-compiled programs trigger some kind of drastic
> performance killer.
>

Well, depending on the version/compilation of g77 you're using, this may
or may not be true.  Several versions of g77 prior to 0.5.20 configured
its version of the libf2c run-time library so that the ALWAYS_FLUSH macro
was defined.  This was done as a result of a belief that many programs
expected output to be flushed to the OS (fflush()) with the hope that
errors would be properly flagged (instead of waiting for subsequent
writes).  Essentially, this difference breaks down to synchronous vs.
ansynchronous error reporting.  The end result (in the case that
ALWAYS_FLUSH is defined) is *reliable and robust* file I/O (errors are
reported immediately) at the cost of speed - something that is especially
noticeable over NFS, and has been duly noted.

Anyway, I've heard of two/three potential solutions to this problem:

	1) make the NFS mount asynchronous (some people think this is a
	Bad Idea)
	2) recompile g77 with the ALWAYS_FLUSH flag disabled.
	3-ish) insert the appropriate CALL FLUSH statements (obviously 
	requires source and a knowledge of its file I/O calls)

This is, of course, assuming that this actually has some bearing on
the problem at hand.  ;)

>
> Maybe someone (Tassos?) could come up with a small piece of code that
> exhibits the problem - in that case it would be possible to
> investigate. But both programs are not open source so it's not easy
> to identify where the problem arises.
>

???  How about just doing several thousand read and/or write calls on
a single variable?  That does a quick and easy job of testing local
vs. network file I/O.


On Wed, 17 Apr 2002, Anastassis Perrakis wrote:

> Being far from a g77/Linux guru I first hope that in preparation for 
> release 5.2 the new g77 will NOT have this problem.

Yes - versions including and post 0.5.2 will not have the ALWAYS_FLUSH
defined (if this is indeed the problem).


On Wed, 17 Apr 2002, Ethan Merritt wrote:
 
>
> I have wondered if the problem is triggered by application programs
> using mmap() to access large files rather than normal read/write/seek
> calls.  I'm not sure how to test this without source code however.
> Does bonnie test mmap access?
> 

Not sure what you mean here... all file access calls go through the
same network layer, neh?

I hope this helps, but let me again stress that I'm not a fortranner - 
I just read the newsgroups.  ;)

	-Tim F

-- 
---------------------------------------------------------

        Tim Fenn
        fenn@brandeis.edu
        Rosenstiel Basic Medical Sciences Research Center
        Brandeis University, Mail Stop 029
        415 South Street
        Waltham, MA 02454
        Phone:  (781) 736-4942
        FAX:  (781) 736-2405

---------------------------------------------------------