The following types of optimization may improve I/O performance:

“Enhancing Performance” in Chapter 10, also provides further information about using FFIO to enhance I/O performance.

The following source program changes may affect the I/O performance of a Fortran program:

I/O optimization can often be accomplished by simply addressing I/O speed. The following storage systems are available, ranked in order of speed:

Fast storage systems are expensive and have smaller capacities. You can specify a fast device through FFIO layers and use several FFIO layers to gain the maximum performance benefit from each storage medium. The remainder of this chapter discusses many of these FFIO optimizations. These easy optimizations are frequently those that yield the highest payoffs.

The FFIO cache layer keeps recently used data in fixed size main memory or cache pages in order to reuse the data directly from these buffers in subsequent references. It can be tuned by selecting the number of cache pages and the size of these pages.

The use of the cache layer is especially effective when access to a file is localized to some regions of the whole file. Well-tuned cached I/O can be an order of magnitude faster than the default I/O.

Even when access is sequential, the cache layer can improve the I/O performance. For good performance, use page sizes large enough to hold the largest records.

The cache layers work with the standard Fortran I/O types and the compiler extensions of BUFFER IN/OUT, READMS/WRITMS, and GETWA/PUTWA.

The following assign command requests 100 pages of 42 blocks each:

assign -F cache:42:100 f:filename

Specifying cache pages of 42 blocks matches the track size of a DD-49 disk.

Scratch files are temporary and are deleted when they are closed. To decrease I/O time, move applications' scratch files from user file systems to high-speed file systems.

When optimizing, you should avoid writing the data to disk. This is especially important if most of the data can be held in main memory.

Fortran lets you open a file with STATUS='SCRATCH'. It also lets you close temporary files by using a STATUS='DELETE' . These files are placed on disk, unless the .scr specification for FFIO or the assign -t command is specified for the file. Files specified as assign -t or .scr are deleted when they are closed.

Several FFIO layers automatically enhance I/O performance by performing asynchronous read-ahead and write-behind. These layers include:

If records are accessed sequentially, the cos and bufa layers will automatically and asynchronously pre-read data ahead of the file position currently being accessed. This behavior can be obtained with the cachea layer with an assign option; in that case, the cachea layer will also detect sequential backward access patterns and pre-read in the reverse direction.

Many user codes access the majority of file records sequentially, even with ACCESS='DIRECT' specified. Asynchronous buffering provides maximum performance when:

Use of automatic read-ahead and write-behind may decrease execution time by half because I/O and CPU processing occur in parallel.

The following assign command specifies a specific cachea layer with 10 pages, each the size of a DD-40 track. Three pages of asynchronous read-ahead are requested. The read-ahead is performed when a sequential read access pattern is detected.

assign -F cachea:48:10:3 f:filename

This command would work for a direct access or sequential Fortran file which has unblocked file structure.

Marking records incurs overhead. If a program reads all of the data in any record it accesses and avoids the use of BACKSPACE, you can make some minor performance savings by eliminating the overhead associated with records. This can be done in several ways, depending on the type of I/O and certain other characteristics.

For example, the following assign statements specify the unblocked file structure:

assign -s unblocked f:filename
assign -s u f:filename
assign -s bin f:filename

The libraries generally have default buffer sizes. The default is suitable for many devices, but major performance improvements can result from requesting an efficient buffer size.

The optimal buffer size for very large files is usually a multiple of a device allocation for the disk. This may be the size of a track on the disk. If optimal size buffers are used and the file is contiguous, disk operations are very efficient. Smaller sizes require more than one operation to access all of the information on the allocation or track. Performance does not improve much with buffers larger than the optimal size, unless striping is specified.

When enough main memory is available to hold the entire file, the buffer size can be selected to be as large as the file for maximum performance.

The maximum length of a formatted record depends on the size of the buffer that the I/O library uses for a file. The size of the buffer depends on the following:

After a request is made, the library usually copies data between its own buffers and the user data area. For small requests, this may result in the blocking of many requests into fewer system requests, but for large requests when blocking is not needed, this is inefficient. You can achieve performance gains by bypassing the library buffers and making system requests to the user data directly.

To bypass the library buffers and to specify a direct system interface, use the assign -s u option or specify the FFIO system, or syscall layer, as is shown in the following assign command examples:

assign  -s u  f:filename
assign  -F system  f:filename
assign  -F syscall  f:filename

The user data should be in multiples of the disk sector size (usually 4096 bytes) for best disk I/O performance.

If library buffers are bypassed, the user data should be on a 4096-byte boundary to prevent I/O performance degradation.

When a program produces a large amount of output used only as input to another program consider using pipes. If both programs can run simultaneously, data can flow directly from one to the next by using a pipe. It is unnecessary to write the data to the disk. See Chapter 4, “Named Pipe Support ”, for details about pipes.

Major performance improvements can result from overlapping CPU work and I/O work. This approach can be used in many high-volume applications; it simultaneously uses as many independent devices as possible.

To use this method, start some I/O operations and then immediately begin computational work without waiting for the I/O operations to complete. When the computational work completes, check on the I/O operations; if they are not completed yet, you must wait. To repeat this cycle, start more I/O and begin more computations.

As an example, assume that you must compute a large matrix. Instead of computing the entire matrix and then writing it out, a better approach is to compute one column at a time and to initiate the output of each column immediately after the column is computed. An example of this follows:

     dimension a(1000,2000)
     do 20 jcol= 1,2000
       do 10 i= 1,1000
         a(i,jcol)= sqrt(exp(ranf()))
10     continue
20   continue
     write(1) a
     end

First, try using the assign -F cos.async f:filename command. If this is not fast enough, rewrite the previous program to overlap I/O with CPU work, as follows:

      dimension a(1000,2000)
      do 20 jcol= 1,2000
        do 10 i= 1,1000
          a(i,jcol)= sqrt(exp(ranf()))
10      continue
        BUFFER OUT(1,0) (a(1,jcol),a(1000,jcol) )
20    continue
      end

The following Fortran statements and library routines can return control to the user after initiating I/O without requiring the I/O to complete: