|
|
IRIX 6.5 » Books » Developer »
MIPSpro Fortran 90 Programmer's I/O Guide
(document number: 007-3695-006 / published: 2002-11-19)
table of contents | additional info | download
find in page
This chapter provides
an overview of buffering and a description of file buffering as it applies
to I/O.
I/O is the process of transferring data between a program and
an external device. The process of optimizing I/O consists primarily of making
the best possible use of the slowest part of the path between the program
and the device.
The slowest part is usually the physical channel, which is often slower
than the CPU or a memory-to-memory data transfer. The time spent in I/O processing
overhead can reduce the amount of time that a channel can be used, thereby
reducing the effective transfer rate. The biggest factor in maximizing this
channel speed is often the reduction of I/O processing overhead.
A
buffer is a temporary storage location for data while the data
is being transferred. A buffer is often used for the following purposes: Small I/O requests can be collected into a buffer, and the
overhead of making many relatively expensive system calls can be greatly reduced.
A collection buffer of this type can be sized and handled so that the
actual physical I/O requests made to the operating system match the physical
characteristics of the device being used.
Many data file structures, such as the f77
and cos file structures, contain control words. During
the write process, a buffer can be used as a work area where control words
can be inserted into the data stream (a process called
blocking). The blocked data is then written to the device. During
the read process, the same buffer work area can be used to examine and remove
these control words before passing the data on to the user (deblocking
).
When data access is random, the same data may be requested
many times. A cache is a buffer that keeps old
requests in the buffer in case these requests are needed again. A cache that
is sufficiently large and/or efficient can avoid a large part of the physical
I/O by having the data ready in a buffer. When the data is often found in
the cache buffer, it is referred to as having a high hit rate.
For example, if the entire file fits in the cache and the file is present
in the cache, no more physical requests are required to perform the I/O. In
this case, the hit rate is 100%.
Running the disks and the CPU in parallel often improves performance;
therefore, it is useful to keep the CPU busy while data is being moved. To
do this when writing, data can be transferred to the buffer at memory-to-memory
copy speed and an asynchronous I/O request can be made. The control is then
immediately returned to the program, which continues to execute as if the
I/O were complete (a process called write-behind).
A similar process can be used while reading; in this process, data is read
into a buffer before the actual request is issued for it. When it is needed,
it is already in the buffer and can be transferred to the user at very high
speed. This is another form or use of a cache.
The I/O path is divided into two parts. One part includes the
user data area, the library buffer, and the system cache. The second part
is referred to as the logical device
, which includes the ultimate I/O device and all of the buffering,
caching, and processing associated with that device. This includes any caching
in the disk controller and the operating system.
Users can directly or indirectly control some buffers. These include
most library buffers and, to some extent, system cache and ldcache
. Some buffering, such as that performed in the IOS, or the disk
controllers, is not under user control.
A well-formed request requires
the following: The size of the request must be a multiple of the sector size
in bytes. For most disk devices, this will be 4096 bytes.
The data that will be transferred must be located on a word
boundary.
The file must be positioned on a sector boundary. This will
be a 4096-byte sector boundary for most disks.
The following sections briefly describe unbuffered I/O, library buffering,
and system cache buffering.
The simplest form of buffering
is none at all; this unbuffered I/O is known as
raw I/O. For sufficiently large, well-formed requests, buffering
is not necessary; it can add unnecessary overhead and delay. The following assign(1) command specifies unbuffered I/O:
Use the assign command to bypass library buffering
for all well-formed requests. The data is transferred directly between the
user data area and the logical device. Requests that are not well formed use
system cache.
The
term library buffering refers to a buffer that the
I/O library associates with a file. When a file is opened, the I/O library
checks the access, form, and any attributes declared on the assign
or asgcmd(1) command to determine
the type of processing that should be used on the file. Buffers are usually
an integral part of the processing.
If the file is assigned with one of the following options, library buffering
is used: | -s blocked | | -F spec (buffering as
defined by spec) | | -s cos | | -s bin | | -s unblocked |
The -F option specifies flexible file I/O (FFIO),
which uses library buffering if the specifications selected include a need
for some buffering. In some cases, more than one set of buffers might be used
in processing a file.
The operating
system or kernel uses a set of buffers in kernel memory for I/O operations.
These are collectively called the system cache. The
I/O library uses system calls to move data between the user memory space and
the system buffer. The system cache ensures that the actual I/O to the logical
device is well formed, and it tries to remember recent data in order to reduce
physical I/O requests. In many cases, though, it is desirable to bypass the
system cache and to perform I/O directly between the user's memory and the
logical device.
For the assign -s cos and assign -s bin
commands, a library buffer ensures that the actual system calls
are well formed. This is not true for the assign -s u option.
If you plan to bypass the system cache, all requests go through the cache
except those that are well-formed.
See the explanation of the -B option on the assign(1) man page for information about bypassing system
buffering on IRIX systems.
The Fortran I/O library automatically selects default buffer sizes.
These defaults can be overridden with the assign command.
The default buffer sizes are as follows (note that one block is 4096
bytes): | Sequential access, formatted | | Default buffer size is 8 blocks.
| | Sequential access, unformatted | | Default buffer size is 8 blocks.
| | Direct access, formatted | | Default buffer size is 16 blocks.
| | Direct access, unformatted | | Default buffer size is 16 blocks. Four buffers of this size
are allocated.
|
MIPSpro Fortran 90 Programmer's I/O Guide
(document number: 007-3695-006 / published: 2002-11-19)
table of contents | additional info | download
Front Matter
About This Guide
Chapter 1. Introduction
Chapter 2. Standard Fortran I/O
Chapter 3. Fortran I/O Extensions
Chapter 4. Named Pipe Support
Chapter 5. System and C I/O
Chapter 6. The assign Environment
Chapter 7. File Structures
Chapter 8. Buffering
Chapter 9. Introduction to FFIO
Chapter 10. Using FFIO
Chapter 11. Foreign File Conversion
Chapter 12. I/O Optimization
Chapter 13. FFIO Layer Reference
Chapter 14. Creating a user Layer
Glossary
Index
home/search |
what's new |
help
|
|
|
