Record-oriented filesystem

In computer science, a record-oriented filesystem is a file system where data are stored as collections of records. This is in contrast to a stream filesystem, where the data are treated as an unformatted stream of bytes.

"Record-oriented file" redirects here. For this feature in the Distributed Data Management Architecture, see Record-oriented file (DDM).

Record-oriented file systems are abstractions of files kept on paper in earlier times. A record might contain data associated with a particular, e.g., building, contact, employee, part, venue.

There are several different possible record formats; the details vary depending on the particular system. In general the formats can be fixed-length or variable length, with different physical organizations or padding mechanisms; metadata may be associated with the file records to define the record length, or the data may be part of the record. Different access methods for records may be provided, for example records may be retrieved in sequential order, by key, or by record number.

Origin and characteristics

Record-oriented filesystems are frequently associated with mainframe operating systems, such as OS/360 and successors^[1] and DOS/360 and successors, and midrange operating systems, such as RSX-11 and VMS. However, they originated earlier in software such as Input/Output Control System (IOCS).^[2] Records, sometimes called logical records, are often written together in blocks, sometimes called physical records; this is the norm for direct access and tape devices, but files on unit record devices normally have only one record per block, that is, they are unblocked.

In a file in a record file system, a systems analyst designs the records that may be used in a file. All application programs accessing the file, whether adding, reading, or updating records share an understanding of the design of the records. In IBM System/360 operating systems there is no restriction on the bit patterns composing the data record, i.e. there is no delimiter character; this is not always true in other software, e.g., certain record types for RCA File Control Processor (FCP) on the 301, RCA 501, 601 and 3301.^{[citation needed]}

A file comes into existence when a file create request is issued to the filesystem. Information about the file may be included with the create request. This information may specify that the file has fixed-length records (all records are the same size) along with the size of the records. Alternatively, the specification may state that the records are of variable length, along with the maximum record length. Additional information including blocking factor, binary vs. text and the maximum number of records may be specified.

An application may be permitted to read only from beginning of a record; the next sequential read returns the next collection of data (record) that the designer intended to be grouped together. The application may also be permitted to write only at the beginning of a record.

Some operating systems require that library routines specific to the record format be included in the program. This means that a program originally expected to read a variable length record file cannot read a fixed length file. These operating systems must provide file system utilities for converting files between one format and another. This means copying the file (which requires additional storage space, time and coordination) may be necessary. Other systems associate the appropriate routine, based on the file organization, at execution time.

In either case, code to manage records is usually provided in protected routines to ensure file integrity.

A common application convention for text files represented as streams is to use a new line delimiter to separate or terminate records, commonly CR, CRLF or LF. Unfortunately, the CPU time required to parse for the record delimiter is significant and the exclusion of the record delimiter pattern from the data is frequently undesirable.

An alternate convention is to include a length field in each record. The writer application is responsible for imposing any record structure and the reader application is responsible for separating out the records.

Advantages and costs

A record-oriented file has several advantages. After a program writes a collection of data as a record the program that reads that record has the understanding of those data as a collection. Often a file will contain several related records in sequence; after the program reads the beginning of the sequence, the next sequential read returns the next collection of data (record) that the writer intended to be grouped together. Another advantage is that the record has a length and there is usually no restriction on the bit patterns composing the data record, i.e. there is no delimiter character.

A second motivator for the idea of record orientation is that it is in some sense the more natural orientation for persistent storage on a non-volatile but slow physical storage device. Most physical storage devices can communicate only in units of a block. Significant portions of modern operating system kernels and associated device drivers are devoted to hiding the naturally structured and delimited nature of physical storage devices. It is not coincidental that record-oriented file systems arose earlier in the history of computing than byte-stream oriented file systems, when the capabilities for abstraction were far less.

There is usually a cost associated with record-oriented files. For fixed-length records, some records may have unused space, while for variable-length records the delimiter or length field takes up space. Variable-length blocks may have overhead due to delimiters or length fields. In addition, there is overhead imposed by the device. On a magnetic tape overhead typically takes the form of an inter-record gap. On a direct access device with fixed-length sectors, there may be unused space in the last sector of a block. On a direct access device with variable-length physical records, that overhead typically takes the form of metadata and inter-record gaps.

See also

• CMS file system
• Data set (IBM mainframe)
• Distributed Data Management Architecture (DDM)
• Files-11
• ISAM/VSAM
• Job Control Language
• Record-oriented file (DDM)
• Stream (computing)