Data Integrity Field

Data Integrity Field (DIF) is an approach to protect data integrity in computer data storage from data corruption. It was proposed in 2003 by the T10 subcommittee of the International Committee for Information Technology Standards.^[1] A similar approach for data integrity was added in 2016 to the NVMe 1.2.1 specification.^[2]

Background

Data availability in storage systems is frequently measured simply in terms of the reliability of the hardware components and the effects of redundant hardware. But the reliability of the software, its ability to detect errors, and its ability to correctly report or apply corrective actions to a failure have a significant bearing on the overall storage system availability.

Packet-based storage transport protocols have CRC protection on command and data payloads. Interconnect buses have parity protection. Memory systems have parity detection/correction schemes. I/O protocol controllers at the transport/interconnect boundaries have internal data path protection. But when DIF was first designed, there was no standard protection for when a data leaves the memory, goes to the disk, remain stored for a long time, and gets read again.^[1]

The data exchange usually takes place between the host CPU and storage disk. There may be a storage data controller in between these two. The controller could be RAID controller or simple storage switches.

In SCSI

DIF was first proposed to Small Computer System Interface (SCSI) as the Data Integrity Field. It extends disk sector from its traditional 512 bytes, to 520 bytes, by adding eight additional protection bytes.^[1] SCSI devices are used in many enterprise storage technologies, such as Fibre Channel. By the time it was incorporated in SBC-2, it had already been renamed Protection Information. The 8 bytes include 2 bytes of CRC "logical block guard", 2 bytes of an arbitrary "logical block application guard", and 4 bytes of "logical block reference tag", which is often the lower 32 bits of LBA.^[3]

SBC-3 expands this technology to Advanced Format (4K) sectors, for 4104-byte sectors in the trailing setup. It also defines three ways for the protection to be "on" instead of the previous one, mainly differing in the handling of the 4-byte part. The most important change is the use of a PROTECTION INFORMATION EXPONENT allowing a different way to lay out this information: instead of expanding sectors at their tail, multiple PIs can be packed together and stored elsewhere.^[4][5][6]

The SCSI READ, WRITE, and VERIFY commands allow access to the protection information.

In SATA

The SATA committee (T13) had a setup similar to the SCSI DIF/PI called the External Path Protection (EPP) in 2008.^[7]

In NVMe

Software support

Oracle Corporation contributed support for SCSI/T10 to the Linux kernel.^[8][9] The main documentation on this topic for Linux is written by Oracle.^[10]

• Electronics portal