TensorFloat-32

Numbering format in Nvidia hardware From Wikipedia, the free encyclopedia

TensorFloat-32 (TF32) is a numeric floating point format designed for Tensor Core running on certain Nvidia GPUs.

Format

The binary format is:

Thumb

The total 19-bit format fits within a double word (32 bits), and while it lacks precision compared with a normal 32-bit IEEE 754 floating-point number, provides much faster computation, up to 8 times on a A100 (compared to a V100 using FP32).[1]

See also

References

Loading related searches...

Wikiwand - on

Seamless Wikipedia browsing. On steroids.