An arithmetic standard has to answer two questions that ordinary mathematics never asks. When the true answer is not representable, which representable value do you return? And when there is no sensible answer at all, what happens to the program? IEEE 754 answers both in a way that looks unremarkable until you notice what it implies: the arithmetic never stops, and the record of everything that went wrong is written to a place almost nobody reads.
The idea
The standard makes floating-point arithmetic total. Every operation returns a value, including the ones that are mathematically undefined, and every exceptional condition is recorded by raising a sticky status flag rather than by transferring control. Testing and setting those flags is a required operation, not an extra. So a conforming implementation always has the diagnosis available. The failure is in the layer above: language bindings put the flags behind machinery so awkward that the totality gets used every day and the diagnosis gets used almost never, which is why a NaN that appears halfway through a computation is so hard to trace back to the operation that made it.
Five ways to round
The standard defines five rounding rules. The first two round to a nearest value and the other three are directed roundings. Round to nearest with ties to even resolves a midpoint by picking the neighbor with an even least significant digit. Round to nearest with ties away from zero resolves it by picking the neighbor of larger magnitude. The three directed modes round toward zero, toward positive infinity, and toward negative infinity.
Ties to even is the default for binary floating point and the recommended default for decimal. Ties away from zero is only required for decimal implementations, which is a small clause with a large consequence: a binary implementation can omit an entire rounding mode and still conform, so code that sets it is not portable in the way code that sets the other four is.
Underneath all five sits the requirement that gives them meaning. Unless specified otherwise, the result of an operation is determined by applying the rounding function to the infinitely precise mathematical result, and that requirement is called correct rounding. The implementation is not permitted to compute an approximation and then round it. It must return the value you would get by computing exactly and rounding once. That single sentence is what makes a floating-point addition a well-defined function rather than a hardware detail, and it is the reason the same expression on two conforming machines gives the same bits when the rounding mode and destination format match.
Five things that can go wrong
The standard defines five exceptions. Each one returns a default value and has a corresponding status flag that is raised when the exception occurs. Invalid operation covers the mathematically undefined cases, such as the square root of a negative number, and returns a quiet NaN. Division by zero fires when an operation on finite operands gives an exact infinite result, and returns a signed infinity. Overflow fires when a finite result is too large to be represented accurately, and in the round-to-nearest modes returns a signed infinity. Underflow fires when a result falls outside the normal range. Inexact fires when the exact unrounded result is not representable exactly, and returns the correctly rounded result.
Two of those deserve a second look. Inexact fires constantly. It is raised by ordinary arithmetic on ordinary values, so a program that traps on it would stop immediately, which tells you the flag is designed to be sampled at the end of a computation rather than watched. Underflow carries a genuine subtlety: a subnormal number always implies an underflow exception, but by default, if the result is exact, no flag is raised. The condition and the flag are not the same thing, and the standard is careful to say so.
The set has been stable across editions. These are the same five exceptions defined in IEEE 754-1985, with one broadening: the division by zero exception was extended to operations other than division, which is how log(0) ends up in the same bucket as 1/0.
The default is that nothing happens
This is the design decision the whole model turns on. No other exception handling is required, but additional non-default alternatives are recommended. The standard suggests optional handling in various forms, including presubstitution of user-defined default values and traps that change the flow of control, and then leaves them optional exactly as they were in 1985.
The result is arithmetic that behaves like a value-returning pipeline instead of a control-flow hazard. A NaN propagates through every subsequent operation and arrives at the end of the computation, where a single check can find it. That is precisely the design that errors as values describes in the large: the error is carried in the return value rather than thrown, and the sticky flags are the accumulator that records which errors occurred along the way. What 754 does not do is give the flags a scope. They are global to a thread and monotonically sticky until cleared, which makes them the opposite of the structured handler a language would otherwise reach for.
Where the flags go to die
C is the language that actually built the interface. The floating-point environment is defined as “the set of floating-point status flags and control modes supported by the implementation,” and fenv.h gives you fetestexcept, feclearexcept, fegetround, and fesetround. It is thread-local, and each thread inherits the initial state of its environment from the parent thread, which is the correct semantics for a per-thread accumulator.
Then comes the gate. Access to and modification of the floating-point environment is only meaningful when #pragma STDC FENV_ACCESS is set to ON. Otherwise the implementation is free to assume that the control modes are always the defaults and that the status flags are never tested or modified, which licenses the optimizer to reorder, fold, and vectorize floating-point code in ways that quietly destroy the flag state you were trying to read. And the pragma is not widely implemented. As cppreference puts it, in practice few current compilers, such as HP aCC, Oracle Studio, and IBM XL, support the pragma explicitly, though most allow meaningful access to the environment anyway.
That combination is the whole problem. The standard requires the flags. The language provides an interface to them. The interface is only guaranteed to mean anything under a pragma that most toolchains do not implement, so using it correctly requires knowing what your specific compiler does with floating-point code at your specific optimization level. Almost nobody pays that cost, which is why the standard information about why a computation produced a NaN is sitting in a register that no layer of the stack ever reads.
The practical read
When a training run produces a NaN loss, the invalid and overflow flags on that thread already know which class of operation caused it. The reason you end up bisecting with print statements instead is not that the information is missing. It is that the path from the hardware flag to your program was made conditional on a pragma. That gap is worth remembering the next time you are chasing an exploding gradient by hand.
Scope of this note
The clause text of IEEE 754 is paywalled. What is quoted here comes from the public summary of the standard and from the cppreference documentation of the C floating-point environment. The claim about language support is specific to C and C++ and to what cppreference documents; this note does not survey how other language runtimes expose or suppress the flags. See IEEE 754 as a document for the same caveat applied to formats and conformance.
Related Notes
- IEEE 754 as a Document - the formats, conformance floor, and what the standard leaves to the language
- Numbers, Overflow, and the Edge of the Type - the same edge for integers, where the four answers differ far more
- Errors as Values vs Control Flow - the design tradeoff the non-stop exception model resolves in one direction
- Exceptions, Handlers, and Non-Local Control - what the optional trap mechanism would have looked like if it had been required
- Normative Versus Informative - the difference between a clause that requires a flag and one that recommends a trap
Sources
- “IEEE 754,” Wikipedia. https://en.wikipedia.org/wiki/IEEE_754 . Backs the five rounding rules and their defaults, the correct-rounding requirement, the five exceptions and their default values, the underflow flag subtlety, the extension of division by zero beyond division, the optionality of traps and alternate handling, and status-flag testing as a required operation.
- “Floating-point environment,” cppreference. https://en.cppreference.com/w/c/numeric/fenv . Backs the definition of the floating-point environment, its thread-local inheritance, the FENV_ACCESS pragma gate and what the implementation may assume without it, and the observation that few compilers support the pragma explicitly.