int foo(int x) { return x + 1 > x; } compiles, at any optimization level worth using, to two instructions: load 1, return. Not a comparison, not an addition. The function is a constant.
Undefined behavior as a contract covers what UB is and why the language has it. This note is about the other half, which is what an optimizer is entitled to do once the contract is on the table. The short version is that the optimizer treats undefined behavior as a fact about your program rather than a hazard in it, and every result below follows mechanically from that.
The idea
A compiler does not detect undefined behavior and then punish you. It assumes UB does not happen, and then propagates that assumption backwards and forwards through the code like any other known fact. This is the crucial asymmetry: an assumption about a later operation constrains what could have been true earlier. Dereferencing a pointer at line 1 makes the pointer non-null at line 1, which makes a null test at line 2 a test on a known value, which makes the branch dead. Nothing checked anything, and nothing was deleted maliciously. A fact was propagated.
The rule that licenses all of it
cppreference’s classification is short: “undefined behavior - There are no restrictions on the behavior of the program.” Not unspecified, which picks one of a set of valid results; not implementation-defined, which requires documentation; not ill-formed, which requires a diagnostic. No restrictions.
The operational consequence is stated on the same page: “Implementations are not required to diagnose undefined behavior (although many simple situations are diagnosed), and the compiled program is not required to do anything meaningful.” And the optimization consequence gets its own heading: “Because correct C++ programs are free of undefined behavior, compilers may produce unexpected results when a program that actually has UB is compiled with optimization enabled.”
Read that sentence as a syllogism rather than a warning. Premise: correct programs contain no UB. Premise: the compiler optimizes correct programs. Conclusion: the compiler may assume, at every point, that no operation ahead of it will be one that has no defined behavior. The assumption is not a heuristic. It is the only thing that makes many ordinary optimizations sound.
Signed overflow, and why it stays undefined
cppreference’s own example annotates return x + 1 > x; as “either true or UB due to signed overflow” and shows the emitted result: mov eax, 1 and ret. Since the only input for which the expression could be false is the one that overflows, and overflow is undefined, the expression is true for every input the compiler must consider.
The LLVM project’s account explains what this is for, and it is not the peephole. “knowing that INT_MAX+1 is undefined allows optimizing” the comparison to true, but the real payoff is loop analysis: with for (i = 0; i <= N; ++i), “the compiler can assume that the loop will iterate exactly N+1 times,” which unlocks unrolling, vectorization, and induction-variable widening. Define overflow and that ends: “if the variable is defined to wrap around on overflow, then the compiler must assume that the loop is possibly infinite,” since N might be INT_MAX.
That is the actual trade, and it is a real one rather than a rationalization. It also has an escape hatch on both ends. “unsigned overflow is guaranteed to be defined as 2’s” complement wrapping, so code that wants wrapping can have it by choosing the type, and “Both Clang and GCC accept the” -fwrapv flag to define signed overflow at the cost of exactly the optimizations above. What the language will not do is give you wrapping semantics and the loop optimizations at once. The full space of choices here is the subject of numbers, overflow, and the edge of the type.
The null check that was not deleted
Null dereference is undefined too, and the LLVM writeup is blunt about how far that goes: “contrary to popular belief, dereferencing a null pointer in C is undefined. It is not defined to trap,” and mapping a page at address zero does not make it defined either.
cppreference shows the consequence directly. Given a function that reads *p and then tests if (!p), it annotates the test with “Either UB above or this branch is never taken” and compiles the whole function to xor eax, eax; ret.
Two optimization orders, one result
The LLVM blog gives the clearest version, “simplified from an exploitable bug that was found in the Linux Kernel.” The function reads
int dead = *P;, then checksif (P == 0) return;, then stores throughP.Run dead-code elimination first and the unused read disappears; the null check is then not redundant and survives, with the comment “Null check not redundant, and is kept.”
Run redundant-null-check elimination first and it reasons that “P was dereferenced by this point, so it can” not be null, so the test becomes
if (false); dead-code elimination then removes the test and the read together, leaving a function that stores throughPwith no check at all.Same source, same standard, two pass orders, two different binaries. The blog’s verdict is the point: both forms are perfectly valid optimized versions of the original, “and both of the optimizations involved are important for the performance of various applications.” Nobody wrote a rule that deletes null checks. The rule that deletes it is the ordinary one that removes a test whose answer is already known.
Two things make this worse in practice than the toy suggests. The passes involved are ordinary and reordering them is normal maintenance, so the same compiler can produce different answers across releases. And “inlining a function often exposes a number of secondary optimization opportunities,” so a check and a dereference written in two different functions, in two different files, can end up adjacent after inlining and interact for the first time in a release build. The pipeline these passes run in, and the representation that makes the reasoning easy, is intermediate representations and SSA.
Why this is a security problem and not a curiosity
The failure mode is specific: the deleted code is usually the check, not the operation. A programmer writes a validation, an optimizer proves the validation always passes because the code below it already assumed so, and the shipped binary has the operation without the guard. The reasoning is sound and the result is an exploitable program.
This is not hypothetical. The LLVM writeup notes that “This problem has bit many projects (including the Linux Kernel, OpenSSL, glibc, etc) and even led to CERT issuing a vulnerability note against GCC,” with the author’s own view that all widely used optimizing C compilers are subject to it rather than GCC alone.
The out-of-bounds case has the same structure and worse consequences. cppreference’s loop annotated “return true in one of the first 4 iterations or UB due to out-of-bounds access” over a four-element table compiles to an unconditional return true, because the only path that reaches the fifth iteration is one that has no behavior. A bounds check written after the access is worth nothing, which is why the exploitation techniques in buffer overflows survive so much defensive code, and why an overflow check that itself overflows is not a check at all, as integer overflow vulnerabilities shows.
The practical rule that falls out of all this is short. Check before you use, never after. Do not compute a value whose validity you intend to test later. And when a check disappears, do not file a compiler bug: read the code above it for the operation that already promised the thing you were about to ask.
Related Notes
- Undefined Behavior as a Contract - what UB is, and why the language keeps it
- Intermediate Representations and SSA - the pass pipeline where the reordering happens
- Buffer Overflows - what an elided bounds check costs in practice
- Integer Overflow Vulnerabilities - the same arithmetic seen from the attacker’s side
- Numbers, Overflow, and the Edge of the Type - the design space between wrapping, trapping, and undefined
- Unsafe Rust and Its Contract - a language that kept the assumptions and confined the places you may make them
Sources
- “Undefined behavior,” cppreference.com. https://en.cppreference.com/w/cpp/language/ub.html . Supports the definition of undefined behavior as having no restrictions on program behavior, implementations not being required to diagnose it, the stated reason optimizers produce unexpected results on programs containing it, and the signed-overflow, out-of-bounds, and null-dereference examples together with the code they compile to.
- Chris Lattner, “What Every C Programmer Should Know About Undefined Behavior #1/3,” LLVM Project Blog, 2011. https://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html . Supports signed overflow licensing the comparison optimization, the loop trip-count assumption and what defining wraparound would cost, unsigned overflow being defined as two’s complement wrapping, the -fwrapv flag in Clang and GCC, and null dereference being undefined rather than defined to trap.
- Chris Lattner, “What Every C Programmer Should Know About Undefined Behavior #2/3,” LLVM Project Blog, 2011. https://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html . Supports the null-check example simplified from an exploitable Linux kernel bug, both pass orderings and their differing output, both forms being valid and both optimizations being important, inlining exposing secondary optimization opportunities, and the list of affected projects with the CERT vulnerability note against GCC.