Floating points are a way of representing numbers based on the idea that the
larger the magnitude of a number, the less we care about knowing its precise
value. If you’re anything like me (until recently) you use floating points regularly in your code
with a rough understanding of what to expect, but don’t understand the specifics
of what floats can and can’t represent. For example what’s the biggest floating
point, or the smallest positive floating point? Or how many times can you add
1.0 to a floating point before something bad happens, and what is that
something bad? How many floating point values are there between 0 and 1? What
about between 1 and 2?
The answer to these questions is, of course, “it depends”, so let’s talk about a
specific floating point standard, the “binary32” type defined in IEEE 754-2008.
This is the commonly-found single-precision floating point type, which
backs the f32
type in rust, and usually backs the float
type in c
(though of course technically this is left unspecified).
From now on this this post, I will refer to this type as simply “float”.
Here’s what a float can represent:
- all powers of 2 from 2^-126 to 2^127
- for each consecutive pair of powers of 2, there are 8,388,607 (that’s 2^23 - 1)
additional numbers, evenly spaced apart
- zero
- infinity
- negative versions of all of the above
The second point is the most important for understanding floating points. Each
successive pair of powers of 2 has 2^23 - 1 floating point values evenly spread
out between them.
There are 2^23 - 1 floats between 0.125 and 0.25, between 1 and 2, between 1024
and 2048, and between 8,388,608 (2^23) and 16,777,216 (2^24). As the numeric
range between consecutive powers of 2 increases, the number of floats between
them stays the same at 2^23 - 1; the floats just get more spread out.
This is the reason that values with lower
magnitudes can be more precisely represented with floating points.
Some implications of this:
- in between 1 and 2, consecutive float values are 2^-23 apart from one
another
- in between 2 and 4, consecutive float values are 2^-22 apart from another
- in between 8,388,608 (2^23) and 16,777,216 (2^24), consecutive float values
are 1 apart from one another
- for each power of 2, there are 8,388,608 (2^23) floats (1 for the power of 2,
and 2^23 - 1 between (exclusive) the power of 2 and the next power of 2)
- the number of positive floats less than 1 is 126 x 2^23 = 1,056,964,608
- there are 126 powers of 2 less than 1
- the number of positive floats greater than or equal to 1 is 128 x 2^23 = 1,073,741,824
- there are 128 powers of 2 greater than or equal to 1 (including 2^0 = 1)
- all floats above 8,388,608 (2^23) are integers
- all floats above 16,777,216 (2^24) are even
- if you attempt to add 1 to 16,777,216 (2^24), the result will be 16,777,216 (2^24)
Here’s how floats are encoded:
Read more...