Why Learn Floating Point? Every scientific calculation, graphics rendering, and physics simulation relies on floating-point arithmetic. Understanding IEEE 754 representation is critical for numerical accuracy, avoiding precision errors, and designing hardware accelerators for AI and scientific computing.
12 min read | Intermediate Level
What You'll Learn
- Understand IEEE 754 floating-point representation
- Convert between decimal and binary floating-point formats
- Distinguish between single-precision (32-bit) and double-precision (64-bit)
- Recognize common floating-point precision issues
What is a Floating Point Number?
A floating-point number is a way of representing real numbers (including fractions and very large/small values) in binary format. It's called "floating-point" because the decimal point can "float" to different positions, allowing representation of a wide range of values.
Floating-point numbers are essential for representing:
- Scientific measurements (0.0000123 meters)
- Large distances (3.84 × 108 meters to the moon)
- Financial calculations with decimals ($123.45)
- Physics simulations and graphics rendering
IEEE 754 Standard
The IEEE 754 standard defines the format for representing and performing arithmetic on floating-point numbers in computers. It specifies two main types:
- Single-precision (32-bit): 1 sign bit + 8 exponent bits + 23 mantissa bits
- Double-precision (64-bit): 1 sign bit + 11 exponent bits + 52 mantissa bits
Floating-Point Structure
A floating-point number consists of three parts:
| Component | Purpose | Bits (Single) | Bits (Double) |
|---|---|---|---|
| Sign bit (S) | Indicates positive (0) or negative (1) | 1 | 1 |
| Exponent (E) | Scales the magnitude (power of 2) | 8 | 11 |
| Mantissa/Significand (M) | Stores the precision (fractional part) | 23 | 52 |
Formula: Value = (-1)S × M × 2E
Where S is the sign bit, M is the mantissa (with implicit leading 1), and E is the exponent (biased).
Example: Representing 3.5
Let's represent 3.5 in IEEE 754 single-precision format:
- Convert to binary: 3.5 in decimal = 11.1 in binary
- Normalize: 11.1 = 1.11 × 21 (move decimal point left until one digit before it)
- Sign bit: S = 0 (positive number)
- Exponent: Actual exponent is 1, bias is 127, so stored exponent = 1 + 127 = 128 = 10000000
- Mantissa: 1.11 → store only fractional part: 11000000000000000000000 (23 bits)
Sign Exponent Mantissa 0 10000000 11000000000000000000000 Complete 32-bit representation: 0 10000000 11000000000000000000000
The leading 1 in the mantissa is implicit (not stored) in IEEE 754 normalized numbers. This gives an extra bit of precision. So 1.11 is stored as just .11.
Single-Precision Representation (32-bit)
Single-precision floating-point uses 32 bits to represent numbers with approximately 7 decimal digits of precision.
Format Breakdown
| Bits | Field | Description |
|---|---|---|
| 31 | Sign | 0 = positive, 1 = negative |
| 30-23 | Exponent | Biased by 127 (range: -126 to +127) |
| 22-0 | Mantissa | 23 bits of fractional precision |
Range: Approximately 1.4 × 10-45 to 3.4 × 1038
Precision: About 7 decimal digits
Exponent Bias
The exponent uses a bias of 127 for single-precision to represent both positive and negative exponents:
- Stored exponent = Actual exponent + 127
- Example: Actual exponent = -3 → Stored as 124 (01111100)
- Example: Actual exponent = +5 → Stored as 132 (10000100)
The bias allows representing negative exponents without using a sign bit for the exponent. Values 0-126 represent negative exponents, 127 is zero, and 128-255 represent positive exponents.
Converting Decimal to Single-Precision
Example: Convert -13.375 to single-precision format:
- Convert to binary: 13.375 = 1101.011
- Normalize: 1101.011 = 1.101011 × 23
- Sign: 1 (negative)
- Exponent: 3 + 127 = 130 = 10000010
- Mantissa: .101011000000000000000000 (23 bits)
Result: 1 10000010 10101100000000000000000
Graphics and Gaming Single-precision floating-point is heavily used in GPUs for 3D graphics and gaming. Vertex positions, texture coordinates, and lighting calculations all use 32-bit floats. Modern GPUs can perform billions of floating-point operations per second (GFLOPS) for realistic rendering.
Double-Precision Representation (64-bit)
Double-precision provides much greater range and precision than single-precision, using 64 bits total.
Format Breakdown
| Bits | Field | Description |
|---|---|---|
| 63 | Sign | 0 = positive, 1 = negative |
| 62-52 | Exponent | Biased by 1023 (11 bits) |
| 51-0 | Mantissa | 52 bits of fractional precision |
Range: Approximately 2.2 × 10-308 to 1.8 × 10308
Precision: About 15-17 decimal digits
Exponent bias: 1023
Scientific Computing Double-precision is essential in scientific simulations (weather forecasting, molecular dynamics, physics), financial calculations, and engineering analysis where high precision is critical. Most scientific programming languages (Python, MATLAB, R) use double-precision by default.
Special Values in IEEE 754
IEEE 754 defines special bit patterns for exceptional values:
| Value | Exponent | Mantissa | Example Use |
|---|---|---|---|
| Zero | All 0s | All 0s | +0.0 or -0.0 (signed zero) |
| Infinity | All 1s | All 0s | Division by zero, overflow |
| NaN (Not a Number) | All 1s | Non-zero | Invalid operations (0/0, √-1) |
| Denormalized | All 0s | Non-zero | Very small numbers near zero |
NaN (Not a Number) has a unique property: NaN ≠ NaN. This is the only value in computing that's not equal to itself. Always use isNaN() function to check for NaN, never use == comparison.
Common Beginner Mistakes
Mistake #1: Assuming floating-point arithmetic is exact
Wrong: Expecting 0.1 + 0.2 == 0.3 in code
Right: 0.1 + 0.2 = 0.30000000000000004 (close, but not exact)
Why? Many decimal fractions cannot be represented exactly in binary floating-point, leading to tiny rounding errors.
Mistake #2: Forgetting the exponent bias
Wrong: Thinking stored exponent 130 means 2130
Right: Actual exponent = 130 - 127 = 3, so it's 23
Why? The bias allows representing negative exponents without a sign bit.
Mistake #3: Comparing floating-point numbers with ==
Wrong: if (x == 0.3) in code after calculations
Right: if (abs(x - 0.3) < epsilon) where epsilon is a small tolerance (e.g., 0.00001)
Why? Accumulated rounding errors make exact comparisons unreliable.
Mistake #4: Mixing single and double precision carelessly
Wrong: Converting double to float and back, expecting no data loss
Right: Understand that float has less precision; converting back won't recover lost bits
Why? Single-precision has only 23 mantissa bits vs 52 in double-precision.
Quick Check Q: What is the exponent bias for IEEE 754 single-precision floating-point?
Show Answer
A: 127
Explanation: Single-precision uses an 8-bit exponent with bias 127. This allows representing exponents from -126 to +127. The formula is: Actual exponent = Stored exponent - 127.
Example: Stored exponent 128 → Actual exponent = 128 - 127 = 1 (meaning 21)
Summary
In this tutorial, you learned:
- Floating-point numbers represent real numbers using sign, exponent, and mantissa fields
- IEEE 754 defines single-precision (32-bit) and double-precision (64-bit) formats
- Single-precision uses 1+8+23 bits with bias 127, double uses 1+11+52 with bias 1023
- The exponent bias allows representing both positive and negative powers of 2
- Floating-point arithmetic is not exact due to limited precision and binary representation
- Special values (zero, infinity, NaN) handle exceptional cases
- Always use tolerance-based comparison for floating-point equality tests