ConceptSystems & CS
Floating point representation
Floating point numbers are represented as a pair of integers consisting of significant digits and a base-2 exponent.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
2Each in their own terms, from the easiest level.
Albert Reuther
A standardized numerical representation for real numbers consisting of an exponent and a mantissa.
Lectures that assume it
7- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Modern Systems Programming in Pure C: Memory Ownership, APIs & Debugging
- Assembly Language & Computer Architecture
- Accuracy of Mathematical Functions in Julia
- GSQ-RCO: Better Non-Uniform GGUF Quantization
- Data Formats & Encoding Part 2
- Efficient Methods and Hardware for Deep Learning
Connections
16The lectures that define it or assume it, and the concepts next to it.
Concepts7
- Binary representation
- Scientific notation
- Correct rounding
- Fast inverse square root
- Mixed precision training
- Pseudodecimals
Show 1 moreShow fewer
Lectures9
- Practical HPC Architectures & Workflows
- Floats and Approximation Methods
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Modern Systems Programming in Pure C: Memory Ownership, APIs & Debugging
- Assembly Language & Computer Architecture
- Accuracy of Mathematical Functions in Julia