ConceptAI & ML
Byte pair encoding
An algorithm that iteratively merges the most common pairs of bytes or symbols to create a vocabulary of compressed tokens.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
2Each in their own terms, from the easiest level.
Andrej Karpathy
Sara Beery
A tokenization method that represents text as sequences of character pairs or subword units.
Connections
12The lectures that define it or assume it, and the concepts next to it.
Concepts8
- Tokenization
- Byte
- Data compression
- BPE splitting regex pattern
- Byte Fallback
- Byte pair encoding (BPE) merging