ConceptAI & ML
Context window
The maximum number of words that a language model can have to predict the next word in a sequence.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Large language models1Working memory equivalents1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
1Lectures that assume it
8- Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- Deep Dive into Large Language Models: Pretraining to RLHF
- Ship Reliable RAG: Evals That Grow with Your Retrieval Complexity
- Emergent Behaviors in Foundational Models
- Transformers that Transform Well Enough to Support Near-Shallow Architectures
- How Always-On Agents Work
- Overview of Transformers
Connections
15The lectures that define it or assume it, and the concepts next to it.
Concepts6
Lectures9
- Introduction to Large Language Models
- Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- Deep Dive into Large Language Models: Pretraining to RLHF
- Ship Reliable RAG: Evals That Grow with Your Retrieval Complexity
- Emergent Behaviors in Foundational Models