ConceptAI & ML
Chain of thought
An emergent or learned search process where an LLM writes out intermediate reasoning steps in text before arriving at a final answer.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
4Each in their own terms, from the easiest level.
Andrej Karpathy
Chains of thought in a model's head are an emergent property of optimization that represents the step-by-step reasoning process.Adam Brown
A prompting technique where a large language model is instructed to think step-by-step to arrive at an answer, significantly improving its reasoning performance.Sasha Rush
A method where the model generates intermediate steps in the process of producing an answer.
Lectures that assume it
1Connections
16The lectures that define it or assume it, and the concepts next to it.
Concepts10
- Language model
- Large language models
- Prompt engineering
- Beam search
- Reinforcement learning
- Guess + check (rejection sampling)
Show 4 moreShow fewer
Lectures6
- Speculations on Test-Time Scaling for Large Language Models (o1)
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- Deep Dive into Large Language Models: Pretraining to RLHF
- Training Sand to Think: Artificial General Intelligence & Future of Physics
- Inference Methods for Deep Learning
- Computational Thinking on Learning Models