ConceptAI & ML
Softmax function
A function that maps arbitrary real-valued vector components to a probability distribution by exponentiating and normalizing.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
1Lectures that assume it
6- Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- How I Learned to Stop Worrying and Love the Transformer
- The Physics of Motivation: An Active Inference Perspective
- Scientific Machine Learning for Electromagnetic Field Computations
- Neuroscience-Inspired Artificial Intelligence
Connections
15The lectures that define it or assume it, and the concepts next to it.
Concepts8
- Dot product
- Exponential function
- Probability distribution
- Attention
- Categorical sampling
- Logits softmax stability shift
Show 2 moreShow fewer
Lectures7
- Natural Language Processing with Deep Learning: Introduction & Word Vectors
- Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- How I Learned to Stop Worrying and Love the Transformer
- The Physics of Motivation: An Active Inference Perspective
- Scientific Machine Learning for Electromagnetic Field Computations