ConceptAI & ML
In-context learning
A technique where language models perform downstream tasks by conditioning on a prompt containing a few input-output examples without updating model parameters.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Language model3Prompting2Auto-regressive language models1Base model1Context window1Few-shot learning1GPT-31Gradient descent1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
6Each in their own terms, from the easiest level.
Ashish Vaswani, Lucas Beyer & William Fedus
The ability of large transformers to respond to their context and learn from it without updating parameters.Andrej Karpathy
The ability of a language model to learn patterns in real time from the provided prompt context without updating its weights.Sara Beery
A technique where a model learns to perform a task by being given examples of that task within the prompt, rather than through parameter updates.Charlie Hou
Using top scoring samples to ask the LLM to generate more data without parameter updates.Phillip Isola
A form of supervised learning where a sequence of input-output example pairs is provided in the prompt, allowing an auto-regressive language model to follow the pattern for a new query.
Lectures that assume it
3Connections
18The lectures that define it or assume it, and the concepts next to it.
Concepts8
Show 2 moreShow fewer
Lectures10
- Deep Dive into Large Language Models: Pretraining to RLHF
- POPRi: Private Federated Learning using Preference-Optimized Synthetic Data
- Transformer Circuits, Induction Heads, In-Context Learning
- Transfer Learning: Models
- Inference Methods for Deep Learning
- Transformers United: DL Models that have revolutionized NLP, CV, RL