ConceptAI & ML
Encoder-decoder architecture
An architecture where an encoder compresses the input into a low-dimensional representation and a decoder reconstructs the output from it.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
4Each in their own terms, from the easiest level.
Brian Yu
An architecture where an encoder encodes an input question into a hidden state, and a decoder decodes that hidden state into the desired output.Hyung Won Chung
An architecture featuring separate encoder and decoder stacks where cross-attention connects the decoder to the final encoder layer output and the output is a sequence.Ashish Vaswani, Lucas Beyer & William Fedus
An architecture where an encoder reads and processes the entire input sequence and a decoder generates the output sequence token by token.
Lectures that assume it
2Connections
16The lectures that define it or assume it, and the concepts next to it.
Concepts10
- Recurrent neural network
- Convolutional neural networks
- Language model
- Language modeling
- LSTM
- Attention mechanism
Show 4 moreShow fewer
Lectures6
- Harvard CS50’s Artificial Intelligence with Python – Full University Course
- Shaping the Future of AI from the History of Transformer Architectures
- Transformers United: DL Models that have revolutionized NLP, CV, RL
- Architectures: Grids
- The Deep Learning Revolution (Turing Award Lecture)
- How I Learned to Stop Worrying and Love the Transformer