ConceptAI & ML
Positional encoding
The addition of location information to input representations to break permutation or translation invariance.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
5Each in their own terms, from the easiest level.
Phillip Isola
Brian Yu
A vector added to an input word representation to retain information about the word's position within a sentence.Ava Amini
A method to encode and capture positional and order information into input embeddings when recurrence is eliminated.Sara Beery
Adding location information to the input of convolutional filters to break shift invariance.Steven Feng
Embeddings added to token representations to give the model a notion of sequence ordering.
Lectures that assume it
2Connections
16The lectures that define it or assume it, and the concepts next to it.
Concepts8
- Embedding
- Vector
- Convolutional neural networks
- Coordinate system
- 2D Fourier feature positional encodings
- Modality-specific positional encodings