Sign inSign in
ConceptAI & ML

Positional encoding

The addition of location information to input representations to break permutation or translation invariance.
as Phillip Isola defines itMITThe wording closest to all 6 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

5

Each in their own terms, from the easiest level.

  • Phillip Isola

    MIT · Undergraduate

  • Brian Yu

    Harvard University · Undergraduate

    A vector added to an input word representation to retain information about the word's position within a sentence.
    • 11:42:51Harvard CS50’s Artificial Intelligence with Python – Full University Course
  • Ava Amini

    MIT · Undergraduate

    A method to encode and capture positional and order information into input embeddings when recurrence is eliminated.
    • 50:01Recurrent Neural Networks, Transformers, and Attention
  • Sara Beery

    MIT · Undergraduate

    Adding location information to the input of convolutional filters to break shift invariance.
  • Steven Feng

    Stanford University · Undergraduate

    Embeddings added to token representations to give the model a notion of sequence ordering.
    • 13:42Overview of Transformers (Stanford CS25: Transformers United V6)

Lectures that assume it

2

Connections

16

The lectures that define it or assume it, and the concepts next to it.

5 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.