Sign inSign in
ConceptAI & ML

Masked language modeling

A pre-training task where a fraction of input sequence tokens are masked, and the model is trained to predict the original masked characters from their positions.
as Drew Jaegle defines itStanford UniversityThe wording closest to all 3 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

3

Each in their own terms, from the easiest level.

  • Ashish Vaswani, Lucas Beyer & William Fedus

    Stanford University · Undergraduate

    A pre-training task where specific words in a sequence are replaced with placeholders, and the model is trained to predict the original words based on the surrounding context.
    • 21:03Transformers United: DL Models that have revolutionized NLP, CV, RL
  • Ming Ding

    Stanford University · Undergraduate

    Masked language modeling is a pretraining task good at understanding text by predicting masked tokens.
    • 4:30From Large Language Models to Large Multimodal Models

Lectures that assume it

2

Connections

10

The lectures that define it or assume it, and the concepts next to it.

3 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.