Sign inSign in
ConceptAI & ML

Vocabulary size in transformer models

The total number of unique tokens in the vocabulary, which dictates the size of the token embedding table rows and the final linear layer.
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Linear layer1Token embedding table1Transformer1

Lecturers

1

    Connections

    3

    The lectures that define it or assume it, and the concepts next to it.

    Concepts2

    1 lecturer. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.