Sign inSign in
ConceptAI & ML

Curriculum-guided layer scaling

A pretraining approach where a model starts at a smaller size and easier data, gradually adding layers throughout training until reaching the final model size on hardest data.
as Steven Feng defines itStanford University
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Language model pretraining1Layer stacking1

Lecturers

1

    Connections

    1

    The lectures that define it or assume it, and the concepts next to it.

    1 lecturer. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.