ConceptAI & ML
Speculative decoding
An approach for faster decoding where a lighter model generates outputs and a heavier model scores and reranks them.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Autoregressive generation2Auto-tuning compiler1Decoder models1Dynamic shapes1Tensor programs1Transformer1
Lecturers
3Each in their own terms, from the easiest level.
Yue Guan
A dynamic shape problem at the operator level that involves complex runtime scheduling at the computation graph level.John Langford
Using a fast approximate model rollout to guess ahead in parallel and verify with a precise model.
Connections
5The lectures that define it or assume it, and the concepts next to it.