Opening Beyond LLMs: Agents, Emergent Abilities, Intermediate-Guided Reasoning, BabyLM
Beyond LLMs: Agents, Emergent Abilities, Intermediate-Guided Reasoning, BabyLM
- Length
- 1 h
- Level
- Graduate
- Recorded
- Stanford University2023
Chapter 1 of 10: Before the lecture
Moves from phase-transition scaling and chain-of-thought to browser agents that run LLMs as computing chips.
Emergent abilities of large language models appear abruptly at a critical scale and cannot be predicted from smaller models. Feng, Garg and Singh, in session 10 of Stanford CS25: Transformers United, start from Wei et al. on emergence. They then compare intermediate-guided reasoning methods: chain-of-thought, tree-of-thoughts, Socratic questioning and program-aided reasoning with code executors. The BabyLM challenge follows, training small models on roughly 100 million words of child-directed language. The second half sets out agent architectures with five levels of autonomy, API and mouse-and-keyboard control, and a browser agent demonstration. Memory and personalization and multi-agent parallelization come next, ending on the problems of reliability and looping.
Emergent abilities of large language models appear abruptly at a critical scale and cannot be predicted from smaller models. Feng, Garg and Singh, in session 10 of Stanford CS25: Transformers United, start from Wei et al. on emergence. They then compare intermediate-guided reasoning methods: chain-of-thought, tree-of-thoughts, Socratic questioning and program-aided reasoning with code executors. The BabyLM challenge follows, training small models on roughly 100 million words of child-directed language. The second half sets out agent architectures with five levels of autonomy, API and mouse-and-keyboard control, and a browser agent demonstration. Memory and personalization and multi-agent parallelization come next, ending on the problems of reliability and looping.
Emergent Abilities of Large Language Models
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Improving Chain-of-Thought Reasoning
This lecture
Talk 10 of 21 of Transformers United.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
Tree of Thoughts
Socratic Questioning
Program-Aided Language Models
Program of Thoughts
Faith and Fate: Computation Graphs
Gilkerson et al., 2017
Gorilla