Opening Guest Lecture: Dorsa Sadigh (UC Berkeley CS 285)
Guest Lecture: Dorsa Sadigh (UC Berkeley CS 285)
- Length
- 1 h 2 min
- Level
- Graduate
- Recorded
- UC Berkeley2023
Chapter 1 of 8: Before the lecture
Sadigh treats pairwise preference queries as feature-space hyperplanes and uses language models as rewards.
Interactive learning in robotics draws on human demonstrations, pairwise comparisons and pre-trained models to acquire policies, reward functions and representations. Sadigh, in a guest session of UC Berkeley's CS 285 on deep reinforcement learning, shows how each pairwise comparison defines a separating hyperplane in feature difference space. Active query selection then shrinks the hypothesis volume efficiently. Large language models serve as proxy rewards for qualities such as fairness in negotiating agents, while Voltron pre-trains visual representations on human videos through language-conditioned masked autoencoding. Further examples cover assisted feeding, autonomous driving and in-context trajectory extrapolation, assuming graduate familiarity with reinforcement learning and Markov decision processes.
Interactive learning in robotics draws on human demonstrations, pairwise comparisons and pre-trained models to acquire policies, reward functions and representations. Sadigh, in a guest session of UC Berkeley's CS 285 on deep reinforcement learning, shows how each pairwise comparison defines a separating hyperplane in feature difference space. Active query selection then shrinks the hypothesis volume efficiently. Large language models serve as proxy rewards for qualities such as fairness in negotiating agents, while Voltron pre-trains visual representations on human videos through language-conditioned masked autoencoding. Further examples cover assisted feeding, autonomous driving and in-context trajectory extrapolation, assuming graduate familiarity with reinforcement learning and Markov decision processes.
Basu et al. HRI 17
Kwon et al. HRI 20
Biyik et al. IJRR 21

This lecture
Lecture 73 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
Kwon et al. ICLR 23
Gandhi et al. CoRL 22
Karamcheti et al. RSS 23
Mirchandani et al. CoRL 23
Sadigh et al. RSS17
Biyik et al. CoRL18
Biyik et al. CDC19
Palan et al. RSS19
Basu et al. IROS19
Biyik et al. RSS20
Myers et al. CoRL21
Biyik et al. ICRA22
ROIAL: Region of Active Learning for Characterizing Exoskeleton Gait Preference Landscapes
Deal or no deal? End-to-end learning for negotiation dialogues.
Targeted Data Acquisition for Evolving Negotiation Agents
Language to Rewards for Robotic Skill Synthesis
Karamcheti et al. RSS23
Minhandani et al. CoRL23
On the Opportunities and Risks of Foundation Models
Language-Driven Representation Learning for Robotics
Reward Design
Commonsense Reasoning
Semantic Manipulation
Teaching Humans
Pattern Machines
Learning Human Preferences
Foundation Models for Robotics
Neural reward work