Sign inSign in
ConceptAI & ML

Bellman equation

An equation expressing the value of a state or state-action pair in terms of the immediate reward and the expected value of subsequent states.
as Sergey Levine defines itUC BerkeleyThe wording closest to all 7 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

6

Each in their own terms, from the easiest level.

  • Pieter Abbeel

    UC Berkeley · Undergraduate

    Equations that express optimal values or Q-values in terms of other optimal values or Q-values one time step ahead.
    • 1:04:41Markov Decision Processes (MDPs) Part 1/2
    • 20:32Markov Decision Processes (MDPs) Part 2/2
  • David Silver

    Google DeepMind · Undergraduate

    The Bellman equation decomposes the value function recursively into an immediate reward plus the discounted value of the successor state.
    • 29:23Markov Processes, Reward Processes & Bellman Equations
  • Russ Tedrake

    MIT · Undergraduate

    A recursive decomposition of the cost-to-go function expressing the cost from a state as the sum of a one-step cost and the optimal cost-to-go from the next state.
    • 45:16Dynamic Programming I
  • Esther Duflo

    MIT · Undergraduate

    An equation that expresses the value function of a problem as the maximum over possible savings choices of instantaneous utility plus expected future value.
  • Fei-Fei Li, Justin Johnson & Serena Yeung

    Stanford University · Undergraduate

    An identity that relates the optimal Q-value at a state-action pair to the immediate reward and the discounted maximum Q-value of the next state.
    • 12:50Deep Reinforcement Learning

Lectures that assume it

18

The first 12 are shown.

Connections

35

The lectures that define it or assume it, and the concepts next to it.

Concepts10

Show 4 more

Lectures25

Show 19 more

6 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.