Sign inSign in
ConceptAI & ML

Stochastic gradient descent

An optimization method that updates parameters using the gradient computed from a single randomly selected data point per iteration.
as Suvrit Sra defines itMITThe wording closest to all 12 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

9

Each in their own terms, from the easiest level.

  • Gilbert Strang

    MIT · Undergraduate

    A gradient descent variant that uses a mini-batch of training samples at each step instead of all the data at once.
    • 3:53Accelerating Gradient Descent: Momentum and Nesterov's Method
    • 13:25Structure of Neural Networks for Deep Learning
    • 0:45Backpropagation: Finding Partial Derivatives
  • Fei-Fei Li, Justin Johnson & Serena Yeung

    Stanford University · Undergraduate

    An approximation of gradient descent where the loss and gradient are computed on a small random subset of training examples called a minibatch.
    • 15:23Training Neural Networks II
    • 1:03:58Loss Functions and Optimization
  • Brian Yu

    Harvard University · Undergraduate

    A variation of gradient descent that calculates the gradient based on one data point at a time.
    • 9:21:21Harvard CS50’s Artificial Intelligence with Python – Full University Course
  • Geoffrey Hinton

    Association for Computing Machinery (ACM) · Undergraduate

    Computing how to change connection strengths based on a small batch of examples rather than the whole training set.
    • 10:13The Deep Learning Revolution (Turing Award Lecture)
  • Yaser Abu-Mostafa

    Caltech · Undergraduate

    A randomized version of gradient descent where parameter updates are computed using one randomly chosen training example at a time.
  • Sara Beery

    MIT · Undergraduate

    An approximation of gradient descent where the gradient is computed on a small subset or batch of data rather than the entire dataset.
    • 6:26How to Train a Neural Net
  • David Silver

    Google DeepMind · Graduate

    An incremental gradient descent method that samples individual states to update parameters rather than computing the full expectation over all states.
    • 19:35Value Function Approximation & Deep Q-Networks (DQN)
  • Chris Ré

    Stanford University · Graduate

    An iterative optimization method that estimates the true gradient by computing the gradient on a single randomly sampled example.
    • 22:30Neural Networks I: Feedforward Architectures & Backpropagation

Lectures that assume it

17

The first 12 are shown.

Connections

39

The lectures that define it or assume it, and the concepts next to it.

Concepts10

Show 4 more

Lectures29

Show 23 more

9 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.