Sign inSign in
ConceptAI & ML

Rectified linear unit

An activation function that computes the maximum of zero and the input, returning the input if positive and zero otherwise.
as Brian Yu defines itHarvard UniversityThe wording closest to all 6 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

6

Each in their own terms, from the easiest level.

  • Ian Goodfellow

    Stanford University · Undergraduate

    A deep convolutional rectified linear unit net outputs a piecewise linear function of the input because it uses rectified linear units.
    • 20:26Adversarial Examples and Adversarial Training
  • Pieter Abbeel

    UC Berkeley · Undergraduate

    An activation function that outputs the input directly if it is positive, otherwise, it outputs zero.
    • 58:41Machine Learning: Optimization and Neural Networks
  • Fei-Fei Li, Justin Johnson & Serena Yeung

    Stanford University · Undergraduate

    An activation function computing max⁡(0,x)\max(0, x) that does not saturate in the positive region, is computationally efficient, and converges faster.
    • 13:35Training Neural Networks I
  • Sara Beery

    MIT · Graduate

    A non-linearity defined as the maximum of zero and the input z, widely used as a default choice in neural networks.
    • 33:55Introduction to Deep Learning
  • Yann LeCun

    New York University · Research

    A simple non-linearity function defined as the positive part of its argument.
    • 17:56Learning Representations: A Challenge for Learning Theory, COLT 2013

Connections

12

The lectures that define it or assume it, and the concepts next to it.

6 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.