Sign inSign in
ConceptAI & ML

Information gain

Information gain measures the reduction in entropy achieved by partitioning the data along a particular attribute, calculated as the difference between the parent entropy and the expected weighted child entropy.
as Pieter Abbeel defines itUC BerkeleyThe wording closest to all 3 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Entropy2Bandit algorithms1Bayesian experimental design1Conditional entropy1Conditional probability1Decision tree1Intrinsic motivation1Latent states1

Opens up

Concepts that name this one as a prerequisite.

Lecturers

3

Each in their own terms, from the easiest level.

  • Sergey Levine

    UC Berkeley · Undergraduate

    A measure of the expected reduction in entropy of a latent variable estimate resulting from taking an action and observing an outcome.
    • 7:08Exploration of advanced topics in deep reinforcement learning
  • Karl Friston

    University College London · Graduate

    Choosing policies that minimize uncertainty and maximize expected information gain about latent states, parameters, or model structures.
    • 51:17The Physics of Motivation: An Active Inference Perspective

Lectures that assume it

2

Connections

12

The lectures that define it or assume it, and the concepts next to it.

3 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.