ConceptAI & ML
Information gain
Information gain measures the reduction in entropy achieved by partitioning the data along a particular attribute, calculated as the difference between the parent entropy and the expected weighted child entropy.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Entropy2Bandit algorithms1Bayesian experimental design1Conditional entropy1Conditional probability1Decision tree1Intrinsic motivation1Latent states1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Sergey Levine
A measure of the expected reduction in entropy of a latent variable estimate resulting from taking an action and observing an outcome.Karl Friston
Choosing policies that minimize uncertainty and maximize expected information gain about latent states, parameters, or model structures.
Lectures that assume it
2Connections
12The lectures that define it or assume it, and the concepts next to it.
Concepts7
- Entropy
- Conditional entropy
- Conditional probability
- Active inference
- Expected free energy
- Information-directed sampling