Entropy
One word, taught differently in 2 fields. Each use is given apart.
In Physics
The negative expected value of the log probability of a distribution, serving as a measure of how random or spread out a random variable is.
In Art & Architecture
A concept from physics representing the state of heat death and loss of organization, used to explain the bleak and decaying nature of contemporary existence and art.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
In Physics
17Each in their own terms, from the easiest level.
Asimina Arvanitaki
A quantity that tells you how much disorder there is in a gas of particles.David Goodstein
The conserved quantity in an ideal engine defined as the ratio of the energy or heat in to the temperature at which it goes in.Ramamurti Shankar
A state variable defined as the change in heat added divided by the temperature in a reversible process.Sean Carroll
A measure of how messy or disorderly things are.Alan Guth
A measure of the number of microscopic quantum states that contribute to a given macroscopic state.Leonard Susskind
Information that is hidden, typically because it is stored in too many degrees of freedom to keep track of.Pieter Abbeel
A measure of uncertainty or expected code length in a probability distribution, calculated as the negative sum of .Catherine Drennan
A measure of the disorder of a system.Joia Mukherjee
The only state of the universe that is sustainable without inputting energy is entropy or chaos.Ahern's Biochemistry #1 - Aqueous Solutions / pH
Disorder or chaos in a system, described here as mixed-upedness.James Mickens
A measure of the randomness or unpredictability of a password, which determines its resistance to guessing.Robert Harle
The expected amount of information you would get from a system, defined as the sum over different options of the probability multiplied by the log of one over the probability.Sergey Levine
The negative expected value of the log probability of x, quantifying how broad a distribution is.Karl Friston
The average surprise or self-information in thermodynamics and physics.Vijay Balasubramanian
The average surprise associated with a probability distribution, defined as the expected value of the negative logarithm of the probabilities of the symbols.Erik Demaine
A probabilistic quantity defined as the sum of probabilities times the log of probabilities, used to formalize uncertainty and information transmission.Ryan O'Donnell
The amount of randomness in a random variable, measured in bits.
In Art & Architecture
1Lectures that assume it
75The first 12 are shown.
- Introduction to Information Theory (University of Cambridge, Lecture 1)
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- The Future of Fundamental Physics: Spacetime and Quantum Amplitudes
- Data Compression and Shannon’s Noiseless Coding Theorem
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- The Linear Model II
- Huffman Coding
- Scaling Down the Laws of Thermodynamics
- Thermodynamics: Now What Happens When You Heat It Up?
- Green HPC: Paving the Way for Sustainable Supercomputing
- The Many Worlds of Quantum Mechanics
- Dynamic Connectivity Lower Bound
Connections
107The lectures that define it or assume it, and the concepts next to it.
Concepts10
Show 4 moreShow fewer
Lectures97
- User Authentication
- Variational Inference, Part 2
- Engine of Nature: The Carnot Engine, Part One
- The Origin of the Universe and the Arrow of Time
- Black-Body Radiation and the Early History of the Universe, Part III
- Me and My Markov Blanket
Show 91 moreShow fewer
- Integer Lower Bounds
- Information Theory: A Sample Lecture
- The Second Law of Thermodynamics and Carnot's Engine
- Entropy
- Thermodynamics: Gibbs Free Energy and Entropy
- The Second Law of Thermodynamics and Entropy
- Kirk Varnedoe on Abstract Art, 1950s 2000s, Part 4: Minimalism and Organicism after 1968
- Deep Reinforcement Learning
- The World As Hologram: Black Hole Complementarity and Hawking Radiation
- Unifying the Cognitive Sciences: The Free Energy Principle
- Entropy: The Carnot Engine and the Flow of Time (The Mechanical Universe, Episode 47)
- Machine Learning: Neural Networks and Decision Trees
- Translational Research and Advocacy
- Introduction to Information Theory
- Curiosity and the Cosmos: CBC IDEAS at Perimeter Institute
- Ahern's Biochemistry #1 - Aqueous Solutions / pH
- Introduction to Information Theory (University of Cambridge, Lecture 1)
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- The Future of Fundamental Physics: Spacetime and Quantum Amplitudes
- Data Compression and Shannon’s Noiseless Coding Theorem
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- The Linear Model II
- Huffman Coding
- Scaling Down the Laws of Thermodynamics
- Thermodynamics: Now What Happens When You Heat It Up?
- Green HPC: Paving the Way for Sustainable Supercomputing
- The Many Worlds of Quantum Mechanics
- Dynamic Connectivity Lower Bound
- The End of the Universe: Astrophysical Scenarios for Cosmic Demise
- Inverse Reinforcement Learning, Part 2
- Entropy Production When Conserved Quantities Fail to Commute
- The Big Picture: Poetic Naturalism from the Core Theory to Consciousness
- The Physics and Philosophy of Time
- String Theory and the Universe: IOP 2010 Newton Medal Lecture
- Transmitting Information Reliably over a Noisy Channel & Shannon’s Noisy Coding Theorem
- Control Hijacking Attacks
- Physical Intelligence: Thermodynamic Roots of Agency
- Main Memory Latency Reduction & Low-Latency DRAM Architectures
- From Voltage-Driven Phase Transition Towards Electrocaloric Devices
- Quantum Darwinism and the Emergence of the Objective, Classical World
- Mutual Information
- Quantum Information Theory
- Quantum "Supremacy"
- A Black Hole Airy Tail: Part 1
- Odd Dynamics of Living Chiral Crystals
- ER = EPR: Wormholes, Horizon Firewalls, and Entanglement
- Emergence of the Classical from within the Quantum Universe
- The Physics of Motivation: An Active Inference Perspective
- Energy and Matter at the Origin of Life
- Enzymes & Metabolism
- Dynamic Optimality I
- Key Events in Evolution
- Succinct Structures II
- Quantum Gravity and Statistical Physics
- Disturbance and Recovery Dynamics in Complex Ecological Systems
- Mathematical Excursions Inspired by Materials Science
- On the Nature of Time According to Modern Physics
- What Is "Quantum" About Quantum Computing?
- Quantum Darwinism and Classical Objectivity: A Collision Model Viewpoint
- Why Does Nature Like the Square Root of Negative One?
- Optimizing Information Processing Using Underdamped Systems
- Inclusive Stochastic Thermodynamics
- Scaling, Criticality, and the Statistical Physics of Biological Networks - Class 1
- Control as Inference, Part 5
- Introduction to Transition Metals
- Potential Energy Surfaces, Transition State Theory, and Reaction Mechanism (CHEM 125, Lecture 37)
- Bayesian Information Engine that Optimally Exploits Noisy Measurements
- Work Trades Off with Complexity in Computationally Restricted Thermodynamics
- Optimal Finite-Time Copying Protocols
- Algorithmic Thermodynamics
- Singularity Theorems, Black Holes, and Conformal Cyclic Cosmology
- Helgoland 2025: Wojciech Zurek
- Deep Inference with Karl J. Friston
- Thermodynamic Consistency of Driven Quantum Optical Master
- Nonequilibrium Thermodynamics of Quantum Coherence Beyond Linear Response
- Solubility and Acid-Base Equilibrium
- Hamiltonian Mechanics, Symplectic Structure & Canonical Transformations
- Statistical Thermodynamics as a Paradigm for Data Science
- Exploration of deep reinforcement learning algorithms
- Exploration of advanced topics in deep reinforcement learning
- Exploration of advanced topics in deep reinforcement learning
- Exploration of advanced topics in deep reinforcement learning
- Quantum Darwinism and Non-Markovian Evolution
- Minimal Architecture for Adaptation in Non-Equilibrium Chemical Signaling Networks
- Sasaki Geometry and Positive Curvature
- Battle of the Big Bang: Alternative Cosmological Theories
- Hitchin Representations of Fuchsian Groups
- Can Quantum Darwinism Arise in Structured Environments?
- Geometric Design Principle of Oscillation-driven Catalytic Molecular Machines
- The Quantum Conspiracy: Re-evaluating Popular Quantum Mechanics Interpretations
- Secrets of the Universe: Minimal Cosmology and the CPT-Symmetric Universe