Concepts
5,907Terms and methods explained by more than one lecturer. Each opens at the minute it is taught.
AI & ML
97- Reinforcement learningA machine learning and psychological approach where an agent learns to make decisions by receiving external rewards or punishments from its environment.
- Supervised learningLearning to predict an output y from an input x given a data set of input-output pairs.
- BackpropagationAn algorithm used to calculate the gradient of a loss function with respect to the weights of a neural network.
- Deep learningA subset of machine learning using hierarchical neural networks to model environments and data.
- HallucinationsInstances where AI models generate factually incorrect or fabricated information.
- Markov decision processA mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
- Unsupervised learningA learning setup where the goal is to learn the underlying hidden structure of unlabeled data.
- Scaling lawsEmpirical power-law relationships predicting model performance as a function of compute, dataset size, and parameter count.
- Differential privacyA mathematical definition of privacy ensuring that an algorithm's output does not reveal whether any single individual's data was included in the dataset.
- Inductive biasThe set of assumptions that a learning algorithm uses to predict outputs for inputs it has not encountered.
- PolicyThe agent's behavior function mapping states to actions.
- Value functionThe expected sum of future rewards starting from state st and following policy pi thereafter.
- BootstrappingUpdating value estimates based on other estimated values rather than final actual outcomes.
- AlignmentThe process of adjusting a pre-trained language model to align with human preferences.
- World modelA component in an AI architecture that predicts future world states resulting from an imagined action sequence.
- VC dimensionA measure of the capacity of a statistical classification model, used in generalization bounds.
- Imitation learningLearning a policy or reward function from expert demonstrations.
- Adversarial examplesInputs to machine learning models that have been intentionally perturbed with small, carefully optimized noise to cause the model to make a misclassification with high confidence.
- Artificial general intelligenceA hypothetical stage of AI development where machines possess human-level intelligence across a broad range of tasks.
- Attention mechanismA mechanism that allows models to focus on particular aspects of the input text by computing weights via queries, keys, and values.
- Kolmogorov complexityThe complexity of something is defined as the length of the shortest explanation or description of that something.
- Rectified linear unitAn activation function that computes the maximum of zero and the input, returning the input if positive and zero otherwise.
- RepresentationA vector embedding representing either a data domain via an encoder function or a single data point.
- RegretThe difference between the reward an agent actually obtained and the reward it could have obtained if it had known the optimal policy from the start.
- Meta-learningLearning to adapt more quickly from past experience of solving other tasks.
- Word embeddingsDense, real-valued vectors representing words in a high-dimensional vector space where similar contexts map to similar vectors.
- Genetic algorithmsA search heuristic that mimics the process of natural selection to generate useful solutions to optimization and search problems.
- Linear probeA technique where a linear projection is trained on top of a frozen representation space to evaluate the quality of learned representations.
- Pre-trainingThe initial stage of training where a model learns a general understanding of the world from large amounts of internet text or private data.
- AutoencoderA neural network architecture that maps input data to a compressed lower-dimensional representation and then decodes it back to reconstruct the original data.
- Self-supervised learningA learning paradigm where systems learn from unlabelled data by predicting missing parts of the input.
- State space modelsA class of models including Mamba that are taking off as alternatives to transformers.
- Credit assignmentThe problem of determining which particular choice or action in a sequence of events precipitated a failure or success outcome.
- DeduplicationThe process of identifying and removing redundant content from a dataset to improve compute efficiency and prevent overfitting.
- Digital twinA computational simulation that assimilates observational data to mimic, predict, and control a physical system in real-time.
- Emergent abilitiesCapabilities that are absent in smaller language models but present in larger ones, appearing suddenly above a certain compute threshold.
- Empirical risk minimizationAn optimization principle in machine learning that trains models by minimizing the error on the training dataset.
- Foundation modelsLarge-scale models trained on vast amounts of data that can be adapted to a wide range of downstream tasks.
- K-means clusteringAn algorithm for clustering data by repeatedly assigning points to clusters and updating the centers of those clusters.
- Membership inference attackDetermining whether a specific data point was part of a model's training set by measuring the model's loss on that point.
- Phase transitionA change in a system where two distinct minimizers of free energy emerge as temperature decreases below a critical value.
- PhonologyThe system of sounds or signs that languages have.
- Post-trainingThe second stage of training where a base model is turned into an assistant by training on conversation datasets.
- Stochastic differential equationA differential equation that includes a stochastic noise term, allowing for modeling of probabilistic dynamics.
- Kernel methodsA class of algorithms for pattern analysis that rely on pre-defined kernel functions to map data into high-dimensional feature spaces.
- AutogradA software library that automatically computes gradients of mathematical functions, essential for training neural networks.
- Bayesian networkGraphical models representing variables and their conditional dependencies via directed acyclic graphs.
- Behavior cloningA form of imitation learning that frames robot control as a supervised learning problem mapping observations to actions.
- Energy-based modelsA class of models that define probability distributions using an unnormalized density function based on an energy function.
- Epistemic uncertaintyUncertainty that captures what the model does not know based on its training data.
- Gradient flowA fast flow on a free energy or a variance lower bound or a marginal likelihood bound.
- LogarithmMathematical operations whose manual calculation can be replaced by tables or built-in functions.
- Lyapunov timeThe characteristic time scale between when a system starts and when it diverges from itself, marking the onset of chaos.
- MarginalizationSumming out variables from a joint distribution to find the probability of a subset of variables.
- Out-of-distribution generalizationThe ability of a machine learning model to correctly classify or perform tasks on inputs that come from a different probability distribution than the training data.
- ResNetA deep neural network architecture that uses skip connections to jump over layers, facilitating the training of very deep networks.
- Speculative decodingAn approach for faster decoding where a lighter model generates outputs and a heavier model scores and reranks them.
- Sufficient statisticA statistic is sufficient for distinguishing two hypotheses if the likelihood ratio depends on the underlying model only through .
- Supervised finetuningTraining a pre-trained model on a smaller, supervised dataset to adapt it for a specific downstream task.
- Universal function approximatorNeural networks are universal function approximators, meaning if you can differentiate it, a neural network can approximate it.
- SurprisalThe negative log probability of an event, representing how unexpected it is.
- Aleatoric uncertaintyStatistical uncertainty inherent in the data-generating process itself, which does not decrease as more data is collected.
- Automatic differentiation (Autodiff)A set of techniques to numerically evaluate the derivative of a function specified by a computer program.
- Belief stateAll the information from the past which is relevant for which future action to take.
- BiasBias measures how far the best possible hypothesis in your hypothesis set is from the target function.
- Bongard problemsA collection of 100 visual puzzle problems created by Mikhail Bongard to test pattern recognition and concept formation.
- CombinatorsThe links that connect nodes in a semantic network.
- Conditional value-at-riskConditional value at risk is the expected loss given that the loss exceeds the value at risk threshold.
- Construct validityWhether an evaluation accurately approximates the abstract construct being targeted.
- Data poisoningA failure mode where evaluation datasets or training methods are artificially constructed to be IID in a way that does not represent real-world deployment.
- DiffeomorphismA smooth bijective map between manifolds whose inverse is also smooth, acting as a symmetry of a dynamical system.
- DP-SGDAn algorithm that applies the Gaussian mechanism to gradients when training deep neural networks on sensitive data.
- Easy problem of consciousnessThe cognitive processing aspects of consciousness, as distinguished from subjective experience.
- Gaussian mechanismA mechanism used for privately approximating a function by adding carefully scaled Gaussian noise.
- Global workspace theoryA theory of consciousness comparing the brain to a theater where conscious contents are broadcast from working memory to unconscious processors.
- Granger causalityIf you can use extra information to improve a prediction, then there's a causal relationship between these two things.
- Influence functionA method to estimate the effect of a data point on a model by considering infinitesimal changes to the training objective.
- Integrated information theoryA theory measuring the interconnectedness and feedback of a system to quantify consciousness using the measure phi.
- JailbreakingA tool or method used to bypass safety guardrails in commercial language models.
- Joint embedding predictive architectureA world model architecture that predicts latent representations instead of raw outputs, focusing on latent structure.
- Linear separabilityThe assumption that data points can be separated by a linear boundary, serving as a simplistic assumption that does not always apply in practice.
- Lipschitz functionA function whose rate of change is bounded everywhere by a constant L.
- Multi-agent reinforcement learningA technique where a large and complex flow system is broken up into sections, and different agents in different sections share information to accelerate training.
- Neural scaling lawsLanguage modeling performance follows a predictable power-law scale with model size in terms of either compute or parameters.
- Phenomenal consciousnessThe second order perception that contents are present and the creation of a sense of present and presence.
- Policy gradient theoremA theorem expressing the policy gradient of a value function as a weighted average of Q-functions.
- Polynomial regressionA form of regression analysis in which the relationship between the independent variable and the dependent variable is modeled as an nth degree polynomial.
- PoolingAn operation that aggregates filter responses over a small region to reduce resolution and build shift and distortion invariance.
- PredictabilityPredictability in science means being able to accurately predict the outcome of an experiment before running it.
- Privacy auditingEmpirically estimating a lower bound on the differential privacy guarantee of a black-box algorithm.
- Probabilistic circuitA classical circuit augmented with coin-flipping gates that output 0 or 1 with probability one half each.
- Riemannian manifoldA smooth manifold equipped with a Riemannian metric that allows for the measurement of distances and angles.
- Scaling hypothesisThe hypothesis that larger data, larger models, and larger compute make AI systems smarter.
- Sparse codingAn unsupervised learning method that represents input data as a sparse linear combination of dictionary elements.
- Spurious correlationsunintended shortcut patterns in data (like hospital markers in medical scans) that models exploit instead of true features.
- System 1 and system 2The expansion of deep learning capabilities from fast, automatic processing to deliberate, reasoning-based processing.
- Translation symmetryA symmetry of space under translation where shifting coordinates by a constant does not change the Lagrangian.