Concepts
5,907Terms and methods explained by more than one lecturer. Each opens at the minute it is taught.
AI & ML
412The first 400 are shown. Search or choose a field to narrow the list.
- Reinforcement learningA machine learning and psychological approach where an agent learns to make decisions by receiving external rewards or punishments from its environment.
- Machine learningThe field of study that gives computers the ability to learn without being explicitly programmed.
- Gradient descentAn iterative optimization algorithm that updates model parameters by moving in the direction opposite to the gradient of the loss function.
- Supervised learningLearning to predict an output y from an input x given a data set of input-output pairs.
- BackpropagationAn algorithm used to calculate the gradient of a loss function with respect to the weights of a neural network.
- Deep learningA subset of machine learning using hierarchical neural networks to model environments and data.
- StateThe value of a system at a given time.
- Large language modelsComputational models trained on massive text corpora to generate text and understand language.
- HallucinationsInstances where AI models generate factually incorrect or fabricated information.
- Markov decision processA mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
- Unsupervised learningA learning setup where the goal is to learn the underlying hidden structure of unlabeled data.
- OverfittingOverfitting occurs when a model fits the training data too well, capturing noise instead of the underlying pattern, leading to poor generalization on out-of-sample data.
- Stochastic gradient descentAn optimization method that updates parameters using the gradient computed from a single randomly selected data point per iteration.
- Reinforcement learning from human feedbackA technique that trains a reward model directly from human feedback and uses reinforcement learning to optimize an agent's policy.
- Convolutional neural networksNeural network architectures specialized for processing grid-like data such as images using convolutions and subsampling.
- Fine-tuningThe process of taking a pre-trained model and further training it on a smaller, task-specific dataset to improve its performance on that task.
- Scaling lawsEmpirical power-law relationships predicting model performance as a function of compute, dataset size, and parameter count.
- Differential privacyA mathematical definition of privacy ensuring that an algorithm's output does not reveal whether any single individual's data was included in the dataset.
- Inductive biasThe set of assumptions that a learning algorithm uses to predict outputs for inputs it has not encountered.
- IndependenceTwo events are independent if the probability of their intersection equals the product of their individual probabilities.
- PolicyThe agent's behavior function mapping states to actions.
- Value functionThe expected sum of future rewards starting from state st and following policy pi thereafter.
- Bayes' ruleA mathematical formula for calculating conditional probabilities by reversing the conditioning, stating P(a|b) = (P(b|a)P(a))/(P(b)).
- Self-attentionA mechanism that allows a model to weigh the importance of different words in an input sequence relative to a target word.
- Jensen's inequalityAn inequality that relates the value of a convex function of an integral to the integral of the convex function.
- Language modelAn autoregressive model over tokens, mapping a token sequence to a distribution over the next token.
- SoftmaxA function that converts a vector of logits into a probability distribution by exponentiating each value and normalizing by the sum.
- TransformerA neural network architecture that uses self-attention mechanisms to process sequences in parallel, often replacing recurrent networks.
- Turing testA test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.
- Zero-sum gameA mathematical representation of a situation in which each participant's gain or loss of utility is exactly balanced by the corresponding losses or gains of the other participants.
- Markov chainA stochastic process where random variables evolve over time in a way that is one step beyond IID, representing the state of a system moving randomly from state to state.
- BootstrappingUpdating value estimates based on other estimated values rather than final actual outcomes.
- ExplorationThe strategy of trying unknown actions to gather information about the environment's dynamics and rewards.
- GeneralizationThe ability of a learning model to perform accurately on unseen data outside the training set.
- Retrieval-augmented generationRetrieval-augmented generation is a technique that augments a language model by retrieving specific or domain knowledge documents to use as context for generating outputs.
- Active learningA machine learning technique where the model is allowed to choose the data from which it learns, often focusing on instances where it is most uncertain.
- AlignmentThe process of adjusting a pre-trained language model to align with human preferences.
- Diffusion modelsGenerative models that add random noise to data in a forward process and learn a reverse denoising process to generate new samples.
- Recurrent neural networkA type of neural network designed to process sequences by maintaining a hidden state that captures information from previous inputs.
- World modelA component in an AI architecture that predicts future world states resulting from an imagined action sequence.
- RegularizationAdding a term or constraint to the loss function to prevent a model from overfitting to training data.
- Transfer learningUsing representations learned from one task to perform more learning on another task.
- VC dimensionA measure of the capacity of a statistical classification model, used in generalization bounds.
- Markov propertyThe property where the future state is conditionally independent of past states given the current state.
- TokenizationThe process for translating strings or text into sequences of tokens and vice versa.
- Value iterationAn algorithm that uses the Bellman equation as an iterative update to find the optimal Q-function.
- Generative modelModels that learn how to generate all of the data by modeling the joint probability distribution of features and labels.
- Loss functionA function that measures the error of a neural network's predictions compared to the true values, which we want to minimize.
- Maximum likelihood estimationEstimating parameters by maximizing the log-likelihood function of the observed data under a parametric model.
- Stationary distributionA probability distribution over the state space that remains unchanged after one step of the Markov chain transition.
- Bellman equationAn equation expressing the value of a state or state-action pair in terms of the immediate reward and the expected value of subsequent states.
- Generative adversarial networksA generative model that learns to generate data from a training distribution through a two-player game between a generator and a discriminator network.
- Imitation learningLearning a policy or reward function from expert demonstrations.
- In-context learningA technique where language models perform downstream tasks by conditioning on a prompt containing a few input-output examples without updating model parameters.
- Adversarial examplesInputs to machine learning models that have been intentionally perturbed with small, carefully optimized noise to cause the model to make a misclassification with high confidence.
- AttentionA mechanism where weights are dynamically determined as a function of the input data.
- Attention mechanismA mechanism that allows models to focus on particular aspects of the input text by computing weights via queries, keys, and values.
- Deep neural networksAn artificial neural network with multiple layers between the input layer and the output layer.
- Kolmogorov complexityThe complexity of something is defined as the length of the shortest explanation or description of that something.
- Learning rateA constant multiplier that determines the step size taken during gradient descent optimization.
- Natural language processingThe field of processing natural language text computationally, facing challenges due to text's discrete and precise nature.
- OptimizationFinding a set of generator, data, latent space, and inverse generator that is optimal for a given dimension of optimality.
- Rectified linear unitAn activation function that computes the maximum of zero and the input, returning the input if positive and zero otherwise.
- RepresentationA vector embedding representing either a data domain via an encoder function or a single data point.
- Model-based reinforcement learningA reinforcement learning approach where an environment model is learned from data and then used for control or planning.
- Activation functionFunctions used in neural networks to introduce non-linearity into the output of a neuron.
- Cross-entropy lossA loss function for classification that measures the difference between the predicted probability distribution and the true label distribution.
- ExploitationExploit the information you've already found to maximize your reward.
- RegretThe difference between the reward an agent actually obtained and the reward it could have obtained if it had known the optimal policy from the start.
- ReturnThe discounted sum of rewards from the current time step to the end of the horizon.
- Cross-attentionAn attention mechanism where queries come from the decoder output while keys and values come from the encoder output.
- Curse of dimensionalityThe exponential growth in computational complexity and required nodes as the dimension of the state space increases in discretized value iteration.
- Direct preference optimizationA method to optimize language models directly on preference data without training a separate reward model.
- InferenceThe process of using an already trained machine learning model to make predictions or classify new data inputs.
- Meta-learningLearning to adapt more quickly from past experience of solving other tasks.
- ModelA system of abstractions that presents selected parts of reality in order to solve specific problems.
- Occam's razorThe learning principle stating that the simplest model that fits the data is also the most plausible explanation.
- Reward modelA separate neural network trained to imitate human preferences and scores for model rollouts.
- Word embeddingsDense, real-valued vectors representing words in a high-dimensional vector space where similar contexts map to similar vectors.
- CompositionCombining two data objects in a way that is distinct from interpolation.
- Cosine similarityThe angle between two vectors used to measure how similar they are.
- Distribution shiftA change in the data distribution between the training phase and the inference/test phase.
- Genetic algorithmsA search heuristic that mimics the process of natural selection to generate useful solutions to optimization and search problems.
- Linear probeA technique where a linear projection is trained on top of a frozen representation space to evaluate the quality of learned representations.
- Offline reinforcement learningThe task of learning a policy from a fixed historical dataset of interactions without interacting further with the environment.
- Policy evaluationThe process of computing or estimating the value function of a given policy.
- Policy gradientA class of reinforcement learning algorithms that optimize the policy parameters directly by maximizing the expected cumulative reward.
- Supervised fine-tuningTraining stage where the model learns how to behave in a certain way using examples of the harness, tools, and ideal responses.
- Conditional independenceX and Y are conditionally independent given Z if and only if their conditional joint probability equals the product of their individual conditional probabilities given Z.
- Linear classifierA computational mechanism that uses a separating hyperplane in a feature space to distinguish between different categories of objects based on linear separability.
- AutoencoderA neural network architecture that maps input data to a compressed lower-dimensional representation and then decodes it back to reconstruct the original data.
- Chain of thoughtAn emergent or learned search process where an LLM writes out intermediate reasoning steps in text before arriving at a final answer.
- Cross-validationA technique where data is split into multiple folds, cycling through each fold as a validation set while training on the rest.
- HyperparametersChoices about the algorithm that are set rather than learned from training data.
- Self-supervised learningA learning paradigm where systems learn from unlabelled data by predicting missing parts of the input.
- State space modelsA class of models including Mamba that are taking off as alternatives to transformers.
- AccuracyThe ratio of correctly labeled samples divided by the total number of samples.
- Admissible heuristicA heuristic is admissible if it is optimistic, meaning it estimates a cost to the goal that is lower than or equal to the true cost.
- Backpropagation through timeThe standard backpropagation algorithm applied to recurrent neural networks unrolled across all time steps of a sequence.
- BacktrackingThe process of backing up to the place where we last made a decision and choosing another branch when hitting a dead end.
- Chain-of-thought promptingA prompting technique that encourages the model to generate intermediate reasoning steps before arriving at the final answer.
- CLIPA model using contrastive objectives and language supervision to learn generalizable semantic concepts across images.
- Constrained optimizationMinimizing or maximizing a function of multiple variables subject to one or more restrictions or constraints.
- Credit assignmentThe problem of determining which particular choice or action in a sequence of events precipitated a failure or success outcome.
- Digital twinA computational simulation that assimilates observational data to mimic, predict, and control a physical system in real-time.
- EmbeddingUsing a trained neural network to give a low-dimensional representation of input data.
- Emergent abilitiesCapabilities that are absent in smaller language models but present in larger ones, appearing suddenly above a certain compute threshold.
- Empirical risk minimizationAn optimization principle in machine learning that trains models by minimizing the error on the training dataset.
- EncoderThe component that infers the distribution over the latent state Z given the history of observations.
- Episodic memoryDeclarative memory of stuff that happened to you in your life that you can remember and tell other people about.
- Exploration vs. exploitationThe trade-off between choosing known actions that yield high rewards (exploitation) and trying new actions to discover potentially better ones (exploration).
- Foundation modelsLarge-scale models trained on vast amounts of data that can be adapted to a wide range of downstream tasks.
- GeneratorAn element of a group with maximum order that generates the entire group when raised to successive powers.
- Linear interpolationA method of transitioning between two points by moving along a straight line.
- Membership inference attackDetermining whether a specific data point was part of a model's training set by measuring the model's loss on that point.
- Mixture of ExpertsA neural network architecture where different subnetworks (experts) specialize in different inputs, coordinated by a gating network or router.
- Monte Carlo tree searchA search algorithm used in decision-making processes, particularly in games, that builds a search tree by simulating random playouts.
- MultimodalityThe ability of large language models to both generate and perceive multiple modalities, such as text, images, and audio.
- Optimal policyA policy that maximizes the sum of rewards given any state we are in.
- PerceptronA simple hypothesis set that computes a linear combination of input attributes compared against a threshold.
- Phase transitionA change in a system where two distinct minimizers of free energy emerge as temperature decreases below a critical value.
- PruningBuilding a full decision tree first and then working upward from the bottom to remove splits that fail a significance criterion.
- RegressionA supervised learning problem where the target labels are real numbers.
- Residual networkThe final concept needed before introducing algorithms, dealing with the capacity of a cut in flow networks.
- Sample complexityThe number of samples required by an algorithm to achieve a certain level of performance with high probability.
- Stochastic differential equationA differential equation that includes a stochastic noise term, allowing for modeling of probabilistic dynamics.
- The bitter lessonThe historical observation in AI that general approaches leveraging computation via search and learning ultimately outperform methods that manually build human knowledge into agents.
- Transformer architectureA neural network architecture that replaces recurrence with self-attention to process tokens across all time positions simultaneously.
- Active inferenceA theoretical framework for cognition and action suggesting that living organisms act to minimize free energy and confirm their predictions about the world.
- Importance samplingA technique to estimate expected values under a target distribution using samples from a different proposal distribution by weighting them.
- Batch normalizationA technique that explicitly normalizes layer activations to have a zero mean and unit variance over a mini-batch.
- Free energy principleA unifying theoretical framework describing self-organizing systems as minimizing variational free energy to resist the second law of thermodynamics.
- Kernel methodsA class of algorithms for pattern analysis that rely on pre-defined kernel functions to map data into high-dimensional feature spaces.
- KL divergenceA measure of how one probability distribution differs from a second, reference probability distribution.
- Multi-head attentionAn attention mechanism where queries, keys, and values are projected into multiple subspaces to allow the model to jointly attend to information from different representation subspaces in parallel.
- Policy iterationAn algorithm that alternates between evaluating the advantage function for a policy and updating the policy to greedily select actions with highest advantage.
- Pseudo-inverseA generalization of the matrix inverse for non-square or singular matrices that provides a solution to the least squares problem.
- Representation learningThe process of mapping raw data to structured abstract representations.
- Advantage functionThe difference between the Q-function and the value function, representing how much better an action is compared to the average performance of the policy in that state.
- Beam searchA heuristic search algorithm that explores a graph by expanding the most promising nodes in a limited set.
- Contrastive learningA learning technique that maps similar data points to nearby embeddings and dissimilar data points to far-away embeddings.
- Latent variable modelModels that condition neural network outputs on unobserved random variables or seeds to represent broad distributions.
- Logistic regressionA linear model that applies a non-linear logistic function to the linear signal to output probabilities.
- Mechanistic interpretabilityMechanistic interpretability tries to understand how individual components and operations in a machine learning model contribute to its overall decision-making process.
- Moravec's paradoxThe observation that high-level reasoning requires very little computation while low-level sensorimotor skills require immense computational resources.
- Positional embeddingA learned vector representation added to each token embedding to give the transformer information about the token's absolute position in the sequence.
- REINFORCE algorithmA Monte Carlo policy gradient method that samples trajectories, estimates the policy gradient, and updates policy parameters via gradient ascent.
- Variational autoencoderA variational autoencoder is an amortized variational inference model that models an input using a latent variable through an encoder and a decoder.
- Vision transformerA vision transformer is an architecture that applies the transformer encoder to image patches for image recognition tasks.
- AI agentsSystems built around large foundation models that use memory, tools, planning, and actions to autonomously execute tasks for users.
- Associative memoryA gigantic network of ideas linked to each other in various ways such as causes, categories, or instances.
- AutogradA software library that automatically computes gradients of mathematical functions, essential for training neural networks.
- Behavior cloningA form of imitation learning that frames robot control as a supervised learning problem mapping observations to actions.
- Categorical perceptionThe phenomenon where perception is altered by learning categories, making differences between categories more discriminable.
- Certainty equivalentA guaranteed cash flow that an investor would accept as an alternative to an expected risky cash flow.
- CompositionalityThe property where complex systems, representations, or ideas are built by combining simpler pieces together.
- Computation graphA directed graph where nodes represent operations or variables, showing how computations are structured and composed.
- Data augmentationA technique to increase the diversity of data available for training models, such as by applying random transformations to existing images.
- Data parallelismConcurrency that arises from performing the same operation on different pieces of data.
- DiscretizationCasting model weights to a lower-bit precision format like 8-bit to reduce memory usage while preserving performance.
- DistillationA technique where a student model is trained to mimic the predictions or output distributions of a teacher model.
- Energy-based modelsA class of models that define probability distributions using an unnormalized density function based on an energy function.
- EvaluationComputing the values of a polynomial at specific points given its coefficients.
- Experience replayA technique in reinforcement learning where the agent stores its experiences in a memory buffer and samples random batches from it to train the network, breaking the correlation between consecutive samples.
- ExtrapolationThe ability of a model to perform well on inputs or conditions outside the range of the training data.
- Feature vectorA vector of numerical features representing an object or sample in the dataset.
- Gaussian process regressionA nonparametric regression method that uses multivariate normal conditionals and a distance-based covariance function to make predictions and quantify uncertainty.
- Generative AITools that are familiar in the United States and used for automated generation of content.
- Gradient flowA fast flow on a free energy or a variance lower bound or a marginal likelihood bound.
- Hardware-software co-designThe joint design and modification of both computer hardware and software architectures to achieve revolutionary performance improvements.
- Hoeffding's inequalityA fundamental inequality from probability theory that bounds the probability that the sum of bounded independent random variables deviates from its expected value.
- Image CaptioningA task where an AI system takes an image as input and generates a natural language description of that image.
- Information gainInformation gain measures the reduction in entropy achieved by partitioning the data along a particular attribute, calculated as the difference between the parent entropy and the expected weighted child entropy.
- Intrinsic valueThe information gain or epistemic value obtained by acting to resolve uncertainty about hidden states.
- Langevin dynamicsA stochastic process combining gradient updates with added Gaussian noise to model the trajectory of weights in noisy optimization.
- Masked language modelingA pre-training task where a fraction of input sequence tokens are masked, and the model is trained to predict the original masked characters from their positions.
- Model selectionSelecting different models to minimize the complexity of parameters in generative models based on accumulated evidence.
- Multi-layer perceptronA neural network composed of multiple layers of neurons feeding into each other sequentially.
- Neural network layerA layer computes a linear transformation of the input vector via a weight matrix and bias vector, followed by a non-linear activation function.
- Neural network trainingThe process of organizing many things, labeling them into categories, and finding matching images to fill up those categories.
- Objective functionA mathematical function that a learning algorithm seeks to minimize or maximize to optimize model performance.
- Odds ratioMeasure comparing how likely an outcome is given one condition versus another condition.
- Out-of-distribution generalizationThe ability of a machine learning model to correctly classify or perform tasks on inputs that come from a different probability distribution than the training data.
- PackingThe practice of arbitrarily concatenating short contexts to fill out long context windows and avoid waste on padded sequences.
- Partially observable Markov decision processA POMDP is a generalization of a Markov decision process that adds unobserved states, observations, and an observation function where the agent maintains a belief state distribution over states.
- Partition functionA normalization constant that ensures a probability density function integrates to one over the entire space.
- RecallThe proportion of items the user would like that are successfully recommended.
- Rejection samplingA sampling method for conditional probabilities where samples are generated from the joint distribution and those that do not match the evidence are discarded.
- ReLUA non-linear activation function defined as the maximum of zero and the input value.
- RMS normRoot Mean Square normalization scales inputs by their root mean square values without subtracting the mean, mapping data points onto a hypersphere in high dimensions.
- Sampling errorDiscrepancies between generated statistical outputs and expected values due to sample size limitations.
- Scaling lawScaling law dictates that increasing compute and model size predictably improves performance.
- SearchGiven a model and a query, finding the best answer to that query by searching through a space of possibilities.
- SolipsismThe philosophical belief that one's own mind is the only thing that can be known to exist.
- Speculative decodingAn approach for faster decoding where a lighter model generates outputs and a heavier model scores and reranks them.
- State value functionThe expected return starting from state s and then following policy pi.
- Sufficient statisticA statistic is sufficient for distinguishing two hypotheses if the likelihood ratio depends on the underlying model only through .
- Taylor expansionRepresenting a function as an infinite sum of terms calculated from the values of its derivatives at a single point.
- Tensor coresA hardware unit and instruction in Volta GPUs that performs 4x4x4 mixed-precision matrix multiply and accumulate operations in a single clock cycle.
- Test-time trainingAdapting or learning with model parameters during inference time rather than only during the initial training phase.
- Transition functionA function that describes the probability of transitioning to a state S' given state S and action A.
- Transition modelA model that maps a state of the world and an action into some distribution over next states.
- Translation invarianceA property of a system where its physical laws or equations remain unchanged under a shift in position.
- Traveling salesperson problemThe problem of finding the shortest possible route to visit a set of different locales or cities and return to the origin.
- Tree of ThoughtsAn extension of chain of thought that considers multiple different reasoning paths with lookahead and backtracking.
- Visual question answeringA task where a model takes an image and a natural language question as inputs and predicts the correct answer from a set of choices.
- Weight decayA regularization method that adds the squared norm of the weights to the in-sample error to penalize large weights and prevent overfitting.
- Distributional shiftThe problem where the distribution of states encountered during test time differs from the distribution of states seen in the training data.
- Inverse reinforcement learningA problem where the goal is to recover an unknown reward function given observed behavior of an agent.
- Generative modelingThe process of mapping abstract representations to data.
- SurprisalThe negative log probability of an event, representing how unexpected it is.
- System identificationThe classical robotics problem of taking data to identify unknown parameters in a known dynamics model.
- Action spaceThe set of all possible actions that an agent can take from a given state.
- HomophilyA biological phenomenon where similar beings tend to sort of hang out with others of their own kind.
- Model-free reinforcement learningReinforcement learning where the agent does not try to explicitly understand the environment's dynamics, going directly to the policy or value function.
- N-step returnsReturns accumulated over N steps instead of a single step to compute targets for Q-learning.
- Natural policy gradientA policy gradient method that multiplies the regular gradient by the inverse of the Fisher Information Matrix to correctly account for parameter sensitivities.
- Optimal controlThe more general problem of selecting controls that optimize some reward or minimize some cost.
- Principle of optimalityThe principle of optimality states that every sub-plan of an optimal plan is itself optimal.
- Probabilistic modelA model that represents a probability distribution over random variables.
- Rational agentAn agent that selects actions it believes will result in high-utility outcomes.
- Reparameterization trickThe reparameterization trick rewrites a random variable from a parameterized Gaussian distribution as a deterministic function of the parameters and an independent standard Gaussian noise variable, enabling low-variance gradient estimation through backpropagation.
- Sigmoid activation functionA mathematical function that squashes input values into a range between zero and one, historically interpreted as a neuron's firing rate.
- Trust regionA region around the current point where the local quadratic approximation is trusted to be accurate.
- A*An algorithm that combines branch and bound, an extended list, and an admissible heuristic.
- Active matterThe study of physics-based models for emergent behavior in systems like flocks of birds, schools of fish, and swarms.
- Actor-criticAn agent that stores both a policy and a value function to get the benefits of both worlds.
- Actor-critic algorithmAn algorithm that combines policy optimization (actor) with value function estimation (critic).
- Adversarial trainingTraining a neural network on a mix of clean and adversarial examples to make it robust against attacks.
- Agent-to-agent communicationMechanisms and protocols that allow different AI agents to exchange information, requests, and sync their states.
- Agentic AIAn advanced phase where AI autonomously performs tasks such as writing code and requirements.
- AI systemsMulti-component frameworks built around large foundation models using mechanisms like model chaining and reflection.
- AmbiguityThe uncertainty about the outcomes given their causes or states.
- AnimismThe worldview that the world is full of persons, only some of whom are human, and life is lived in relationship with others.
- Approximation errorThe error that occurs when a target function is approximated by the best element in a chosen function class.
- Artificial lifeA field of research exploring artificial agents, systems, and evolution to study life, cognition, and complex systems.
- Automatic differentiation (Autodiff)A set of techniques to numerically evaluate the derivative of a function specified by a computer program.
- BabyLM challengeAn effort to train LLMs using similar amounts of text data that human babies are exposed to while growing up.
- BaselineA value subtracted from the cumulative future reward to reduce variance in the policy gradient estimator.
- Batch gradient descentGradient descent where weight updates are computed using all examples in the training set at once.
- Bayesian inferenceA method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available.
- Bayesian optimizationBayesian optimization uses Gaussian processes to assist in optimizing a function of interest that is opaque and expensive to evaluate.
- Belief distributionThe conditional probability distribution of a hidden state variable given all the evidence collected to date.
- Bongard problemsA collection of 100 visual puzzle problems created by Mikhail Bongard to test pattern recognition and concept formation.
- Bradley-Terry modelA model from economics literature that connects a score or utility function to pairwise preferences using a logistic function.
- CapabilityAn unforgeable token or key that gives the holder the right to perform a specific action or access a resource.
- Cayley graphA graph that graphically represents a group with vertices corresponding to group elements and directed edges representing right-multiplication by generators.
- Closed-loop controlA control strategy where the agent observes a state and produces a policy that maps states to actions dynamically.
- Cognitive mapAn internal mental representation of the spatial layout of an environment that allows for flexible navigation.
- Cognitive scienceA shift away from behavioral science and structuralism towards studying the internal workings and mechanisms of the mind.
- ConfabulationWhen models invent non-existent objects, actions, or facts, also known as hallucinations.
- Construct validityWhether an evaluation accurately approximates the abstract construct being targeted.
- Continual learningInfinite and permanent fundamental self-improvement where systems learn constantly from every new interaction.
- Contrastive lossA loss function used to distinguish data points by pulling positive pairs together and pushing negative pairs apart.
- ControlNetAn auxiliary network added to a base diffusion model to inject structural control signals like poses or edges.
- Convolutional networkA neural network architecture that uses sparse, weight-sharing matrices to perform operations similar to filters on input data.
- Covariance functionA covariance function defines the covariance between the values of a Gaussian process at any two input points.
- Cross entropyA measure of the difference between two probability distributions based on logarithmic expected values.
- CuriosityThe drive to seek information, which is often missing in artificial intelligence research.
- D-separationA criterion to decide if two variables are conditionally independent given a set of evidence variables by checking all undirected paths between them.
- DataData generated by implicit generative models that can be manipulated with special operators like interpolation and composition.
- Data poisoningA failure mode where evaluation datasets or training methods are artificially constructed to be IID in a way that does not represent real-world deployment.
- DenoisingThe process of removing noise from an image, which can be achieved by following the gradient of the score function.
- Density estimationMethods that work on the basis of estimating data distributions to produce kinds of data that humans tend to produce.
- Differential dynamic programmingAn extension of LQR for nonlinear systems that uses second-order dynamics approximations, equivalent to Newton's method.
- Discriminative modelModels like CLIP that compute similarity scores between images and texts rather than generating natural language responses.
- Double descentThe phenomenon where test error initially decreases, then increases, and then decreases again as model capacity increases.
- DP-SGDAn algorithm that applies the Gaussian mechanism to gradients when training deep neural networks on sensitive data.
- Dual normA norm associated with another norm that appears in the dual formulation of the steepest descent update.
- Dynamic time WarpingAn algorithm used to align two or more temporal sequences that may vary in speed or phase.
- Dynamics modelA representation that defines the transition probabilities from a current state and action to subsequent states.
- Easy problem of consciousnessThe cognitive processing aspects of consciousness, as distinguished from subjective experience.
- Effective depthThe depth in the search tree that corresponds to the cost of the optimal solution divided by the minimum arc cost.
- EmpowermentAn intrinsic reward formulation that maximizes the mutual information between an agent's actions and outcomes to maximize control over the world.
- ER=EPRThe conjecture that an Einstein-Rosen bridge (wormhole) is equivalent to quantum entanglement (EPR pairs).
- Euclidean normThe square root of the sum of the squares of a vector's entries, measuring its length from the origin.
- Evidence lower boundA lower bound to the data likelihood under a model, used as an objective in variational autoencoders.
- Expectile lossA loss function that weights positive and negative residuals asymmetrically, allowing estimation of values other than the mean.
- Explanation-based learningA learning paradigm that uses explanations of examples to generalize knowledge, historically related to proof systems in logic and games.
- F1 scoreThe harmonic mean of precision and recall used to prevent systems from gaming either metric alone.
- FaithfulnessA metric measuring how much of the LLM's answer is grounded in the retrieved context rather than hallucinated.
- Few-shot learningAn evaluation setting where models are tested on their ability to learn new classes from very few examples per class.
- Flynn effectThe substantial and long-documented increase in average intelligence test scores over generations.
- Forward modelA mapping from the input x to what would be ideally observed by sensors if there were no noise.
- Function approximationUsing parametric functions to generalize value or policy estimates across large or continuous state and action spaces.
- Gauss-Newton methodAn optimization algorithm that approximates the Hessian of a composite objective function by ignoring the second-order curvature of the model and assuming a squared error loss.
- Gaussian policyA policy for continuous action spaces parameterized by a mean that is a linear combination of features and a variance.
- Gaussian processA method of constructing function spaces by sampling random functions and conditioning them on the training data.
- Global workspace theoryA theory of consciousness comparing the brain to a theater where conscious contents are broadcast from working memory to unconscious processors.
- GlottochronologyA method of estimating the time when two related languages diverged from a common ancestor based on vocabulary changes.
- Goodhart's lawOnce a measure of some quality is turned into a target to optimize, it is no longer a good measure of quality.
- Gram matrixA matrix computed from feature vectors across spatial dimensions that measures co-occurrence statistics of different features.
- Health indexA composite measure evaluating the healthiness of a well by comparing current time periods to known good and bad historical periods.
- Hinge lossA loss function used to push distance computations through to make the optimization landscape smooth and well-behaved.
- Hopfield networkA recurrent neural network that acts as a content-addressable memory by settling into attractor states corresponding to stored patterns.
- Implicit Q-learningAn offline reinforcement learning algorithm that avoids querying out-of-sample actions during updates.
- Influence functionA method to estimate the effect of a data point on a model by considering infinitesimal changes to the training objective.
- Integrated information theoryA theory measuring the interconnectedness and feedback of a system to quantify consciousness using the measure phi.
- Intentional fallacyThe error of judging a literary work based on the author's intended meaning rather than the text itself.
- Intrinsic motivationAn objective function that incentivizes policies to explore novel states without relying on external rewards.
- Invariant measureA probability distribution that remains unchanged under the evolution of the system, acting like a histogram of sampled peaks and minima.
- Iterative LQRAn algorithm that iteratively linearizes dynamics and applies LQR to solve trajectory optimization problems.
- JailbreakingA tool or method used to bypass safety guardrails in commercial language models.
- Join operationAn operation that links a tree rooted at V as a child of W by splitting and concatenating Euler tour BSTs.
- Kernel fusionCombining multiple separate kernels into a single kernel to reduce memory roundtrips and avoid redundant global memory accesses.
- Knowledge distillationA project addressing the difficulty of using massive models by transferring knowledge to smaller models.
- Koopman operatorAn infinite-dimensional linear operator that operates on the Hilbert space of observable functions of a state.
- Language modelingA task where a network reads a sequence of text and learns to predict the next character or word.
- Latent spaceA lower-dimensional space that represents the underlying factors of variation in the data.
- Latent space interpolationFinding a reasonable path between points in a latent space that stays within the trained natural image manifold.
- Latent variablesVariables that correspond to the original training data learned by a generative model.
- Layer normalizationA technique to normalize the inputs of a layer to have a specific mean and variance, often followed by learnable scaling and shifting parameters.
- Leaky abstractionAn abstraction that fails to hide implementation details and instead seeps them, yielding brittle and unwieldy systems.
- Least squares policy iterationAn algorithm that combines policy evaluation via least squares Q-learning with greedy policy improvement for control.
- Least squares temporal-differenceAn algorithm that computes weights directly by solving for the fixed point of temporal-difference targets using least squares.
- Levels of autonomyA scale from level 0 to 5 categorizing the degree of control held by a human versus an automated driving system or agent.
- Linear layerA computational building block that computes a weighted sum of inputs plus a bias term.
- LogitsThe raw, unnormalized output values of a model before applying a softmax function.
- Long-term depressionA mechanism for synaptic weakening in the cerebellum, often triggered by the interaction of climbing fiber and granule cell inputs.
- Long-term memory for agentsPersistent storage mechanisms for AI agents using embeddings and vector databases, analogous to a computer disk.
- Low-rank adaptationA parameter-efficient fine-tuning method where a small set of additional parameters are added to many layers inside an LLM.
- Manifold hypothesisThe assumption that high-dimensional data lies on or near a lower-dimensional manifold.
- MemorizationThe phenomenon that LLMs repeat sections of their training data verbatim when being prompted in the correct way.
- Message passingAn organizational style where data objects store their own operations directly rather than looking them up in a central table.
- MNISTA publicly available database of hand-written digits used as a standard task for testing machine learning algorithms.
- ModalityA type of sensory data or information stream such as vision, touch, or audio.
- Monte Carlo methodsLearning methods that update value estimates by waiting until the end of the episode and using the actual total return.
- Monte Carlo policy evaluationA method to evaluate a policy by running rollouts and summing the rewards obtained along each trajectory.
- Multi-agent reinforcement learningA technique where a large and complex flow system is broken up into sections, and different agents in different sections share information to accelerate training.
- Multi-agent systemSystems composed of multiple interacting intelligent agents that can operate in parallel and specialize in different tasks.
- Multi-class classificationA supervised learning task where the model predicts one of discrete labels for an input .
- Multi-headed attentionUsing multiple self-attention layers simultaneously to allow a model to pay attention to different facets of the input data at the same time.
- Multidimensional scalingThe problem of finding positions in a high-dimensional space given only pairwise distances.
- MutexA mutex (mutual exclusion object) is a synchronization primitive used to protect shared data by allowing only one thread to access the critical section at a time
- MythA widely held but false belief or idea according to the Oxford Dictionary.
- Neural netsA class of machine learning architectures that use stacks of linear transformations interleaved with pointwise nonlinearities.
- Neural scaling lawsLanguage modeling performance follows a predictable power-law scale with model size in terms of either compute or parameters.
- Niche constructionThe process by which organisms modify their own and each other's habitats and niches, as seen in new symbioses.
- Online learningThe requirement that the brain must continuously learn from incoming data in real time without storing raw sensory inputs.
- OptionA way of behaving that terminates, serving as an abstraction for action in reinforcement learning.
- Parameter estimationA statistical modeling objective where variables represent parameters of a model to be estimated from data.
- Perceptual inferenceTesting hypotheses in relation to predictions under those hypotheses in relation to actual sensory evidence.
- Permutation invarianceA property where permuting the adjacency matrix and node attributes of a graph results in the same output function value.
- PerplexityA measure of the quality of a language model's predictions.
- Phenomenal consciousnessThe second order perception that contents are present and the creation of a sense of present and presence.
- Plan divergenceA looping problem where an AI agent diverges from its task, makes mistakes, and continues randomly without knowing how to correct itself.
- PlanningUsing an environmental model to perform look-ahead, compute, and construct a value function or policy.
- Polynomial kernelA kernelization algorithm that outputs an instance of size polynomial in the parameter k.
- PolysemyThe capacity for a word or phrase to have multiple distinct meanings.
- Power lawA mathematical relationship where loudness is proportional to physical intensity raised to a fractional power.
- Power usage effectivenessA metric standing for power usage effectiveness that measures the amount of overhead for power and cooling in a data center.
- Pre-layer normA layer normalization placement where normalization is applied to the residual path before sub-layers rather than after them.
- Prior distributionA probability distribution that expresses one's beliefs about a quantity before some evidence is taken into account.
- Procedural memoryNon-declarative memory of knowing how to do things without being fully aware of or able to explain how it works.
- Projected gradient ascentPeriodically projecting the generated image during optimization onto a nicer set of images with properties like spatial smoothness.
- Proximal policy optimizationA reinforcement learning algorithm used to fine-tune the sampling policy so that model outputs achieve high rewards.
- Quantum reference framesReference frames used by systems to compute next actions relative to some set point or semantics encoded by the frame.
- Quantum theoryA framework where a universe consists of information-encoding states evolving unitarily in universal time with finite action.
- Query vectorA vector formed from the first two elements of a trigram by rotating them to query language profiles.
- RecoverabilityThe existence of an algorithm that guarantees a consistent and unbiased estimate of a target quantity from datasets affected by missing data.
- Reduced SVDA simplified form of singular value decomposition that drops irrelevant information corresponding to zero singular values.
- Reference-free metricsSystem evaluation methods where human judges directly rate system outputs without a gold-standard reference.
- Replay bufferA memory component inside a DQN agent that stores past experiences (state, action, reward, next state) for training.
- Residual neural networkResNet is a residual neural network and these are common types of neural networks used for image classification.
- Reward functionA scalar-valued function of the state and action that indicates which states and actions are better.
- Reward hackingA phenomenon where the specified reward does not provide the behavior that was really hoped to be achieved.
- Reward learningLearning the feature weights of a reward function from expert demonstrations so that optimization or search recovers the demonstrated behavior.
- Reward modelingThe process of reconstructing scalar reward values from large datasets of human choices to reflect human preferences.
- RLHFTrying to approximate a complex distribution by squashing specific parts according to human preference.
- RobustnessThe ability of a model to maintain performance on out-of-distribution or corrupted data.
- Roofline modelA visual model that lets you plot compute throughput of a given hardware against its memory bandwidth.
- Root cause analysisA problem-solving process that helped connect sensor anomalies to stator deformation and cooling system failures.
- ROUGEA family of metrics designed to evaluate the quality of machine generated text, such as summaries.
- Scaling hypothesisThe hypothesis that larger data, larger models, and larger compute make AI systems smarter.
- SeparabilityA property of data indicating whether a hyperplane can completely separate the samples of different classes.
- Shift cipherA substitution cipher where each letter in the plaintext is shifted a fixed number of positions down the alphabet.
- Shortcut learningA phenomenon where machine learning models perform well on a dataset by exploiting simple correlations rather than learning the intended underlying concept.
- Sliding window attentionAn attention mechanism where each token only attends to a local neighborhood of surrounding tokens to increase efficiency.
- Socratic questioningA divide-and-conquer algorithm using large language models to propose subproblems and recursively backtrack to solve a complex original problem.
- Solomonoff inductionA mathematical theory of universal inductive inference based on algorithmic probability and Occam's razor.
- Sparse representationA representation where many components of the vector are zero, typical of ReLU activations in high dimensions.
- SparsificationTechniques to reduce general 3SAT instances to instances with a bounded or sparse number of clauses.
- Statistical errorThe error determined by the statistical procedure used to select a function from a class based on random data.
- Steepest descentAn optimization method where the next iterate moves in the direction of the negative gradient of the objective function.
- Stochastic optimizationOptimization methods that make random decisions, such as genetic algorithms or simulated annealing.
- Stochastic parrotsA characterization of language models as probabilistic text generators without understanding.
- Subjective experienceA way of talking about counterfactuals that explain how our perceptions can go wrong, rather than referring to an inner mental theater.
- Support constraintConstraints that restrict the optimized policy to stay within the support of a base policy rather than enforcing a strict distributional distance.
- Surrogate modelA machine learning model trained on massive HPC simulation datasets to approximate complex physics simulators rapidly.
- Symbolic AIAn approach to artificial intelligence that programs explicit representations and rules containing specific named objects and symbols.
- SynonymySynonyms are words that have the same or similar meaning, though they may not represent the exact same meaning or operationalization.
- System 1 and system 2The expansion of deep learning capabilities from fast, automatic processing to deliberate, reasoning-based processing.
- Tanh activation functionAn activation function that squashes numbers to , is zero-centered, but still kills gradients when saturated.
- Teacher forcingA training technique for sequence models where the ground-truth output from the previous time step is used as the input for the current time step.
- Temporal logicA formal system for representing propositions qualified in terms of time, used in verification to describe liveness properties.
- The Turing testAn empirical test proposed by Alan Turing to replace the ambiguous question 'can a machine think?'
- Thermodynamic equilibriumA state where nothing changes anymore and molecules become the same temperature as the ambient surroundings.
- Threat multiplierClimate change acts as a threat multiplier that makes existing geopolitical and environmental problems worse.
- Topic modelingAn unsupervised machine learning technique used to discover hidden thematic structures in a corpus.
- TransfusionAn architecture that combines autoregressive training on discrete tokens with diffusion on continuous tokens starting from a pre-trained LLM.