Concepts
5,907Terms and methods explained by more than one lecturer. Each opens at the minute it is taught.
AI & ML
586The first 400 are shown. Search or choose a field to narrow the list.
- Reinforcement learningA machine learning and psychological approach where an agent learns to make decisions by receiving external rewards or punishments from its environment.
- Machine learningThe field of study that gives computers the ability to learn without being explicitly programmed.
- Gradient descentAn iterative optimization algorithm that updates model parameters by moving in the direction opposite to the gradient of the loss function.
- Supervised learningLearning to predict an output y from an input x given a data set of input-output pairs.
- Chain ruleA formula for computing the derivative of the composition of two or more functions.
- BackpropagationAn algorithm used to calculate the gradient of a loss function with respect to the weights of a neural network.
- Deep learningA subset of machine learning using hierarchical neural networks to model environments and data.
- StateThe value of a system at a given time.
- Large language modelsComputational models trained on massive text corpora to generate text and understand language.
- HallucinationsInstances where AI models generate factually incorrect or fabricated information.
- Markov decision processA mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
- Unsupervised learningA learning setup where the goal is to learn the underlying hidden structure of unlabeled data.
- OverfittingOverfitting occurs when a model fits the training data too well, capturing noise instead of the underlying pattern, leading to poor generalization on out-of-sample data.
- Stochastic gradient descentAn optimization method that updates parameters using the gradient computed from a single randomly selected data point per iteration.
- Reinforcement learning from human feedbackA technique that trains a reward model directly from human feedback and uses reinforcement learning to optimize an agent's policy.
- Convolutional neural networksNeural network architectures specialized for processing grid-like data such as images using convolutions and subsampling.
- Fine-tuningThe process of taking a pre-trained model and further training it on a smaller, task-specific dataset to improve its performance on that task.
- Scaling lawsEmpirical power-law relationships predicting model performance as a function of compute, dataset size, and parameter count.
- Artificial intelligenceThe simulation of human intelligence in machines that are programmed to think like humans as best as possible and mimic human action.
- Inductive biasThe set of assumptions that a learning algorithm uses to predict outputs for inputs it has not encountered.
- IndependenceTwo events are independent if the probability of their intersection equals the product of their individual probabilities.
- PolicyThe agent's behavior function mapping states to actions.
- Value functionThe expected sum of future rewards starting from state st and following policy pi thereafter.
- Bayes' ruleA mathematical formula for calculating conditional probabilities by reversing the conditioning, stating P(a|b) = (P(b|a)P(a))/(P(b)).
- Self-attentionA mechanism that allows a model to weigh the importance of different words in an input sequence relative to a target word.
- Jensen's inequalityAn inequality that relates the value of a convex function of an integral to the integral of the convex function.
- Language modelAn autoregressive model over tokens, mapping a token sequence to a distribution over the next token.
- SoftmaxA function that converts a vector of logits into a probability distribution by exponentiating each value and normalizing by the sum.
- TransformerA neural network architecture that uses self-attention mechanisms to process sequences in parallel, often replacing recurrent networks.
- Turing testA test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.
- Zero-sum gameA mathematical representation of a situation in which each participant's gain or loss of utility is exactly balanced by the corresponding losses or gains of the other participants.
- Conditional probabilityThe probability of an event X occurring given that another event Y has already occurred.
- Markov chainA stochastic process where random variables evolve over time in a way that is one step beyond IID, representing the state of a system moving randomly from state to state.
- Product ruleA rule stating that the joint probability of two variables is equal to the product of a marginal distribution and a conditional distribution.
- BootstrappingUpdating value estimates based on other estimated values rather than final actual outcomes.
- ExplorationThe strategy of trying unknown actions to gather information about the environment's dynamics and rewards.
- GeneralizationThe ability of a learning model to perform accurately on unseen data outside the training set.
- Retrieval-augmented generationRetrieval-augmented generation is a technique that augments a language model by retrieving specific or domain knowledge documents to use as context for generating outputs.
- SamplingThe procedure involving a sampling technique and estimated sample size.
- Active learningA machine learning technique where the model is allowed to choose the data from which it learns, often focusing on instances where it is most uncertain.
- AlignmentThe process of adjusting a pre-trained language model to align with human preferences.
- Diffusion modelsGenerative models that add random noise to data in a forward process and learn a reverse denoising process to generate new samples.
- Recurrent neural networkA type of neural network designed to process sequences by maintaining a hidden state that captures information from previous inputs.
- World modelA component in an AI architecture that predicts future world states resulting from an imagined action sequence.
- RegularizationAdding a term or constraint to the loss function to prevent a model from overfitting to training data.
- Transfer learningUsing representations learned from one task to perform more learning on another task.
- VC dimensionA measure of the capacity of a statistical classification model, used in generalization bounds.
- Markov propertyThe property where the future state is conditionally independent of past states given the current state.
- TokenizationThe process for translating strings or text into sequences of tokens and vice versa.
- Value iterationAn algorithm that uses the Bellman equation as an iterative update to find the optimal Q-function.
- Breadth-first searchAn algorithm for traversing or searching tree or graph data structures that explores neighbor nodes first, before moving to the next level neighbors.
- Generative modelModels that learn how to generate all of the data by modeling the joint probability distribution of features and labels.
- Loss functionA function that measures the error of a neural network's predictions compared to the true values, which we want to minimize.
- Maximum likelihood estimationEstimating parameters by maximizing the log-likelihood function of the observed data under a parametric model.
- Stationary distributionA probability distribution over the state space that remains unchanged after one step of the Markov chain transition.
- Bellman equationAn equation expressing the value of a state or state-action pair in terms of the immediate reward and the expected value of subsequent states.
- Generative adversarial networksA generative model that learns to generate data from a training distribution through a two-player game between a generator and a discriminator network.
- HeuristicsRules of thumb or tricks used by a computer program to guide decision-making and problem-solving.
- Imitation learningLearning a policy or reward function from expert demonstrations.
- In-context learningA technique where language models perform downstream tasks by conditioning on a prompt containing a few input-output examples without updating model parameters.
- Adversarial examplesInputs to machine learning models that have been intentionally perturbed with small, carefully optimized noise to cause the model to make a misclassification with high confidence.
- Artificial general intelligenceA hypothetical stage of AI development where machines possess human-level intelligence across a broad range of tasks.
- AttentionA mechanism where weights are dynamically determined as a function of the input data.
- Attention mechanismA mechanism that allows models to focus on particular aspects of the input text by computing weights via queries, keys, and values.
- Deep neural networksAn artificial neural network with multiple layers between the input layer and the output layer.
- Kolmogorov complexityThe complexity of something is defined as the length of the shortest explanation or description of that something.
- Learning rateA constant multiplier that determines the step size taken during gradient descent optimization.
- Natural language processingThe field of processing natural language text computationally, facing challenges due to text's discrete and precise nature.
- OptimizationFinding a set of generator, data, latent space, and inverse generator that is optimal for a given dimension of optimality.
- Rectified linear unitAn activation function that computes the maximum of zero and the input, returning the input if positive and zero otherwise.
- RepresentationA vector embedding representing either a data domain via an encoder function or a single data point.
- State spaceThe set of all possible states that a stochastic process or system can occupy.
- Support vector machinesA supervised learning classification approach that finds a decision boundary maximizing the distance between different groups of data points.
- Model-based reinforcement learningA reinforcement learning approach where an environment model is learned from data and then used for control or planning.
- Activation functionFunctions used in neural networks to introduce non-linearity into the output of a neuron.
- Cross-entropy lossA loss function for classification that measures the difference between the predicted probability distribution and the true label distribution.
- ExploitationExploit the information you've already found to maximize your reward.
- RegretThe difference between the reward an agent actually obtained and the reward it could have obtained if it had known the optimal policy from the start.
- ReturnThe discounted sum of rewards from the current time step to the end of the horizon.
- Alpha-beta pruningAn optimized search algorithm for minimax that eliminates branches in the search tree that do not need to be evaluated.
- Cross-attentionAn attention mechanism where queries come from the decoder output while keys and values come from the encoder output.
- Curse of dimensionalityThe exponential growth in computational complexity and required nodes as the dimension of the state space increases in discretized value iteration.
- Depth-first searchA search strategy that explores as far as possible along each branch before backtracking.
- Direct preference optimizationA method to optimize language models directly on preference data without training a separate reward model.
- DropoutA regularization technique that randomly shuts off a subset of neurons during training to prevent overfitting.
- InferenceThe process of using an already trained machine learning model to make predictions or classify new data inputs.
- Meta-learningLearning to adapt more quickly from past experience of solving other tasks.
- ModelA system of abstractions that presents selected parts of reality in order to solve specific problems.
- Occam's razorThe learning principle stating that the simplest model that fits the data is also the most plausible explanation.
- Positional encodingThe addition of location information to input representations to break permutation or translation invariance.
- Q-learningA reinforcement learning method that uses a function approximator to estimate the action-value function.
- Reward modelA separate neural network trained to imitate human preferences and scores for model rollouts.
- Word embeddingsDense, real-valued vectors representing words in a high-dimensional vector space where similar contexts map to similar vectors.
- AxonA wire-like structure that serves as the output of a neuron.
- ClusteringAn unsupervised learning problem where data is partitioned into clusters or groups based on features or properties.
- CompositionCombining two data objects in a way that is distinct from interpolation.
- Cosine similarityThe angle between two vectors used to measure how similar they are.
- Genetic algorithmsA search heuristic that mimics the process of natural selection to generate useful solutions to optimization and search problems.
- Linear probeA technique where a linear projection is trained on top of a frozen representation space to evaluate the quality of learned representations.
- LocalizationA computer vision task that involves both classifying an image and drawing a bounding box around the object of interest.
- Max poolingA pooling operation that slides a filter over spatial regions and extracts the maximum value within each region.
- Offline reinforcement learningThe task of learning a policy from a fixed historical dataset of interactions without interacting further with the environment.
- Prompt engineeringThe process of structuring prompts to get an AI system to produce desired outputs.
- SimulationAn animated model used to experiment with rules when physical experiments require hundreds of robots.
- Policy evaluationThe process of computing or estimating the value function of a given policy.
- Policy gradientA class of reinforcement learning algorithms that optimize the policy parameters directly by maximizing the expected cumulative reward.
- Supervised fine-tuningTraining stage where the model learns how to behave in a certain way using examples of the harness, tools, and ideal responses.
- Conditional independenceX and Y are conditionally independent given Z if and only if their conditional joint probability equals the product of their individual conditional probabilities given Z.
- Linear classifierA computational mechanism that uses a separating hyperplane in a feature space to distinguish between different categories of objects based on linear separability.
- Pre-trainingThe initial stage of training where a model learns a general understanding of the world from large amounts of internet text or private data.
- AutoencoderA neural network architecture that maps input data to a compressed lower-dimensional representation and then decodes it back to reconstruct the original data.
- Chain of thoughtAn emergent or learned search process where an LLM writes out intermediate reasoning steps in text before arriving at a final answer.
- Cross-validationA technique where data is split into multiple folds, cycling through each fold as a validation set while training on the rest.
- HyperparametersChoices about the algorithm that are set rather than learned from training data.
- Self-supervised learningA learning paradigm where systems learn from unlabelled data by predicting missing parts of the input.
- State space modelsA class of models including Mamba that are taking off as alternatives to transformers.
- AccuracyThe ratio of correctly labeled samples divided by the total number of samples.
- Admissible heuristicA heuristic is admissible if it is optimistic, meaning it estimates a cost to the goal that is lower than or equal to the true cost.
- Backpropagation through timeThe standard backpropagation algorithm applied to recurrent neural networks unrolled across all time steps of a sequence.
- BacktrackingThe process of backing up to the place where we last made a decision and choosing another branch when hitting a dead end.
- Chain-of-thought promptingA prompting technique that encourages the model to generate intermediate reasoning steps before arriving at the final answer.
- CLIPA model using contrastive objectives and language supervision to learn generalizable semantic concepts across images.
- Computer visionTechnology that enables computers to derive meaningful information from digital images, videos, and other visual inputs.
- Constrained optimizationMinimizing or maximizing a function of multiple variables subject to one or more restrictions or constraints.
- ConstraintsLinear equations or inequalities that restrict the possible values of the variables in an optimization problem.
- ControlControl asks what is the optimal policy, or which direction should you walk to get the most reward.
- DeduplicationThe process of identifying and removing redundant content from a dataset to improve compute efficiency and prevent overfitting.
- EmbeddingUsing a trained neural network to give a low-dimensional representation of input data.
- Emergent abilitiesCapabilities that are absent in smaller language models but present in larger ones, appearing suddenly above a certain compute threshold.
- Empirical risk minimizationAn optimization principle in machine learning that trains models by minimizing the error on the training dataset.
- EncoderThe component that infers the distribution over the latent state Z given the history of observations.
- Encoder-decoder architectureAn architecture where an encoder compresses the input into a low-dimensional representation and a decoder reconstructs the output from it.
- Episodic memoryDeclarative memory of stuff that happened to you in your life that you can remember and tell other people about.
- Exploration vs. exploitationThe trade-off between choosing known actions that yield high rewards (exploitation) and trying new actions to discover potentially better ones (exploration).
- Foundation modelsLarge-scale models trained on vast amounts of data that can be adapted to a wide range of downstream tasks.
- GeneratorAn element of a group with maximum order that generates the entire group when raised to successive powers.
- K-means clusteringAn algorithm for clustering data by repeatedly assigning points to clusters and updating the centers of those clusters.
- Linear interpolationA method of transitioning between two points by moving along a straight line.
- McGurk effectA perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception.
- MinimaxAn algorithm for deterministic two-player zero-sum games where one player maximizes score and the other minimizes it.
- Minimax algorithmA logical, deterministic algorithm used in game trees where one player tries to maximize their score and the other tries to minimize it.
- Mixture of ExpertsA neural network architecture where different subnetworks (experts) specialize in different inputs, coordinated by a gating network or router.
- Monte Carlo tree searchA search algorithm used in decision-making processes, particularly in games, that builds a search tree by simulating random playouts.
- MultimodalityThe ability of large language models to both generate and perceive multiple modalities, such as text, images, and audio.
- Optimal policyA policy that maximizes the sum of rewards given any state we are in.
- PerceptronA simple hypothesis set that computes a linear combination of input attributes compared against a threshold.
- Phase transitionA change in a system where two distinct minimizers of free energy emerge as temperature decreases below a critical value.
- PhonologyThe system of sounds or signs that languages have.
- Post-trainingThe second stage of training where a base model is turned into an assistant by training on conversation datasets.
- PruningBuilding a full decision tree first and then working upward from the bottom to remove splits that fail a significance criterion.
- RegressionA supervised learning problem where the target labels are real numbers.
- Residual networkThe final concept needed before introducing algorithms, dealing with the capacity of a cut in flow networks.
- Stochastic differential equationA differential equation that includes a stochastic noise term, allowing for modeling of probabilistic dynamics.
- The bitter lessonThe historical observation in AI that general approaches leveraging computation via search and learning ultimately outperform methods that manually build human knowledge into agents.
- Transformer architectureA neural network architecture that replaces recurrence with self-attention to process tokens across all time positions simultaneously.
- ValueThe expected utility an agent will receive starting from a given state and acting optimally.
- Importance samplingA technique to estimate expected values under a target distribution using samples from a different proposal distribution by weighting them.
- Batch normalizationA technique that explicitly normalizes layer activations to have a zero mean and unit variance over a mini-batch.
- Free energy principleA unifying theoretical framework describing self-organizing systems as minimizing variational free energy to resist the second law of thermodynamics.
- Kernel methodsA class of algorithms for pattern analysis that rely on pre-defined kernel functions to map data into high-dimensional feature spaces.
- KL divergenceA measure of how one probability distribution differs from a second, reference probability distribution.
- Marginal distributionThe probability distribution of a subset of a collection of random variables, obtained by summing out the other variables in the joint distribution.
- Multi-head attentionAn attention mechanism where queries, keys, and values are projected into multiple subspaces to allow the model to jointly attend to information from different representation subspaces in parallel.
- Policy iterationAn algorithm that alternates between evaluating the advantage function for a policy and updating the policy to greedily select actions with highest advantage.
- Pseudo-inverseA generalization of the matrix inverse for non-square or singular matrices that provides a solution to the least squares problem.
- Representation learningThe process of mapping raw data to structured abstract representations.
- Sampling biasA phenomenon where learning produces a similarly biased outcome because the data is sampled in a biased way.
- Sigmoid functionA function that maps real numbers to a range between 0 and 1, defined as 1 over 1 plus e to the negative Z.
- UtilityThe balance of pleasure over pain and happiness over suffering.
- A* searchA search algorithm that expands the node with the lowest combined value of path cost and heuristic estimate .
- Advantage functionThe difference between the Q-function and the value function, representing how much better an action is compared to the average performance of the policy in that state.
- Backtracking searchA search algorithm for CSPs that builds a solution incrementally by assigning values to variables and backtracking when a constraint is violated.
- Beam searchA heuristic search algorithm that explores a graph by expanding the most promising nodes in a limited set.
- Contrastive learningA learning technique that maps similar data points to nearby embeddings and dissimilar data points to far-away embeddings.
- Early stoppingA regularization technique based on validation that stops training before overfitting begins to degrade performance.
- Exploding gradientsA problem where weight matrices are very large, causing gradients to grow exponentially during backpropagation and making network training unstable.
- FilteringA technique in backtracking that rules out candidates for unassigned variables to detect potential failures early.
- Hidden Markov modelA generative probabilistic model used to recover a hidden sequence of states from noisy observations over time.
- Latent variable modelModels that condition neural network outputs on unobserved random variables or seeds to represent broad distributions.
- LikelihoodThe probability of the data given the hypothesis, serving as a plausible error measure.
- Logistic regressionA linear model that applies a non-linear logistic function to the linear signal to output probabilities.
- Mechanistic interpretabilityMechanistic interpretability tries to understand how individual components and operations in a machine learning model contribute to its overall decision-making process.
- Moravec's paradoxThe observation that high-level reasoning requires very little computation while low-level sensorimotor skills require immense computational resources.
- Positional embeddingA learned vector representation added to each token embedding to give the transformer information about the token's absolute position in the sequence.
- REINFORCE algorithmA Monte Carlo policy gradient method that samples trajectories, estimates the policy gradient, and updates policy parameters via gradient ascent.
- Semantic segmentationA computer vision task where the goal is to classify every single pixel in an input image into a specific category without distinguishing between separate instances of the same class.
- Value of informationComputations that determine the gain in utility associated with acquiring specific evidence before making a decision.
- Vanishing gradientsA problem where weight matrices are very small, causing gradients to shrink exponentially close to zero during backpropagation and preventing stable training.
- Variational autoencoderA variational autoencoder is an amortized variational inference model that models an input using a latent variable through an encoder and a decoder.
- Vision transformerA vision transformer is an architecture that applies the transformer encoder to image patches for image recognition tasks.
- AI agentsSystems built around large foundation models that use memory, tools, planning, and actions to autonomously execute tasks for users.
- Associative memoryA gigantic network of ideas linked to each other in various ways such as causes, categories, or instances.
- AutogradA software library that automatically computes gradients of mathematical functions, essential for training neural networks.
- Axioms of probabilityFundamental rules governing probability values, such as ranging between 0 and 1 inclusive.
- Bayesian networkGraphical models representing variables and their conditional dependencies via directed acyclic graphs.
- Camera obscuraAn optical device consisting of a darkened box or room with a lens or pinhole that projects an inverted image of the scene outside onto a surface inside.
- Categorical perceptionThe phenomenon where perception is altered by learning categories, making differences between categories more discriminable.
- Certainty equivalentA guaranteed cash flow that an investor would accept as an alternative to an expected risky cash flow.
- Chain rule of probabilityA formula stating that any joint distribution can be factored as the product of conditional probabilities of each variable given all preceding variables.
- ClassificationA type of machine learning that automates predicting an output category from an input.
- Cognitive behavioral therapyA structured, evidence-based, time-limited psychotherapy focusing on the relationship between thoughts, emotions, and behaviors.
- CompositionalityThe property where complex systems, representations, or ideas are built by combining simpler pieces together.
- Computation graphA directed graph where nodes represent operations or variables, showing how computations are structured and composed.
- Constraint graphA graph representation of a CSP where nodes are variables and edges represent constraints between them.
- Convolutional layerLayers that apply the same calculation in translation-invariant ways across different regions of an image using parameter sharing.
- Data augmentationA technique to increase the diversity of data available for training models, such as by applying random transformations to existing images.
- Data parallelismConcurrency that arises from performing the same operation on different pieces of data.
- Deep Q learningQ-learning where the function approximator is a deep neural network.
- DendritesThe inputs to a neuron where stuff comes in.
- DiscretizationCasting model weights to a lower-bit precision format like 8-bit to reduce memory usage while preserving performance.
- DistillationA technique where a student model is trained to mimic the predictions or output distributions of a teacher model.
- Energy-based modelsA class of models that define probability distributions using an unnormalized density function based on an energy function.
- Epistemic uncertaintyUncertainty that captures what the model does not know based on its training data.
- EvaluationComputing the values of a polynomial at specific points given its coefficients.
- Evaluation functionA function that estimates the value of a non-terminal state in a game tree, replacing the need to search to the actual terminal leaves.
- Exhaustive searchA method to find the simplest tree by evaluating every possible binary tree topology for a given set of taxa.
- Experience replayA technique in reinforcement learning where the agent stores its experiences in a memory buffer and samples random batches from it to train the network, breaking the correlation between consecutive samples.
- Expert systemsAI systems that rely on hardcoded knowledge and pre-programmed solutions.
- Exponential moving averageA technique that replaces a variable's value with a weighted average of its previous values, decaying exponentially over time.
- ExtrapolationThe ability of a model to perform well on inputs or conditions outside the range of the training data.
- Feature vectorA vector of numerical features representing an object or sample in the dataset.
- Gaussian process regressionA nonparametric regression method that uses multivariate normal conditionals and a distance-based covariance function to make predictions and quantify uncertainty.
- Generative AITools that are familiar in the United States and used for automated generation of content.
- Gradient clippingA heuristic technique that scales down the gradient norm if it exceeds a predefined threshold to prevent exploding gradients.
- Graphics processing unitA processor originally developed for computer graphics that features thousands of cores and is well-suited for parallelizable algorithms.
- Hoeffding's inequalityA fundamental inequality from probability theory that bounds the probability that the sum of bounded independent random variables deviates from its expected value.
- Huffman codingAn entropy encoding algorithm that uses shorter bit strings for more common inputs and longer bit strings for less common inputs.
- Image CaptioningA task where an AI system takes an image as input and generates a natural language description of that image.
- Information gainInformation gain measures the reduction in entropy achieved by partitioning the data along a particular attribute, calculated as the difference between the parent entropy and the expected weighted child entropy.
- Intrinsic valueThe information gain or epistemic value obtained by acting to resolve uncertainty about hidden states.
- Linear regressionA statistical approach to prediction that fits coefficients to a linear combination of various variables.
- LogarithmMathematical operations whose manual calculation can be replaced by tables or built-in functions.
- Long short-term memoryA specific type of RNN architecture designed to learn long-term dependencies by using gating mechanisms to control information flow.
- LotteryA situation with uncertain prizes represented as a probabilistic distribution over outcomes.
- LSTMA type of recurrent neural network architecture designed to maintain long-term dependencies by using gating mechanisms to control information flow.
- Lyapunov timeThe characteristic time scale between when a system starts and when it diverges from itself, marking the onset of chaos.
- MarginalizationSumming out variables from a joint distribution to find the probability of a subset of variables.
- Markov assumptionThe assumption that the future state of a system depends only on its current state, not on the sequence of events that preceded it.
- Masked language modelingA pre-training task where a fraction of input sequence tokens are masked, and the model is trained to predict the original masked characters from their positions.
- Model selectionSelecting different models to minimize the complexity of parameters in generative models based on accumulated evidence.
- Multi-layer perceptronA neural network composed of multiple layers of neurons feeding into each other sequentially.
- Neural network layerA layer computes a linear transformation of the input vector via a weight matrix and bias vector, followed by a non-linear activation function.
- Neural network trainingThe process of organizing many things, labeling them into categories, and finding matching images to fill up those categories.
- Objective functionA mathematical function that a learning algorithm seeks to minimize or maximize to optimize model performance.
- Odds ratioMeasure comparing how likely an outcome is given one condition versus another condition.
- PackingThe practice of arbitrarily concatenating short contexts to fill out long context windows and avoid waste on padded sequences.
- Partially observable Markov decision processA POMDP is a generalization of a Markov decision process that adds unobserved states, observations, and an observation function where the agent maintains a belief state distribution over states.
- Perfect information gameA game where each player has complete information about the state of the game at every decision point.
- PredictionPrediction basically means how well will I do if I follow my current policy.
- PretrainingPretraining teaches a neural network to mimic its training data from the internet by predicting the next word given a block of text.
- PyTorch tensorsMulti-dimensional arrays provided by the PyTorch library that allow efficient manipulation and GPU acceleration.
- Quality-adjusted life yearMeasures of the quality of life years used as the underpinnings of government healthcare priorities.
- Refractory periodA period within which it is difficult or impossible to initiate another action potential.
- Rejection samplingA sampling method for conditional probabilities where samples are generated from the joint distribution and those that do not match the evidence are discarded.
- ReLUA non-linear activation function defined as the maximum of zero and the input value.
- Residual connectionA connection that adds the input of a layer to its output, allowing the network to learn residual mappings.
- ResNetA deep neural network architecture that uses skip connections to jump over layers, facilitating the training of very deep networks.
- RMS normRoot Mean Square normalization scales inputs by their root mean square values without subtracting the mean, mapping data points onto a hypersphere in high dimensions.
- Sampling errorDiscrepancies between generated statistical outputs and expected values due to sample size limitations.
- Scaling lawScaling law dictates that increasing compute and model size predictably improves performance.
- SearchGiven a model and a query, finding the best answer to that query by searching through a space of possibilities.
- SolipsismThe philosophical belief that one's own mind is the only thing that can be known to exist.
- Speech recognitionA task involving the conversion of sound waves into text, which includes pre-processing and acoustic modeling.
- State value functionThe expected return starting from state s and then following policy pi.
- Supervised finetuningTraining a pre-trained model on a smaller, supervised dataset to adapt it for a specific downstream task.
- Taylor expansionRepresenting a function as an infinite sum of terms calculated from the values of its derivatives at a single point.
- Temporal difference learningA learning method that updates value estimates incrementally based on each observed transition rather than waiting for the end of an episode.
- Tensor coresA hardware unit and instruction in Volta GPUs that performs 4x4x4 mixed-precision matrix multiply and accumulate operations in a single clock cycle.
- Test-time trainingAdapting or learning with model parameters during inference time rather than only during the initial training phase.
- Toeplitz matrixA matrix in which each descending diagonal from left to right is constant, representing linear shift-invariant operations.
- TokenizerA tokenizer converts raw text as a string to some sequence of integers according to some vocabulary of possible elements.
- Transition functionA function that describes the probability of transitioning to a state S' given state S and action A.
- Transition modelA model that maps a state of the world and an action into some distribution over next states.
- Translation invarianceA property of a system where its physical laws or equations remain unchanged under a shift in position.
- Traveling salesperson problemThe problem of finding the shortest possible route to visit a set of different locales or cities and return to the origin.
- Tree of ThoughtsAn extension of chain of thought that considers multiple different reasoning paths with lookahead and backtracking.
- Universal function approximatorNeural networks are universal function approximators, meaning if you can differentiate it, a neural network can approximate it.
- Visual question answeringA task where a model takes an image and a natural language question as inputs and predicts the correct answer from a set of choices.
- Weight decayA regularization method that adds the squared norm of the weights to the in-sample error to penalize large weights and prevent overfitting.
- Working memoryThe ability to hold information in mind for a short time and manipulate that information.
- Distributional shiftThe problem where the distribution of states encountered during test time differs from the distribution of states seen in the training data.
- Inverse reinforcement learningA problem where the goal is to recover an unknown reward function given observed behavior of an agent.
- Autoregressive modelsA model that generates a sequence by predicting the next element based on the previous elements in the sequence.
- Byte pair encodingAn algorithm that iteratively merges the most common pairs of bytes or symbols to create a vocabulary of compressed tokens.
- Generative modelingThe process of mapping abstract representations to data.
- Legendre polynomialsPolynomial solutions to Legendre's differential equation that satisfy physical boundary conditions for integer .
- Object detectionA task where given an input image and a fixed set of categories, the algorithm draws a bounding box around every instance of those categories in the image.
- One-hot encodingA method of encoding categorical integer data into a binary vector where all elements are zero except for the dimension corresponding to the integer index, which is set to one.
- System identificationThe classical robotics problem of taking data to identify unknown parameters in a known dynamics model.
- Action spaceThe set of all possible actions that an agent can take from a given state.
- Adversarial searchA search technique for planning in environments where other agents are actively working against the primary agent's goals.
- AI winterA period of reduced funding and interest in artificial intelligence research following a period of over-optimistic expectations.
- Arc consistencyA property of a constraint satisfaction problem where for every arc between two variables, every value in the domain of the first variable has a corresponding value in the domain of the second variable that satisfies the constraint.
- Bias variance decompositionExpressing the expected loss in terms of the bias of the error and the variance of the error.
- Computational graphA directed graph representing the sequence of operations used to compute a function.
- Conditional probability tableA collection of distributions over a variable for each combination of its parents' values inside a node.
- DichotomyA set of binary labels assigned to specific points in a space.
- Explicit constraintA constraint specified by explicitly listing all allowed tuples of values for the involved variables.
- Hill climbingA local search algorithm that starts from a state and iteratively moves to the neighboring state with the highest objective value until no higher neighbor exists.
- HomophilyA biological phenomenon where similar beings tend to sort of hang out with others of their own kind.
- Implicit constraintA constraint defined by a function or code snippet that checks if a given assignment of variables is valid.
- Inference by enumerationAn exact inference algorithm that sums over hidden variables in the joint distribution to answer queries.
- Iterative deepeningA search strategy that repeatedly performs a depth-limited search, increasing the depth limit with each iteration until time runs out.
- Learning rate decayA training strategy where the learning rate is systematically decreased over time, typically by a factor of 10, during the later stages of training to allow the model to fine-tune its parameters.
- N-step returnsReturns accumulated over N steps instead of a single step to compute targets for Q-learning.
- Natural policy gradientA policy gradient method that multiplies the regular gradient by the inverse of the Fisher Information Matrix to correctly account for parameter sensitivities.
- Off-policy learningA learning paradigm where an agent learns the optimal policy while following a different, potentially suboptimal, behavior policy.
- Optimal controlThe more general problem of selecting controls that optimize some reward or minimize some cost.
- Principle of optimalityThe principle of optimality states that every sub-plan of an optimal plan is itself optimal.
- Probabilistic inferenceThe process of computing a desired probability from other known probabilities, such as deriving a conditional probability from a joint distribution.
- Probabilistic modelA model that represents a probability distribution over random variables.
- Probability distributionA function that takes a random variable and gives the probability for each of the possible values in its domain.
- Prompt injectionA vulnerability where an attacker manipulates inputs to make a model execute unintended instructions.
- Rational agentAn agent that selects actions it believes will result in high-utility outcomes.
- Reparameterization trickThe reparameterization trick rewrites a random variable from a parameterized Gaussian distribution as a deterministic function of the parameters and an independent standard Gaussian noise variable, enabling low-variance gradient estimation through backpropagation.
- Sigmoid activation functionA mathematical function that squashes input values into a range between zero and one, historically interpreted as a neuron's firing rate.
- Support vectorsThe data points with positive Lagrange multipliers that lie exactly on the margin boundary and define the separating hyperplane.
- Trust regionA region around the current point where the local quadratic approximation is trusted to be accurate.
- 2.5D sketchThe second process in David Marr's framework where surfaces, depth information, layers, and discontinuities are pieced together.
- A*An algorithm that combines branch and bound, an extended list, and an admissible heuristic.
- AbductionA process that is neither deduction nor induction, but rather reasoning from observed effects back to likely causes.
- ActivationThe weighted sum of feature values in a linear classifier.
- Active matterThe study of physics-based models for emergent behavior in systems like flocks of birds, schools of fish, and swarms.
- Actor-critic algorithmAn algorithm that combines policy optimization (actor) with value function estimation (critic).
- AdaGradAn optimization algorithm that scales the learning rate per parameter by dividing by the square root of the sum of historical squared gradients.
- AdamAn optimization algorithm that combines momentum (first moment estimates) and adaptive scaling (second moment estimates of squared gradients).
- AdmissibilityThe property where the estimated distance between any node X and the goal is less than or equal to the actual distance between X and the goal.
- Adversarial trainingTraining a neural network on a mix of clean and adversarial examples to make it robust against attacks.
- Affine transformationA transformation that preserves lines and parallelism, allowing ellipsoids to be converted into spheres.
- AgentAn entity that perceives its environment through sensors and acts upon it through actuators.
- Agent-to-agent communicationMechanisms and protocols that allow different AI agents to exchange information, requests, and sync their states.
- Agentic AIAn advanced phase where AI autonomously performs tasks such as writing code and requirements.
- AI systemsMulti-component frameworks built around large foundation models using mechanisms like model chaining and reflection.
- Aleatoric uncertaintyStatistical uncertainty inherent in the data-generating process itself, which does not decrease as more data is collected.
- Alignment problemHow we can get an artificial system to have the right values and do the right things that we would like it to do.
- AmbiguityThe uncertainty about the outcomes given their causes or states.
- Analytical engineCharles Babbage's proposed general-purpose mechanical computer that could execute programmed instructions from punched cards.
- AnalyticsThe union of data science and optimization together.
- AnimismThe worldview that the world is full of persons, only some of whom are human, and life is lived in relationship with others.
- Approximate Q-learningQ-learning where the Q-value function is parameterized by weights over features rather than stored in a table.
- Artificial lifeA field of research exploring artificial agents, systems, and evolution to study life, cognition, and complex systems.
- Augmented intelligenceA third type of AI where humans retain the agency to set direction alongside bots and other humans.
- BabyLM challengeAn effort to train LLMs using similar amounts of text data that human babies are exposed to while growing up.
- BaselineA value subtracted from the cumulative future reward to reduce variance in the policy gradient estimator.
- Baseline functionA function subtracted from the policy gradient to reduce variance without changing the expectation of the gradient.
- Batch gradient descentGradient descent where weight updates are computed using all examples in the training set at once.
- Bayesian inferenceA method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available.
- Belief distributionThe conditional probability distribution of a hidden state variable given all the evidence collected to date.
- Belief stateAll the information from the past which is relevant for which future action to take.
- BiasBias measures how far the best possible hypothesis in your hypothesis set is from the target function.
- Bias-variance tradeoffTest error can be decomposed into train error (bias) plus the difference between test error and train error (variance).
- Binary classificationCategorizing data into one of two mutually exclusive classes, such as plus one or minus one.
- Binary constraintA constraint that involves exactly two variables.
- Bitter lessonThe historical observation in AI that general methods leveraging computation and data scaling consistently outperform methods relying on human-crafted modeling assumptions and inductive biases.
- Block sizeThe maximum context length used by the transformer to make predictions.
- Character-level language modelA model that treats every input line as an example and treats them all as sequences of individual characters.
- Chinese room argumentA thought experiment arguing that a machine can simulate understanding without actually possessing it.
- Classification problemA machine learning task where the goal is to assign input data to one of several predefined categories.
- Cognitive mapAn internal mental representation of the spatial layout of an environment that allows for flexible navigation.
- Cognitive scienceA shift away from behavioral science and structuralism towards studying the internal workings and mechanisms of the mind.
- CombinatorsThe links that connect nodes in a semantic network.
- Common CrawlAn organization that regularly crawls the internet and provides open repositories of web crawl data.
- Concentrated log likelihoodA profile likelihood involving just the remaining parameter after plugging in the MLE for another parameter.
- Conditional independence in Bayes netsArcs formally encode conditional independence, though they are often intuitively thought of as encoding causality.
- Conditional probability distributionA probability distribution for a node given the values of its parents in the Bayesian network.
- Conditional value-at-riskConditional value at risk is the expected loss given that the loss exceeds the value at risk threshold.
- Conjugate priorA prior where updating with new data points only requires changing parameters rather than recomputing the entire posterior function.
- Constitutional AIAn approach developed by Anthropic that aims to align large language models with a specific constitution.
- Constraint propagationThe process of looking at local constraints as we go to prevent downstream problems in search.
- Contextual banditsA variation of bandits where not only is there an action to choose, but there is also some state information.
- Continual learningInfinite and permanent fundamental self-improvement where systems learn constantly from every new interaction.
- Contrastive divergenceAn approximation algorithm used to estimate gradients for training Restricted Boltzmann Machines.
- Correspondence problemThe challenge for the brain to decide what portion of one image matches with what portion of the other image.
- Covariance functionA covariance function defines the covariance between the values of a Gaussian process at any two input points.
- Cross entropyA measure of the difference between two probability distributions based on logarithmic expected values.
- CrossoverA recombination process where the front part of one chromosome and the back part of another are fused together.
- CuriosityThe drive to seek information, which is often missing in artificial intelligence research.
- D-separationA criterion to decide if two variables are conditionally independent given a set of evidence variables by checking all undirected paths between them.
- Decoder-only architectureAn architecture using a single stack with causal self-attention that handles both inputs and targets via concatenation to generate output sequences.
- DecodingFinding the single most likely state sequence or trajectory given the observed evidence using argmax.
- Delta ()A value used in backpropagation calculated using partial derivatives to determine the direction and magnitude of weight updates.
- DenoisingThe process of removing noise from an image, which can be achieved by following the gradient of the score function.
- Density estimationMethods that work on the basis of estimating data distributions to produce kinds of data that humans tend to produce.
- Differential dynamic programmingAn extension of LQR for nonlinear systems that uses second-order dynamics approximations, equivalent to Newton's method.
- DisentanglementA machine learning concept and process of separating distinct underlying factors of variation in data.
- Domain randomizationA data augmentation technique where simulation environments are randomized to bridge the gap between simulation and reality.
- Dominant driving forceThe primary force that governs a trend, filtering out secondary forces when predicting future trajectories.
- Double descentThe phenomenon where test error initially decreases, then increases, and then decreases again as model capacity increases.
- Dual normA norm associated with another norm that appears in the dual formulation of the steepest descent update.
- Dynamic time WarpingAn algorithm used to align two or more temporal sequences that may vary in speed or phase.
- Dynamics modelA representation that defines the transition probabilities from a current state and action to subsequent states.
- Effective depthThe depth in the search tree that corresponds to the cost of the optimal solution divided by the minimum arc cost.
- Electron massA characteristic number of an electron that determines the rate at which its amplitude rotates in time.
- EM algorithmAn algorithm used to implement the estimation of unknown parameters by treating missing data as latent variables.
- EmpowermentAn intrinsic reward formulation that maximizes the mutual information between an agent's actions and outcomes to maximize control over the world.
- Encoder-only architectureA Transformer architecture that processes input sequences into a single vector representation, suitable for classification tasks rather than sequence generation.
- Euclidean distanceThe square of the distance between two points in a flat space, given by the sum of the squares of the coordinate differences.
- Evidence lower boundA lower bound to the data likelihood under a model, used as an objective in variational autoencoders.
- Expectation-maximization (EM) algorithmAn iterative method to find maximum likelihood estimates of parameters in statistical models with hidden variables.
- Face detectionThe task of detecting human faces in images using statistical machine learning techniques.