Concepts
5,907Terms and methods explained by more than one lecturer. Each opens at the minute it is taught.
AI & ML
88- Reinforcement learningA machine learning and psychological approach where an agent learns to make decisions by receiving external rewards or punishments from its environment.
- Machine learningThe field of study that gives computers the ability to learn without being explicitly programmed.
- Supervised learningLearning to predict an output y from an input x given a data set of input-output pairs.
- Chain ruleA formula for computing the derivative of the composition of two or more functions.
- Deep learningA subset of machine learning using hierarchical neural networks to model environments and data.
- StateThe value of a system at a given time.
- Large language modelsComputational models trained on massive text corpora to generate text and understand language.
- HallucinationsInstances where AI models generate factually incorrect or fabricated information.
- Markov decision processA mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
- Unsupervised learningA learning setup where the goal is to learn the underlying hidden structure of unlabeled data.
- Convolutional neural networksNeural network architectures specialized for processing grid-like data such as images using convolutions and subsampling.
- Artificial intelligenceThe simulation of human intelligence in machines that are programmed to think like humans as best as possible and mimic human action.
- PolicyThe agent's behavior function mapping states to actions.
- Value functionThe expected sum of future rewards starting from state st and following policy pi thereafter.
- Product ruleA rule stating that the joint probability of two variables is equal to the product of a marginal distribution and a conditional distribution.
- ExplorationThe strategy of trying unknown actions to gather information about the environment's dynamics and rewards.
- GeneralizationThe ability of a learning model to perform accurately on unseen data outside the training set.
- SamplingThe procedure involving a sampling technique and estimated sample size.
- Active learningA machine learning technique where the model is allowed to choose the data from which it learns, often focusing on instances where it is most uncertain.
- Markov propertyThe property where the future state is conditionally independent of past states given the current state.
- Generative adversarial networksA generative model that learns to generate data from a training distribution through a two-player game between a generator and a discriminator network.
- HeuristicsRules of thumb or tricks used by a computer program to guide decision-making and problem-solving.
- AttentionA mechanism where weights are dynamically determined as a function of the input data.
- Deep neural networksAn artificial neural network with multiple layers between the input layer and the output layer.
- Model-based reinforcement learningA reinforcement learning approach where an environment model is learned from data and then used for control or planning.
- ExploitationExploit the information you've already found to maximize your reward.
- ReturnThe discounted sum of rewards from the current time step to the end of the horizon.
- InferenceThe process of using an already trained machine learning model to make predictions or classify new data inputs.
- ModelA system of abstractions that presents selected parts of reality in order to solve specific problems.
- Reward modelA separate neural network trained to imitate human preferences and scores for model rollouts.
- AxonA wire-like structure that serves as the output of a neuron.
- Prompt engineeringThe process of structuring prompts to get an AI system to produce desired outputs.
- SimulationAn animated model used to experiment with rules when physical experiments require hundreds of robots.
- Pre-trainingThe initial stage of training where a model learns a general understanding of the world from large amounts of internet text or private data.
- Constrained optimizationMinimizing or maximizing a function of multiple variables subject to one or more restrictions or constraints.
- ControlControl asks what is the optimal policy, or which direction should you walk to get the most reward.
- Credit assignmentThe problem of determining which particular choice or action in a sequence of events precipitated a failure or success outcome.
- EmbeddingUsing a trained neural network to give a low-dimensional representation of input data.
- EncoderThe component that infers the distribution over the latent state Z given the history of observations.
- Episodic memoryDeclarative memory of stuff that happened to you in your life that you can remember and tell other people about.
- GeneratorAn element of a group with maximum order that generates the entire group when raised to successive powers.
- McGurk effectA perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception.
- MinimaxAn algorithm for deterministic two-player zero-sum games where one player maximizes score and the other minimizes it.
- Minimax algorithmA logical, deterministic algorithm used in game trees where one player tries to maximize their score and the other tries to minimize it.
- PerceptronA simple hypothesis set that computes a linear combination of input attributes compared against a threshold.
- Post-trainingThe second stage of training where a base model is turned into an assistant by training on conversation datasets.
- PruningBuilding a full decision tree first and then working upward from the bottom to remove splits that fail a significance criterion.
- Transformer architectureA neural network architecture that replaces recurrence with self-attention to process tokens across all time positions simultaneously.
- Sampling biasA phenomenon where learning produces a similarly biased outcome because the data is sampled in a biased way.
- Value of informationComputations that determine the gain in utility associated with acquiring specific evidence before making a decision.
- Associative memoryA gigantic network of ideas linked to each other in various ways such as causes, categories, or instances.
- Camera obscuraAn optical device consisting of a darkened box or room with a lens or pinhole that projects an inverted image of the scene outside onto a surface inside.
- Cognitive behavioral therapyA structured, evidence-based, time-limited psychotherapy focusing on the relationship between thoughts, emotions, and behaviors.
- CompositionalityThe property where complex systems, representations, or ideas are built by combining simpler pieces together.
- DendritesThe inputs to a neuron where stuff comes in.
- Hardware-software co-designThe joint design and modification of both computer hardware and software architectures to achieve revolutionary performance improvements.
- LogarithmMathematical operations whose manual calculation can be replaced by tables or built-in functions.
- Partially observable Markov decision processA POMDP is a generalization of a Markov decision process that adds unobserved states, observations, and an observation function where the agent maintains a belief state distribution over states.
- PredictionPrediction basically means how well will I do if I follow my current policy.
- Refractory periodA period within which it is difficult or impossible to initiate another action potential.
- Speech recognitionA task involving the conversion of sound waves into text, which includes pre-processing and acoustic modeling.
- Sufficient statisticA statistic is sufficient for distinguishing two hypotheses if the likelihood ratio depends on the underlying model only through .
- Model-free reinforcement learningReinforcement learning where the agent does not try to explicitly understand the environment's dynamics, going directly to the policy or value function.
- Optimal controlThe more general problem of selecting controls that optimize some reward or minimize some cost.
- AbductionA process that is neither deduction nor induction, but rather reasoning from observed effects back to likely causes.
- Actor-criticAn agent that stores both a policy and a value function to get the benefits of both worlds.
- Analytical engineCharles Babbage's proposed general-purpose mechanical computer that could execute programmed instructions from punched cards.
- BiofeedbackA technique that provides feedback on physiological processes to help individuals regulate oxygenated blood flow to their frontal cortex.
- Closed-loop controlA control strategy where the agent observes a state and produces a policy that maps states to actions dynamically.
- ConfabulationWhen models invent non-existent objects, actions, or facts, also known as hallucinations.
- DataData generated by implicit generative models that can be manipulated with special operators like interpolation and composition.
- DecodingFinding the single most likely state sequence or trajectory given the observed evidence using argmax.
- Electron massA characteristic number of an electron that determines the rate at which its amplitude rotates in time.
- Explanation-based learningA learning paradigm that uses explanations of examples to generalize knowledge, historically related to proof systems in logic and games.
- Flynn effectThe substantial and long-documented increase in average intelligence test scores over generations.
- GroundingFinding a unique mapping or pairing between one modality and another.
- LiDARA laser range finding system that measures distance by timing the reflection of laser beams.
- Linear separabilityThe assumption that data points can be separated by a linear boundary, serving as a simplistic assumption that does not always apply in practice.
- MNISTA publicly available database of hand-written digits used as a standard task for testing machine learning algorithms.
- Neural netsA class of machine learning architectures that use stacks of linear transformations interleaved with pointwise nonlinearities.
- Planning problemA class of search problems where what you are interested in is finding the sequence of actions or path from the start state to the goal.
- Protein folding problemThe problem of predicting the 3D structure of a protein directly from its one-dimensional amino acid sequence.
- Pygmalion effectThe effect where believing someone has a certain characteristic causes them to behave as if they have that characteristic.
- RewardThe equivalent of costs from search in MDPs, received as big rewards at the end or small living rewards each step.
- Semantic networkA cognitive model of knowledge represented as nodes connected by relationships.
- Shift cipherA substitution cipher where each letter in the plaintext is shifted a fixed number of positions down the alphabet.
- Threat multiplierClimate change acts as a threat multiplier that makes existing geopolitical and environmental problems worse.
- ValidityThe extent to which a measurement captures what it was intended to measure.