Gradient · Reference

Glossary

Every abbreviation used across the curriculum, spelled out in full — with a link back to the chapter where it’s introduced.

127 abbreviations

3DGS3D Gaussian Splatting
3D Gaussian Splatting (3DGS) fundamentals Tile-based differentiable rasterization in 3DGS Train and render a 3D Gaussian Splatting scene
A2CAdvantage Actor-Critic
Actor-Critic architectures (A2C/A3C)
AdaBoostAdaptive Boosting
Ensembles II: Boosting & AdaBoost Build a high-performance tabular ML pipeline
AdamAdaptive Moment Estimation
Optimizers: Momentum, RMSProp, Adam & AdamW
AdamWAdam with Decoupled Weight Decay
Optimizers: Momentum, RMSProp, Adam & AdamW Weight decay vs. L2 regularization
ANNApproximate Nearest Neighbor
Vector databases & ANN search (HNSW & IVF-PQ)
ARIMAAutoRegressive Integrated Moving Average
Time series forecasting: ARIMA & exponential smoothing
AUCArea Under the Curve
ROC curves, AUC & PR curves
autogradAutomatic Differentiation
Build a computational graph & autograd engine Backpropagation, step by step
AWQActivation-aware Weight Quantization
Quantization (AWQ, GPTQ, GGUF & BitNet 1.58-bit)Build and deploy a monitored, high-throughput AI serving cluster
BCEBinary Cross-Entropy
Binary cross-entropy loss
BERTBidirectional Encoder Representations from Transformers
Interleaved multimodal generation Transfer learning & feature extraction
BF16Brain Floating Point 16-bit
Mixed precision training (FP16 & BF16)
BitNet1.58-bit Ternary Neural Network
Quantization (AWQ, GPTQ, GGUF & BitNet 1.58-bit)
BPEByte-Pair Encoding
Tokenization & Byte-Pair Encoding (BPE)
CLIPContrastive Language–Image Pretraining
Contrastive learning & CLIP Vision-Language Models (VLM) assembled
CNNConvolutional Neural Network
A minimal CNN architecture Train a CNN and inspect its learned feature filters
CoTChain-of-Thought
Reasoning models & test-time compute scaling Train a reasoning model using GRPO & verifiable rewards
DBSCANDensity-Based Spatial Clustering of Applications with Noise
Density-based clustering: DBSCAN & HDBSCAN Build a customer segmentation & recommendation engine
DDIMDenoising Diffusion Implicit Models
Deterministic sampling with DDIM
DDPDistributed Data Parallel
Large-scale distributed training (DDP, FSDP, ZeRO & Megatron-LM)
DDPMDenoising Diffusion Probabilistic Models
Denoising Diffusion Probabilistic Models (DDPM)
DiTDiffusion Transformer
Diffusion Transformers (DiT / Flux / SD3)Build a flow-matching & DiT image synthesizer
DPODirect Preference Optimization
Direct Preference Optimization (DPO)
DQNDeep Q-Network
Deep Q-Networks (DQN)
ELBOEvidence Lower Bound
Variational Autoencoders (VAE)
F1F1 Score
Confusion matrix, precision, recall & F1
FAISSFacebook AI Similarity Search
Vector databases & ANN search (HNSW & IVF-PQ)
FGSMFast Gradient Sign Method
Adversarial attacks & robustness (FGSM & PGD)
FlashAttentionFast, Memory-Efficient Exact Attention with IO-Awareness
FlashAttention (1, 2, 3) & GPU memory hierarchies
FNFalse Negative
Confusion matrix, precision, recall & F1
FPFalse Positive
Confusion matrix, precision, recall & F1
FP1616-bit Floating Point
Mixed precision training (FP16 & BF16)
FP3232-bit Single Precision Floating Point
Mixed precision training (FP16 & BF16)
FPRFalse Positive Rate
ROC curves, AUC & PR curves
FSDPFully Sharded Data Parallel
Large-scale distributed training (DDP, FSDP, ZeRO & Megatron-LM)
GANGenerative Adversarial Network
VQ-VAE & discrete latent codebooks
GBMGradient Boosting Machine
Gradient Boosting Machines (GBM)
GeLUGaussian Error Linear Unit
Activation functions: Sigmoid, Tanh, ReLU, GeLU & Swish
GGUFGPT-Generated Unified Format (quantization format)
Quantization (AWQ, GPTQ, GGUF & BitNet 1.58-bit)
GloVeGlobal Vectors for Word Representation
Word embeddings (Word2Vec & GloVe)
GPTGenerative Pre-trained Transformer
Tokenization & Byte-Pair Encoding (BPE)The modern LLM post-training pipeline
GPTQAccurate Post-Training Quantization for Generative Pre-trained Transformers
Quantization (AWQ, GPTQ, GGUF & BitNet 1.58-bit)
GRPOGroup Relative Policy Optimization
Group Relative Policy Optimization (GRPO / DeepSeek-R1)Train a reasoning model using GRPO & verifiable rewards
GRUGated Recurrent Unit
The Gated Recurrent Unit (GRU)
HBMHigh Bandwidth Memory
FlashAttention (1, 2, 3) & GPU memory hierarchies GPU memory hierarchies & kernel profiling
HDBSCANHierarchical Density-Based Spatial Clustering of Applications with Noise
Density-based clustering: DBSCAN & HDBSCAN Build a customer segmentation & recommendation engine
HNSWHierarchical Navigable Small World
Vector databases & ANN search (HNSW & IVF-PQ)
IoUIntersection over Union
Object detection fundamentals (Bounding Boxes & IoU)
IVF-PQInverted File with Product Quantization
Vector databases & ANN search (HNSW & IVF-PQ)
JSONJavaScript Object Notation
Function calling & tool use
k-NNk-Nearest Neighbors
k-Nearest Neighbors (k-NN)
KL divergenceKullback–Leibler Divergence
KL divergence & mutual information Variational Autoencoders (VAE)
KTOKahneman-Tversky Optimization
Advanced preference alignment (KTO & SimPO)
KV cacheKey-Value Cache
High-throughput serving: PagedAttention & vLLM Build and deploy a monitored, high-throughput AI serving cluster
L1Lasso Regularization / Manhattan Norm
Lasso regression (L1 regularization)
L2Ridge Regularization / Euclidean Norm
Ridge regression (L2 regularization)
LDALinear Discriminant Analysis
Linear Discriminant Analysis (LDA)
LightGBMLight Gradient Boosting Machine
Modern boosters: XGBoost & LightGBM
LOFLocal Outlier Factor
Anomaly detection: Isolation Forests & LOF
LoRALow-Rank Adaptation
Transfer learning & feature extraction The modern LLM post-training pipeline
LSTMLong Short-Term Memory
Vanishing gradients in RNNs & LSTMs
MAEMean Absolute Error
Loss functions: MSE vs. MAE vs. Huber
MambaSelective State-Space Architecture
Selective state spaces (Mamba / Mamba-2)Transformer vs. Mamba on a 128k sequence benchmark
MAPMaximum A Posteriori
Maximum A Posteriori (MAP) estimation Build a Bayesian spam classifier & anomaly detector
MCTSMonte Carlo Tree Search
Process Reward Models (PRM) & Monte Carlo Tree Search
MDPMarkov Decision Process
Markov Decision Processes (MDP)
MEMITMass-Editing Memory in a Transformer
Model editing (ROME & MEMIT)
MLEMaximum Likelihood Estimation
Maximum Likelihood Estimation (MLE)Build a Bayesian spam classifier & anomaly detector
MLPMulti-Layer Perceptron
Multi-Layer Perceptrons (MLP)The forward pass
MoEMixture of Experts
Large-scale distributed training (DDP, FSDP, ZeRO & Megatron-LM)
MSEMean Squared Error
Loss functions: MSE vs. MAE vs. Huber
NeRFNeural Radiance Field
Neural Radiance Fields (NeRF)Instant-NGP & spatial multiresolution hash grids
NMSNon-Maximum Suppression
Object detection fundamentals (Bounding Boxes & IoU)
ODEOrdinary Differential Equation
Continuous-time flow matching Deterministic sampling with DDIM
OLSOrdinary Least Squares
Linear regression & ordinary least squares
PagedAttentionPaged Attention Memory Management (vLLM)
High-throughput serving: PagedAttention & vLLM
PCAPrincipal Component Analysis
PCA: directions of maximum variance Image compression & latent extraction via SVD/PCA
PGDProjected Gradient Descent
Adversarial attacks & robustness (FGSM & PGD)
PPOProximal Policy Optimization
Proximal Policy Optimization (PPO)RLHF: Bradley-Terry reward modeling
PR curvePrecision-Recall Curve
ROC curves, AUC & PR curves
PRMProcess Reward Model
Process Reward Models (PRM) & Monte Carlo Tree Search
PSIPopulation Stability Index
Model monitoring, drift detection & canary rollouts
RAGRetrieval-Augmented Generation
Retrieval-Augmented Generation (RAG)Build a multimodal visual question answering assistant
RBFRadial Basis Function
The kernel trick (RBF & Polynomial)
ReActReasoning and Acting
The ReAct paradigm: Reasoning & Acting
ReflexionSelf-Reflection in Agents
Environment feedback & self-reflection (Reflexion)
ReLURectified Linear Unit
Activation functions: Sigmoid, Tanh, ReLU, GeLU & Swish
ResNetResidual Neural Network
ResNet & residual skip connections
RLAIFReinforcement Learning from AI Feedback
Constitutional AI & RLAIF
RLHFReinforcement Learning from Human Feedback
RLHF: Bradley-Terry reward modeling
RLVRReinforcement Learning with Verifiable Rewards
RL with Verifiable Rewards (RLVR)Train a reasoning model using GRPO & verifiable rewards
RMSNormRoot Mean Square Normalization
RMSNorm
RMSPropRoot Mean Square Propagation
Optimizers: Momentum, RMSProp, Adam & AdamW
RNNRecurrent Neural Network
Recurrent Neural Networks (RNN)
ROCReceiver Operating Characteristic
ROC curves, AUC & PR curves
ROMERank-One Model Editing
Model editing (ROME & MEMIT)
RoPERotary Position Embedding
Rotary Position Embedding (RoPE)Context window scaling (YaRN & LongRoPE)
SAESparse Autoencoder
Sparse Autoencoders (SAE) & monosemanticity Red-team, interpret, and safety-patch a model
ScaNNScalable Nearest Neighbors
Vector databases & ANN search (HNSW & IVF-PQ)
SDFSigned Distance Function
Representing 3D data: Point clouds, meshes & SDFs
SFTSupervised Fine-Tuning
Instruction tuning & Supervised Fine-Tuning (SFT)The modern LLM post-training pipeline
SGDStochastic Gradient Descent
Stochastic & mini-batch gradient descent
SimPOSimple Preference Optimization
Advanced preference alignment (KTO & SimPO)
SRAMStatic Random-Access Memory
FlashAttention (1, 2, 3) & GPU memory hierarchies GPU memory hierarchies & kernel profiling
SSMState-Space Model
State-space models (SSM & S4)Selective state spaces (Mamba / Mamba-2)
SVDSingular Value Decomposition
Singular Value Decomposition (SVD)Image compression & latent extraction via SVD/PCA
SVMSupport Vector Machine
Support Vector Machines (SVM)The kernel trick (RBF & Polynomial)
SWE-benchSoftware Engineering Benchmark
Agent benchmarking & SWE-bench evaluation
t-SNEt-Distributed Stochastic Neighbor Embedding
Non-linear dimensionality reduction: t-SNE
TNTrue Negative
Confusion matrix, precision, recall & F1
TPTrue Positive
Confusion matrix, precision, recall & F1
TPRTrue Positive Rate
ROC curves, AUC & PR curves
U-NetU-Shaped Convolutional Network
Semantic segmentation & the U-Net
UCBUpper Confidence Bound
Multi-armed bandits (Epsilon-greedy & UCB)
UMAPUniform Manifold Approximation and Projection
UMAP (Uniform Manifold Approximation & Projection)Build a customer segmentation & recommendation engine
VAEVariational Autoencoder
Variational Autoencoders (VAE)
ViTVision Transformer
Transfer learning & feature extraction
VLAVision-Language-Action (Robotics)
Embodied AI & Vision-Language-Action (VLA) models Build and deploy a monitored, high-throughput AI serving cluster
vLLMVirtual Large Language Model (Serving Engine)
High-throughput serving: PagedAttention & vLLM Build and deploy a monitored, high-throughput AI serving cluster
VLMVision-Language Model
Vision-Language Models (VLM) assembled Build a multimodal visual question answering assistant
VQ-VAEVector-Quantized Variational Autoencoder
VQ-VAE & discrete latent codebooks
XGBoosteXtreme Gradient Boosting
Modern boosters: XGBoost & LightGBM
XORExclusive OR
The perceptron & the XOR problem
YaRNYet another RoPE extensioN method
Context window scaling (YaRN & LongRoPE)
YOLOYou Only Look Once
One-stage vs. two-stage detectors (YOLO vs. Faster R-CNN)
ZeROZero Redundancy Optimizer
Large-scale distributed training (DDP, FSDP, ZeRO & Megatron-LM)