Machine learning, deep learning, explainable AI, and multimodal AI — every formula visualized, every chapter tested.
Drag a point along a curve, watch the tangent line, and find the exact spot where the slope hits zero.
Extend downhill descent to multiple parameters simultaneously on a 3D loss surface.
Drag a point through linked functions and watch slopes multiply through a computational graph.
Matrix derivatives, vector-valued transformations, and second-order curvature.
Approximate complex loss landscapes with linear tangents and quadratic bowls.
Split a single convex bowl into non-convex wells and saddle points.
Take steps against the gradient, tune learning rates, and watch overshooting vs. divergence.
Give gradient descent inertia to damp oscillations in narrow ravines.
Optimize objective functions subject to equality and inequality boundary constraints.
Construct a scalar automatic differentiation engine from scratch.
L1, L2, Linf norms, dot products, and cosine distance in feature space.
Watch a matrix rotate, stretch, shear, and project data vectors.
Geometric perspectives on Ax=b and projecting onto column spaces.
Signed volume scaling, singularities, and when a linear system cannot be inverted.
Find invariant axes where a matrix acts as a pure scalar stretch.
Map the shape and spread of high-dimensional data clouds.
Decompose any matrix into rotate-stretch-rotate factors.
Compress information by truncating small singular values with the Eckart-Young theorem.
Derive principal component analysis by maximizing variance on orthogonal axes.
Compress high-resolution imagery and extract eigenfaces interactively.
Build discrete and continuous distributions that strictly sum to one.
Compute first and second moments and measure joint linear variability.
Slice joint probability tables and integrate out nuisance variables.
Update prior beliefs into posterior distributions as new evidence arrives.
Explore the core distributions that govern machine learning models.
Find the parameter values that maximize the likelihood of observed data.
Inject prior knowledge into likelihood estimation to prevent overfitting.
Quantify surprise in bits using Shannon entropy.
Measure relative entropy between true and approximate probability distributions.
Implement full MLE and MAP estimation pipelines with Bayesian updates.
Fit a line by solving the normal equations and inspect residual sum of squares.
Analyze outlier sensitivity across squared, absolute, and robust losses.
Penalize weight magnitudes to stabilize collinear predictors.
Drive sparse feature selection through diamond-shaped geometric constraints.
Combine L1 sparsity with L2 grouping effect.
Squash continuous linear outputs into class probabilities.
Derive log-loss for classification and analyze gradients near saturation.
Generalize binary classification to K classes with normalized exponentials.
Maximize between-class separation while minimizing within-class variance.
Turn Part I's Bayes' rule into a full classifier, one independent feature at a time.
Construct and regularize a multi-class probabilistic linear classifier.
Same dataset, several models from this part, one scoreboard.
Recursively partition feature space using Shannon entropy and Gini impurity.
Prune overgrown decision trees to restore validation generalization.
Reduce variance by aggregating decorrelated bootstrap trees.
Sequentially re-weight misclassified instances with weak learners.
Fit consecutive regression trees to the pseudo-residuals of any differentiable loss.
Second-order Taylor expansions, histogram binning, and tree regularizers.
Construct maximum-margin separating hyperplanes using support vectors.
Map non-linearly separable data into infinite-dimensional Hilbert spaces.
Non-parametric instance-based voting across spatial distance metrics.
Benchmark Random Forest, XGBoost, and Kernel SVMs on complex datasets.
Iteratively update cluster centroids and Voronoi partition cells.
Probabilistic centroid seeding and inertia curve inflection analysis.
Dendrogram hierarchies with single, complete, and Ward's minimum variance linkage.
Discover arbitrary non-spherical cluster geometries and isolate noise points.
Preserve local neighborhood probabilities using Student-t kernel matching.
Fuzzy simplicial sets and preserving global topological structure.
Isolate outliers via random recursive partitioning and local density ratios.
Factorize sparse user-item interaction matrices with alternating least squares.
Decompose trend, seasonality, and autoregressive lags for future projection.
Unify UMAP, HDBSCAN, and collaborative filtering into a complete pipeline.
Same messy dataset, run through kNN, Naive Bayes, and DBSCAN — see which one actually fits its shape.
Diagnose classifier performance on severely imbalanced datasets.
Sweep decision thresholds and trace out discrimination ability.
Mitigate validation variance and prevent subtle data leakage.
Decompose expected test error into irreducible noise, bias, and variance.
Standardize, normalize, and apply Power/Box-Cox transformations.
One-hot, ordinal, target encoding, and weight-of-evidence techniques.
Variance thresholds, mutual information scores, and recursive feature elimination.
Automate parameter sweeps using Optuna and Hyperband search strategies.
Calibrate uncalibrated confidence outputs into true posterior probabilities.
Execute complete cross-validation, hyperparameter tuning, and probability calibration.
A single artificial neuron and why non-linear separation requires depth.
Stack hidden layers to approximate arbitrary continuous functions.
Compare non-linearities and their derivative saturation profiles.
Trace matrix multiplications, biases, and activations layer by layer.
See why backpropagation is the chain rule applied backward through layers.
Walk through analytical gradient computations by hand and verify against autograd.
Explore saddle points, plateaus, ravines, and local minima.
Batch size tradeoffs, noisy gradient estimates, and generalization.
Race adaptive optimizers down complex loss surfaces.
Write Tensor abstractions, layers, loss functions, and AdamW optimizer.
Preserve activation and gradient variance across deep layers.
Analyze spectral norms, gradient explosion, and gradient clipping.
Normalize activations across mini-batches to accelerate convergence.
Normalize across feature channels for small batches and sequences.
Streamline normalization by scaling by root-mean-square without mean subtraction.
Stochastically deactivate neurons during training to prevent co-adaptation.
Understand why decoupled weight decay is essential when using Adam.
Warm up learning rates and decay smoothly with cosine annealing.
Accelerate training and conserve memory with dynamic loss scaling.
Overcome vanishing gradients using modern initialization, RMSNorm, and AdamW.
Slide 2D kernels across images with stride, dilation, and padding.
Downsample feature maps with max and average pooling operations.
Assemble convolutions, activations, and pooling into a working classifier.
Allow gradient signals to pass unimpeded through 100+ layer networks.
Depthwise separable convolutions for efficient edge computing.
Repurpose pretrained vision representations for downstream tasks.
Predict bounding box coordinates and filter overlaps using NMS.
Compare single-pass grid predictions with region proposal networks.
Encoder-decoder architecture with contracting and expansive skip paths.
Train a CNN live in the browser and visualize convolutional kernel activations.
Take a network that already works, and specialize it to a new task in a fraction of the steps.
Variable context length, token order, and temporal dependencies.
Propagate hidden states step-by-step through recurrent loops.
Protect long-term sequence memories with forget, input, and output gates.
Streamline recurrence by merging cell states and hidden states.
Compress source sequences into fixed vectors and decode target sequences.
Map discrete vocabulary tokens into dense continuous semantic vector spaces.
Segment raw text into subword token vocabularies.
Dynamic alignment allowing every token to query the entire sequence.
Compute Query-Key-Value dot products across parallel projection subspaces.
Assemble Multi-Head Attention, LayerNorm, Residual connections, and MLPs.
Inject token order into permutation-invariant attention matrices.
Rotate query and key vectors in 2D coordinate pairs to preserve relative distances.
Interpolate RoPE frequencies to extend context length from 8k to 128k+ tokens.
Quantify the O(N^2) memory and compute cost of full self-attention.
Tiling attention computations between SRAM and HBM using online softmax.
Swap softmax for kernel feature maps to achieve O(N) linear time attention.
Discretize continuous linear dynamical systems into efficient sequence models.
Input-dependent time-varying parameters and hardware-accelerated associative scans.
Combine Transformer attention layers with Mamba SSM blocks.
Profile speed, memory consumption, and needle-in-a-haystack retrieval accuracy.
Compress images through a latent bottleneck and reconstruct them.
Sample continuous latents using the reparameterization trick and ELBO loss.
Quantize continuous vectors to discrete codebook entries for generative modeling.
Track exact probability densities through invertible neural network layers.
Follow gradients of log-density via Langevin dynamics to denoise samples.
Step-by-step Gaussian forward corruption and learned reverse denoising.
Accelerate reverse diffusion using non-Markovian deterministic ODE trajectories.
Learn straight optimal transport velocity fields between noise and data distributions.
Replace convolutional U-Net denoisers with scalable Transformer backbones.
Train a velocity vector field and sample generated images with ODE solvers.
Represent geometry through discrete point sets, polygon meshes, and signed distance fields.
Pinhole cameras, intrinsic/extrinsic matrices, and casting rays into 3D space.
Synthesize novel views via continuous volumetric rendering functions.
Accelerate neural field training from hours to seconds with hash encodings.
Represent 3D scenes as millions of anisotropic 3D ellipsoidal Gaussians.
Sort Gaussians and project them in real time with adaptive densification.
Generate full 3D meshes from single 2D images with feedforward triplane networks.
Optimize 3D Gaussians from multi-view photographs and render in real time.
Project images, text, and audio into a unified semantic metric space.
InfoNCE loss, symmetric cross-entropy, and zero-shot image classification.
Condition generation by querying visual features with text tokens.
Connect vision encoders with autoregressive language models.
Convert 2D image patches into embedding tokens with linear and Perceiver projectors.
Generate mixed sequences of text and image tokens in a single stream.
Extract mel-spectrograms and align acoustic representations with language.
Process multi-frame video sequences across spatial and temporal dimensions.
Ground multimodal responses with dense vector search and reranking.
Combine visual embeddings, dense retrieval, and a VLM into a working assistant.
Audio in, retrieval in the middle, an image out — every modality from this course, in one pipeline.
Balance exploration vs. exploitation across uncertain payoff distributions.
States, actions, state transitions, rewards, and discount factors.
Recursive decomposition of expected cumulative future rewards.
Solve exact optimal policies when transition dynamics are fully known.
Learn optimal action-value tables through off-policy bootstrapping.
Stabilize neural Q-function estimation with experience replay and target networks.
Directly ascend expected reward gradients using the log-derivative trick.
Reduce policy gradient variance by subtracting value function baselines.
Clip probability ratio updates to prevent destructive policy collapse.
Train an autonomous agent live from scratch using Proximal Policy Optimization.
Deconstruct the journey from pretraining to SFT, preference alignment, and reasoning RL.
Curate high-quality conversational datasets and apply loss masking.
Fit reward models to human pairwise preference data and align policies via PPO.
Derive closed-form preference optimization without training an explicit reward model.
Align on unpaired feedback and reference-free target margins.
Eliminate critic networks by normalizing advantages across sampled groups.
Deterministic programmatic reward verification for math and competitive coding.
Trade inference latency for accuracy via long Chain-of-Thought (CoT) and self-reflection.
Score reasoning step-by-step and search token trajectories with MCTS.
Train a compact model to produce verifiable step-by-step mathematical reasoning.
LoRA, a system prompt, and a KV-cache — the whole gap between a capstone and a product.
Emit structured JSON tool arguments and parse external environment outputs.
Interleave Thought, Action, and Observation loops to solve multi-step problems.
Decompose ambiguous goals into ordered subtasks with dynamic replanning.
Manage context windows and retrieve episodic memories dynamically.
Analyze compiler errors and terminal logs to self-correct execution mistakes.
Coordinator-worker architectures, agent role specialization, and debate protocols.
Evaluate software engineering agents on real-world GitHub issues.
Isolate agent code execution within secure Docker/WASM sandboxes.
Design approval checkpoints, budget thresholds, and user interventions.
Construct an agent that reads files, modifies code, runs tests, and fixes bugs.
Automated principle-based self-critique and revision without human annotations.
Generate input perturbations that fool classifiers while remaining imperceptible.
Audit safety filters against indirect injections, base64 obfuscation, and persona hijacks.
Compute demographic parity, equalized odds, and calibration disparity.
Probe internal representations using linear probing and activation patching.
Decompose polysemantic hidden activations into clean monosemantic feature dictionaries.
Trace specific attention head circuits responsible for in-context pattern replication.
Locate and update specific factual associations directly in MLP weights.
Measure internal model uncertainty to flag hallucinations before output generation.
Extract features with an SAE, discover a vulnerability, and patch it in place.
Map visual camera observations directly to low-level robotic motor torques (RT-2/OpenVLA/pi0).
Predict smooth, multi-modal continuous physical action trajectories using flow matching.
Bridge the reality gap with domain randomization and high-throughput GPU physics.
Understand SRAM vs. HBM bandwidth, Tensor Core utilization, and arithmetic intensity.
Compress model weights into FP8, INT4, and ternary 1.58-bit representations.
Manage dynamic KV-cache memory blocks without fragmentation to achieve 10x throughput.
Draft multiple future tokens concurrently and verify in parallel to reduce latency.
Distribute massive models across clusters using Tensor, Pipeline, and Sequence Parallelism.
Index millions of embeddings and query nearest neighbors in sub-millisecond time.
Ensure point-in-time correctness and synchronize offline training with online serving.
Detect covariate and concept drift with Kolmogorov-Smirnov and PSI statistics.
Deploy a vLLM serving pipeline with INT4 AWQ quantization, Prometheus metrics, and drift alarms.
Why the most accurate model on the leaderboard is often the least explainable.
Highlight exactly which pixels actually moved a prediction.
See what a CNN was visually "looking at" the moment it decided.
Walk a straight line from a blank baseline to the real input, and integrate the gradient the whole way there.
Explain any black-box model by locally approximating it with something simple.
A game-theoretic, provably fair way to split credit among features.
Shapley values made practical and applied to an actual trained classifier.
"What's the smallest change that would have flipped this decision?"
Saliency, SHAP, and a counterfactual on the same case, compared side by side.
Sweep one feature across its whole range and watch the model's average answer trace a curve.
Read off which feature did the most work — for free, from a forest you already trained.
Replace a linear local approximation with a simple if-then rule that's true almost everywhere nearby.
Saliency, SHAP, PDP, tree importance, and an Anchor rule, on the exact same prediction.
Turn a handful of dots and lines into an adjacency matrix, and see why a grid convolution has nowhere to slide.
Watch a node's feature update by averaging its neighbors', layer after layer, until far-apart nodes start to look alike.
Sample a handful of neighbors instead of all of them, then apply the same rule to a node the network has never seen.
Give every neighbor its own attention weight instead of a flat average, and watch the important ones dominate.
Wire message passing and attention together to guess a tiny molecule's property from its atoms and bonds alone.
Cut an image into patches, and feed them to the exact same block that already reads sentences.
Hide a word in the middle of a sentence and watch a model reconstruct it from both directions at once.
Two networks compete — one forges, one detects — and both get better together.
Collapse an entire denoising trajectory onto one endpoint, and generate a sample in a single jump.
Steer the reverse-noise process with a caption, and watch the same denoiser draw a different picture.
Add a router that sends each token to only two of dozens of feedforward blocks.
Freeze a huge weight matrix, and learn only a tiny low-rank update instead.
Change nothing but the input, and watch a frozen model's answer change anyway.
Fit a straight line through a log-log plot, and predict a loss you haven't trained for yet.
Change one hyperparameter, keep everything else fixed, and see why "everything else" is the hard part to guarantee.