Type a sentence, get back the photo it describes. That's the entire product — and every piece needed to build it was already built, one chapter at a time, earlier in this part.
Pick a query — the same three captions from Chapter 1, standing in for whatever a user might type. The engine embeds it, searches every image by distance in the shared space, and returns whichever one is closest. That's the whole search engine: an embedding, a nearest-neighbor scan, and a ranking.
Type a query , embed it into the shared space (Chapters 1 and 2), and rank every image by distance:
- — the user's typed query, embedded into the shared image-text space.
- — one image in the indexed collection being ranked against the query.
- — the full collection of indexed images being searched.
- — every indexed image, sorted by distance to the query — nearest (best match) first.
- There's no separate search algorithm beyond this
Embed the query, rank by distance — that's the entire mechanism, with nothing hidden behind it.
- Real systems scale it up, not change it
A real system indexes millions of images instead of three, and uses an approximate nearest-neighbor structure instead of scanning every one — but the underlying question, "which indexed vector sits closest to my query's vector," never changes.
Check the full ranking, not just the winner. Every query's true image wins by a margin of more than units over the runner-up — the same wide, unambiguous gap Chapter 1 first pointed out, now doing the actual job of a product feature instead of just illustrating a concept.
Query: "a bird flying," embedded at :
- Compute distance to every indexed image
- Bird photo:
- Dog photo:
- Cat photo:
- Sort and return the top result
The bird photo wins by a margin of over the next-best result — the same wide-margin pattern every query in this space produces.
- This is the whole engine
Embed the query (Chapters 1–2), rank by distance (Chapter 1), done. Everything else — a real text encoder, a real image index, approximate search over millions of vectors — is engineering on top of exactly this idea, not a different one.
Click the image the search engine should return for “a bird flying.”
Every chapter in this part fed into this one: a shared space to make text and images comparable (Chapter 1), a loss that places that space without hand labels (Chapter 2), attention to connect the two modalities at a finer grain than "whole image vs. whole caption" (Chapters 3–4), and retrieval to ground an answer in something looked up rather than merely remembered (Chapter 5). None of it required a new kind of mathematics — every formula in this part was a derivative, a softmax, or a distance, applied across a boundary between two different kinds of data instead of within one. Multimodal AI returns later in the course to add a third and fourth modality — audio and video — and a generative direction this part never tried: producing a new image instead of just retrieving one.