Part XV — Multimodal Foundation Models: CLIP, Cross-Attention & VLMs · Chapter 10

Build a multimodal visual question answering assistant

Hook

Type a sentence, get back the photo it describes. That's the entire product — and every piece needed to build it was already built, one chapter at a time, earlier in this part.

Intuition
Image: dogImage: catImage: birdCaption: a dog runningCaption: a cat sleepingCaption: a bird flying
top match: "Image: dog" (distance 0.28, margin 2.56 over the runner-up)

Pick a query — the same three captions from Chapter 1, standing in for whatever a user might type. The engine embeds it, searches every image by distance in the shared space, and returns whichever one is closest. That's the whole search engine: an embedding, a nearest-neighbor scan, and a ranking.

Formalize

Type a query qq, embed it into the shared space (Chapters 1 and 2), and rank every image by distance:

results(q)=sortimageindex  qimage\text{results}(q) = \text{sort}_{\,\text{image} \in \text{index}}\; \lVert q - \text{image} \rVert
  • qq — the user's typed query, embedded into the shared image-text space.
  • image\text{image} — one image in the indexed collection being ranked against the query.
  • index\text{index} — the full collection of indexed images being searched.
  • results(q)\text{results}(q) — every indexed image, sorted by distance to the query — nearest (best match) first.
  1. There's no separate search algorithm beyond this

    Embed the query, rank by distance — that's the entire mechanism, with nothing hidden behind it.

  2. Real systems scale it up, not change it

    A real system indexes millions of images instead of three, and uses an approximate nearest-neighbor structure instead of scanning every one — but the underlying question, "which indexed vector sits closest to my query's vector," never changes.

Play
Image: dogImage: catImage: birdCaption: a dog runningCaption: a cat sleepingCaption: a bird flying
ranked: Image: cat (0.36), Image: dog (2.78), Image: bird (3.79)

Check the full ranking, not just the winner. Every query's true image wins by a margin of more than 22 units over the runner-up — the same wide, unambiguous gap Chapter 1 first pointed out, now doing the actual job of a product feature instead of just illustrating a concept.

Worked example

Query: "a bird flying," embedded at (3.7,0.4)(3.7, 0.4):

  1. Compute distance to every indexed image
    • Bird photo: 0.50.5
    • Dog photo: 3.113.11
    • Cat photo: 3.753.75
  2. Sort and return the top result

    The bird photo wins by a margin of 3.110.5=2.613.11 - 0.5 = 2.61 over the next-best result — the same wide-margin pattern every query in this space produces.

  3. This is the whole engine

    Embed the query (Chapters 1–2), rank by distance (Chapter 1), done. Everything else — a real text encoder, a real image index, approximate search over millions of vectors — is engineering on top of exactly this idea, not a different one.

Checkpoint

Click the image the search engine should return for “a bird flying.”

Image: dogImage: catImage: birdCaption: a dog runningCaption: a cat sleepingCaption: a bird flying
click an image
Click an image to try it
Summary
results(q)=sortimageindex  qimage\text{results}(q) = \text{sort}_{\,\text{image} \in \text{index}}\; \lVert q - \text{image} \rVert

Every chapter in this part fed into this one: a shared space to make text and images comparable (Chapter 1), a loss that places that space without hand labels (Chapter 2), attention to connect the two modalities at a finer grain than "whole image vs. whole caption" (Chapters 3–4), and retrieval to ground an answer in something looked up rather than merely remembered (Chapter 5). None of it required a new kind of mathematics — every formula in this part was a derivative, a softmax, or a distance, applied across a boundary between two different kinds of data instead of within one. Multimodal AI returns later in the course to add a third and fourth modality — audio and video — and a generative direction this part never tried: producing a new image instead of just retrieving one.