MODEL EXPLORER / MULTIMODAL MILESTONES
Meet in a shared image–text space
Where do the image and text meet?
RECORDED NUMERICAL EXAMPLE
Pair the batch in both directions.
Recorded similarities for three synthetic image/token pairs. Compare pairing order and temperature using the same untrained encoder weights.
Rows are images; columns are texts in the displayed order. Diagonal targets encode the supplied pairing, not model knowledge. Temperature rescales the same cosines before both softmax directions. These random encoders have no learned image–text alignment.
Min 0.1821 · Max 0.2774
Recorded cell values
| Row / col | 0 | 1 | 2 |
|---|---|---|---|
| 0 | 0.2413 | 0.1821 | 0.2245 |
| 1 | 0.257 | 0.194 | 0.2446 |
| 2 | 0.2774 | 0.2201 | 0.2607 |
Min 0.1811 · Max 0.4513
Recorded cell values
| Row / col | 0 | 1 | 2 |
|---|---|---|---|
| 0 | 0.4513 | 0.1938 | 0.3549 |
| 1 | 0.4456 | 0.1811 | 0.3733 |
| 2 | 0.4486 | 0.1979 | 0.3535 |
Min 0.2493 · Max 0.4406
Recorded cell values
| Row / col | 0 | 1 | 2 |
|---|---|---|---|
| 0 | 0.2547 | 0.3188 | 0.4265 |
| 1 | 0.2561 | 0.3033 | 0.4406 |
| 2 | 0.2493 | 0.3324 | 0.4183 |
Each heatmap uses its own scale. Mint is positive, rust is negative, and the lightest color is zero. Table values are rounded to four significant digits.
Bars share one scale within this example. Values are rounded to four significant digits.
- Temperature
- 0.07
- Symmetric contrastive loss
- 1.163
- Mean image-row entropy · nats
- 1.043
Verified by the local recipe
- Both projected modalities have unit norm; logits are scaled pairwise cosines
- Joint pair permutations preserve the objective; text-only permutations move similarity columns
- The text attention is causal and the symmetric loss equals both diagonal log-probabilities
- Both encoders and the learned log-temperature receive gradients; a small SGD step lowers this batch loss
Download the source and run.py to reproduce these checks. Choosing an example here replays recorded values.
THE SAME MODEL, ON YOUR MACHINE
Pick up where
the graph leaves off.
Trace two independent encoders into normalized embeddings, then compare both directions of the image–text contrastive objective.
Use in VS Code ↗1. Save these files in one folder
2. Reproduce the example
Use Python 3.13 in a dedicated environment, matching the tested recipe. These commands are for macOS / Linux shells.
python3.13 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python run.pyExpected output: [3,3]
Input 1: [3,3,8,8] · float32 randn
Input 2: [3,4] · int64 ones · CPU / eval · seed 0
3. Open the source in TensorViz
Install the preview, open this folder in local VS Code, and choose Open as Graph above TinyCLIP in model.py. Select your Python environment. Save before choosing Validate.
Installation and supported setup ↗
The recipe was run with Python 3.13.13 and Torch 2.7.1. This is an untrained example; a successful shape check does not measure accuracy.
ABOUT THIS EXAMPLE
Trace two independent encoders into normalized embeddings, then compare both directions of the image–text contrastive objective.
Training method · 2021
CLIP
One width-eight, two-head block per encoder. Four image patches plus a class token; four text positions with fixed final EOS pooling. Linear projections into four shared features and learned log-temperature.
Source, capture & limitations +
Reduced untrained ViT/text towers with a vocabulary of sixteen, fixed-length inputs, GELU and no tokenizer or augmentation pipeline. Text pooling assumes EOS occupies the final position. Synthetic pairs teach the objective, not image–text understanding or zero-shot accuracy.
Original TensorViz teaching example. PyTorch provides the underlying operators.
No separate redistribution license has been declared for these project examples.
- Content revision
f82eeaabb5d24e21- Source SHA-256
a7e72c3bbad2e899038b77cab7a58e4115cbe191500180e55a0058316f782711- Captured
- 2026-09-17 · Python 3.13.13 / Torch 2.7.1
Reduced image/text encoders, normalized cosine scores, symmetric contrastive loss, pairing permutations, causal text visibility and gradient checks. Synthetic images and token IDs; no trained semantic alignment.
Read the model manifest ↗