START HERE
Explore graph ResidualClassifier4-step tour
The residual path
What happens where the skip path rejoins?
TensorViz
MODEL EXPLORER / CURATED BY TENSORVIZ
A model worth
looking inside.
Follow a connection. Unfold a block.
Find the idea behind the architecture.
CHOOSE A LEARNING PATH
11 lessons
From attention to a modern decoder
Start with attention, change one design choice at a time, then connect the pieces in LLaMA.
Follow the path ↗6 lessonsFrom pixels to spatial understanding
Follow convolution, residual paths, U-shaped skips and image patches through landmark vision designs.
Follow the path ↗5 lessonsHow models learn and adapt
Explore alignment, bidirectional pretraining, image–text pairing, low-rank adaptation and reasoning post-training.
Follow the path ↗4 lessonsFrom adversarial learning to diffusion
Compare the GAN game with noise prediction, latent-space denoising and a Transformer diffusion backbone.
Follow the path ↗3 lessonsThree different ways to use context
Inspect selective recurrent state, Evoformer sequence/pair coupling and promptable mask decoding.
Follow the path ↗A teaching order, not a claim that each later model adopted every earlier idea.
29 models to explore
TENSOR SHAPES
Explore graph SmallCNN4-step tour
From pixels to features
Where does an image become a classifier input?
TensorViz
ATTENTION
Explore graph TokenMixer4-step tour
Inside self-attention
How do queries, keys and values meet?
TensorViz
MODERN LLMS
Explore graph RMSNorm4-step tour
Scale without centering
Component · 2019
What does normalization change in one token vector?
TensorViz
MODERN LLMS
Explore graph SwiGLU4-step tour
A gate inside the feed-forward block
Component · 2020
Why does this feed-forward block split into two branches?
TensorViz
MODERN LLMS
Explore graph Transformer decoder4-step tour
One token at a time
Architecture · 2017
How does a decoder form a next-token distribution?
TensorViz
MODERN LLMS
Explore graph Pre-LN / Post-LN4-step tour
Where the normalization goes
Component · 2020
What happens to the shortcut when normalization moves?
TensorViz
MODERN LLMS
Explore graph MHA / MQA / GQA4-step tour
Eight queries, fewer key/value heads
Component · 2023
Which attention heads can share a key/value cache?
TensorViz
MODERN LLMS
Explore graph RoPE / RoFormer4-step tour
Put position into attention
Component · 2021
Where does token position enter the attention score?
TensorViz
ATTENTION SYSTEMS
Explore graph FlashAttention4-step tour
Compute attention one tile at a time
Inference method · 2022
Can exact attention avoid storing the entire score matrix?
TensorViz
ATTENTION SYSTEMS
Explore graph Multi-head Latent Attention4-step tour
Compress the cache, keep position separate
Component · 2024
What must MLA store to avoid reconstructing every key and value?
TensorViz
MODERN LLMS
Explore graph Sparse MoE / Switch / Mixtral4-step tour
Let a router choose the experts
Component · 2017
How can a token use only part of a larger network?
TensorViz
MODERN LLMS
Explore graph LoRA4-step tour
Learn a small update to frozen weights
Training method · 2021
How can two small matrices adapt a frozen linear layer?
TensorViz
MODERN LLMS
Explore graph LLaMA architecture synthesis4-step tour
Bring the modern decoder pieces together
Architecture · 2023
Where do RMSNorm, RoPE and SwiGLU meet in an actual decoder block?
TensorViz
VISION MILESTONES
Explore graph AlexNet4-step tour
The convolutional network that changed ImageNet
Architecture · 2012
How do five convolutions turn an image into a classification vector?
TensorViz
VISION MILESTONES
Explore graph ResNet-184-step tour
Follow shortcuts through eighteen layers
Architecture · 2015
What can flow through a shortcut when a residual branch changes?
TensorViz
VISION MILESTONES
Explore graph U-Net4-step tour
Bring spatial detail back into the decoder
Architecture · 2015
How does a segmentation decoder recover features lost during downsampling?
TensorViz
VISION MILESTONES
Explore graph Vision Transformer4-step tour
Turn an image into tokens
Architecture · 2020
Where does an image become a sequence for a Transformer?
TensorViz
LANGUAGE MILESTONES
Explore graph Bahdanau additive attention4-step tour
Choose source context for the next decoder step
Component · 2014
How does a recurrent decoder decide which source positions to consult?
TensorViz
LANGUAGE MILESTONES
Explore graph BERT pretraining4-step tour
Learn from both sides of a masked token
Architecture · 2018
What changes when the encoder can see both left and right context?
TensorViz
GENERATIVE MODELS
Explore graph Generative adversarial networks4-step tour
Train a generator against a discriminator
Training method · 2014
Which network is being updated, and what is it trying to change?
TensorViz
GENERATIVE MODELS
Explore graph Denoising diffusion probabilistic models4-step tour
Learn to reverse a noise schedule
Training method · 2020
How does a model learn to reverse corruption?
TensorViz
GENERATIVE MODELS
Explore graph Latent diffusion4-step tour
Move denoising into a compressed space
Architecture · 2021
Why denoise a latent, and where does text enter?
TensorViz
GENERATIVE MODELS
Explore graph Diffusion Transformer (DiT)4-step tour
Use a Transformer as the diffusion backbone
Architecture · 2022
What changes when a Transformer denoises latent patches?
TensorViz
MULTIMODAL MILESTONES
Explore graph CLIP4-step tour
Meet in a shared image–text space
Training method · 2021
Where do the image and text meet?
TensorViz
MULTIMODAL MILESTONES
Explore graph Segment Anything · promptable decoder4-step tour
Turn a prompt into candidate masks
Component · 2023
How do image features and a user prompt become a mask?
TensorViz
ALTERNATIVE ARCHITECTURES
Explore graph Mamba selective state-space models4-step tour
Carry context through a selective state
Architecture · 2023
How does the input decide what a recurrent state retains?
TensorViz
ALTERNATIVE ARCHITECTURES
Explore graph AlphaFold2 · Evoformer coupling4-step tour
Exchange alignment and residue-pair information
Component · 2021
How do sequence and pair representations interact before structure is produced?
TensorViz
LANGUAGE MILESTONES
Explore graph DeepSeek-R1 post-training4-step tour
See what changes during reasoning post-training
Training method · 2025
What changes in the learning process while the base architecture stays the same?
TensorViz
No models match yet.
Try another idea, or return to the full collection.
BRING IT BACK TO YOUR PROJECT
Understand it here.
Keep exploring locally.
Every model includes its source and a small reproduction recipe. Open your own PyTorch class with TensorViz in VS Code.
Use TensorViz in VS Code