MODEL EXPLORER / VISION MILESTONES
Follow shortcuts through eighteen layers
What can flow through a shortcut when a residual branch changes?
RECORDED NUMERICAL EXAMPLE
A shortcut around two convolutions.
Recorded intervention on an untrained identity-shortcut basic block. The full graph preserves ResNet-18's stage pattern, while this comparison isolates a single residual addition.
The first basic-block form is tested on fixed synthetic features while its final convolution weights are scaled. The shortcut remains fixed. Original ResNet basic blocks apply ReLU after addition, so a zero branch returns ReLU(x), not every signed input unchanged. This demonstrates arithmetic and a gradient path, not an empirical training-stability result.
Min -1.4 · Max 0.6
Recorded cell values
| Row / col | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|---|
| 0 | -1.4 | -1.368 | -1.337 | -1.305 | -1.273 | -1.241 | -1.21 | -1.178 |
| 1 | -1.146 | -1.114 | -1.083 | -1.051 | -1.019 | -0.9873 | -0.9556 | -0.9238 |
| 2 | -0.8921 | -0.8603 | -0.8286 | -0.7968 | -0.7651 | -0.7333 | -0.7016 | -0.6698 |
| 3 | -0.6381 | -0.6063 | -0.5746 | -0.5429 | -0.5111 | -0.4794 | -0.4476 | -0.4159 |
| 4 | -0.3841 | -0.3524 | -0.3206 | -0.2889 | -0.2571 | -0.2254 | -0.1937 | -0.1619 |
| 5 | -0.1302 | -0.09841 | -0.06667 | -0.03492 | -0.003175 | 0.02857 | 0.06032 | 0.09206 |
| 6 | 0.1238 | 0.1556 | 0.1873 | 0.219 | 0.2508 | 0.2825 | 0.3143 | 0.346 |
| 7 | 0.3778 | 0.4095 | 0.4413 | 0.473 | 0.5048 | 0.5365 | 0.5683 | 0.6 |
Min 0 · Max 0
Recorded cell values
| Row / col | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Min 0 · Max 0.6
Recorded cell values
| Row / col | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 5 | 0 | 0 | 0 | 0 | 0 | 0.02857 | 0.06032 | 0.09206 |
| 6 | 0.1238 | 0.1556 | 0.1873 | 0.219 | 0.2508 | 0.2825 | 0.3143 | 0.346 |
| 7 | 0.3778 | 0.4095 | 0.4413 | 0.473 | 0.5048 | 0.5365 | 0.5683 | 0.6 |
Each heatmap uses its own scale. Mint is positive, rust is negative, and the lightest color is zero. Table values are rounded to four significant digits.
Bars share one scale within this example. Values are rounded to four significant digits.
- Residual branch scale
- 0
- Elements clipped by output ReLU
- 256
Verified by the local recipe
- A zero residual branch leaves ReLU(x), including the original post-addition activation
- The shortcut carries unit gradients on positive inputs when the residual branch is zero
- Stride-two projection shortcuts match the residual branch channel and spatial dimensions
- Four [2,2,2,2] stages produce 16/8/4/2-pixel maps before global averaging
Download the source and run.py to reproduce these checks. Choosing an example here replays recorded values.
THE SAME MODEL, ON YOUR MACHINE
Pick up where
the graph leaves off.
Explore the full ResNet-18 block pattern at reduced width, with identity and projection shortcuts, batch normalization and post-addition ReLU.
Use in VS Code ↗1. Save these files in one folder
2. Reproduce the example
Use Python 3.13 in a dedicated environment, matching the tested recipe. These commands are for macOS / Linux shells.
python3.13 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python run.pyExpected output: [1,10]
Input 1: [1,3,64,64] · float32 randn · CPU / eval · seed 0
3. Open the source in TensorViz
Install the preview, open this folder in local VS Code, and choose Open as Graph above ReducedResNet18 in model.py. Select your Python environment. Save before choosing Validate.
Installation and supported setup ↗
The recipe was run with Python 3.13.13 and Torch 2.7.1. This is an untrained example; a successful shape check does not measure accuracy.
ABOUT THIS EXAMPLE
Explore the full ResNet-18 block pattern at reduced width, with identity and projection shortcuts, batch normalization and post-addition ReLU.
Architecture · 2015
ResNet-18
Original [2,2,2,2] basic-block stage pattern, 7×7 stride-2 stem and max pool, widths reduced to 8/16/32/64. Uses batch normalization, stride-2 projection shortcuts and ReLU after addition.
Source, capture & limitations +
Untrained 64×64 reference with reduced widths and ten output classes. Unlike the earlier generic residual example, this follows the ResNet-18 basic-block pattern. Numerical shortcut checks do not establish training performance or accuracy; BatchNorm runs with initial evaluation statistics.
Original TensorViz teaching example. PyTorch provides the underlying operators.
No separate redistribution license has been declared for these project examples.
- Content revision
22389a64ed4ca366- Source SHA-256
7d174000085621c697f0c024c9f4c596f637ffdbcab9ed091cdf6b4e4a173005- Captured
- 2026-09-17 · Python 3.13.13 / Torch 2.7.1
Reduced-width ResNet-18 CPU forward, zero-branch and shortcut gradient checks, projected-shape agreement and stage-shape/global-average verification.
Read the model manifest ↗