summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-13 14:50:09 +0100
committerskal <pascal.massimino@gmail.com>2026-02-13 14:50:09 +0100
commit4d1df90124418a57cb9ed62eab25497d53a47c8c (patch)
tree1f7ccd9a3cea593cbb39c53d839cdd8be97c050f
parentc01f3e6b676134fe01036aabbd063b690c9fd1ed (diff)
CNN v2 web tool: Fix layer naming and visualization bugs
- Align layer naming with codebase: Layer 0/1/2 (not Layer 1/2/3) - Split static features: Static 0-3 (p0-p3) and Static 4-7 (uv,sin,bias) - Fix Layer 2 not appearing: removed isOutput filter from layerOutputs - Fix canvas context switching: force clear before recreation - Disable static buttons in weights mode - Add ASCII pipeline diagram to CNN_V2.md Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
-rw-r--r--doc/CNN_V2.md54
-rw-r--r--doc/CNN_V2_WEB_TOOL.md19
-rw-r--r--tools/cnn_v2_test/index.html138
3 files changed, 165 insertions, 46 deletions
diff --git a/doc/CNN_V2.md b/doc/CNN_V2.md
index b0aa24c..e56b022 100644
--- a/doc/CNN_V2.md
+++ b/doc/CNN_V2.md
@@ -32,6 +32,60 @@ Input RGBD → Static Features Compute → CNN Layers → Output RGBA
└─ computed once/frame ─┘ └─ multi-pass ─┘
```
+**Detailed Data Flow:**
+
+```
+ ┌─────────────────────────────────────────┐
+ │ Static Features (computed once) │
+ │ 8D: p0,p1,p2,p3,uv_x,uv_y,sin10x,bias │
+ └──────────────┬──────────────────────────┘
+ │
+ │ 8D (broadcast to all layers)
+ ├───────────────────────────┐
+ │ │
+ ┌──────────────┐ │ │
+ │ Input RGBD │──────────────┤ │
+ │ 4D │ 4D │ │
+ └──────────────┘ │ │
+ ▼ │
+ ┌────────────┐ │
+ │ Layer 0 │ (12D input) │
+ │ (CNN) │ = 4D + 8D │
+ │ 12D → 4D │ │
+ └─────┬──────┘ │
+ │ 4D output │
+ │ │
+ ├───────────────────────────┘
+ │ │
+ ▼ │
+ ┌────────────┐ │
+ │ Layer 1 │ (12D input) │
+ │ (CNN) │ = 4D + 8D │
+ │ 12D → 4D │ │
+ └─────┬──────┘ │
+ │ 4D output │
+ │ │
+ ├───────────────────────────┘
+ ▼ │
+ ... │
+ │ │
+ ▼ │
+ ┌────────────┐ │
+ │ Layer N │ (12D input) │
+ │ (output) │◄──────────────────┘
+ │ 12D → 4D │
+ └─────┬──────┘
+ │ 4D (RGBA)
+ ▼
+ Output
+```
+
+**Key Points:**
+- Static features computed once, broadcast to all CNN layers
+- Each layer: previous 4D output + 8D static → 12D input → 4D output
+- Ping-pong buffering between layers
+- Layer 0 special case: uses input RGBD instead of previous layer output
+
**Static Features Texture:**
- Name: `static_features`
- Format: `texture_storage_2d<rgba32uint, write>` (4×u32)
diff --git a/doc/CNN_V2_WEB_TOOL.md b/doc/CNN_V2_WEB_TOOL.md
index 81549ab..a5162a6 100644
--- a/doc/CNN_V2_WEB_TOOL.md
+++ b/doc/CNN_V2_WEB_TOOL.md
@@ -6,7 +6,7 @@ Browser-based WebGPU tool for validating CNN v2 inference with layer visualizati
---
-## Status (2026-02-13)
+## Status (2026-02-13 evening)
**Working:**
- ✅ WebGPU initialization and device setup
@@ -16,13 +16,20 @@ Browser-based WebGPU tool for validating CNN v2 inference with layer visualizati
- ✅ Mode switching (Activations/Weights tabs)
- ✅ Canvas context management (2D for weights, WebGPU for activations)
- ✅ Weight visualization infrastructure (layer selection, grid layout)
+- ✅ Layer naming matches codebase convention (Layer 0, Layer 1, Layer 2)
+- ✅ Static features split visualization (Static 0-3, Static 4-7)
+- ✅ All layers visible including output layer (Layer 2)
-**Not Working:**
-- ❌ Layer activation visualization (all black)
-- ❌ Weight kernel display (canvases empty, but logging shows execution)
+**Recent Fixes:**
+- Fixed Layer 2 not appearing (was excluded from layerOutputs due to isOutput check)
+- Fixed canvas context switching (force clear before recreation)
+- Added Static 0-3 / Static 4-7 buttons to view all 8 static feature channels
+- Aligned naming with train_cnn_v2.py/.wgsl: Layer 0, Layer 1, Layer 2 (not Layer 1, 2, 3)
+- Disabled Static buttons in weights mode (no learnable weights)
-**Partially Working:**
-- ⚠️ Texture readback pipeline (UV gradient test works, data reads fail)
+**Known Issues:**
+- Layer activation visualization may show black if texture data not properly unpacked
+- Weight kernel display depends on correct 2D context creation after canvas recreation
---
diff --git a/tools/cnn_v2_test/index.html b/tools/cnn_v2_test/index.html
index 199deea..e2b0a04 100644
--- a/tools/cnn_v2_test/index.html
+++ b/tools/cnn_v2_test/index.html
@@ -4,10 +4,14 @@
CNN v2 Testing Tool - WebGPU-based inference validator
Architecture:
- - Static features (8D): p0-p3 (parametric), uv_x, uv_y, sin(10*uv_x), bias
+ - Static features (8D): p0-p3 (parametric), uv_x, uv_y, sin(10*uv_x), bias (NOT a CNN layer)
- Layer 0: input RGBD (4D) + static (8D) = 12D → 4 channels
- - Layer 1+: previous (4D) + static (8D) = 12D → 4 channels
- - All layers: uniform 12D input, 4D output (ping-pong buffer)
+ - Layer 1+: previous layer (4D) + static (8D) = 12D → 4 channels
+ - All CNN layers: uniform 12D input, 4D output (ping-pong buffer)
+
+ Naming convention (matches train_cnn_v2.py / .wgsl / .cc):
+ - UI shows: "Static 0-3", "Static 4-7", "Layer 0", "Layer 1", "Layer 2"
+ - weights.layers[] array: Layer 0 = weights.layers[0], Layer 1 = weights.layers[1]
Features:
- Side panel: .bin metadata display, weight statistics per layer
@@ -161,6 +165,14 @@
border-color: #4a9eff;
color: #1a1a1a;
}
+ .layer-buttons button:disabled {
+ opacity: 0.3;
+ cursor: not-allowed;
+ }
+ .layer-buttons button:disabled:hover {
+ border-color: #404040;
+ background: #1a1a1a;
+ }
.layer-grid {
display: grid;
grid-template-columns: 1fr 1fr;
@@ -686,12 +698,12 @@ class CNNTester {
<tbody>
`;
- // Display layers as "Layer 1", "Layer 2", etc. (matching visualization button labels)
+ // Display layers as "Layer 0", "Layer 1", etc. (matching codebase convention)
for (let i = 0; i < layers.length; i++) {
const l = layers[i];
html += `
<tr>
- <td>Layer ${i + 1}</td>
+ <td>Layer ${i}</td>
<td>${l.inChannels}→${l.outChannels} (${l.kernelSize}×${l.kernelSize})</td>
<td>${l.weightCount}</td>
<td>${l.min.toFixed(3)}</td>
@@ -885,7 +897,7 @@ class CNNTester {
staticPass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8));
staticPass.end();
- // Copy static features to persistent storage (Layer 0)
+ // Copy static features to persistent storage (visualization index 0, shown as Static 0-3 / Static 4-7)
encoder.copyTextureToTexture(
{ texture: staticTex },
{ texture: layerTextures[0] },
@@ -940,16 +952,17 @@ class CNNTester {
[srcTex, dstTex] = [dstTex, srcTex];
- // Copy layer output to persistent storage for visualization
+ // Copy CNN layer output to persistent storage for visualization
+ // i=0: Layer 0 → layerTextures[1]
+ // i=1: Layer 1 → layerTextures[2], etc.
encoder.copyTextureToTexture(
{ texture: srcTex },
- { texture: layerTextures[i + 1] }, // +1 because layerTextures[0] is static features
+ { texture: layerTextures[i + 1] },
[width, height]
);
- if (!isOutput) {
- this.layerOutputs.push(layerTextures[i + 1]);
- }
+ // Always push layer outputs for visualization (including output layer)
+ this.layerOutputs.push(layerTextures[i + 1]);
}
const modeBuffer = this.device.createBuffer({
@@ -1014,12 +1027,19 @@ class CNNTester {
</div>
`;
- html += `<div style="font-size: 9px; color: #808080; margin-bottom: 8px; padding-bottom: 8px; border-bottom: 1px solid #404040;">Static features (8D: p0-p3 + spatial) + ${this.weights.layers.length} CNN layers. All layers: 12D→4D.</div>`;
+ html += `<div style="font-size: 9px; color: #808080; margin-bottom: 8px; padding-bottom: 8px; border-bottom: 1px solid #404040;">Static features (8D, split 0-3/4-7) + ${this.weights.layers.length} CNN layers (Layer 0-${this.weights.layers.length - 1}). All CNN layers: 12D→4D.</div>`;
html += '<div class="layer-buttons">';
- for (let i = 0; i < this.layerOutputs.length; i++) {
- // Visualization layers: Static features (i=0), CNN Layer 1 (i=1), CNN Layer 2 (i=2), ...
- const label = i === 0 ? 'Static' : `Layer ${i}`;
+ // Static features: two buttons for 8D (0-3 and 4-7)
+ const staticDisabled = (this.vizMode === 'weights') ? ' disabled' : '';
+ html += `<button onclick="tester.visualizeLayer(0, 0)" id="layerBtn0_0"${staticDisabled}>Static 0-3</button>`;
+ html += `<button onclick="tester.visualizeLayer(0, 4)" id="layerBtn0_4"${staticDisabled}>Static 4-7</button>`;
+
+ // CNN layers: Layer 0, Layer 1, Layer 2, ...
+ for (let i = 1; i < this.layerOutputs.length; i++) {
+ // i=1: Layer 0 (weights.layers[0])
+ // i=2: Layer 1 (weights.layers[1]), etc.
+ const label = `Layer ${i - 1}`;
html += `<button onclick="tester.visualizeLayer(${i})" id="layerBtn${i}">${label}</button>`;
}
html += '</div>';
@@ -1034,8 +1054,8 @@ class CNNTester {
// Create initial canvases
this.recreateCanvases();
- // Auto-select static features (Layer 0)
- this.visualizeLayer(0);
+ // Auto-select static features (channels 0-3)
+ this.visualizeLayer(0, 0);
}
setVizMode(mode) {
@@ -1047,14 +1067,28 @@ class CNNTester {
document.getElementById('vizModeWeights').style.borderColor = mode === 'weights' ? '#4a9eff' : '#404040';
document.getElementById('vizModeWeights').style.color = mode === 'weights' ? '#1a1a1a' : '#e0e0e0';
+ // Update Static button states
+ const staticBtn0 = document.getElementById('layerBtn0_0');
+ const staticBtn4 = document.getElementById('layerBtn0_4');
+ if (staticBtn0) staticBtn0.disabled = (mode === 'weights');
+ if (staticBtn4) staticBtn4.disabled = (mode === 'weights');
+
// Clear and recreate canvas grid
this.recreateCanvases();
- // Re-visualize current layer
+ // Re-visualize current layer (switch to Layer 0 if Static was active in weights mode)
const activeBtn = document.querySelector('.layer-buttons button.active');
if (activeBtn) {
- const layerIdx = parseInt(activeBtn.id.replace('layerBtn', ''));
- this.visualizeLayer(layerIdx);
+ const isStatic = activeBtn.id.startsWith('layerBtn0_');
+ if (isStatic && mode === 'weights') {
+ this.visualizeLayer(1); // Switch to Layer 0 (first CNN layer)
+ } else if (!isStatic) {
+ const layerIdx = parseInt(activeBtn.id.replace('layerBtn', ''));
+ this.visualizeLayer(layerIdx);
+ } else {
+ const channelOffset = parseInt(activeBtn.id.split('_')[1]);
+ this.visualizeLayer(0, channelOffset);
+ }
}
}
@@ -1062,6 +1096,13 @@ class CNNTester {
const grid = document.getElementById('layerGrid');
if (!grid) return;
+ // Force removal of old canvases to clear any WebGPU contexts
+ const oldCanvases = grid.querySelectorAll('canvas');
+ oldCanvases.forEach(canvas => {
+ canvas.width = 0;
+ canvas.height = 0;
+ });
+
grid.innerHTML = '';
for (let c = 0; c < 4; c++) {
const div = document.createElement('div');
@@ -1074,25 +1115,31 @@ class CNNTester {
}
}
- async visualizeLayer(layerIdx) {
+ async visualizeLayer(layerIdx, channelOffset = 0) {
if (!this.layerOutputs || layerIdx >= this.layerOutputs.length) {
this.log(`Cannot visualize layer ${layerIdx}: no data`, 'error');
return;
}
// Update button states
- for (let i = 0; i < this.layerOutputs.length; i++) {
- const btn = document.getElementById(`layerBtn${i}`);
- if (btn) btn.classList.toggle('active', i === layerIdx);
+ document.querySelectorAll('.layer-buttons button').forEach(btn => btn.classList.remove('active'));
+ if (layerIdx === 0) {
+ // Static features
+ const btnId = `layerBtn0_${channelOffset}`;
+ const btn = document.getElementById(btnId);
+ if (btn) btn.classList.add('active');
+ } else {
+ const btn = document.getElementById(`layerBtn${layerIdx}`);
+ if (btn) btn.classList.add('active');
}
- const layerName = layerIdx === 0 ? 'Static Features' : `Layer ${layerIdx}`;
+ const layerName = layerIdx === 0 ? `Static Features (${channelOffset}-${channelOffset + 3})` : `Layer ${layerIdx - 1}`;
// Check mode
if (this.vizMode === 'weights' && layerIdx > 0) {
- // Map visualization layer to weight array index
- // Visualization Layer 1 (layerIdx=1) → weights.layers[0] (CNN Layer 1 weights)
- // Visualization Layer 2 (layerIdx=2) → weights.layers[1] (CNN Layer 2 weights)
+ // Map visualization index to weights.layers[] array index:
+ // layerIdx=1 → Layer 0 → weights.layers[0]
+ // layerIdx=2 → Layer 1 → weights.layers[1], etc.
this.visualizeWeights(layerIdx - 1);
return;
}
@@ -1117,10 +1164,14 @@ class CNNTester {
this.log(`Visualizing ${layerName} activations (${width}×${height})`);
// Update channel labels based on layer type
- // Static features: 8 channels (p0,p1,p2,p3,uv_x,uv_y,sin10_x,bias)
- // CNN layers: 4 channels per layer (uniform)
+ // Static features (layerIdx=0): 8 channels split into two views
+ // CNN layers (layerIdx≥1): 4 channels per layer
+ const staticLabels = [
+ ['Ch0 (p0)', 'Ch1 (p1)', 'Ch2 (p2)', 'Ch3 (p3)'],
+ ['Ch4 (uv_x)', 'Ch5 (uv_y)', 'Ch6 (sin10_x)', 'Ch7 (bias)']
+ ];
const channelLabels = layerIdx === 0
- ? ['Ch0 (p0)', 'Ch1 (p1)', 'Ch2 (p2)', 'Ch3 (p3)']
+ ? staticLabels[channelOffset / 4]
: ['Ch0', 'Ch1', 'Ch2', 'Ch3'];
for (let c = 0; c < 4; c++) {
@@ -1153,9 +1204,9 @@ class CNNTester {
continue;
}
+ // Set canvas size BEFORE getting context
canvas.width = width;
canvas.height = height;
- this.log(`Canvas ${c}: ${width}×${height}`);
const ctx = canvas.getContext('webgpu');
if (!ctx) {
@@ -1165,6 +1216,7 @@ class CNNTester {
try {
ctx.configure({ device: this.device, format: this.format });
+ this.log(`Canvas ${c}: ${width}×${height}, WebGPU context configured`);
} catch (e) {
this.log(`Failed to configure canvas ${c}: ${e.message}`, 'error');
continue;
@@ -1175,8 +1227,9 @@ class CNNTester {
size: 8,
usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST
});
- // Use normal channel (0-3), or debug modes (10=UV, 11=raw, 12=unpacked)
- const paramsData = new Float32Array([c, vizScale]);
+ // Use channel index with offset for static features
+ const actualChannel = channelOffset + c;
+ const paramsData = new Float32Array([actualChannel, vizScale]);
this.device.queue.writeBuffer(paramsBuffer, 0, paramsData);
const bindGroup = this.device.createBindGroup({
@@ -1212,17 +1265,17 @@ class CNNTester {
}
visualizeWeights(cnnLayerIdx) {
- // cnnLayerIdx is index into weights.layers[] array
- // Display as "Layer N" where N = cnnLayerIdx + 1 (e.g., weights.layers[0] = Layer 1)
+ // cnnLayerIdx is index into weights.layers[] array (0-indexed)
+ // Display as "Layer N" matching codebase convention
+ // Example: weights.layers[0] = Layer 0, weights.layers[1] = Layer 1
const layer = this.weights.layers[cnnLayerIdx];
if (!layer) {
- this.log(`CNN Layer ${cnnLayerIdx + 1} not found`, 'error');
+ this.log(`Layer ${cnnLayerIdx} not found`, 'error');
return;
}
const { kernelSize, inChannels, outChannels, weightOffset, min, max } = layer;
- const displayLayerNum = cnnLayerIdx + 1;
- this.log(`Visualizing Layer ${displayLayerNum} weights: ${inChannels}→${outChannels}, ${kernelSize}×${kernelSize}, offset=${weightOffset}`);
+ this.log(`Visualizing Layer ${cnnLayerIdx} weights: ${inChannels}→${outChannels}, ${kernelSize}×${kernelSize}, offset=${weightOffset}`);
this.log(`Weight range: [${min.toFixed(3)}, ${max.toFixed(3)}]`);
// Update channel labels to show output channels
@@ -1240,8 +1293,13 @@ class CNNTester {
continue;
}
- // Need to clear WebGPU context and use 2D
+ // Get 2D context (canvases recreated by recreateCanvases())
const ctx = canvas.getContext('2d', { willReadFrequently: false });
+ if (!ctx) {
+ this.log(`Failed to get 2D context for canvas ${outCh}`, 'error');
+ continue;
+ }
+
const cellSize = 20; // Pixels per weight
const padding = 2;
const gridWidth = inChannels * (kernelSize * cellSize + padding);