diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-13 14:50:09 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-13 14:50:09 +0100 |
| commit | 4d1df90124418a57cb9ed62eab25497d53a47c8c (patch) | |
| tree | 1f7ccd9a3cea593cbb39c53d839cdd8be97c050f | |
| parent | c01f3e6b676134fe01036aabbd063b690c9fd1ed (diff) | |
CNN v2 web tool: Fix layer naming and visualization bugs
- Align layer naming with codebase: Layer 0/1/2 (not Layer 1/2/3)
- Split static features: Static 0-3 (p0-p3) and Static 4-7 (uv,sin,bias)
- Fix Layer 2 not appearing: removed isOutput filter from layerOutputs
- Fix canvas context switching: force clear before recreation
- Disable static buttons in weights mode
- Add ASCII pipeline diagram to CNN_V2.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
| -rw-r--r-- | doc/CNN_V2.md | 54 | ||||
| -rw-r--r-- | doc/CNN_V2_WEB_TOOL.md | 19 | ||||
| -rw-r--r-- | tools/cnn_v2_test/index.html | 138 |
3 files changed, 165 insertions, 46 deletions
diff --git a/doc/CNN_V2.md b/doc/CNN_V2.md index b0aa24c..e56b022 100644 --- a/doc/CNN_V2.md +++ b/doc/CNN_V2.md @@ -32,6 +32,60 @@ Input RGBD → Static Features Compute → CNN Layers → Output RGBA └─ computed once/frame ─┘ └─ multi-pass ─┘ ``` +**Detailed Data Flow:** + +``` + ┌─────────────────────────────────────────┐ + │ Static Features (computed once) │ + │ 8D: p0,p1,p2,p3,uv_x,uv_y,sin10x,bias │ + └──────────────┬──────────────────────────┘ + │ + │ 8D (broadcast to all layers) + ├───────────────────────────┐ + │ │ + ┌──────────────┐ │ │ + │ Input RGBD │──────────────┤ │ + │ 4D │ 4D │ │ + └──────────────┘ │ │ + ▼ │ + ┌────────────┐ │ + │ Layer 0 │ (12D input) │ + │ (CNN) │ = 4D + 8D │ + │ 12D → 4D │ │ + └─────┬──────┘ │ + │ 4D output │ + │ │ + ├───────────────────────────┘ + │ │ + ▼ │ + ┌────────────┐ │ + │ Layer 1 │ (12D input) │ + │ (CNN) │ = 4D + 8D │ + │ 12D → 4D │ │ + └─────┬──────┘ │ + │ 4D output │ + │ │ + ├───────────────────────────┘ + ▼ │ + ... │ + │ │ + ▼ │ + ┌────────────┐ │ + │ Layer N │ (12D input) │ + │ (output) │◄──────────────────┘ + │ 12D → 4D │ + └─────┬──────┘ + │ 4D (RGBA) + ▼ + Output +``` + +**Key Points:** +- Static features computed once, broadcast to all CNN layers +- Each layer: previous 4D output + 8D static → 12D input → 4D output +- Ping-pong buffering between layers +- Layer 0 special case: uses input RGBD instead of previous layer output + **Static Features Texture:** - Name: `static_features` - Format: `texture_storage_2d<rgba32uint, write>` (4×u32) diff --git a/doc/CNN_V2_WEB_TOOL.md b/doc/CNN_V2_WEB_TOOL.md index 81549ab..a5162a6 100644 --- a/doc/CNN_V2_WEB_TOOL.md +++ b/doc/CNN_V2_WEB_TOOL.md @@ -6,7 +6,7 @@ Browser-based WebGPU tool for validating CNN v2 inference with layer visualizati --- -## Status (2026-02-13) +## Status (2026-02-13 evening) **Working:** - ✅ WebGPU initialization and device setup @@ -16,13 +16,20 @@ Browser-based WebGPU tool for validating CNN v2 inference with layer visualizati - ✅ Mode switching (Activations/Weights tabs) - ✅ Canvas context management (2D for weights, WebGPU for activations) - ✅ Weight visualization infrastructure (layer selection, grid layout) +- ✅ Layer naming matches codebase convention (Layer 0, Layer 1, Layer 2) +- ✅ Static features split visualization (Static 0-3, Static 4-7) +- ✅ All layers visible including output layer (Layer 2) -**Not Working:** -- ❌ Layer activation visualization (all black) -- ❌ Weight kernel display (canvases empty, but logging shows execution) +**Recent Fixes:** +- Fixed Layer 2 not appearing (was excluded from layerOutputs due to isOutput check) +- Fixed canvas context switching (force clear before recreation) +- Added Static 0-3 / Static 4-7 buttons to view all 8 static feature channels +- Aligned naming with train_cnn_v2.py/.wgsl: Layer 0, Layer 1, Layer 2 (not Layer 1, 2, 3) +- Disabled Static buttons in weights mode (no learnable weights) -**Partially Working:** -- ⚠️ Texture readback pipeline (UV gradient test works, data reads fail) +**Known Issues:** +- Layer activation visualization may show black if texture data not properly unpacked +- Weight kernel display depends on correct 2D context creation after canvas recreation --- diff --git a/tools/cnn_v2_test/index.html b/tools/cnn_v2_test/index.html index 199deea..e2b0a04 100644 --- a/tools/cnn_v2_test/index.html +++ b/tools/cnn_v2_test/index.html @@ -4,10 +4,14 @@ CNN v2 Testing Tool - WebGPU-based inference validator Architecture: - - Static features (8D): p0-p3 (parametric), uv_x, uv_y, sin(10*uv_x), bias + - Static features (8D): p0-p3 (parametric), uv_x, uv_y, sin(10*uv_x), bias (NOT a CNN layer) - Layer 0: input RGBD (4D) + static (8D) = 12D → 4 channels - - Layer 1+: previous (4D) + static (8D) = 12D → 4 channels - - All layers: uniform 12D input, 4D output (ping-pong buffer) + - Layer 1+: previous layer (4D) + static (8D) = 12D → 4 channels + - All CNN layers: uniform 12D input, 4D output (ping-pong buffer) + + Naming convention (matches train_cnn_v2.py / .wgsl / .cc): + - UI shows: "Static 0-3", "Static 4-7", "Layer 0", "Layer 1", "Layer 2" + - weights.layers[] array: Layer 0 = weights.layers[0], Layer 1 = weights.layers[1] Features: - Side panel: .bin metadata display, weight statistics per layer @@ -161,6 +165,14 @@ border-color: #4a9eff; color: #1a1a1a; } + .layer-buttons button:disabled { + opacity: 0.3; + cursor: not-allowed; + } + .layer-buttons button:disabled:hover { + border-color: #404040; + background: #1a1a1a; + } .layer-grid { display: grid; grid-template-columns: 1fr 1fr; @@ -686,12 +698,12 @@ class CNNTester { <tbody> `; - // Display layers as "Layer 1", "Layer 2", etc. (matching visualization button labels) + // Display layers as "Layer 0", "Layer 1", etc. (matching codebase convention) for (let i = 0; i < layers.length; i++) { const l = layers[i]; html += ` <tr> - <td>Layer ${i + 1}</td> + <td>Layer ${i}</td> <td>${l.inChannels}→${l.outChannels} (${l.kernelSize}×${l.kernelSize})</td> <td>${l.weightCount}</td> <td>${l.min.toFixed(3)}</td> @@ -885,7 +897,7 @@ class CNNTester { staticPass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8)); staticPass.end(); - // Copy static features to persistent storage (Layer 0) + // Copy static features to persistent storage (visualization index 0, shown as Static 0-3 / Static 4-7) encoder.copyTextureToTexture( { texture: staticTex }, { texture: layerTextures[0] }, @@ -940,16 +952,17 @@ class CNNTester { [srcTex, dstTex] = [dstTex, srcTex]; - // Copy layer output to persistent storage for visualization + // Copy CNN layer output to persistent storage for visualization + // i=0: Layer 0 → layerTextures[1] + // i=1: Layer 1 → layerTextures[2], etc. encoder.copyTextureToTexture( { texture: srcTex }, - { texture: layerTextures[i + 1] }, // +1 because layerTextures[0] is static features + { texture: layerTextures[i + 1] }, [width, height] ); - if (!isOutput) { - this.layerOutputs.push(layerTextures[i + 1]); - } + // Always push layer outputs for visualization (including output layer) + this.layerOutputs.push(layerTextures[i + 1]); } const modeBuffer = this.device.createBuffer({ @@ -1014,12 +1027,19 @@ class CNNTester { </div> `; - html += `<div style="font-size: 9px; color: #808080; margin-bottom: 8px; padding-bottom: 8px; border-bottom: 1px solid #404040;">Static features (8D: p0-p3 + spatial) + ${this.weights.layers.length} CNN layers. All layers: 12D→4D.</div>`; + html += `<div style="font-size: 9px; color: #808080; margin-bottom: 8px; padding-bottom: 8px; border-bottom: 1px solid #404040;">Static features (8D, split 0-3/4-7) + ${this.weights.layers.length} CNN layers (Layer 0-${this.weights.layers.length - 1}). All CNN layers: 12D→4D.</div>`; html += '<div class="layer-buttons">'; - for (let i = 0; i < this.layerOutputs.length; i++) { - // Visualization layers: Static features (i=0), CNN Layer 1 (i=1), CNN Layer 2 (i=2), ... - const label = i === 0 ? 'Static' : `Layer ${i}`; + // Static features: two buttons for 8D (0-3 and 4-7) + const staticDisabled = (this.vizMode === 'weights') ? ' disabled' : ''; + html += `<button onclick="tester.visualizeLayer(0, 0)" id="layerBtn0_0"${staticDisabled}>Static 0-3</button>`; + html += `<button onclick="tester.visualizeLayer(0, 4)" id="layerBtn0_4"${staticDisabled}>Static 4-7</button>`; + + // CNN layers: Layer 0, Layer 1, Layer 2, ... + for (let i = 1; i < this.layerOutputs.length; i++) { + // i=1: Layer 0 (weights.layers[0]) + // i=2: Layer 1 (weights.layers[1]), etc. + const label = `Layer ${i - 1}`; html += `<button onclick="tester.visualizeLayer(${i})" id="layerBtn${i}">${label}</button>`; } html += '</div>'; @@ -1034,8 +1054,8 @@ class CNNTester { // Create initial canvases this.recreateCanvases(); - // Auto-select static features (Layer 0) - this.visualizeLayer(0); + // Auto-select static features (channels 0-3) + this.visualizeLayer(0, 0); } setVizMode(mode) { @@ -1047,14 +1067,28 @@ class CNNTester { document.getElementById('vizModeWeights').style.borderColor = mode === 'weights' ? '#4a9eff' : '#404040'; document.getElementById('vizModeWeights').style.color = mode === 'weights' ? '#1a1a1a' : '#e0e0e0'; + // Update Static button states + const staticBtn0 = document.getElementById('layerBtn0_0'); + const staticBtn4 = document.getElementById('layerBtn0_4'); + if (staticBtn0) staticBtn0.disabled = (mode === 'weights'); + if (staticBtn4) staticBtn4.disabled = (mode === 'weights'); + // Clear and recreate canvas grid this.recreateCanvases(); - // Re-visualize current layer + // Re-visualize current layer (switch to Layer 0 if Static was active in weights mode) const activeBtn = document.querySelector('.layer-buttons button.active'); if (activeBtn) { - const layerIdx = parseInt(activeBtn.id.replace('layerBtn', '')); - this.visualizeLayer(layerIdx); + const isStatic = activeBtn.id.startsWith('layerBtn0_'); + if (isStatic && mode === 'weights') { + this.visualizeLayer(1); // Switch to Layer 0 (first CNN layer) + } else if (!isStatic) { + const layerIdx = parseInt(activeBtn.id.replace('layerBtn', '')); + this.visualizeLayer(layerIdx); + } else { + const channelOffset = parseInt(activeBtn.id.split('_')[1]); + this.visualizeLayer(0, channelOffset); + } } } @@ -1062,6 +1096,13 @@ class CNNTester { const grid = document.getElementById('layerGrid'); if (!grid) return; + // Force removal of old canvases to clear any WebGPU contexts + const oldCanvases = grid.querySelectorAll('canvas'); + oldCanvases.forEach(canvas => { + canvas.width = 0; + canvas.height = 0; + }); + grid.innerHTML = ''; for (let c = 0; c < 4; c++) { const div = document.createElement('div'); @@ -1074,25 +1115,31 @@ class CNNTester { } } - async visualizeLayer(layerIdx) { + async visualizeLayer(layerIdx, channelOffset = 0) { if (!this.layerOutputs || layerIdx >= this.layerOutputs.length) { this.log(`Cannot visualize layer ${layerIdx}: no data`, 'error'); return; } // Update button states - for (let i = 0; i < this.layerOutputs.length; i++) { - const btn = document.getElementById(`layerBtn${i}`); - if (btn) btn.classList.toggle('active', i === layerIdx); + document.querySelectorAll('.layer-buttons button').forEach(btn => btn.classList.remove('active')); + if (layerIdx === 0) { + // Static features + const btnId = `layerBtn0_${channelOffset}`; + const btn = document.getElementById(btnId); + if (btn) btn.classList.add('active'); + } else { + const btn = document.getElementById(`layerBtn${layerIdx}`); + if (btn) btn.classList.add('active'); } - const layerName = layerIdx === 0 ? 'Static Features' : `Layer ${layerIdx}`; + const layerName = layerIdx === 0 ? `Static Features (${channelOffset}-${channelOffset + 3})` : `Layer ${layerIdx - 1}`; // Check mode if (this.vizMode === 'weights' && layerIdx > 0) { - // Map visualization layer to weight array index - // Visualization Layer 1 (layerIdx=1) → weights.layers[0] (CNN Layer 1 weights) - // Visualization Layer 2 (layerIdx=2) → weights.layers[1] (CNN Layer 2 weights) + // Map visualization index to weights.layers[] array index: + // layerIdx=1 → Layer 0 → weights.layers[0] + // layerIdx=2 → Layer 1 → weights.layers[1], etc. this.visualizeWeights(layerIdx - 1); return; } @@ -1117,10 +1164,14 @@ class CNNTester { this.log(`Visualizing ${layerName} activations (${width}×${height})`); // Update channel labels based on layer type - // Static features: 8 channels (p0,p1,p2,p3,uv_x,uv_y,sin10_x,bias) - // CNN layers: 4 channels per layer (uniform) + // Static features (layerIdx=0): 8 channels split into two views + // CNN layers (layerIdx≥1): 4 channels per layer + const staticLabels = [ + ['Ch0 (p0)', 'Ch1 (p1)', 'Ch2 (p2)', 'Ch3 (p3)'], + ['Ch4 (uv_x)', 'Ch5 (uv_y)', 'Ch6 (sin10_x)', 'Ch7 (bias)'] + ]; const channelLabels = layerIdx === 0 - ? ['Ch0 (p0)', 'Ch1 (p1)', 'Ch2 (p2)', 'Ch3 (p3)'] + ? staticLabels[channelOffset / 4] : ['Ch0', 'Ch1', 'Ch2', 'Ch3']; for (let c = 0; c < 4; c++) { @@ -1153,9 +1204,9 @@ class CNNTester { continue; } + // Set canvas size BEFORE getting context canvas.width = width; canvas.height = height; - this.log(`Canvas ${c}: ${width}×${height}`); const ctx = canvas.getContext('webgpu'); if (!ctx) { @@ -1165,6 +1216,7 @@ class CNNTester { try { ctx.configure({ device: this.device, format: this.format }); + this.log(`Canvas ${c}: ${width}×${height}, WebGPU context configured`); } catch (e) { this.log(`Failed to configure canvas ${c}: ${e.message}`, 'error'); continue; @@ -1175,8 +1227,9 @@ class CNNTester { size: 8, usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST }); - // Use normal channel (0-3), or debug modes (10=UV, 11=raw, 12=unpacked) - const paramsData = new Float32Array([c, vizScale]); + // Use channel index with offset for static features + const actualChannel = channelOffset + c; + const paramsData = new Float32Array([actualChannel, vizScale]); this.device.queue.writeBuffer(paramsBuffer, 0, paramsData); const bindGroup = this.device.createBindGroup({ @@ -1212,17 +1265,17 @@ class CNNTester { } visualizeWeights(cnnLayerIdx) { - // cnnLayerIdx is index into weights.layers[] array - // Display as "Layer N" where N = cnnLayerIdx + 1 (e.g., weights.layers[0] = Layer 1) + // cnnLayerIdx is index into weights.layers[] array (0-indexed) + // Display as "Layer N" matching codebase convention + // Example: weights.layers[0] = Layer 0, weights.layers[1] = Layer 1 const layer = this.weights.layers[cnnLayerIdx]; if (!layer) { - this.log(`CNN Layer ${cnnLayerIdx + 1} not found`, 'error'); + this.log(`Layer ${cnnLayerIdx} not found`, 'error'); return; } const { kernelSize, inChannels, outChannels, weightOffset, min, max } = layer; - const displayLayerNum = cnnLayerIdx + 1; - this.log(`Visualizing Layer ${displayLayerNum} weights: ${inChannels}→${outChannels}, ${kernelSize}×${kernelSize}, offset=${weightOffset}`); + this.log(`Visualizing Layer ${cnnLayerIdx} weights: ${inChannels}→${outChannels}, ${kernelSize}×${kernelSize}, offset=${weightOffset}`); this.log(`Weight range: [${min.toFixed(3)}, ${max.toFixed(3)}]`); // Update channel labels to show output channels @@ -1240,8 +1293,13 @@ class CNNTester { continue; } - // Need to clear WebGPU context and use 2D + // Get 2D context (canvases recreated by recreateCanvases()) const ctx = canvas.getContext('2d', { willReadFrequently: false }); + if (!ctx) { + this.log(`Failed to get 2D context for canvas ${outCh}`, 'error'); + continue; + } + const cellSize = 20; // Pixels per weight const padding = 2; const gridWidth = inChannels * (kernelSize * cellSize + padding); |
