diff options
| -rw-r--r-- | doc/CNN_V2_WEB_TOOL.md | 295 | ||||
| -rw-r--r-- | tools/cnn_v2_test/index.html | 622 |
2 files changed, 904 insertions, 13 deletions
diff --git a/doc/CNN_V2_WEB_TOOL.md b/doc/CNN_V2_WEB_TOOL.md new file mode 100644 index 0000000..2fbc70e --- /dev/null +++ b/doc/CNN_V2_WEB_TOOL.md @@ -0,0 +1,295 @@ +# CNN v2 Web Testing Tool + +Browser-based WebGPU tool for validating CNN v2 inference with layer visualization and weight inspection. + +**Location:** `tools/cnn_v2_test/index.html` + +--- + +## Status (2026-02-13) + +**Working:** +- ✅ WebGPU initialization and device setup +- ✅ Binary weight file parsing (.bin format) +- ✅ Weight statistics (min/max per layer) +- ✅ UI layout with collapsible panels +- ✅ Mode switching (Activations/Weights tabs) +- ✅ Canvas context management (2D for weights, WebGPU for activations) +- ✅ Weight visualization infrastructure (layer selection, grid layout) + +**Not Working:** +- ❌ Layer activation visualization (all black) +- ❌ Weight kernel display (canvases empty, but logging shows execution) + +**Partially Working:** +- ⚠️ Texture readback pipeline (UV gradient test works, data reads fail) + +--- + +## Architecture + +### File Structure +- Single-file HTML tool (~1100 lines) +- Embedded shaders: STATIC_SHADER, CNN_SHADER, DISPLAY_SHADER, LAYER_VIZ_SHADER +- Pure WebGPU (no external dependencies) + +### Key Components + +**1. Weight Parsing** +- Reads binary format: header (16B) + layer info (20B×N) + f16 weights +- Computes min/max per layer via f16 unpacking +- Stores `{ layers[], weights[], fileSize }` + +**2. CNN Pipeline** +- Static features computation (RGBD + UV + sin + bias → 7D packed) +- Layer-by-layer convolution with storage buffer weights +- Ping-pong buffers for intermediate results +- Copy to persistent textures for visualization + +**3. Visualization Modes** + +**Activations Mode:** +- 4 grayscale views per layer (channels 0-3) +- WebGPU compute → unpack f16 → scale → grayscale +- Auto-scale: Layer 0 (static) = 1.0, CNN layers = 0.2 + +**Weights Mode:** +- 2D canvas rendering per output channel +- Shows all input kernels horizontally +- Normalized by layer min/max → [0, 1] → grayscale +- 20px cells, 2px padding between kernels + +### Texture Management + +**Persistent Storage (layerTextures[]):** +- One texture per layer output (static + all CNN layers) +- `rgba32uint` format (packed f16 data) +- `COPY_DST` usage for storing results + +**Compute Buffers (computeTextures[]):** +- 2 textures for ping-pong computation +- Reused across all layers +- `COPY_SRC` usage for copying to persistent storage + +**Pipeline:** +``` +Static pass → copy to layerTextures[0] +For each CNN layer i: + Compute (ping-pong) → copy to layerTextures[i+1] +``` + +--- + +## Known Issues + +### Issue #1: Layer Activations Show Black + +**Symptom:** +- All 4 channel canvases render black +- UV gradient test (debug mode 10) works +- Raw packed data test (mode 11) shows black +- Unpacked f16 test (mode 12) shows black + +**Diagnosis:** +- Texture access works (UV gradient visible) +- Texture data is all zeros (packed.x = 0) +- Textures being read are empty + +**Root Cause:** +- `copyTextureToTexture` operations may not be executing +- Possible ordering issue (copies not submitted before visualization) +- Alternative: textures created with wrong usage flags + +**Investigation Steps Taken:** +1. Added `onSubmittedWorkDone()` wait before visualization +2. Verified texture creation with `COPY_SRC` and `COPY_DST` flags +3. Confirmed separate texture allocation per layer (no aliasing) +4. Added debug shader modes to isolate issue + +**Next Steps:** +- Verify encoder contains copy commands (add debug logging) +- Check if compute passes actually write data (add known-value test) +- Test copyTextureToTexture in isolation +- Consider CPU readback to verify texture contents + +### Issue #2: Weight Visualization Empty + +**Symptom:** +- Canvases created with correct dimensions (logged) +- No visual output (black canvases) +- Console logs show method execution + +**Potential Causes:** +1. Weight indexing calculation incorrect +2. Canvas not properly attached to DOM when rendering +3. 2D context operations not flushing +4. Min/max normalization producing black (all values equal?) + +**Debug Added:** +- Comprehensive logging of dimensions, indices, ranges +- Canvas context check before rendering + +**Next Steps:** +- Add test rendering (fixed gradient) to verify 2D context works +- Log sample weight values to verify data access +- Check if canvas is visible in DOM inspector +- Verify min/max calculation produces valid range + +--- + +## UI Layout + +### Header +- Controls: Blend slider, Depth input, View mode display +- Drop zone for .bin weight files + +### Content Area +**Left:** Main canvas (CNN output display) +**Right:** Sidebar with panels: + +1. **Layer Visualization Panel** (top, flex: 1) + - Mode tabs: Activations / Weights + - Layer selection buttons (Static L0, Layer 1, Layer 2, ...) + - 2×2 grid of channel views + +2. **Weights Info Panel** (bottom, collapsible) + - File size, layer count + - Per-layer table: size, weight count, min/max + - Click header to collapse/expand + +### Footer +- Status line (GPU timing, dimensions, mode) +- Console log (scrollable, color-coded) + +--- + +## Shader Details + +### LAYER_VIZ_SHADER + +**Purpose:** Display single channel from packed layer texture + +**Inputs:** +- `@binding(0) layer_tex: texture_2d<u32>` - Packed f16 layer data +- `@binding(1) viz_params: vec2<f32>` - (channel_idx, scale) + +**Debug Modes:** +- Channel 10: UV gradient (texture coordinate test) +- Channel 11: Raw packed u32 data +- Channel 12: First unpacked f16 value + +**Normal Operation:** +- Unpack all 8 f16 channels from rgba32uint +- Select channel by index (0-7) +- Apply scale factor (1.0 for static, 0.2 for CNN) +- Clamp to [0, 1] and output grayscale + +**Scale Rationale:** +- Static features (RGBD, UV): already in [0, 1] range +- CNN activations: post-ReLU [0, ~5], need scaling for visibility + +--- + +## Binary Weight Format + +**Header (16 bytes):** +``` +u32 magic; // 0x32_4E_4E_43 ("CNN2") +u32 version; // Format version +u32 num_layers; // Layer count +u32 total_weights;// Total f16 weight count +``` + +**Layer Info (20 bytes × N):** +``` +u32 kernel_size; // 3, 5, 7, etc. +u32 in_channels; // Input channel count +u32 out_channels; // Output channel count +u32 weight_offset; // Offset in f16 units +u32 weight_count; // Number of f16 weights +``` + +**Weights (variable):** +- Packed f16 pairs as u32 (lo 16 bits, hi 16 bits) +- Sequential storage: [layer0_weights][layer1_weights]... + +--- + +## Testing Workflow + +### Load & Parse +1. Drop PNG image → displays original +2. Drop .bin weights → parses and shows info table +3. Auto-runs CNN pipeline + +### Verify Pipeline +1. Check console for "Running CNN pipeline" +2. Verify "Completed in Xms" +3. Check "Layer visualization ready: N layers" + +### Debug Activations +1. Select "Activations" tab +2. Click layer buttons to switch +3. Check console for texture/canvas logs +4. If black: note which debug modes work (UV vs data) + +### Debug Weights +1. Select "Weights" tab +2. Click Layer 1 or Layer 2 (Layer 0 has no weights) +3. Check console for "Visualizing Layer N weights" +4. Check canvas dimensions logged +5. Verify weight range is non-trivial (not [0, 0]) + +--- + +## Integration with Main Project + +**Training Pipeline:** +```bash +# Generate weights +./training/train_cnn_v2.py --export-binary + +# Test in browser +open tools/cnn_v2_test/index.html +# Drop: workspaces/main/cnn_v2_weights.bin +# Drop: training/input/test.png +``` + +**Validation:** +- Compare against demo CNNv2Effect (visual check) +- Verify layer count matches binary file +- Check weight ranges match training logs + +--- + +## Future Enhancements + +- [ ] Fix layer activation visualization (black texture issue) +- [ ] Fix weight kernel display (empty canvas issue) +- [ ] Add per-channel auto-scaling (compute min/max from visible data) +- [ ] Export rendered outputs (download PNG) +- [ ] Side-by-side comparison with original +- [ ] Heatmap mode (color-coded activations) +- [ ] Weight statistics overlay (mean, std, sparsity) +- [ ] Batch processing (multiple images in sequence) +- [ ] Integration with Python training (live reload) + +--- + +## Code Metrics + +- Total lines: ~1100 +- JavaScript: ~700 lines +- WGSL shaders: ~300 lines +- HTML/CSS: ~100 lines + +**Dependencies:** None (pure WebGPU + HTML5) + +--- + +## Related Files + +- `doc/CNN_V2.md` - CNN v2 architecture and design +- `doc/CNN_TEST_TOOL.md` - C++ offline testing tool (deprecated) +- `training/train_cnn_v2.py` - Training script with binary export +- `workspaces/main/cnn_v2_weights.bin` - Trained weights diff --git a/tools/cnn_v2_test/index.html b/tools/cnn_v2_test/index.html index 9c28455..bfc91c5 100644 --- a/tools/cnn_v2_test/index.html +++ b/tools/cnn_v2_test/index.html @@ -3,10 +3,10 @@ <!-- CNN v2 Testing Tool - WebGPU-based inference validator - TODO: - - Side panel: .bin metadata display, weight statistics, validation - - Layer inspection: R/G/B/A plane split, intermediate layer visualization - - Activation heatmaps for debugging + Features: + - Side panel: .bin metadata display, weight statistics per layer + - Layer inspection: 4-channel grayscale split, intermediate layer visualization + - View modes: CNN output, original, diff (×10) --> <head> <meta charset="UTF-8"> @@ -55,6 +55,11 @@ .drop-zone:hover { border-color: #606060; background: #252525; } .drop-zone.active { border-color: #4a9eff; background: #1a2a3a; } .drop-zone.error { border-color: #ff4a4a; background: #3a1a1a; } + .content { + flex: 1; + display: flex; + overflow: hidden; + } .main { flex: 1; display: flex; @@ -78,6 +83,103 @@ pointer-events: none; z-index: 10; } + .sidebar { + width: 350px; + background: #2a2a2a; + border-left: 1px solid #404040; + overflow-y: auto; + display: flex; + flex-direction: column; + gap: 16px; + padding: 16px; + } + .panel { + border: 1px solid #404040; + border-radius: 4px; + overflow: hidden; + } + .panel.collapsed .panel-content { + display: none; + } + .panel-header { + background: #1a1a1a; + padding: 8px 12px; + font-size: 12px; + font-weight: bold; + border-bottom: 1px solid #404040; + } + .panel-content { + padding: 12px; + font-size: 11px; + } + .panel-content table { + width: 100%; + border-collapse: collapse; + } + .panel-content th { + text-align: left; + padding: 4px; + font-size: 10px; + color: #808080; + border-bottom: 1px solid #404040; + } + .panel-content td { + padding: 4px; + font-size: 10px; + } + .panel-content tr:hover { + background: #1a1a1a; + } + .layer-buttons { + display: flex; + flex-wrap: wrap; + gap: 6px; + margin-bottom: 12px; + } + .layer-buttons button { + background: #1a1a1a; + border: 1px solid #404040; + color: #e0e0e0; + padding: 6px 12px; + font-size: 10px; + font-family: 'Courier New', monospace; + cursor: pointer; + transition: all 0.2s; + } + .layer-buttons button:hover { + border-color: #606060; + background: #252525; + } + .layer-buttons button.active { + background: #4a9eff; + border-color: #4a9eff; + color: #1a1a1a; + } + .layer-grid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 8px; + } + .layer-view { + aspect-ratio: 1; + background: #1a1a1a; + border: 1px solid #404040; + display: flex; + flex-direction: column; + overflow: hidden; + } + .layer-view-label { + background: #2a2a2a; + padding: 4px; + font-size: 9px; + text-align: center; + border-bottom: 1px solid #404040; + } + .layer-view canvas { + width: 100%; + height: 100%; + image-rendering: pixelated; + } canvas { max-width: 100%; max-height: 100%; @@ -134,8 +236,26 @@ </div> <div class="drop-zone" id="weightsDrop">Drop .bin Weights</div> </div> - <div class="main" id="mainDrop"> - <canvas id="canvas"></canvas> + <div class="content"> + <div class="main" id="mainDrop"> + <canvas id="canvas"></canvas> + </div> + <div class="sidebar"> + <div class="panel" style="flex: 1; display: flex; flex-direction: column; min-height: 0;"> + <div class="panel-header">Layer Visualization</div> + <div class="panel-content" id="layerViz" style="flex: 1; overflow-y: auto;"> + <p style="color: #808080; text-align: center;">Load image + weights</p> + </div> + </div> + <div class="panel" id="weightsInfoPanel"> + <div class="panel-header" style="cursor: pointer; user-select: none;" onclick="tester.toggleWeightsInfo()"> + Weights Info <span id="weightsInfoToggle">▼</span> + </div> + <div class="panel-content" id="weightsInfo"> + <p style="color: #808080; text-align: center;">No weights loaded</p> + </div> + </div> + </div> </div> <div class="footer"> <div class="footer-top"> @@ -329,6 +449,71 @@ fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { } }`; +const LAYER_VIZ_SHADER = ` +@group(0) @binding(0) var layer_tex: texture_2d<u32>; +@group(0) @binding(1) var<uniform> viz_params: vec2<f32>; // x=channel_idx, y=scale + +@vertex +fn vs_main(@builtin(vertex_index) idx: u32) -> @builtin(position) vec4<f32> { + var pos = array<vec2<f32>, 6>( + vec2<f32>(-1.0, -1.0), vec2<f32>(1.0, -1.0), vec2<f32>(-1.0, 1.0), + vec2<f32>(-1.0, 1.0), vec2<f32>(1.0, -1.0), vec2<f32>(1.0, 1.0) + ); + return vec4<f32>(pos[idx], 0.0, 1.0); +} + +@fragment +fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { + let coord = vec2<i32>(pos.xy); + let dims = textureDimensions(layer_tex); + + let channel = u32(viz_params.x); + + // DEBUG MODE 1: Texture coordinates (channel 10) + if (channel == 10u) { + let uv = vec2<f32>(f32(coord.x) / f32(dims.x), f32(coord.y) / f32(dims.y)); + return vec4<f32>(uv.x, uv.y, 0.0, 1.0); + } + + let packed = textureLoad(layer_tex, coord, 0); + + // DEBUG MODE 2: Raw packed data (channel 11) + if (channel == 11u) { + let raw_val = f32(packed.x) / 4294967295.0; + return vec4<f32>(raw_val, raw_val, raw_val, 1.0); + } + + let v0 = unpack2x16float(packed.x); + let v1 = unpack2x16float(packed.y); + let v2 = unpack2x16float(packed.z); + let v3 = unpack2x16float(packed.w); + + // DEBUG MODE 3: First unpacked value (channel 12) + if (channel == 12u) { + return vec4<f32>(v0.x, v0.x, v0.x, 1.0); + } + + var channels: array<f32, 8>; + channels[0] = v0.x; + channels[1] = v0.y; + channels[2] = v1.x; + channels[3] = v1.y; + channels[4] = v2.x; + channels[5] = v2.y; + channels[6] = v3.x; + channels[7] = v3.y; + + let scale = viz_params.y; + + let idx = min(channel, 7u); + let raw = channels[idx]; + + // Apply scale: multiply and clamp to [0, 1] + let val = clamp(raw * scale, 0.0, 1.0); + + return vec4<f32>(val, val, val, 1.0); +}`; + class CNNTester { constructor() { this.canvas = document.getElementById('canvas'); @@ -404,14 +589,48 @@ class CNNTester { const weightsOffset = 16 + numLayers * 20; const weights = new Uint32Array(buffer.slice(weightsOffset)); - // Verify weights are non-zero + // Calculate min/max per layer + for (let i = 0; i < numLayers; i++) { + const layer = layers[i]; + let min = Infinity, max = -Infinity; + const startIdx = layer.weightOffset; + const endIdx = startIdx + layer.weightCount; + + for (let j = startIdx; j < endIdx; j++) { + const pairIdx = Math.floor(j / 2); + const packed = weights[pairIdx]; + const unpacked = this.unpackF16(packed); + const val = (j % 2 === 0) ? unpacked[0] : unpacked[1]; + min = Math.min(min, val); + max = Math.max(max, val); + } + + layer.min = min; + layer.max = max; + this.log(` Layer ${i} range: [${min.toFixed(4)}, ${max.toFixed(4)}]`); + } + let nonZero = 0; for (let i = 0; i < weights.length; i++) { if (weights[i] !== 0) nonZero++; } this.log(` Weight buffer: ${weights.length} u32 (${nonZero} non-zero)`); - return { layers, weights }; + return { layers, weights, fileSize: buffer.byteLength }; + } + + unpackF16(packed) { + const lo = packed & 0xFFFF; + const hi = (packed >> 16) & 0xFFFF; + const toFloat = (bits) => { + const sign = (bits >> 15) & 1; + const exp = (bits >> 10) & 0x1F; + const frac = bits & 0x3FF; + if (exp === 0) return (sign ? -1 : 1) * Math.pow(2, -14) * (frac / 1024); + if (exp === 31) return frac ? NaN : (sign ? -Infinity : Infinity); + return (sign ? -1 : 1) * Math.pow(2, exp - 15) * (1 + frac / 1024); + }; + return [toFloat(lo), toFloat(hi)]; } async loadImage(file) { @@ -434,6 +653,7 @@ class CNNTester { this.weights = this.parseWeights(buffer); this.weightsBuffer = buffer; this.log(`Loaded weights: ${file.name} (${this.weights.layers.length} layers, ${(buffer.byteLength/1024).toFixed(1)} KB)`); + this.updateWeightsPanel(); if (this.image) { this.setStatus(`Ready: ${this.image.width}×${this.image.height}`); this.run(); @@ -442,6 +662,49 @@ class CNNTester { } } + updateWeightsPanel() { + const panel = document.getElementById('weightsInfo'); + const { layers, fileSize } = this.weights; + + let html = ` + <div style="margin-bottom: 12px;"> + <div><strong>File Size:</strong> ${(fileSize / 1024).toFixed(2)} KB</div> + <div><strong>Layers:</strong> ${layers.length}</div> + </div> + <table> + <thead> + <tr> + <th>Layer</th> + <th>Size</th> + <th>Weights</th> + <th>Min</th> + <th>Max</th> + </tr> + </thead> + <tbody> + `; + + for (let i = 0; i < layers.length; i++) { + const l = layers[i]; + html += ` + <tr> + <td>${i + 1}</td> + <td>${l.inChannels}→${l.outChannels} (${l.kernelSize}×${l.kernelSize})</td> + <td>${l.weightCount}</td> + <td>${l.min.toFixed(3)}</td> + <td>${l.max.toFixed(3)}</td> + </tr> + `; + } + + html += ` + </tbody> + </table> + `; + + panel.innerHTML = html; + } + displayOriginal() { if (!this.image || !this.device) return; @@ -547,19 +810,32 @@ class CNNTester { const staticTex = this.device.createTexture({ size: [width, height], format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING + usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC }); - const layerTextures = [ + // Create one texture per layer output (static + all CNN layers) + this.layerOutputs = []; + const numLayers = this.weights.layers.length + 1; // +1 for static features + const layerTextures = []; + for (let i = 0; i < numLayers; i++) { + layerTextures.push(this.device.createTexture({ + size: [width, height], + format: 'rgba32uint', + usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST + })); + } + + // Ping-pong buffers for computation + const computeTextures = [ this.device.createTexture({ size: [width, height], format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING + usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC }), this.device.createTexture({ size: [width, height], format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING + usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC }) ]; @@ -606,8 +882,16 @@ class CNNTester { staticPass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8)); staticPass.end(); + // Copy static features to persistent storage (Layer 0) + encoder.copyTextureToTexture( + { texture: staticTex }, + { texture: layerTextures[0] }, + [width, height] + ); + this.layerOutputs.push(layerTextures[0]); + let srcTex = staticTex; - let dstTex = layerTextures[0]; + let dstTex = computeTextures[0]; for (let i = 0; i < this.weights.layers.length; i++) { const layer = this.weights.layers[i]; @@ -652,6 +936,17 @@ class CNNTester { cnnPass.end(); [srcTex, dstTex] = [dstTex, srcTex]; + + // Copy layer output to persistent storage for visualization + encoder.copyTextureToTexture( + { texture: srcTex }, + { texture: layerTextures[i + 1] }, // +1 because layerTextures[0] is static features + [width, height] + ); + + if (!isOutput) { + this.layerOutputs.push(layerTextures[i + 1]); + } } const modeBuffer = this.device.createBuffer({ @@ -689,10 +984,311 @@ class CNNTester { this.device.queue.submit([encoder.finish()]); + // Wait for GPU to finish before visualizing layers + await this.device.queue.onSubmittedWorkDone(); + const t1 = performance.now(); const mode = ['CNN Output', 'Original', 'Diff (×10)'][this.viewMode]; this.setStatus(`GPU: ${(t1-t0).toFixed(1)}ms | ${width}×${height} | ${mode}`); this.log(`Completed in ${(t1-t0).toFixed(1)}ms`); + + // Update layer visualization panel + this.updateLayerVizPanel(); + } + + updateLayerVizPanel() { + const panel = document.getElementById('layerViz'); + + if (!this.layerOutputs || this.layerOutputs.length === 0) { + panel.innerHTML = '<p style="color: #808080; text-align: center;">No layers to visualize</p>'; + return; + } + + let html = ` + <div style="display: flex; gap: 8px; margin-bottom: 12px; border-bottom: 1px solid #404040; padding-bottom: 8px;"> + <button onclick="tester.setVizMode('activations')" id="vizModeActivations" style="flex:1; padding: 6px; background: #4a9eff; border: 1px solid #4a9eff; color: #1a1a1a; cursor: pointer; font-family: 'Courier New'; font-size: 10px;">Activations</button> + <button onclick="tester.setVizMode('weights')" id="vizModeWeights" style="flex:1; padding: 6px; background: #1a1a1a; border: 1px solid #404040; color: #e0e0e0; cursor: pointer; font-family: 'Courier New'; font-size: 10px;">Weights</button> + </div> + `; + + html += '<div class="layer-buttons">'; + for (let i = 0; i < this.layerOutputs.length; i++) { + const label = i === 0 ? 'Static (L0)' : `Layer ${i}`; + html += `<button onclick="tester.visualizeLayer(${i})" id="layerBtn${i}">${label}</button>`; + } + html += '</div>'; + + html += '<div class="layer-grid" id="layerGrid"></div>'; + + panel.innerHTML = html; + + this.log(`Layer visualization ready: ${this.layerOutputs.length} layers`); + this.vizMode = 'activations'; + + // Create initial canvases + this.recreateCanvases(); + + // Auto-select static features (Layer 0) + this.visualizeLayer(0); + } + + setVizMode(mode) { + this.vizMode = mode; + document.getElementById('vizModeActivations').style.background = mode === 'activations' ? '#4a9eff' : '#1a1a1a'; + document.getElementById('vizModeActivations').style.borderColor = mode === 'activations' ? '#4a9eff' : '#404040'; + document.getElementById('vizModeActivations').style.color = mode === 'activations' ? '#1a1a1a' : '#e0e0e0'; + document.getElementById('vizModeWeights').style.background = mode === 'weights' ? '#4a9eff' : '#1a1a1a'; + document.getElementById('vizModeWeights').style.borderColor = mode === 'weights' ? '#4a9eff' : '#404040'; + document.getElementById('vizModeWeights').style.color = mode === 'weights' ? '#1a1a1a' : '#e0e0e0'; + + // Clear and recreate canvas grid + this.recreateCanvases(); + + // Re-visualize current layer + const activeBtn = document.querySelector('.layer-buttons button.active'); + if (activeBtn) { + const layerIdx = parseInt(activeBtn.id.replace('layerBtn', '')); + this.visualizeLayer(layerIdx); + } + } + + recreateCanvases() { + const grid = document.getElementById('layerGrid'); + if (!grid) return; + + grid.innerHTML = ''; + for (let c = 0; c < 4; c++) { + const div = document.createElement('div'); + div.className = 'layer-view'; + div.innerHTML = ` + <div class="layer-view-label" id="channelLabel${c}">Ch ${c}</div> + <canvas id="layerCanvas${c}"></canvas> + `; + grid.appendChild(div); + } + } + + async visualizeLayer(layerIdx) { + if (!this.layerOutputs || layerIdx >= this.layerOutputs.length) { + this.log(`Cannot visualize layer ${layerIdx}: no data`, 'error'); + return; + } + + // Update button states + for (let i = 0; i < this.layerOutputs.length; i++) { + const btn = document.getElementById(`layerBtn${i}`); + if (btn) btn.classList.toggle('active', i === layerIdx); + } + + const layerName = layerIdx === 0 ? 'Static Features' : `Layer ${layerIdx}`; + + // Check mode + if (this.vizMode === 'weights' && layerIdx > 0) { + this.visualizeWeights(layerIdx - 1); // Layer 1 → weights.layers[0] + return; + } + + if (this.vizMode === 'weights' && layerIdx === 0) { + this.log('Static features have no weights', 'error'); + // Clear canvases + for (let c = 0; c < 4; c++) { + const canvas = document.getElementById(`layerCanvas${c}`); + if (canvas) { + const ctx = canvas.getContext('2d'); + ctx.fillStyle = '#1a1a1a'; + ctx.fillRect(0, 0, canvas.width, canvas.height); + } + } + return; + } + + const layerTex = this.layerOutputs[layerIdx]; + const { width, height } = this.image; + + this.log(`Visualizing ${layerName} activations (${width}×${height})`); + + // Update channel labels based on layer type + const channelLabels = layerIdx === 0 + ? ['R', 'G', 'B', 'D'] // Static features: RGBA (R,G,B,Depth,UV_X,UV_Y,sin,bias) + : ['Ch0', 'Ch1', 'Ch2', 'Ch3']; // CNN layers + + for (let c = 0; c < 4; c++) { + const label = document.getElementById(`channelLabel${c}`); + if (label) label.textContent = channelLabels[c]; + } + + // Create layer viz pipeline if needed + if (!this.layerVizPipeline) { + this.layerVizPipeline = this.device.createRenderPipeline({ + layout: 'auto', + vertex: { + module: this.device.createShaderModule({ code: LAYER_VIZ_SHADER }), + entryPoint: 'vs_main' + }, + fragment: { + module: this.device.createShaderModule({ code: LAYER_VIZ_SHADER }), + entryPoint: 'fs_main', + targets: [{ format: this.format }] + } + }); + this.log('Created layer visualization pipeline'); + } + + // Render each channel to its canvas + for (let c = 0; c < 4; c++) { + const canvas = document.getElementById(`layerCanvas${c}`); + if (!canvas) { + this.log(`Canvas layerCanvas${c} not found`, 'error'); + continue; + } + + canvas.width = width; + canvas.height = height; + this.log(`Canvas ${c}: ${width}×${height}`); + + const ctx = canvas.getContext('webgpu'); + if (!ctx) { + this.log(`Failed to get WebGPU context for channel ${c}`, 'error'); + continue; + } + + try { + ctx.configure({ device: this.device, format: this.format }); + } catch (e) { + this.log(`Failed to configure canvas ${c}: ${e.message}`, 'error'); + continue; + } + + const vizScale = layerIdx === 0 ? 1.0 : 0.2; // Static: 1.0, CNN layers: 0.2 (assumes ~5 max) + const paramsBuffer = this.device.createBuffer({ + size: 8, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST + }); + // Use normal channel (0-3), or debug modes (10=UV, 11=raw, 12=unpacked) + const paramsData = new Float32Array([c, vizScale]); + this.device.queue.writeBuffer(paramsBuffer, 0, paramsData); + + const bindGroup = this.device.createBindGroup({ + layout: this.layerVizPipeline.getBindGroupLayout(0), + entries: [ + { binding: 0, resource: layerTex.createView() }, + { binding: 1, resource: { buffer: paramsBuffer } } + ] + }); + + const encoder = this.device.createCommandEncoder(); + const renderPass = encoder.beginRenderPass({ + colorAttachments: [{ + view: ctx.getCurrentTexture().createView(), + loadOp: 'clear', + clearValue: { r: 1.0, g: 0.0, b: 1.0, a: 1.0 }, // Magenta clear for debugging + storeOp: 'store' + }] + }); + + renderPass.setPipeline(this.layerVizPipeline); + renderPass.setBindGroup(0, bindGroup); + renderPass.draw(6); + renderPass.end(); + + this.device.queue.submit([encoder.finish()]); + this.log(`Submitted render for channel ${c}`); + } + + // Wait for all renders to complete + await this.device.queue.onSubmittedWorkDone(); + this.log(`Rendered 4 channels for ${layerName}`); + } + + visualizeWeights(cnnLayerIdx) { + const layer = this.weights.layers[cnnLayerIdx]; + if (!layer) { + this.log(`Layer ${cnnLayerIdx} not found`, 'error'); + return; + } + + const { kernelSize, inChannels, outChannels, weightOffset, min, max } = layer; + this.log(`Visualizing Layer ${cnnLayerIdx + 1} weights: ${inChannels}→${outChannels}, ${kernelSize}×${kernelSize}, offset=${weightOffset}`); + this.log(`Weight range: [${min.toFixed(3)}, ${max.toFixed(3)}]`); + + // Update channel labels to show output channels + const channelLabels = [`Out 0`, `Out 1`, `Out 2`, `Out 3`]; + for (let c = 0; c < 4; c++) { + const label = document.getElementById(`channelLabel${c}`); + if (label) label.textContent = c < outChannels ? channelLabels[c] : '-'; + } + + // Render each output channel's weights + for (let outCh = 0; outCh < 4; outCh++) { + const canvas = document.getElementById(`layerCanvas${outCh}`); + if (!canvas) { + this.log(`Canvas ${outCh} not found`, 'error'); + continue; + } + + // Need to clear WebGPU context and use 2D + const ctx = canvas.getContext('2d', { willReadFrequently: false }); + const cellSize = 20; // Pixels per weight + const padding = 2; + const gridWidth = inChannels * (kernelSize * cellSize + padding); + const gridHeight = kernelSize * cellSize; + + canvas.width = gridWidth; + canvas.height = gridHeight; + this.log(`Canvas ${outCh}: ${gridWidth}×${gridHeight}`); + + ctx.fillStyle = '#1a1a1a'; + ctx.fillRect(0, 0, canvas.width, canvas.height); + + if (outCh >= outChannels) { + this.log(`Channel ${outCh} >= outChannels ${outChannels}, skipping`); + continue; + } + + // Draw kernels for each input channel + for (let inCh = 0; inCh < inChannels; inCh++) { + const xOffset = inCh * (kernelSize * cellSize + padding); + + for (let ky = 0; ky < kernelSize; ky++) { + for (let kx = 0; kx < kernelSize; kx++) { + const spatialIdx = ky * kernelSize + kx; + const wIdx = weightOffset + + outCh * inChannels * kernelSize * kernelSize + + inCh * kernelSize * kernelSize + + spatialIdx; + + const weight = this.getWeightValue(wIdx); + const normalized = (weight - min) / (max - min); // [0, 1] + const intensity = Math.floor(normalized * 255); + + ctx.fillStyle = `rgb(${intensity}, ${intensity}, ${intensity})`; + ctx.fillRect( + xOffset + kx * cellSize, + ky * cellSize, + cellSize - 1, + cellSize - 1 + ); + } + } + } + this.log(`Rendered output channel ${outCh}`); + } + + this.log(`Weight visualization complete`); + } + + getWeightValue(idx) { + const pairIdx = Math.floor(idx / 2); + const packed = this.weights.weights[pairIdx]; + const unpacked = this.unpackF16(packed); + return (idx % 2 === 0) ? unpacked[0] : unpacked[1]; + } + + toggleWeightsInfo() { + const panel = document.getElementById('weightsInfoPanel'); + const toggle = document.getElementById('weightsInfoToggle'); + panel.classList.toggle('collapsed'); + toggle.textContent = panel.classList.contains('collapsed') ? '▶' : '▼'; } updateDisplay() { |