diff options
Diffstat (limited to 'tools/cnn_v2_test')
| -rw-r--r-- | tools/cnn_v2_test/README.md | 251 | ||||
| -rw-r--r-- | tools/cnn_v2_test/index.html | 2049 |
2 files changed, 0 insertions, 2300 deletions
diff --git a/tools/cnn_v2_test/README.md b/tools/cnn_v2_test/README.md deleted file mode 100644 index d41a00f..0000000 --- a/tools/cnn_v2_test/README.md +++ /dev/null @@ -1,251 +0,0 @@ -# CNN v2 Testing Tool - -WebGPU-based browser tool for testing trained CNN v2 weights. - ---- - -## Features - -- Drag-drop PNG images and `.bin` weights (or click to browse) -- Real-time CNN inference with WebGPU compute shaders -- View modes: CNN output, original input, difference (×10) -- Adjustable blend amount and depth -- Data-driven pipeline (supports variable layer count) -- GPU timing display -- **Left Panel:** Weights info + kernel visualization (1px/weight, all layers) -- **Right Panel:** Layer activation viewer with 4-channel split + 4× zoom - ---- - -## Requirements - -- Browser with WebGPU support: - - Chrome/Edge 113+ (enable `chrome://flags/#enable-unsafe-webgpu` if needed) - - Safari 18+ (macOS Ventura+) -- Trained CNN v2 weights in binary format (`.bin`) -- Test images (PNG format) - ---- - -## Usage - -### 1. Open Tool - -```bash -open tools/cnn_v2_test/index.html -``` - -Or use a local server to avoid CORS: -```bash -python3 -m http.server 8000 -# Open http://localhost:8000/tools/cnn_v2_test/ -``` - -### 2. Load Data - -1. **Drop `.bin` weights** into left sidebar zone (or click to browse) -2. **Drop PNG image** anywhere in center canvas area -3. CNN runs automatically when both loaded - -### 3. Layout - -**Left Sidebar:** -- Weights drop zone (click or drag-drop `.bin` files) -- Weights info panel (layer specs, ranges, file size) -- Weights visualization (click Layer 0/1/2 buttons) - - 1 pixel per weight, all input channels horizontally - - Output channels (Out 0-3) stacked vertically - -**Center Canvas:** -- Main output view (CNN result, original, or diff) -- Keyboard: `SPACE` = original, `D` = diff (×10) - -**Right Sidebar:** -- Layer selection buttons (Static 0-3/4-7, Layer 0/1/2) -- 4 small activation views (Ch0/1/2/3) in a row -- Large zoom view below (4× magnification, follows mouse) - -**Header Controls:** -- **Blend:** Mix between original (0.0) and CNN output (1.0) -- **Depth:** Uniform depth value for all pixels (0.0–1.0) -- **View:** Current display mode - -**Footer:** -- Status: GPU timing (ms), image dimensions, view mode -- Console: Timestamped event log (file loads, errors) - ---- - -## Preparing Test Data - -### Export Weights - -```bash -# From trained checkpoint -./training/export_cnn_v2_weights.py \ - checkpoints/checkpoint_epoch_100.pth \ - --output-weights tools/cnn_v2_test/test_weights.bin -``` - -Binary format: 16-byte header + 20 bytes per layer + f16 weights (~3.2 KB for 3-layer model) - -### Test Images - -Use training images or any PNG: -```bash -# Copy test image -cp training/input/test.png tools/cnn_v2_test/ -``` - -**Note:** Grayscale images automatically converted to RGB. - ---- - -## Validation - -### Visual Comparison - -Compare browser output with C++ tool: - -```bash -# Generate C++ output -./build/cnn_test training/input/test.png /tmp/cpp_output.png - -# Load same image in browser tool -# Visually compare outputs -``` - -### GPU Timing - -Expected performance: -- 512×512: ~1-2 ms (integrated GPU) -- 1024×1024: ~3-5 ms -- 1920×1080: ~5-8 ms - -Slower than expected? Check: -- WebGPU enabled in browser -- Dedicated GPU selected (if available) -- No background tabs consuming GPU - ---- - -## Troubleshooting - -### "WebGPU not supported" - -- Update browser to latest version -- Enable WebGPU flag: `chrome://flags/#enable-unsafe-webgpu` -- Try Safari 18+ (native WebGPU on macOS) - -### "Invalid .bin file" - -- Check magic number: `hexdump -C weights.bin | head` -- Should start with: `43 4e 4e 32` ('CNN2') -- Re-export weights: `./training/export_cnn_v2_weights.py` - -### Black output / incorrect colors - -- Check blend slider (set to 1.0 for full CNN output) -- Verify training converged (loss < 0.01) -- Compare with C++ tool output - -### Shader compilation errors - -Open browser console (F12) for detailed errors. Common issues: -- Image too large (>4096×4096 not tested) -- Unsupported texture format (rare on modern GPUs) - ---- - -## Architecture - -**Pipeline:** -1. **Static Features Pass** - Generate 8D features (RGBD, UV, sin, bias) -2. **CNN Layer Passes** - Compute N layers with ping-pong textures -3. **Display Pass** - Unpack and render with view mode - -**Textures:** -- Input: RGBA8 (original image) -- Depth: R32F (uniform depth) -- Static features: RGBA32Uint (8×f16 packed) -- Layer buffers: RGBA32Uint (ping-pong) - -**Data-Driven Execution:** -- Layer count read from binary header -- Per-layer params (kernel size, channels, offsets) from binary -- Single CNN shader dispatched N times - ---- - -## Implemented Features - -**✓ Weights Metadata Panel:** -- Layer descriptions (kernel size, channels, weight count) -- Weight statistics (min/max per layer) -- File size and layer count - -**✓ Weights Visualization:** -- Per-layer kernel heatmaps (1px/weight) -- All input channels displayed horizontally -- Output channels stacked vertically -- Normalized grayscale display - -**✓ Layer Activation Viewer:** -- Static features (8D split into 0-3 and 4-7 views) -- All CNN layer outputs (Layer 0/1/2...) -- 4-channel split view (grayscale per channel) -- Mouse-driven 4× zoom view - -## TODO - -**Future Enhancements:** -- Weight distribution histograms per layer -- Activation statistics (min/max/mean overlay) -- Side-by-side diff mode (browser vs C++ output) -- Export rendered layers as PNG - ---- - -## Extensions (v2+) - -Planned enhancements: - -**Variable Feature Count:** -- Binary v2: Add `num_features` to header -- Shader: Dynamic feature array or multiple textures - -**Multi-Scale Input (Mip Levels):** -- Uncomment mip bindings in static shader -- No binary format change needed - -**8-bit Quantized Weights:** -- Binary version bump (format field already present) -- Add quantization codepath in `get_weight()` function -- 2× size reduction (~1.6 KB) - -**Pre-defined Test Images:** -- Dropdown menu with training/input/*.png -- Requires local file server - ---- - -## Size - -- HTML structure: ~2 KB -- CSS styling: ~2 KB -- JavaScript logic: ~10 KB (includes zoom + weights viz) -- Static shader: ~1 KB -- CNN shader: ~3 KB -- Display shader: ~1 KB -- Layer viz shader: ~2 KB -- Zoom shader: ~1 KB -- **Total: ~22 KB** (single file, no dependencies) - ---- - -## See Also - -- `doc/CNN_V2.md` - Architecture and design -- `doc/HOWTO.md` - Training workflows -- `training/export_cnn_v2_weights.py` - Binary format -- `src/effects/cnn_v2_effect.cc` - C++ reference implementation diff --git a/tools/cnn_v2_test/index.html b/tools/cnn_v2_test/index.html deleted file mode 100644 index e226d0c..0000000 --- a/tools/cnn_v2_test/index.html +++ /dev/null @@ -1,2049 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> -<!-- - CNN v2 Testing Tool - WebGPU-based inference validator - - Architecture: - - Static features (8D): p0-p3 (parametric), uv_x, uv_y, sin(10*uv_x), bias (NOT a CNN layer) - - Layer 0: input RGBD (4D) + static (8D) = 12D → 4 channels - - Layer 1+: previous layer (4D) + static (8D) = 12D → 4 channels - - All CNN layers: uniform 12D input, 4D output (ping-pong buffer) - - Naming convention (matches train_cnn_v2.py / .wgsl / .cc): - - UI shows: "Static 0-3", "Static 4-7", "Layer 0", "Layer 1", "Layer 2" - - weights.layers[] array: Layer 0 = weights.layers[0], Layer 1 = weights.layers[1] - - Features: - - Input: PNG images or video files (MP4, WebM, etc.) - - Video playback: Play/Pause, frame-by-frame navigation (◄/► buttons) - - Video mode: Non-realtime processing (drops frames if CNN slower than playback) - - Side panel: .bin metadata display, weight statistics per layer - - Layer inspection: 4-channel grayscale split, intermediate layer visualization - - View modes: CNN output, original, diff (×10) - - Optimization: Layer viz updates only on pause/seek during video playback - - WGSL Shader Reuse: - - CNN_SHADER (inference), STATIC_SHADER, LAYER_VIZ_SHADER are inline for single-file deployment - - Can extract to .wgsl files for: better IDE support, testing, cross-tool reuse - - Tradeoff: extraction needs fetch() or build step, breaks single-file portability - - C++ sync: manual (WGSL ≠ GLSL) but logic identical ---> -<head> - <meta charset="UTF-8"> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <title>CNN v2 Testing Tool</title> - <style> - * { margin: 0; padding: 0; box-sizing: border-box; } - body { - font-family: 'Courier New', monospace; - background: #1a1a1a; - color: #e0e0e0; - display: flex; - flex-direction: column; - height: 100vh; - overflow: hidden; - } - .header { - background: #2a2a2a; - padding: 16px; - border-bottom: 1px solid #404040; - display: flex; - align-items: center; - gap: 24px; - flex-wrap: wrap; - } - h1 { font-size: 18px; } - .controls { - display: flex; - gap: 16px; - align-items: center; - flex-wrap: wrap; - } - .control-group { - display: flex; - gap: 8px; - align-items: center; - } - .control-group label { font-size: 12px; } - input[type="range"] { width: 120px; } - input[type="number"] { width: 60px; background: #1a1a1a; color: #e0e0e0; border: 1px solid #404040; padding: 4px; } - .drop-zone { - border: 3px dashed #606060; - padding: 20px; - text-align: center; - cursor: pointer; - transition: all 0.2s; - font-size: 13px; - font-weight: bold; - background: #252525; - border-radius: 6px; - color: #4a9eff; - } - button { - background: #1a1a1a; - border: 1px solid #404040; - color: #e0e0e0; - padding: 6px 12px; - font-size: 12px; - font-family: 'Courier New', monospace; - cursor: pointer; - transition: all 0.2s; - border-radius: 4px; - } - button:hover { border-color: #606060; background: #252525; } - button:disabled { opacity: 0.3; cursor: not-allowed; } - video { display: none; } - .drop-zone:hover { border-color: #4a9eff; background: #2a3545; } - .drop-zone.active { border-color: #4a9eff; background: #1a2a3a; } - .drop-zone.error { border-color: #ff4a4a; background: #3a1a1a; } - .content { - flex: 1; - display: flex; - overflow: hidden; - gap: 1px; - background: #404040; - } - .left-sidebar { - width: 315px; - background: #2a2a2a; - overflow-y: auto; - display: flex; - flex-direction: column; - gap: 16px; - padding: 16px; - } - .main { - flex: 1; - display: flex; - justify-content: center; - align-items: center; - padding: 24px; - overflow: auto; - position: relative; - background: #1a1a1a; - } - .video-controls-float { - position: absolute; - top: 16px; - left: 50%; - transform: translateX(-50%); - display: flex; - gap: 8px; - background: rgba(42, 42, 42, 0.95); - padding: 8px 12px; - border-radius: 4px; - border: 1px solid #404040; - z-index: 100; - } - .bottom-controls-float { - position: absolute; - bottom: 16px; - left: 50%; - transform: translateX(-50%); - display: flex; - gap: 16px; - align-items: center; - background: rgba(42, 42, 42, 0.95); - padding: 8px 16px; - border-radius: 4px; - border: 1px solid #404040; - z-index: 100; - } - .bottom-controls-float .control-group { - display: flex; - gap: 8px; - align-items: center; - } - .bottom-controls-float #videoControls { - display: flex; - gap: 8px; - align-items: center; - padding-right: 16px; - border-right: 1px solid #404040; - } - .main.drop-active::after { - content: 'Drop PNG/video here'; - position: absolute; - inset: 24px; - display: flex; - align-items: center; - justify-content: center; - border: 3px dashed #4a9eff; - background: rgba(74, 158, 255, 0.1); - font-size: 24px; - color: #4a9eff; - pointer-events: none; - z-index: 10; - } - .sidebar { - width: 400px; - background: #2a2a2a; - overflow-y: auto; - display: flex; - flex-direction: column; - gap: 16px; - padding: 16px; - } - .panel { - border: 1px solid #404040; - border-radius: 4px; - overflow: hidden; - } - .panel.collapsed .panel-content { - display: none; - } - .panel-header { - background: #1a1a1a; - padding: 8px 12px; - font-size: 12px; - font-weight: bold; - border-bottom: 1px solid #404040; - } - .panel-content { - padding: 12px; - font-size: 11px; - } - .panel-content table { - width: 100%; - border-collapse: collapse; - } - .panel-content th { - text-align: left; - padding: 4px; - font-size: 10px; - color: #808080; - border-bottom: 1px solid #404040; - } - .panel-content td { - padding: 4px; - font-size: 10px; - } - .panel-content tr:hover { - background: #1a1a1a; - } - .layer-buttons { - display: flex; - flex-wrap: wrap; - gap: 6px; - margin-bottom: 12px; - } - .layer-buttons button { - background: #1a1a1a; - border: 1px solid #404040; - color: #e0e0e0; - padding: 6px 12px; - font-size: 10px; - font-family: 'Courier New', monospace; - cursor: pointer; - transition: all 0.2s; - } - .layer-buttons button:hover { - border-color: #606060; - background: #252525; - } - .layer-buttons button.active { - background: #4a9eff; - border-color: #4a9eff; - color: #1a1a1a; - } - .layer-buttons button:disabled { - opacity: 0.3; - cursor: not-allowed; - } - .layer-buttons button:disabled:hover { - border-color: #404040; - background: #1a1a1a; - } - .layer-grid { - display: grid; - grid-template-columns: repeat(4, 1fr); - gap: 4px; - margin-bottom: 12px; - } - .layer-view { - aspect-ratio: 1; - background: #1a1a1a; - border: 1px solid #404040; - display: flex; - flex-direction: column; - overflow: hidden; - } - .layer-preview { - background: #1a1a1a; - border: 1px solid #404040; - display: flex; - flex-direction: column; - overflow: hidden; - margin-top: 8px; - } - .layer-preview canvas { - width: 100%; - height: 100%; - image-rendering: pixelated; - } - .layer-view.active { - border: 2px solid #ffffff; - } - .layer-view canvas { - cursor: pointer; - } - .layer-view-label { - background: #2a2a2a; - padding: 4px; - font-size: 9px; - text-align: center; - border-bottom: 1px solid #404040; - } - .layer-view canvas { - width: 100%; - height: 100%; - image-rendering: pixelated; - } - canvas { - max-width: 100%; - max-height: 100%; - image-rendering: pixelated; - box-shadow: 0 4px 12px rgba(0,0,0,0.5); - } - .footer { - background: #2a2a2a; - border-top: 1px solid #404040; - font-size: 11px; - display: flex; - flex-direction: column; - gap: 8px; - } - .footer-top { - padding: 12px 16px 0; - display: flex; - justify-content: space-between; - } - .status { color: #4a9eff; } - .shortcuts { color: #808080; } - .console { - background: #1a1a1a; - padding: 8px 16px; - font-family: 'Courier New', monospace; - font-size: 10px; - color: #808080; - max-height: 100px; - overflow-y: auto; - border-top: 1px solid #404040; - } - .console-line { margin: 2px 0; } - .console-line.error { color: #ff4a4a; } - .console-line.info { color: #4a9eff; } - </style> -</head> -<body> - <div class="header"> - <h1>CNN v2 Testing Tool</h1> - </div> - <video id="videoSource" muted loop></video> - <div class="content"> - <div class="left-sidebar"> - <input type="file" id="weightsFile" accept=".bin" style="display: none;"> - <div class="drop-zone" id="weightsDrop" onclick="document.getElementById('weightsFile').click()"> - Drop .bin Weights or Click to Browse - </div> - <div class="panel" id="weightsInfoPanel"> - <div class="panel-header">Weights Info</div> - <div class="panel-content" id="weightsInfo"> - <p style="color: #808080; text-align: center;">No weights loaded</p> - </div> - </div> - <div class="panel" id="weightsVizPanel" style="display: none;"> - <div class="panel-header">Weights Visualization</div> - <div class="panel-content" id="weightsViz"> - <div class="layer-buttons" id="weightsLayerButtons"></div> - <canvas id="weightsCanvas" style="width: 100%; image-rendering: pixelated; border: 1px solid #404040;"></canvas> - </div> - </div> - <div class="panel"> - <div class="panel-content"> - <label for="mipLevel" style="font-size: 11px;">Mip Level:</label> - <select id="mipLevel" style="width: 100%; background: #1a1a1a; color: #e0e0e0; border: 1px solid #404040; padding: 4px; margin-top: 4px;"> - <option value="0">Mip 0 (original)</option> - <option value="1">Mip 1 (half res)</option> - <option value="2">Mip 2 (quarter res)</option> - </select> - </div> - </div> - </div> - <div class="main" id="mainDrop"> - <div class="bottom-controls-float"> - <div id="videoControls"> - <button id="playPauseBtn" disabled>Play</button> - <button id="stepBackBtn" disabled>◄ Frame</button> - <button id="stepForwardBtn" disabled>Frame ►</button> - </div> - <div class="control-group"> - <label>Blend:</label> - <input type="range" id="blend" min="0" max="1" step="0.01" value="1.0"> - <span id="blendValue">1.0</span> - </div> - <div class="control-group"> - <label>Depth:</label> - <input type="range" id="depth" min="0" max="1" step="0.01" value="1.0"> - <span id="depthValue">1.0</span> - </div> - <button id="savePngBtn">Save PNG</button> - </div> - <canvas id="canvas"></canvas> - </div> - <div class="sidebar"> - <div class="panel" style="flex: 1; display: flex; flex-direction: column; min-height: 0;"> - <div class="panel-header">Layer Visualization</div> - <div class="panel-content" id="layerViz" style="flex: 1; overflow: hidden;"> - <p style="color: #808080; text-align: center;">Load image + weights</p> - </div> - </div> - </div> - </div> - <div class="footer"> - <div class="footer-top"> - <span class="status" id="status">Drop PNG/video anywhere to begin</span> - <span class="shortcuts">[SPACE] Original | [D] Diff (×10)</span> - </div> - <div class="console" id="console"></div> - </div> - - <script> -// ============================================================================ -// EMBEDDED WEIGHTS & CONSTANTS -// ============================================================================ - -// Default pre-trained weights (base64-encoded binary format) -// Version 2: 4 layers (3×3, 5×5, 3×3, 3×3), 2496 f16 weights, mip_level=2 -const DEFAULT_WEIGHTS_B64 = 'Q05OMgIAAAAEAAAAwAkAAAIAAAADAAAADAAAAAQAAAAAAAAAsAEAAAUAAAAMAAAABAAAALABAACwBAAAAwAAAAwAAAAEAAAAYAYAALABAAADAAAADAAAAAQAAAAQCAAAsAEAAAU3faplMDmtR7gnMLqt6bSrLM4RCa/En4q257kVsmWz57aSHJMxz6wILJC0tLdBriWww7IULUehCClCo60dBiu1nWqsf60ZKn6ktCWKjrswATSfLwQunzJjKKWkN6hxLTMwbS2DJvgvUjFDL1YsQDFFL78ysC5OL/cvxC2kJ6qh0i1BLH2rzCrcKFUoeixTqwwopjD+rXmewCY6sYUtXCwwsaKqGjBcqoykKigRJYStaqjMp+siPi1BLI+tGatfK5Ii6C1qLY0tYSGFKz4wpzNdH1QuJDKmMJi0lLVAs0y2Q7YWtY21fLXusf+n8LDSsaethK3drB4rtSROKYOrLK53qrqu0REYLEUuVy1qEqohDSzgqk4sDKKSKi0clKcVKvupJ69rKTmw8q7qptatQK7OsFUw5Z5JKJ4udSp9LLQeui87LbcxljEgJ6Iw75jDLfUvIjCxnh0g763Lq/ItMqzDqP0sXCRcqnkl9qDlJUStSyR8oTuwA616IrAnNqo5JS4qDKeILmahyaHZI48tryiajuEs0aghLBcuny+aovQpAhj6Kqkwdy+8MZ0wLzBvKBStsrRAKJez+raaKAotBiVSqZqyk7b2sHO1e7cJsfGmQLACpWizBLP9LnWxYLWoJPeb/CY5ISokXqynJ4qtG6K1qpesL6zGqYssIDJRpnErRi3RL9kh1zBFLPkdGSNvKtEuvyywmgilbC43LNovbywCKj4pFzEbMmMuly2gMFYscCgzliIomSqZnpSnyK3hJJKsAasgJGMrfCyNqXwpqaYNq14wiyzWLrSn/yLbqm+tnauOpkKtRKdCrBcYQS0dnGAveqeBrD8sMiGpLkAugzEaLM6lLzAkL5YydzYnqGo15zh2MuSwJK0nqxI04jZ5LAs2TjilNeSc3yANLecrCzBCprUvfjUHMWCuFrAkItyq/an0JSUnvKnrrAosv5CRrTGvQKesntuur6v2rsyxzbCAsHYn1y5GrAGsASYUmawrpSLooRSy86sBqmaxAq67sD0lJalOKxOtkqx8H+wqgygMLhup8SzNKZuhcafWKUKs567KI1opDCsoplatAykJpc+skavUrK4p2iznLlMqcig4Le6mDKiaJpIsMiOgLGOtQqI7sFGworKfsTOq86ZIlru0dLCEoMqq4KzsI6I2MzixMocqSym8MwQtT7Njqrwy26rEthe2nTGxL/Gq+az8MPg1Tq6EqXmslqyArkKs/S73MqEwmyuzrUUxejLhKYaw0yUlMzgxAZULsZ4rhq8ssgarCjDTrPop0ywBLswwjbT7MMAxdq2fsEC04DZoOIovG7G4LwM1gTNnKDsuEbByrzyxvLLBKJgkGDQANSMy66wVrM21ebURriAluK5quFa3wLBsK2wvaDU7OEg3RDGWKVUzpTfPNG+tbrGcr3ytRKosr7yuCbB2rV6gZq3msWmtjqvmoNurP6YXrOIpf6l/J2irl6/iqK2jy6MCLkkhjSDQoAWWACo1JrWjP6nvKvmthay+KJ6rUqoKqaatHKyJrUOarydBo5yu/CUaKFoxFCW1CNgpri2WK02kgqvYqkotwqlIrdiiEa1aKZ2tXa6mrkax4KkYKp2vcKgErYsi2RvbqWapU6EAnMyqtyPBpYwdZyVZkwGl1yhhJ2QBPaUJqMmMJJ54IikpcqmUHzmacCDzq1Cr3yR9n8aizKlWKFiogapBFlknrimnHmemDqbVKHciNRyII5AsxZ0+Lf0Xmyh7LMIqDS2KK9EkxyxRHKgp2iL9K0QfxCwGLLEuwiqrLcWob6xpppasp6+lotypGrC9qdmpPKUuplagES2cpSyrsSyHJTMi3Kk4KWAlSCaqKNMtR626rKaoj6koI1wqeivGI9cpuqQ9KQUkZyEJKOmquyW0JymirSjhprWgkBpKLFykzZyloWSrNKxrGaCtMi1MqL6t56lLqu+wbbTetYkqYDR1rB0wqir/sWQwNas8N9E4wq+9I6WwT6xuMDy1yC9tM/Kwka+btK8vJisnIJWeUa30LRkwDaqIsNqzWK9lLnEzKjEMqYMuWy8uMs0qI6xKLjcvxicEqYCv06zrrLusKK/lMeMz8CyCMmqxO7AtNpW38zFzL5i2Wq19tkCuBaTlt8Kv85Mlsg6wWLfgstutzDJVNAqZxCywrQgspDYOMS0mGbQCuf63QS7GJ4GsBLizuRS0mKyiKKMkBbLXseCufCr4qKUpah7Vqh8tV6eqLLQoGy1bMNEu6i4fMD4wZSvbjwOpmCBzLMmeJKddoYqkIic6qpqRY6nNqDiwIq5dqcmndqbnKnGkSCjmKBUsriySrHWsZyTaG7smSKxAIwolIi2zLX6unK5KqXCwKq03qyarcKWMqQmmd6tIodWtH6UvLg2tTadPJOOp2iGgny0ufyy+L7AvNClhpiEpC6qMqqMp7KTopJ4mmB2ylM6mrKhfKiQrTyiiKdGoQqjKJ6Umxip/qDiq/ChgKtmqIiwOr+CunZF7Kfot36poqkcthCx+Ksapg5T5pn0oNqOPq4osMSbSqQQmGqgXKhEl3yV1piyswazLK7QoQBTaqU8lIS13Ldch+qQqJ2AsPKfmp3Ink5Z2HhosR5z4qLIoGqkNLCct2Ck3KPGnUC0oJBQq7agOKyaq0qsqpAap8SylLg4qriy6M3MqKCtdKpMjSi86KigsGCz/n2erEyu7J/QRVCkpILUwcC35LI8qxiw6Knoq5jAAKo8wnieqLF0vVTAYMZw4Jyx2t/ayTjGWMoGzKbwus1w4QRxeJse1dTGSNJGwmCrEJV8uQKygKe4gjSqkrLeydiaMroS0FrQms8Uygi28qe2uXS2Ko4q1d7ZxszEpiDSBMoc0STWpNc0xJKSvrMWm6bCKsOC3CrEOJNC1Ga5Qubi7U6/+NRQ0AqnSuFoySDmKtJS0b7KcNAMmqi45IbMvGzjeMg2qSioPKVWtSK6EpaA1UTckMt2m16nwM5E2oDHBsZ+pniVpMc4vQy1epXkqHifBl7Mu36T/KzQorix4JAOmWyqJFVUqq67doiot2CxYME8i2JxVKhQt5ioYJsWp1KiSpL0lhq1JpWAgbCweKW2o1CrCIMsrcghkHUqW3hiTI5osYqMlB+WaLy0uKNUooKx4qdEezqRlJEapyKuUoEmoZyT7nqcoo6v3n4yqZaGcpNElwij3IkinQiAFIFQK2ygqIoKsiZxEI6ukqCf7KFSkgqSTqjEq8JZLJPufXKmFkaEj36lCKj2qURxfKkQouaqQhRIrGSmepKin7Cl8KEcuKI+ip4Evz6xIF0woVK/yHLyfLSj0ny+oWywSJHWmQaEomWos6ZTMpPWlY61pqLelZqYGpAidcyzQE5kneBr1pnQkJSwIqWYpIabdKA8oHKroGeCnYplOKzAmC51LJ0emp6o+rXAofCkCKV4w4x1sKCYjrKAgKa0r+BcPJDMmP6o2JW4pIqqtm4srTqgHlLWlsBBepaqrKq27rBat9aTlot8qkaw2o5sl76ivKDkjNyjzKKWY5KlHrQCr8SjxquarXqrlKB2xyyfZL1Sqq7LWpxA04zZwMkyvUiyHMig1ay+GJqenVq1Ao1awVLHQnrEqxTD/LO8kKB+NH1grfKsPsY6u+aIELLaj4LBmLBU0wDOlM8ksdKjbqPSqQykHJmYodC+WMcYuSCJ7psYvNDTaLqWw/qy7Myw4xjTnMIouQTV9OJ81YSlbLiIx3TVuMUcokrDzI0ow8CQQr9IvDyxsLnk0OTVhLmmobLAULN4zkyyZsGC0LK01L3Upw52Jroywlix0MCwr5qkQJkot9aWzsYuui66HrHykMa9ZsDet96yBqXWvXbAXsraxIqgpsVOvtq5frF+iZa2WqROwcaP+qX2w+aW3rxWpI7Bwrlqu5K0LrxexX7DUrfOvhK3QrUGwP7BrsY2tU6yWr8qkpK18rn2rHCbloYmfaqM1nfSr7Sn1qjuk2KT2qyem4KXJJ4MdxaidqPWsa58zKTSsoKXAJUymz6rJpv+oGKsOJo2hSicHqA4oOiiRmr4k0BxBq8Ui16jTKvyq7ijmqHcpZanhHnGfMikxIiEk7S4Yq90sfKWSoZyntKg/qh+nJiifnAyvlKeXJMIdViKeoxEjLKvZpXymAqkhraCofK5SnTGmLqdkq7mjYCD8qV0qQKo0qrUo+KsZKVSs0iaULFUI8qS0mlWtiiqbGBegACwBoAErhaW1qMwqHSxfKVKpp6x7poiweKxCrdkivK48sJewrKdArHYnqyhoHbUnsagYK58qSjAgMcUwsCt0K/4rLC7mJGwtvStOMFQu0SzuJQUsBTBMLswqcJyEnVQsESn3ox2z9ai/qFqwES7tKP0vSChMoqQwVzR4LKaT+y/NK06q2y0LIi2wHrIcKZuzsrSHn/6xkrPssAovJzEipEQiDbDjr3SqIis5LGIoOSm6p1apeqGGrtAqJzCIJRuptqrApiktWTAwMB4xQizXKoIgASFFsLwweTHbLdQtqyzXoKYtay3SLeOke6wgoPWr/SpFKUEmDacWptSoMChJKm6s6azkHe+mfzFKKyamfi6bK/wr5atPqEMxUTAlKSeueiRxoSQjQqxQLRavgauKriOssymXLZOooa97pFoufTSppqgoVq05tEg196yCsQIy7bEitAItJ7RgtUEzxjGML/QmEKIlrPgjPDFaoTYoPDFcJRavtK4XrKmsk6zjsCwsTa4UsPQs9jI/I3ct1C6cMV+b5y7wJZ0tYTF9MGojdS/oLTShziM/MVmnxC8FKJUwRCUxIz8wiS4QLWipLCCYq9EseabMKnEll6kPqIawRq+xGcgjyCkgqKed7SB6qZcr6CwJLW+st6ePq7WuHycUrhqsSq7zsKuZtimgCXCrmKkqnIGp4LHNsX2wnqyBsH2xIbDhpwCzra1ss44wTCypKDCyyK23LRiwYKKPMJmxcaqZKcshCCYipoyxNa1Nsbwozi1+MB8lQ5mtsDel3jDnlbutxiPzsWmp5SpTHaqys7EstauTPqoRsOosf6g3sLOgeaAfKUIsWi/BJdosUSzdMM4pSy3kpGM0DjWvLWw0cjR4MWWqQaYMLo2rZSijJjstZiFaLBadMq0TseyjYi0VGsQt8yo5oZCgti/HMLciM6r3KgMk8K6OqKup9q0srT0xcaWMMMwra67qrhSfsZ3GrrIj2a2+pqSvdrEcrRQ0IDhgMB+PCDWVM8qjnJ5ZKOmw4C0dMGyuG6DGMQUvrq+Oq4UsTSzHMRg2ibbXs+Axa7N5sAqqnSoerQUmky8oKIiuUjGsoBitdKy9q6iw661pqg4thKnpkYmt+a3gseypGp5Co22fM6YSKJap66hwopmsmqhlrCMkZyiLL4KnGKupKvUmyCQbLFUrbSZerKahlaRoqCYm5SqYKW0rcS8WrAUkzaMcGlqpRK3bnresXy18IXapEKqHKFssXKCpKMUrfamapf4tKjBiKJGoU54HK+8q5qq4qVuiZiy4JuEsTixNMFQnlSSIIw4k1KzxpbMlDqyKqz6gra4SpcOw3a3Vq+qqC6tOq22eORvnpC8hRadkka2q/K7HHUiowawpqPInLyA0qYMlsihUqGGkWCb7K1WdWK5Dr5EhnKv5KHKlXqYnJ/2l9i0YKUYuMzHxpyCs/ChMkPEtwanxoFQqJi3Uq7Mseq3arXskWKc5pOAc7CZcqCwc5w7qKO4f3iaKIDsq/KRgLpWsQqn5rYYkxCWPoU0bx6hzGdkkqibtofEoxy8GpUupSCTiKiwvpij7LbiulqkErXetejFkL2+upqtUp0OwiLAPsdCpxLIlrKOyQ7C2r3utIg0drZEl2y6oLkquoaX4rCysAa9GDRCwKrHDsNivAbHsqtioqiGvrqgJE66Kqw4rzKyDKgaomp6TK2EsDyc0oOSol6NZJkmsvyxorMss5pR0KBquEixPpjsgXCpsnXQocq2MrfGmoivvLBeacahmLROpe6kcGCSfdC03qL6i6yitHHohrxzqq4UiP6JMqF8qThOshWAVUqHupDsoohQuJSkv/ywqLiwlNjG7o++hxi3vIKmleCdyrH6wYatdsPWsjLCNol+sSTDpryCptbBDK+qs4zBpLGc0Nqc1rdo09jX5MqsrHi2xKOad8igwJxAoeSsiqgkqdChcLOYxJzGlMkAsUzCuKzskTjAOKhuplqjHqf8wzDKYIGefNDISqd8pIC23Ltwu7zC9KgMsQDL/JcgrryYzLJ0oTSoyqpkmLax+KuejVyqxr08ulZ2XpyQr5yxRsEMpwzD0KmEqoihRC6mwF6xOplwmjSSmpMep0SvhpOEndCluqLyvtCGgo3unOyy9IXKtmZ9yIK8hlqohrEUtxh0XKH0sGi18p6coHa3Tow6psqa/JRUMU6yiKbUoXigQpo2i7C18q3ur6CnWrSateC3/KY+jlCJ6o6qr+x8VJUkSFadyAgGpji0xraytBSd+rYksTqDAHQAtxSjkqMAmNqxhqNesEi5uKsqlFqo9Kg6seizOrdusAasErjmtoKv8rb8ph6cYLnMmcKlCLJ6pjiuIKpkpKK1UKvyq3RhVpZac+izlrYitWB+DrI4omKOZKikiZS1Fqicf+q25rJmsqKrYrNGt0JWRLWel2KfLqQ=='; - -// Reusable fullscreen quad vertex shader (2 triangles covering NDC) -const FULLSCREEN_QUAD_VS = ` -@vertex -fn vs_main(@builtin(vertex_index) idx: u32) -> @builtin(position) vec4<f32> { - var pos = array<vec2<f32>, 6>( - vec2<f32>(-1.0, -1.0), vec2<f32>(1.0, -1.0), vec2<f32>(-1.0, 1.0), - vec2<f32>(-1.0, 1.0), vec2<f32>(1.0, -1.0), vec2<f32>(1.0, 1.0) - ); - return vec4<f32>(pos[idx], 0.0, 1.0); -}`; - -// ============================================================================ -// WGSL SHADERS -// ============================================================================ - -// Static features: 7D parametric features (RGBD + UV + sin(10*uv_x) + bias) -const STATIC_SHADER = ` -@group(0) @binding(0) var input_tex: texture_2d<f32>; -@group(0) @binding(1) var linear_sampler: sampler; -@group(0) @binding(2) var depth_tex: texture_2d<f32>; -@group(0) @binding(3) var output_tex: texture_storage_2d<rgba32uint, write>; -@group(0) @binding(4) var<uniform> mip_level: u32; - -@compute @workgroup_size(8, 8) -fn main(@builtin(global_invocation_id) id: vec3<u32>) { - let coord = vec2<i32>(id.xy); - let dims = textureDimensions(input_tex); - if (coord.x >= i32(dims.x) || coord.y >= i32(dims.y)) { return; } - - // Use normalized UV coords with linear sampler (bilinear filtering) - let uv = (vec2<f32>(coord) + 0.5) / vec2<f32>(dims); - let rgba = textureSampleLevel(input_tex, linear_sampler, uv, f32(mip_level)); - - let p0 = rgba.r; - let p1 = rgba.g; - let p2 = rgba.b; - let p3 = textureLoad(depth_tex, coord, 0).r; - - let uv_x = f32(coord.x) / f32(dims.x); - let uv_y = f32(coord.y) / f32(dims.y); - let sin20_y = sin(20.0 * uv_y); - let bias = 1.0; - - let packed = vec4<u32>( - pack2x16float(vec2<f32>(p0, p1)), - pack2x16float(vec2<f32>(p2, p3)), - pack2x16float(vec2<f32>(uv_x, uv_y)), - pack2x16float(vec2<f32>(sin20_y, bias)) - ); - textureStore(output_tex, coord, packed); -}`; - -const CNN_SHADER = ` -struct LayerParams { - kernel_size: u32, - in_channels: u32, - out_channels: u32, - weight_offset: u32, - is_output_layer: u32, - blend_amount: f32, - is_layer_0: u32, -} - -@group(0) @binding(0) var static_features: texture_2d<u32>; -@group(0) @binding(1) var layer_input: texture_2d<u32>; -@group(0) @binding(2) var output_tex: texture_storage_2d<rgba32uint, write>; -@group(0) @binding(3) var<storage, read> weights_buffer: array<u32>; -@group(0) @binding(4) var<uniform> params: LayerParams; -@group(0) @binding(5) var original_input: texture_2d<f32>; - -fn unpack_static_features(coord: vec2<i32>) -> array<f32, 8> { - let packed = textureLoad(static_features, coord, 0); - let v0 = unpack2x16float(packed.x); - let v1 = unpack2x16float(packed.y); - let v2 = unpack2x16float(packed.z); - let v3 = unpack2x16float(packed.w); - return array<f32, 8>(v0.x, v0.y, v1.x, v1.y, v2.x, v2.y, v3.x, v3.y); -} - -fn unpack_layer_channels(coord: vec2<i32>) -> vec4<f32> { - let packed = textureLoad(layer_input, coord, 0); - let v0 = unpack2x16float(packed.x); - let v1 = unpack2x16float(packed.y); - return vec4<f32>(v0.x, v0.y, v1.x, v1.y); -} - -fn pack_channels(values: vec4<f32>) -> vec4<u32> { - return vec4<u32>( - pack2x16float(vec2<f32>(values.x, values.y)), - pack2x16float(vec2<f32>(values.z, values.w)), - 0u, - 0u - ); -} - -fn get_weight(idx: u32) -> f32 { - let pair_idx = idx / 2u; - let packed = weights_buffer[pair_idx]; - let unpacked = unpack2x16float(packed); - return select(unpacked.y, unpacked.x, (idx & 1u) == 0u); -} - -@compute @workgroup_size(8, 8) -fn main(@builtin(global_invocation_id) id: vec3<u32>) { - let coord = vec2<i32>(id.xy); - let dims = textureDimensions(static_features); - if (coord.x >= i32(dims.x) || coord.y >= i32(dims.y)) { return; } - - let kernel_size = params.kernel_size; - let in_channels = params.in_channels; // Always 12 (4 prev + 8 static) - let out_channels = params.out_channels; // Always 4 - let weight_offset = params.weight_offset; - let is_output = params.is_output_layer != 0u; - let kernel_radius = i32(kernel_size / 2u); - - let static_feat = unpack_static_features(coord); - - var output: vec4<f32> = vec4<f32>(0.0); - for (var c: u32 = 0u; c < 4u; c++) { - var sum: f32 = 0.0; - for (var ky: i32 = -kernel_radius; ky <= kernel_radius; ky++) { - for (var kx: i32 = -kernel_radius; kx <= kernel_radius; kx++) { - let sample_coord = coord + vec2<i32>(kx, ky); - let clamped = vec2<i32>( - clamp(sample_coord.x, 0, i32(dims.x) - 1), - clamp(sample_coord.y, 0, i32(dims.y) - 1) - ); - let static_local = unpack_static_features(clamped); - let layer_local = unpack_layer_channels(clamped); - - let ky_idx = u32(ky + kernel_radius); - let kx_idx = u32(kx + kernel_radius); - let spatial_idx = ky_idx * kernel_size + kx_idx; - - // Previous layer channels (4D) - for (var i: u32 = 0u; i < 4u; i++) { - let w_idx = weight_offset + - c * in_channels * kernel_size * kernel_size + - i * kernel_size * kernel_size + spatial_idx; - sum += get_weight(w_idx) * layer_local[i]; - } - - // Static features (8D) - for (var i: u32 = 0u; i < 8u; i++) { - let w_idx = weight_offset + - c * in_channels * kernel_size * kernel_size + - (4u + i) * kernel_size * kernel_size + spatial_idx; - sum += get_weight(w_idx) * static_local[i]; - } - } - } - - if (is_output || params.is_layer_0 != 0u) { - output[c] = 1.0 / (1.0 + exp(-sum)); // Sigmoid [0,1] - } else { - output[c] = max(0.0, sum); // ReLU - } - } - - if (is_output) { - let original = textureLoad(original_input, coord, 0).rgb; - let result_rgb = vec3<f32>(output.x, output.y, output.z); - let blended = mix(original, result_rgb, params.blend_amount); - output.x = blended.r; - output.y = blended.g; - output.z = blended.b; - } - - textureStore(output_tex, coord, pack_channels(output)); -}`; - -const DISPLAY_SHADER = ` -@group(0) @binding(0) var result_tex: texture_2d<u32>; -@group(0) @binding(1) var original_tex: texture_2d<f32>; -@group(0) @binding(2) var<uniform> mode: u32; - -@vertex -fn vs_main(@builtin(vertex_index) idx: u32) -> @builtin(position) vec4<f32> { - var pos = array<vec2<f32>, 6>( - vec2<f32>(-1.0, -1.0), vec2<f32>(1.0, -1.0), vec2<f32>(-1.0, 1.0), - vec2<f32>(-1.0, 1.0), vec2<f32>(1.0, -1.0), vec2<f32>(1.0, 1.0) - ); - return vec4<f32>(pos[idx], 0.0, 1.0); -} - -@fragment -fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { - let coord = vec2<i32>(pos.xy); - let packed = textureLoad(result_tex, coord, 0); - let v0 = unpack2x16float(packed.x); - let v1 = unpack2x16float(packed.y); - let result = vec3<f32>(v0.x, v0.y, v1.x); - - if (mode == 0u) { - return vec4<f32>(result, 1.0); - } else if (mode == 1u) { - let original = textureLoad(original_tex, coord, 0).rgb; - return vec4<f32>(original, 1.0); - } else { - let original = textureLoad(original_tex, coord, 0).rgb; - let diff = abs(result - original) * 10.0; - return vec4<f32>(diff, 1.0); - } -}`; - -const LAYER_VIZ_SHADER = ` -@group(0) @binding(0) var layer_tex: texture_2d<u32>; -@group(0) @binding(1) var<uniform> viz_params: vec2<f32>; // x=channel_idx, y=scale - -@vertex -fn vs_main(@builtin(vertex_index) idx: u32) -> @builtin(position) vec4<f32> { - var pos = array<vec2<f32>, 6>( - vec2<f32>(-1.0, -1.0), vec2<f32>(1.0, -1.0), vec2<f32>(-1.0, 1.0), - vec2<f32>(-1.0, 1.0), vec2<f32>(1.0, -1.0), vec2<f32>(1.0, 1.0) - ); - return vec4<f32>(pos[idx], 0.0, 1.0); -} - -@fragment -fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { - let coord = vec2<i32>(pos.xy); - let dims = textureDimensions(layer_tex); - - let channel = u32(viz_params.x); - - // DEBUG MODE 1: Texture coordinates (channel 10) - if (channel == 10u) { - let uv = vec2<f32>(f32(coord.x) / f32(dims.x), f32(coord.y) / f32(dims.y)); - return vec4<f32>(uv.x, uv.y, 0.0, 1.0); - } - - let packed = textureLoad(layer_tex, coord, 0); - - // DEBUG MODE 2: Raw packed data (channel 11) - if (channel == 11u) { - let raw_val = f32(packed.x) / 4294967295.0; - return vec4<f32>(raw_val, raw_val, raw_val, 1.0); - } - - let v0 = unpack2x16float(packed.x); - let v1 = unpack2x16float(packed.y); - let v2 = unpack2x16float(packed.z); - let v3 = unpack2x16float(packed.w); - - // DEBUG MODE 3: First unpacked value (channel 12) - if (channel == 12u) { - return vec4<f32>(v0.x, v0.x, v0.x, 1.0); - } - - var channels: array<f32, 8>; - channels[0] = v0.x; - channels[1] = v0.y; - channels[2] = v1.x; - channels[3] = v1.y; - channels[4] = v2.x; - channels[5] = v2.y; - channels[6] = v3.x; - channels[7] = v3.y; - - let scale = viz_params.y; - - let idx = min(channel, 7u); - let raw = channels[idx]; - - // Apply scale: multiply and clamp to [0, 1] - let val = clamp(raw * scale, 0.0, 1.0); - - return vec4<f32>(val, val, val, 1.0); -}`; - -class CNNTester { - constructor() { - this.canvas = document.getElementById('canvas'); - this.status = document.getElementById('status'); - this.console = document.getElementById('console'); - this.image = null; - this.video = document.getElementById('videoSource'); - this.weights = null; - this.viewMode = 0; - this.blendAmount = 1.0; - this.depth = 1.0; - this.currentLayerIdx = null; - this.currentChannelOffset = null; - this.isVideo = false; - this.fps = 30; - this.isProcessing = false; - this.mipLevel = 0; - this.selectedChannel = 0; - this.init(); - } - - log(msg, type = 'info') { - const line = document.createElement('div'); - line.className = `console-line ${type}`; - line.textContent = `[${new Date().toLocaleTimeString()}] ${msg}`; - this.console.appendChild(line); - this.console.scrollTop = this.console.scrollHeight; - } - - async init() { - if (!navigator.gpu) { - this.setStatus('WebGPU not supported', true); - this.log('WebGPU not supported in this browser', 'error'); - return; - } - - try { - this.adapter = await navigator.gpu.requestAdapter(); - this.device = await this.adapter.requestDevice(); - this.context = this.canvas.getContext('webgpu'); - this.format = navigator.gpu.getPreferredCanvasFormat(); - this.log('WebGPU initialized successfully'); - } catch (e) { - this.setStatus(`GPU init failed: ${e.message}`, true); - this.log(`GPU initialization failed: ${e.message}`, 'error'); - } - } - - setStatus(msg, isError = false) { - this.status.textContent = msg; - this.status.style.color = isError ? '#ff4a4a' : '#4a9eff'; - } - - // Get current source dimensions (video or image) - getDimensions() { - if (this.isVideo) { - return { width: this.video.videoWidth, height: this.video.videoHeight }; - } - return { width: this.image.width, height: this.image.height }; - } - - // Enable/disable video playback controls - setVideoControlsEnabled(enabled) { - ['playPauseBtn', 'stepBackBtn', 'stepForwardBtn'].forEach(id => - document.getElementById(id).disabled = !enabled - ); - } - - parseWeights(buffer) { - const view = new DataView(buffer); - const magic = view.getUint32(0, true); - if (magic !== 0x32_4E_4E_43) { - throw new Error('Invalid .bin file (bad magic)'); - } - - const version = view.getUint32(4, true); - const numLayers = view.getUint32(8, true); - const totalWeights = view.getUint32(12, true); - - // Version 2: added mip_level field (20-byte header) - let mipLevel = 0; - let headerSize = 16; - if (version === 2) { - mipLevel = view.getUint32(16, true); - headerSize = 20; - this.log(`Binary header: version=${version}, layers=${numLayers}, weights=${totalWeights}, mip_level=${mipLevel}`); - } else if (version === 1) { - this.log(`Binary header: version=${version}, layers=${numLayers}, weights=${totalWeights}`); - } else { - throw new Error(`Unsupported binary version: ${version}`); - } - - const layers = []; - for (let i = 0; i < numLayers; i++) { - const offset = headerSize + i * 20; - const layer = { - kernelSize: view.getUint32(offset, true), - inChannels: view.getUint32(offset + 4, true), - outChannels: view.getUint32(offset + 8, true), - weightOffset: view.getUint32(offset + 12, true), - weightCount: view.getUint32(offset + 16, true), - }; - layers.push(layer); - this.log(` Layer ${i}: ${layer.inChannels}→${layer.outChannels}, kernel=${layer.kernelSize}×${layer.kernelSize}, weights=${layer.weightCount}`); - } - - const weightsOffset = headerSize + numLayers * 20; - const weights = new Uint32Array(buffer.slice(weightsOffset)); - - // Calculate min/max per layer - for (let i = 0; i < numLayers; i++) { - const layer = layers[i]; - let min = Infinity, max = -Infinity; - const startIdx = layer.weightOffset; - const endIdx = startIdx + layer.weightCount; - - for (let j = startIdx; j < endIdx; j++) { - const pairIdx = Math.floor(j / 2); - const packed = weights[pairIdx]; - const unpacked = this.unpackF16(packed); - const val = (j % 2 === 0) ? unpacked[0] : unpacked[1]; - min = Math.min(min, val); - max = Math.max(max, val); - } - - layer.min = min; - layer.max = max; - this.log(` Layer ${i} range: [${min.toFixed(4)}, ${max.toFixed(4)}]`); - } - - let nonZero = 0; - for (let i = 0; i < weights.length; i++) { - if (weights[i] !== 0) nonZero++; - } - this.log(` Weight buffer: ${weights.length} u32 (${nonZero} non-zero)`); - - return { version, layers, weights, mipLevel, fileSize: buffer.byteLength }; - } - - unpackF16(packed) { - const lo = packed & 0xFFFF; - const hi = (packed >> 16) & 0xFFFF; - const toFloat = (bits) => { - const sign = (bits >> 15) & 1; - const exp = (bits >> 10) & 0x1F; - const frac = bits & 0x3FF; - if (exp === 0) return (sign ? -1 : 1) * Math.pow(2, -14) * (frac / 1024); - if (exp === 31) return frac ? NaN : (sign ? -Infinity : Infinity); - return (sign ? -1 : 1) * Math.pow(2, exp - 15) * (1 + frac / 1024); - }; - return [toFloat(lo), toFloat(hi)]; - } - - async loadImage(file) { - const img = await createImageBitmap(file); - this.image = img; - this.isVideo = false; - this.canvas.width = img.width; - this.canvas.height = img.height; - this.setVideoControlsEnabled(false); - this.log(`Loaded image: ${file.name} (${img.width}×${img.height})`); - if (this.weights) { - this.setStatus(`Ready: ${img.width}×${img.height}`); - this.run(); - } else { - this.setStatus(`Image loaded (${img.width}×${img.height}) - drop .bin weights to process`); - this.displayOriginal(); - } - } - - // Video loading: wait for metadata, then first frame decode (readyState≥2) - async loadVideo(file) { - return new Promise((resolve, reject) => { - this.video.src = URL.createObjectURL(file); - - this.video.onloadedmetadata = () => { - const w = this.video.videoWidth; - const h = this.video.videoHeight; - if (w === 0 || h === 0) { - reject(new Error('Video has invalid dimensions')); - return; - } - - this.isVideo = true; - this.canvas.width = w; - this.canvas.height = h; - this.fps = 30; - this.log(`Loaded video: ${file.name} (${w}×${h}, ${this.video.duration.toFixed(1)}s)`); - this.setVideoControlsEnabled(true); - - // Set up event handlers - this.video.onpause = () => { document.getElementById('playPauseBtn').textContent = 'Play'; }; - this.video.onplay = () => { document.getElementById('playPauseBtn').textContent = 'Pause'; this.playbackLoop(); }; - - // Wait for first frame to be decoded before displaying - const displayFirstFrame = () => { - this.video.onseeked = () => { if (!this.isProcessing) this.processVideoFrame(); }; - if (this.video.readyState >= 2) { // HAVE_CURRENT_DATA or better - if (this.weights) { - this.setStatus(`Ready: ${w}×${h}`); - this.processVideoFrame().then(() => resolve()); - } else { - this.setStatus(`Video loaded - drop .bin weights to process`); - this.displayOriginal(); - resolve(); - } - } else { - setTimeout(displayFirstFrame, 50); // Poll until frame ready - } - }; - - this.video.onseeked = displayFirstFrame; - this.video.currentTime = 0; - }; - - this.video.onerror = () => reject(new Error('Failed to load video')); - }); - } - - // Video playback loop (non-realtime, drops frames if CNN slow) - playbackLoop() { - if (this.video.paused || this.video.ended) return; - if (!this.isProcessing) this.processVideoFrame(); - requestAnimationFrame(() => this.playbackLoop()); - } - - // Process current video frame through CNN pipeline - async processVideoFrame() { - if (!this.weights || this.isProcessing) return; - this.isProcessing = true; - await this.run(); - this.isProcessing = false; - } - - // Video controls - togglePlayPause() { - this.video.paused ? this.video.play() : this.video.pause(); - } - - stepFrame(direction) { - if (!this.isVideo) return; - this.video.pause(); - this.video.currentTime = Math.max(0, Math.min(this.video.duration, - this.video.currentTime + direction / this.fps)); - } - - async loadWeights(file) { - const buffer = await file.arrayBuffer(); - this.weights = this.parseWeights(buffer); - this.weightsBuffer = buffer; - this.mipLevel = this.weights.mipLevel; // Set mip level from binary format - this.log(`Loaded weights: ${file.name} (${this.weights.layers.length} layers, ${(buffer.byteLength/1024).toFixed(1)} KB)`); - - // Update UI dropdown to reflect loaded mip level - const mipLevelSelect = document.getElementById('mipLevel'); - if (mipLevelSelect) { - mipLevelSelect.value = this.mipLevel.toString(); - } - - this.updateWeightsPanel(); - if (this.image) { - this.setStatus(`Ready: ${this.image.width}×${this.image.height}`); - this.run(); - } else { - this.setStatus('Weights loaded - drop PNG image to process'); - } - } - - updateWeightsPanel() { - const panel = document.getElementById('weightsInfo'); - const { version, layers, mipLevel, fileSize } = this.weights; - - let html = ` - <div style="margin-bottom: 12px;"> - <div><strong>File Size:</strong> ${(fileSize / 1024).toFixed(2)} KB</div> - <div><strong>Version:</strong> ${version}</div> - <div><strong>CNN Layers:</strong> ${layers.length}</div> - <div><strong>Mip Level:</strong> ${mipLevel} (p0-p3 features)</div> - <div style="font-size: 9px; color: #808080; margin-top: 4px;">Static features (input) + ${layers.length} conv layers</div> - </div> - <table> - <thead> - <tr> - <th>Layer</th> - <th>Size</th> - <th>Weights</th> - <th>Min</th> - <th>Max</th> - </tr> - </thead> - <tbody> - `; - - // Display layers as "Layer 0", "Layer 1", etc. (matching codebase convention) - for (let i = 0; i < layers.length; i++) { - const l = layers[i]; - html += ` - <tr> - <td>Layer ${i}</td> - <td>${l.inChannels}→${l.outChannels} (${l.kernelSize}×${l.kernelSize})</td> - <td>${l.weightCount}</td> - <td>${l.min.toFixed(3)}</td> - <td>${l.max.toFixed(3)}</td> - </tr> - `; - } - - html += ` - </tbody> - </table> - `; - - panel.innerHTML = html; - - // Show weights visualization panel and create layer buttons - const weightsVizPanel = document.getElementById('weightsVizPanel'); - weightsVizPanel.style.display = 'block'; - - const weightsLayerButtons = document.getElementById('weightsLayerButtons'); - let buttonsHtml = ''; - for (let i = 0; i < layers.length; i++) { - buttonsHtml += `<button onclick="tester.visualizeWeights(${i})" id="weightsBtn${i}">Layer ${i}</button>`; - } - weightsLayerButtons.innerHTML = buttonsHtml; - - // Auto-select first layer - this.visualizeWeights(0); - } - - generateMipmaps(texture, width, height) { - if (!this.mipmapPipeline) { - const mipmapShader = FULLSCREEN_QUAD_VS + ` - @group(0) @binding(0) var src: texture_2d<f32>; - @fragment - fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { - let coord = vec2<i32>(i32(pos.x) * 2, i32(pos.y) * 2); - var sum = vec4<f32>(0.0); - for (var y: i32 = 0; y < 2; y++) { - for (var x: i32 = 0; x < 2; x++) { - sum += textureLoad(src, coord + vec2<i32>(x, y), 0); - } - } - return sum * 0.25; - } - `; - this.mipmapPipeline = this.device.createRenderPipeline({ - layout: 'auto', - vertex: { module: this.device.createShaderModule({ code: mipmapShader }), entryPoint: 'vs_main' }, - fragment: { - module: this.device.createShaderModule({ code: mipmapShader }), - entryPoint: 'fs_main', - targets: [{ format: 'rgba8unorm' }] - } - }); - } - - const encoder = this.device.createCommandEncoder(); - - for (let mip = 1; mip < 3; mip++) { - const mipWidth = Math.max(1, width >> mip); - const mipHeight = Math.max(1, height >> mip); - - const bindGroup = this.device.createBindGroup({ - layout: this.mipmapPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: texture.createView({ baseMipLevel: mip - 1, mipLevelCount: 1 }) } - ] - }); - - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: texture.createView({ baseMipLevel: mip, mipLevelCount: 1 }), - loadOp: 'clear', - storeOp: 'store' - }] - }); - - renderPass.setPipeline(this.mipmapPipeline); - renderPass.setBindGroup(0, bindGroup); - renderPass.setViewport(0, 0, mipWidth, mipHeight, 0, 1); - renderPass.draw(6); - renderPass.end(); - } - - this.device.queue.submit([encoder.finish()]); - } - - displayOriginal() { - const source = this.isVideo ? this.video : this.image; - if (!source || !this.device) return; - - const { width, height } = this.getDimensions(); - this.context.configure({ device: this.device, format: this.format }); - - const inputTex = this.device.createTexture({ - size: [width, height], - format: 'rgba8unorm', - usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST | GPUTextureUsage.RENDER_ATTACHMENT - }); - - this.device.queue.copyExternalImageToTexture( - { source: source }, - { texture: inputTex }, - [width, height] - ); - - const simpleShader = FULLSCREEN_QUAD_VS + ` - @group(0) @binding(0) var tex: texture_2d<f32>; - @fragment - fn fs_main(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> { - let coord = vec2<i32>(pos.xy); - return textureLoad(tex, coord, 0); - } - `; - - const pipeline = this.device.createRenderPipeline({ - layout: 'auto', - vertex: { module: this.device.createShaderModule({ code: simpleShader }), entryPoint: 'vs_main' }, - fragment: { - module: this.device.createShaderModule({ code: simpleShader }), - entryPoint: 'fs_main', - targets: [{ format: this.format }] - } - }); - - const bindGroup = this.device.createBindGroup({ - layout: pipeline.getBindGroupLayout(0), - entries: [{ binding: 0, resource: inputTex.createView() }] - }); - - const encoder = this.device.createCommandEncoder(); - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: this.context.getCurrentTexture().createView(), - loadOp: 'clear', - storeOp: 'store' - }] - }); - renderPass.setPipeline(pipeline); - renderPass.setBindGroup(0, bindGroup); - renderPass.draw(6); - renderPass.end(); - - this.device.queue.submit([encoder.finish()]); - } - - // Run CNN inference pipeline on current source (image or video frame) - async run() { - const t0 = performance.now(); - const source = this.isVideo ? this.video : this.image; - if (!source) return; - const { width, height } = this.getDimensions(); - - this.context.configure({ device: this.device, format: this.format }); - - // Create persistent input texture for original view with mipmaps - if (this.inputTexture) this.inputTexture.destroy(); - this.inputTexture = this.device.createTexture({ - size: [width, height], - format: 'rgba8unorm', - mipLevelCount: 3, - usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST | GPUTextureUsage.RENDER_ATTACHMENT - }); - - this.device.queue.copyExternalImageToTexture( - { source: source }, - { texture: this.inputTexture, mipLevel: 0 }, - [width, height] - ); - - // Generate mipmaps - this.generateMipmaps(this.inputTexture, width, height); - - const staticTex = this.device.createTexture({ - size: [width, height], - format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC - }); - - // Create one texture per layer output (static + all CNN layers) - this.layerOutputs = []; - const numLayers = this.weights.layers.length + 1; // +1 for static features - const layerTextures = []; - for (let i = 0; i < numLayers; i++) { - layerTextures.push(this.device.createTexture({ - size: [width, height], - format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST - })); - } - - // Ping-pong buffers for computation - const computeTextures = [ - this.device.createTexture({ - size: [width, height], - format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC - }), - this.device.createTexture({ - size: [width, height], - format: 'rgba32uint', - usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_SRC - }) - ]; - - const weightsGPU = this.device.createBuffer({ - size: this.weightsBuffer.byteLength, - usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST - }); - this.device.queue.writeBuffer(weightsGPU, 0, this.weightsBuffer); - const staticPipeline = this.device.createComputePipeline({ - layout: 'auto', - compute: { module: this.device.createShaderModule({ code: STATIC_SHADER }), entryPoint: 'main' } - }); - - const cnnPipeline = this.device.createComputePipeline({ - layout: 'auto', - compute: { module: this.device.createShaderModule({ code: CNN_SHADER }), entryPoint: 'main' } - }); - - const displayPipeline = this.device.createRenderPipeline({ - layout: 'auto', - vertex: { module: this.device.createShaderModule({ code: DISPLAY_SHADER }), entryPoint: 'vs_main' }, - fragment: { - module: this.device.createShaderModule({ code: DISPLAY_SHADER }), - entryPoint: 'fs_main', - targets: [{ format: this.format }] - } - }); - - const encoder = this.device.createCommandEncoder(); - - const mipLevelBuffer = this.device.createBuffer({ - size: 4, - usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST - }); - this.device.queue.writeBuffer(mipLevelBuffer, 0, new Uint32Array([this.mipLevel])); - - if (!this.pointSampler) { - this.pointSampler = this.device.createSampler({ - magFilter: 'linear', - minFilter: 'linear', - mipmapFilter: 'linear' - }); - } - - // Extract depth from alpha channel (or 1.0 if no alpha) - const depthTex = this.device.createTexture({ - size: [width, height, 1], - format: 'r32float', - usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST - }); - - // Read image data to extract alpha channel - const tempCanvas = document.createElement('canvas'); - tempCanvas.width = width; - tempCanvas.height = height; - const tempCtx = tempCanvas.getContext('2d'); - tempCtx.drawImage(source, 0, 0, width, height); - const imageData = tempCtx.getImageData(0, 0, width, height); - const pixels = imageData.data; - - // Extract alpha channel (RGBA format: every 4th byte) - const depthData = new Float32Array(width * height); - for (let i = 0; i < width * height; i++) { - depthData[i] = pixels[i * 4 + 3] / 255.0; // Alpha channel [0, 255] → [0, 1] - } - - this.device.queue.writeTexture( - { texture: depthTex }, - depthData, - { bytesPerRow: width * 4 }, - [width, height, 1] - ); - - const staticBG = this.device.createBindGroup({ - layout: staticPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: this.inputTexture.createView() }, - { binding: 1, resource: this.pointSampler }, - { binding: 2, resource: depthTex.createView() }, // Depth from alpha (matches training) - { binding: 3, resource: staticTex.createView() }, - { binding: 4, resource: { buffer: mipLevelBuffer } } - ] - }); - - const staticPass = encoder.beginComputePass(); - staticPass.setPipeline(staticPipeline); - staticPass.setBindGroup(0, staticBG); - staticPass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8)); - staticPass.end(); - - // Copy static features to persistent storage (visualization index 0, shown as Static 0-3 / Static 4-7) - encoder.copyTextureToTexture( - { texture: staticTex }, - { texture: layerTextures[0] }, - [width, height] - ); - this.layerOutputs.push(layerTextures[0]); - - let srcTex = staticTex; - let dstTex = computeTextures[0]; - - for (let i = 0; i < this.weights.layers.length; i++) { - const layer = this.weights.layers[i]; - const isOutput = i === this.weights.layers.length - 1; - - // Calculate absolute weight offset in f16 units (add header offset) - // Version 1: 4 u32 header, Version 2: 5 u32 header - const headerSizeU32 = (this.weights.version === 1) ? 4 : 5; - const headerOffsetU32 = headerSizeU32 + this.weights.layers.length * 5; // Header + layer info in u32 - const absoluteWeightOffset = headerOffsetU32 * 2 + layer.weightOffset; // Convert to f16 units - - const paramsData = new Uint32Array(7); - paramsData[0] = layer.kernelSize; - paramsData[1] = layer.inChannels; - paramsData[2] = layer.outChannels; - paramsData[3] = absoluteWeightOffset; // Use absolute offset - paramsData[4] = isOutput ? 1 : 0; - paramsData[6] = (i === 0) ? 1 : 0; // is_layer_0 flag - - const paramsView = new Float32Array(paramsData.buffer); - paramsView[5] = this.blendAmount; - - const paramsBuffer = this.device.createBuffer({ - size: 28, - usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST - }); - this.device.queue.writeBuffer(paramsBuffer, 0, paramsData); - - const cnnBG = this.device.createBindGroup({ - layout: cnnPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: layerTextures[0].createView() }, - { binding: 1, resource: srcTex.createView() }, - { binding: 2, resource: dstTex.createView() }, - { binding: 3, resource: { buffer: weightsGPU } }, - { binding: 4, resource: { buffer: paramsBuffer } }, - { binding: 5, resource: this.inputTexture.createView() } - ] - }); - - const cnnPass = encoder.beginComputePass(); - cnnPass.setPipeline(cnnPipeline); - cnnPass.setBindGroup(0, cnnBG); - cnnPass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8)); - cnnPass.end(); - - [srcTex, dstTex] = [dstTex, srcTex]; - - // Copy CNN layer output to persistent storage for visualization - // i=0: Layer 0 → layerTextures[1] - // i=1: Layer 1 → layerTextures[2], etc. - encoder.copyTextureToTexture( - { texture: srcTex }, - { texture: layerTextures[i + 1] }, - [width, height] - ); - - // Always push layer outputs for visualization (including output layer) - this.layerOutputs.push(layerTextures[i + 1]); - } - - const modeBuffer = this.device.createBuffer({ - size: 4, - usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST - }); - this.device.queue.writeBuffer(modeBuffer, 0, new Uint32Array([this.viewMode])); - - // Store result texture and display pipeline for view mode switching - this.resultTexture = srcTex; - this.displayPipeline = displayPipeline; - this.modeBuffer = modeBuffer; - - const displayBG = this.device.createBindGroup({ - layout: displayPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: srcTex.createView() }, - { binding: 1, resource: this.inputTexture.createView() }, - { binding: 2, resource: { buffer: modeBuffer } } - ] - }); - this.displayBindGroup = displayBG; - - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: this.context.getCurrentTexture().createView(), - loadOp: 'clear', - storeOp: 'store' - }] - }); - renderPass.setPipeline(displayPipeline); - renderPass.setBindGroup(0, displayBG); - renderPass.draw(6); - renderPass.end(); - - this.device.queue.submit([encoder.finish()]); - - // Wait for GPU to finish before visualizing layers - await this.device.queue.onSubmittedWorkDone(); - - const t1 = performance.now(); - const mode = ['CNN Output', 'Original', 'Diff (×10)'][this.viewMode]; - this.setStatus(`GPU: ${(t1-t0).toFixed(1)}ms | ${width}×${height} | ${mode}`); - this.log(`Completed in ${(t1-t0).toFixed(1)}ms`); - - // Update layer visualization panel - this.updateLayerVizPanel(); - } - - updateLayerVizPanel() { - const panel = document.getElementById('layerViz'); - - if (!this.layerOutputs || this.layerOutputs.length === 0) { - panel.innerHTML = '<p style="color: #808080; text-align: center;">No layers to visualize</p>'; - return; - } - - // Only rebuild panel structure if layer count changed - const needsRebuild = !this.lastLayerCount || this.lastLayerCount !== this.layerOutputs.length; - - if (needsRebuild) { - let html = '<div class="layer-buttons">'; - html += `<button onclick="tester.visualizeLayer(0, 0)" id="layerBtn0_0">Static 0-3</button>`; - html += `<button onclick="tester.visualizeLayer(0, 4)" id="layerBtn0_4">Static 4-7</button>`; - - for (let i = 1; i < this.layerOutputs.length; i++) { - const label = `Layer ${i - 1}`; - html += `<button onclick="tester.visualizeLayer(${i})" id="layerBtn${i}">${label}</button>`; - } - html += `<button onclick="tester.saveCompositedLayer()" style="margin-left: 20px; background: #28a745;">Save Composited</button>`; - html += '</div>'; - - html += '<div class="layer-grid" id="layerGrid"></div>'; - html += '<div class="layer-preview"><div class="layer-view-label" id="previewLabel">Ch0</div><canvas id="previewCanvas"></canvas></div>'; - - panel.innerHTML = html; - this.log(`Layer visualization ready: ${this.layerOutputs.length} layers`); - this.recreateCanvases(); - this.lastLayerCount = this.layerOutputs.length; - } - - // Update current visualization - if (this.currentLayerIdx !== null) { - this.visualizeLayer(this.currentLayerIdx, this.currentChannelOffset || 0); - } else { - this.visualizeLayer(0, 0); - } - } - - recreateCanvases() { - const grid = document.getElementById('layerGrid'); - if (!grid) return; - - // Force removal of old canvases to clear any WebGPU contexts - const oldCanvases = grid.querySelectorAll('canvas'); - oldCanvases.forEach(canvas => { - canvas.width = 0; - canvas.height = 0; - }); - - grid.innerHTML = ''; - for (let c = 0; c < 4; c++) { - const div = document.createElement('div'); - div.className = 'layer-view'; - div.innerHTML = ` - <div class="layer-view-label" id="channelLabel${c}">Ch ${c}</div> - <canvas id="layerCanvas${c}"></canvas> - `; - div.onclick = () => this.selectChannel(c); - grid.appendChild(div); - } - this.selectedChannel = 0; - } - - async visualizeLayer(layerIdx, channelOffset = 0) { - if (!this.layerOutputs || layerIdx >= this.layerOutputs.length) { - this.log(`Cannot visualize layer ${layerIdx}: no data`, 'error'); - return; - } - - // Store current selection - this.currentLayerIdx = layerIdx; - this.currentChannelOffset = channelOffset; - - // Update button states - document.querySelectorAll('.layer-buttons button').forEach(btn => btn.classList.remove('active')); - if (layerIdx === 0) { - // Static features - const btnId = `layerBtn0_${channelOffset}`; - const btn = document.getElementById(btnId); - if (btn) btn.classList.add('active'); - } else { - const btn = document.getElementById(`layerBtn${layerIdx}`); - if (btn) btn.classList.add('active'); - } - - const layerName = layerIdx === 0 ? `Static Features (${channelOffset}-${channelOffset + 3})` : `Layer ${layerIdx - 1}`; - const layerTex = this.layerOutputs[layerIdx]; - const { width, height } = this.getDimensions(); - - // Update channel labels based on layer type - // Static features (layerIdx=0): 8 channels split into two views - // CNN layers (layerIdx≥1): 4 channels per layer - const staticLabels = [ - ['Ch0 (p0)', 'Ch1 (p1)', 'Ch2 (p2)', 'Ch3 (p3)'], - ['Ch4 (uv_x)', 'Ch5 (uv_y)', 'Ch6 (sin10_x)', 'Ch7 (bias)'] - ]; - const channelLabels = layerIdx === 0 - ? staticLabels[channelOffset / 4] - : ['Ch0', 'Ch1', 'Ch2', 'Ch3']; - - for (let c = 0; c < 4; c++) { - const label = document.getElementById(`channelLabel${c}`); - if (label) label.textContent = channelLabels[c]; - } - - // Create layer viz pipeline if needed - if (!this.layerVizPipeline) { - this.layerVizPipeline = this.device.createRenderPipeline({ - layout: 'auto', - vertex: { - module: this.device.createShaderModule({ code: LAYER_VIZ_SHADER }), - entryPoint: 'vs_main' - }, - fragment: { - module: this.device.createShaderModule({ code: LAYER_VIZ_SHADER }), - entryPoint: 'fs_main', - targets: [{ format: this.format }] - } - }); - this.log('Created layer visualization pipeline'); - } - - // Render each channel to its canvas - for (let c = 0; c < 4; c++) { - const canvas = document.getElementById(`layerCanvas${c}`); - if (!canvas) { - this.log(`Canvas layerCanvas${c} not found`, 'error'); - continue; - } - - // Set canvas size BEFORE getting context - canvas.width = width; - canvas.height = height; - - const ctx = canvas.getContext('webgpu'); - if (!ctx) { - this.log(`Failed to get WebGPU context for channel ${c}`, 'error'); - continue; - } - - try { - ctx.configure({ device: this.device, format: this.format }); - } catch (e) { - this.log(`Failed to configure canvas ${c}: ${e.message}`, 'error'); - continue; - } - - const vizScale = 1.0; // Always 1.0, shader clamps to [0,1] - const paramsBuffer = this.device.createBuffer({ - size: 8, - usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST - }); - // Use channel index with offset for static features - const actualChannel = channelOffset + c; - const paramsData = new Float32Array([actualChannel, vizScale]); - this.device.queue.writeBuffer(paramsBuffer, 0, paramsData); - - const bindGroup = this.device.createBindGroup({ - layout: this.layerVizPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: layerTex.createView() }, - { binding: 1, resource: { buffer: paramsBuffer } } - ] - }); - - const encoder = this.device.createCommandEncoder(); - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: ctx.getCurrentTexture().createView(), - loadOp: 'clear', - clearValue: { r: 1.0, g: 0.0, b: 1.0, a: 1.0 }, // Magenta clear for debugging - storeOp: 'store' - }] - }); - - renderPass.setPipeline(this.layerVizPipeline); - renderPass.setBindGroup(0, bindGroup); - renderPass.draw(6); - renderPass.end(); - - this.device.queue.submit([encoder.finish()]); - } - - // Wait for all renders to complete - await this.device.queue.onSubmittedWorkDone(); - - // Update active channel highlighting and preview - this.updateChannelSelection(); - await this.renderChannelPreview(); - } - - selectChannel(channelIdx) { - this.selectedChannel = channelIdx; - this.updateChannelSelection(); - this.renderChannelPreview(); - } - - updateChannelSelection() { - const grid = document.getElementById('layerGrid'); - if (!grid) return; - - const views = grid.querySelectorAll('.layer-view'); - views.forEach((view, idx) => { - view.classList.toggle('active', idx === this.selectedChannel); - }); - } - - async renderChannelPreview() { - const previewCanvas = document.getElementById('previewCanvas'); - const previewLabel = document.getElementById('previewLabel'); - if (!previewCanvas || !this.device) return; - - const { width, height } = this.getDimensions(); - previewCanvas.width = width; - previewCanvas.height = height; - - const ctx = previewCanvas.getContext('webgpu'); - if (!ctx) return; - - try { - ctx.configure({ device: this.device, format: this.format }); - } catch (e) { - return; - } - - // Update label - const channelLabel = document.getElementById(`channelLabel${this.selectedChannel}`); - if (channelLabel && previewLabel) { - previewLabel.textContent = channelLabel.textContent; - } - - // Render selected channel - const layerIdx = this.currentLayerIdx; - const channelOffset = this.currentChannelOffset; - const layerTex = this.layerOutputs[layerIdx]; - if (!layerTex) return; - - // Always 1.0, shader clamps to [0,1] - show exact layer values - const vizScale = 1.0; - const actualChannel = channelOffset + this.selectedChannel; - - const paramsBuffer = this.device.createBuffer({ - size: 8, - usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST - }); - const paramsData = new Float32Array([actualChannel, vizScale]); - this.device.queue.writeBuffer(paramsBuffer, 0, paramsData); - - const bindGroup = this.device.createBindGroup({ - layout: this.layerVizPipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: layerTex.createView() }, - { binding: 1, resource: { buffer: paramsBuffer } } - ] - }); - - const encoder = this.device.createCommandEncoder(); - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: ctx.getCurrentTexture().createView(), - loadOp: 'clear', - storeOp: 'store' - }] - }); - - renderPass.setPipeline(this.layerVizPipeline); - renderPass.setBindGroup(0, bindGroup); - renderPass.draw(6); - renderPass.end(); - - this.device.queue.submit([encoder.finish()]); - } - - visualizeWeights(cnnLayerIdx) { - const layer = this.weights.layers[cnnLayerIdx]; - if (!layer) { - this.log(`Layer ${cnnLayerIdx} not found`, 'error'); - return; - } - - // Update button states - document.querySelectorAll('#weightsLayerButtons button').forEach(btn => btn.classList.remove('active')); - const btn = document.getElementById(`weightsBtn${cnnLayerIdx}`); - if (btn) btn.classList.add('active'); - - const { kernelSize, inChannels, outChannels, weightOffset, min, max } = layer; - - const canvas = document.getElementById('weightsCanvas'); - const ctx = canvas.getContext('2d', { willReadFrequently: false }); - - // 1 pixel per weight, show all input channels horizontally - const width = inChannels * kernelSize; - const height = outChannels * kernelSize; - - canvas.width = width; - canvas.height = height; - - ctx.fillStyle = '#1a1a1a'; - ctx.fillRect(0, 0, width, height); - - // Stack output channels vertically - for (let outCh = 0; outCh < outChannels; outCh++) { - const yOffset = outCh * kernelSize; - - for (let inCh = 0; inCh < inChannels; inCh++) { - const xOffset = inCh * kernelSize; - - for (let ky = 0; ky < kernelSize; ky++) { - for (let kx = 0; kx < kernelSize; kx++) { - const spatialIdx = ky * kernelSize + kx; - const wIdx = weightOffset + - outCh * inChannels * kernelSize * kernelSize + - inCh * kernelSize * kernelSize + - spatialIdx; - - const weight = this.getWeightValue(wIdx); - const normalized = (weight - min) / (max - min); - const intensity = Math.floor(normalized * 255); - - ctx.fillStyle = `rgb(${intensity}, ${intensity}, ${intensity})`; - ctx.fillRect(xOffset + kx, yOffset + ky, 1, 1); - } - } - } - } - } - - getWeightValue(idx) { - const pairIdx = Math.floor(idx / 2); - const packed = this.weights.weights[pairIdx]; - const unpacked = this.unpackF16(packed); - return (idx % 2 === 0) ? unpacked[0] : unpacked[1]; - } - - toggleWeightsInfo() { - const panel = document.getElementById('weightsInfoPanel'); - const toggle = document.getElementById('weightsInfoToggle'); - panel.classList.toggle('collapsed'); - toggle.textContent = panel.classList.contains('collapsed') ? '▶' : '▼'; - } - - updateDisplay() { - if (!this.displayPipeline || !this.displayBindGroup) return; - - this.device.queue.writeBuffer(this.modeBuffer, 0, new Uint32Array([this.viewMode])); - - const encoder = this.device.createCommandEncoder(); - const renderPass = encoder.beginRenderPass({ - colorAttachments: [{ - view: this.context.getCurrentTexture().createView(), - loadOp: 'clear', - storeOp: 'store' - }] - }); - renderPass.setPipeline(this.displayPipeline); - renderPass.setBindGroup(0, this.displayBindGroup); - renderPass.draw(6); - renderPass.end(); - - this.device.queue.submit([encoder.finish()]); - } - - async savePNG() { - if (!this.image && !this.isVideo) { - this.log('No image loaded', 'error'); - return; - } - - if (!this.resultTexture) { - this.log('No result to save', 'error'); - return; - } - - try { - const { width, height } = this.getDimensions(); - - // GPU readback from result texture - const bytesPerRow = width * 16; // 4×u32 per pixel - const paddedBytesPerRow = Math.ceil(bytesPerRow / 256) * 256; - const bufferSize = paddedBytesPerRow * height; - - const stagingBuffer = this.device.createBuffer({ - size: bufferSize, - usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ - }); - - const encoder = this.device.createCommandEncoder(); - encoder.copyTextureToBuffer( - { texture: this.resultTexture }, - { buffer: stagingBuffer, bytesPerRow: paddedBytesPerRow, rowsPerImage: height }, - { width, height, depthOrArrayLayers: 1 } - ); - this.device.queue.submit([encoder.finish()]); - - await stagingBuffer.mapAsync(GPUMapMode.READ); - const mapped = new Uint8Array(stagingBuffer.getMappedRange()); - - // Unpack f16 to RGBA8 - const pixels = new Uint8Array(width * height * 4); - for (let y = 0; y < height; y++) { - const rowOffset = y * paddedBytesPerRow; - for (let x = 0; x < width; x++) { - const pixelOffset = rowOffset + x * 16; - const data = new Uint32Array(mapped.buffer, mapped.byteOffset + pixelOffset, 4); - - // Unpack f16 (first 4 channels only) - const unpack = (u32, idx) => { - const h = (idx === 0) ? (u32 & 0xFFFF) : ((u32 >> 16) & 0xFFFF); - const sign = (h >> 15) & 1; - const exp = (h >> 10) & 0x1F; - const frac = h & 0x3FF; - if (exp === 0) return 0; - if (exp === 31) return sign ? 0 : 255; - const e = exp - 15; - const val = (1 + frac / 1024) * Math.pow(2, e); - return Math.max(0, Math.min(255, Math.round(val * 255))); - }; - - const outIdx = (y * width + x) * 4; - pixels[outIdx + 0] = unpack(data[0], 0); // R - pixels[outIdx + 1] = unpack(data[0], 1); // G - pixels[outIdx + 2] = unpack(data[1], 0); // B - pixels[outIdx + 3] = 255; // A - } - } - - stagingBuffer.unmap(); - stagingBuffer.destroy(); - - // Create blob from pixels - const canvas = document.createElement('canvas'); - canvas.width = width; - canvas.height = height; - const ctx = canvas.getContext('2d'); - const imageData = new ImageData(new Uint8ClampedArray(pixels), width, height); - ctx.putImageData(imageData, 0, 0); - - const blob = await new Promise(resolve => canvas.toBlob(resolve, 'image/png')); - const url = URL.createObjectURL(blob); - const a = document.createElement('a'); - const mode = ['cnn', 'original', 'diff'][this.viewMode]; - a.href = url; - a.download = `output_${width}x${height}_${mode}.png`; - a.click(); - URL.revokeObjectURL(url); - - this.log(`Saved PNG: ${a.download}`); - this.setStatus(`Saved: ${a.download}`); - } catch (err) { - this.log(`Failed to save PNG: ${err.message}`, 'error'); - this.setStatus(`Save failed: ${err.message}`, true); - } - } - - async saveCompositedLayer() { - if (!this.currentLayerIdx) { - this.log('No layer selected for compositing', 'error'); - return; - } - - try { - const canvases = []; - for (let i = 0; i < 4; i++) { - const canvas = document.getElementById(`layerCanvas${i}`); - if (!canvas) { - this.log(`Canvas layerCanvas${i} not found`, 'error'); - return; - } - canvases.push(canvas); - } - - const width = canvases[0].width; - const height = canvases[0].height; - const compositedWidth = width * 4; - - // Create composited canvas - const compositedCanvas = document.createElement('canvas'); - compositedCanvas.width = compositedWidth; - compositedCanvas.height = height; - const ctx = compositedCanvas.getContext('2d'); - - // Composite horizontally - for (let i = 0; i < 4; i++) { - ctx.drawImage(canvases[i], i * width, 0); - } - - // Convert to grayscale - const imageData = ctx.getImageData(0, 0, compositedWidth, height); - const pixels = imageData.data; - for (let i = 0; i < pixels.length; i += 4) { - const gray = 0.299 * pixels[i] + 0.587 * pixels[i + 1] + 0.114 * pixels[i + 2]; - pixels[i] = pixels[i + 1] = pixels[i + 2] = gray; - } - ctx.putImageData(imageData, 0, 0); - - // Save as PNG - const blob = await new Promise(resolve => compositedCanvas.toBlob(resolve, 'image/png')); - const url = URL.createObjectURL(blob); - const a = document.createElement('a'); - a.href = url; - a.download = `composited_layer${this.currentLayerIdx - 1}_${compositedWidth}x${height}.png`; - a.click(); - URL.revokeObjectURL(url); - - this.log(`Saved composited layer: ${a.download}`); - this.setStatus(`Saved: ${a.download}`); - } catch (err) { - this.log(`Failed to save composited layer: ${err.message}`, 'error'); - this.setStatus(`Compositing failed: ${err.message}`, true); - } - } -} - -const tester = new CNNTester(); - -// Load default weights on startup -(async () => { - try { - const binaryString = atob(DEFAULT_WEIGHTS_B64); - const bytes = new Uint8Array(binaryString.length); - for (let i = 0; i < binaryString.length; i++) { - bytes[i] = binaryString.charCodeAt(i); - } - await tester.loadWeights({ name: 'default.bin', arrayBuffer: () => Promise.resolve(bytes.buffer) }); - tester.log('Loaded default weights'); - } catch (err) { - tester.log(`Failed to load default weights: ${err.message}`, 'error'); - } -})(); - -function setupDropZone(id, callback) { - const zone = document.getElementById(id); - ['dragenter', 'dragover', 'dragleave', 'drop'].forEach(e => { - zone.addEventListener(e, ev => { ev.preventDefault(); ev.stopPropagation(); }); - }); - ['dragenter', 'dragover'].forEach(e => zone.addEventListener(e, () => zone.classList.add('active'))); - ['dragleave', 'drop'].forEach(e => zone.addEventListener(e, () => zone.classList.remove('active'))); - zone.addEventListener('drop', e => { - const file = e.dataTransfer.files[0]; - if (file) callback(file).catch(err => { - zone.classList.add('error'); - tester.setStatus(err.message, true); - tester.log(err.message, 'error'); - setTimeout(() => zone.classList.remove('error'), 2000); - }); - }); -} - -// Whole window drop for PNG images and videos -const mainArea = document.getElementById('mainDrop'); -['dragenter', 'dragover', 'dragleave', 'drop'].forEach(e => { - mainArea.addEventListener(e, ev => { ev.preventDefault(); ev.stopPropagation(); }); -}); -['dragenter', 'dragover'].forEach(e => mainArea.addEventListener(e, () => mainArea.classList.add('drop-active'))); -['dragleave', 'drop'].forEach(e => mainArea.addEventListener(e, () => mainArea.classList.remove('drop-active'))); -mainArea.addEventListener('drop', e => { - const file = e.dataTransfer.files[0]; - if (file) { - if (file.type.startsWith('image/')) { - tester.loadImage(file).catch(err => { - tester.setStatus(err.message, true); - tester.log(err.message, 'error'); - }); - } else if (file.type.startsWith('video/')) { - tester.loadVideo(file).catch(err => { - tester.setStatus(err.message, true); - tester.log(err.message, 'error'); - }); - } - } -}); - -// Weights drop zone -setupDropZone('weightsDrop', f => tester.loadWeights(f)); - -// Weights file input -document.getElementById('weightsFile').addEventListener('change', e => { - const file = e.target.files[0]; - if (file) { - tester.loadWeights(file).catch(err => { - tester.setStatus(err.message, true); - tester.log(err.message, 'error'); - }); - } -}); - -document.getElementById('blend').addEventListener('input', e => { - tester.blendAmount = parseFloat(e.target.value); - document.getElementById('blendValue').textContent = e.target.value; - if ((tester.image || tester.isVideo) && tester.weights) { - tester.log(`Blend changed to ${e.target.value}`); - tester.run(); - } -}); - -document.getElementById('depth').addEventListener('input', e => { - tester.depth = parseFloat(e.target.value); - document.getElementById('depthValue').textContent = e.target.value; - if ((tester.image || tester.isVideo) && tester.weights) tester.run(); -}); - -document.getElementById('mipLevel').addEventListener('change', e => { - tester.mipLevel = parseInt(e.target.value); - tester.log(`Mip level changed to ${e.target.value}`); - if ((tester.image || tester.isVideo) && tester.weights) tester.run(); -}); - -document.getElementById('playPauseBtn').addEventListener('click', () => tester.togglePlayPause()); -document.getElementById('stepBackBtn').addEventListener('click', () => tester.stepFrame(-1)); -document.getElementById('stepForwardBtn').addEventListener('click', () => tester.stepFrame(1)); -document.getElementById('savePngBtn').addEventListener('click', () => tester.savePNG()); - -document.addEventListener('keydown', e => { - if (e.code === 'Space') { - e.preventDefault(); - if (tester.viewMode === 1) { - tester.viewMode = 0; - } else { - tester.viewMode = 1; - } - const modeName = ['CNN Output', 'Original', 'Diff (×10)'][tester.viewMode]; - if ((tester.image || tester.isVideo) && tester.weights) { - tester.log(`View mode: ${modeName}`); - tester.updateDisplay(); - const width = tester.isVideo ? tester.video.videoWidth : tester.image.width; - const height = tester.isVideo ? tester.video.videoHeight : tester.image.height; - tester.setStatus(`${width}×${height} | ${modeName}`); - } - } else if (e.code === 'KeyD') { - e.preventDefault(); - if (tester.viewMode === 2) { - tester.viewMode = 0; - } else { - tester.viewMode = 2; - } - const modeName = ['CNN Output', 'Original', 'Diff (×10)'][tester.viewMode]; - if ((tester.image || tester.isVideo) && tester.weights) { - tester.log(`View mode: ${modeName}`); - tester.updateDisplay(); - const width = tester.isVideo ? tester.video.videoWidth : tester.image.width; - const height = tester.isVideo ? tester.video.videoHeight : tester.image.height; - tester.setStatus(`${width}×${height} | ${modeName}`); - } - } -}); - </script> -</body> -</html> |