path: root/doc/CNN_V2_WEB_TOOL.md

# CNN v2 Web Testing Tool

Browser-based WebGPU tool for validating CNN v2 inference with layer visualization and weight inspection.

**Location:** `tools/cnn_v2_test/index.html`

---

## Status (2026-02-13)

**Working:**
- ✅ WebGPU initialization and device setup
- ✅ Binary weight file parsing (.bin format)
- ✅ Weight statistics (min/max per layer)
- ✅ UI layout with collapsible panels
- ✅ Mode switching (Activations/Weights tabs)
- ✅ Canvas context management (2D for weights, WebGPU for activations)
- ✅ Weight visualization infrastructure (layer selection, grid layout)

**Not Working:**
- ❌ Layer activation visualization (all black)
- ❌ Weight kernel display (canvases empty, but logging shows execution)

**Partially Working:**
- ⚠️ Texture readback pipeline (UV gradient test works, data reads fail)

---

## Architecture

### File Structure
- Single-file HTML tool (~1100 lines)
- Embedded shaders: STATIC_SHADER, CNN_SHADER, DISPLAY_SHADER, LAYER_VIZ_SHADER
- Pure WebGPU (no external dependencies)

### Key Components

**1. Weight Parsing**
- Reads binary format: header (16B) + layer info (20B×N) + f16 weights
- Computes min/max per layer via f16 unpacking
- Stores `{ layers[], weights[], fileSize }`

**2. CNN Pipeline**
- Static features computation (RGBD + UV + sin + bias → 7D packed)
- Layer-by-layer convolution with storage buffer weights
- Ping-pong buffers for intermediate results
- Copy to persistent textures for visualization

**3. Visualization Modes**

**Activations Mode:**
- 4 grayscale views per layer (channels 0-3)
- WebGPU compute → unpack f16 → scale → grayscale
- Auto-scale: Layer 0 (static) = 1.0, CNN layers = 0.2

**Weights Mode:**
- 2D canvas rendering per output channel
- Shows all input kernels horizontally
- Normalized by layer min/max → [0, 1] → grayscale
- 20px cells, 2px padding between kernels

### Texture Management

**Persistent Storage (layerTextures[]):**
- One texture per layer output (static + all CNN layers)
- `rgba32uint` format (packed f16 data)
- `COPY_DST` usage for storing results

**Compute Buffers (computeTextures[]):**
- 2 textures for ping-pong computation
- Reused across all layers
- `COPY_SRC` usage for copying to persistent storage

**Pipeline:**
```
Static pass → copy to layerTextures[0]
For each CNN layer i:
  Compute (ping-pong) → copy to layerTextures[i+1]
```

---

## Known Issues

### Issue #1: Layer Activations Show Black

**Symptom:**
- All 4 channel canvases render black
- UV gradient test (debug mode 10) works
- Raw packed data test (mode 11) shows black
- Unpacked f16 test (mode 12) shows black

**Diagnosis:**
- Texture access works (UV gradient visible)
- Texture data is all zeros (packed.x = 0)
- Textures being read are empty

**Root Cause:**
- `copyTextureToTexture` operations may not be executing
- Possible ordering issue (copies not submitted before visualization)
- Alternative: textures created with wrong usage flags

**Investigation Steps Taken:**
1. Added `onSubmittedWorkDone()` wait before visualization
2. Verified texture creation with `COPY_SRC` and `COPY_DST` flags
3. Confirmed separate texture allocation per layer (no aliasing)
4. Added debug shader modes to isolate issue

**Next Steps:**
- Verify encoder contains copy commands (add debug logging)
- Check if compute passes actually write data (add known-value test)
- Test copyTextureToTexture in isolation
- Consider CPU readback to verify texture contents

### Issue #2: Weight Visualization Empty

**Symptom:**
- Canvases created with correct dimensions (logged)
- No visual output (black canvases)
- Console logs show method execution

**Potential Causes:**
1. Weight indexing calculation incorrect
2. Canvas not properly attached to DOM when rendering
3. 2D context operations not flushing
4. Min/max normalization producing black (all values equal?)

**Debug Added:**
- Comprehensive logging of dimensions, indices, ranges
- Canvas context check before rendering

**Next Steps:**
- Add test rendering (fixed gradient) to verify 2D context works
- Log sample weight values to verify data access
- Check if canvas is visible in DOM inspector
- Verify min/max calculation produces valid range

---

## UI Layout

### Header
- Controls: Blend slider, Depth input, View mode display
- Drop zone for .bin weight files

### Content Area
**Left:** Main canvas (CNN output display)
**Right:** Sidebar with panels:

1. **Layer Visualization Panel** (top, flex: 1)
   - Mode tabs: Activations / Weights
   - Layer selection buttons (Static L0, Layer 1, Layer 2, ...)
   - 2×2 grid of channel views

2. **Weights Info Panel** (bottom, collapsible)
   - File size, layer count
   - Per-layer table: size, weight count, min/max
   - Click header to collapse/expand

### Footer
- Status line (GPU timing, dimensions, mode)
- Console log (scrollable, color-coded)

---

## Shader Details

### LAYER_VIZ_SHADER

**Purpose:** Display single channel from packed layer texture

**Inputs:**
- `@binding(0) layer_tex: texture_2d<u32>` - Packed f16 layer data
- `@binding(1) viz_params: vec2<f32>` - (channel_idx, scale)

**Debug Modes:**
- Channel 10: UV gradient (texture coordinate test)
- Channel 11: Raw packed u32 data
- Channel 12: First unpacked f16 value

**Normal Operation:**
- Unpack all 8 f16 channels from rgba32uint
- Select channel by index (0-7)
- Apply scale factor (1.0 for static, 0.2 for CNN)
- Clamp to [0, 1] and output grayscale

**Scale Rationale:**
- Static features (RGBD, UV): already in [0, 1] range
- CNN activations: post-ReLU [0, ~5], need scaling for visibility

---

## Binary Weight Format

**Header (16 bytes):**
```
u32 magic;        // 0x32_4E_4E_43 ("CNN2")
u32 version;      // Format version
u32 num_layers;   // Layer count
u32 total_weights;// Total f16 weight count
```

**Layer Info (20 bytes × N):**
```
u32 kernel_size;   // 3, 5, 7, etc.
u32 in_channels;   // Input channel count
u32 out_channels;  // Output channel count
u32 weight_offset; // Offset in f16 units
u32 weight_count;  // Number of f16 weights
```

**Weights (variable):**
- Packed f16 pairs as u32 (lo 16 bits, hi 16 bits)
- Sequential storage: [layer0_weights][layer1_weights]...

---

## Testing Workflow

### Load & Parse
1. Drop PNG image → displays original
2. Drop .bin weights → parses and shows info table
3. Auto-runs CNN pipeline

### Verify Pipeline
1. Check console for "Running CNN pipeline"
2. Verify "Completed in Xms"
3. Check "Layer visualization ready: N layers"

### Debug Activations
1. Select "Activations" tab
2. Click layer buttons to switch
3. Check console for texture/canvas logs
4. If black: note which debug modes work (UV vs data)

### Debug Weights
1. Select "Weights" tab
2. Click Layer 1 or Layer 2 (Layer 0 has no weights)
3. Check console for "Visualizing Layer N weights"
4. Check canvas dimensions logged
5. Verify weight range is non-trivial (not [0, 0])

---

## Integration with Main Project

**Training Pipeline:**
```bash
# Generate weights
./training/train_cnn_v2.py --export-binary

# Test in browser
open tools/cnn_v2_test/index.html
# Drop: workspaces/main/cnn_v2_weights.bin
# Drop: training/input/test.png
```

**Validation:**
- Compare against demo CNNv2Effect (visual check)
- Verify layer count matches binary file
- Check weight ranges match training logs

---

## Future Enhancements

- [ ] Fix layer activation visualization (black texture issue)
- [ ] Fix weight kernel display (empty canvas issue)
- [ ] Add per-channel auto-scaling (compute min/max from visible data)
- [ ] Export rendered outputs (download PNG)
- [ ] Side-by-side comparison with original
- [ ] Heatmap mode (color-coded activations)
- [ ] Weight statistics overlay (mean, std, sparsity)
- [ ] Batch processing (multiple images in sequence)
- [ ] Integration with Python training (live reload)

---

## Code Metrics

- Total lines: ~1100
- JavaScript: ~700 lines
- WGSL shaders: ~300 lines
- HTML/CSS: ~100 lines

**Dependencies:** None (pure WebGPU + HTML5)

---

## Related Files

- `doc/CNN_V2.md` - CNN v2 architecture and design
- `doc/CNN_TEST_TOOL.md` - C++ offline testing tool (deprecated)
- `training/train_cnn_v2.py` - Training script with binary export
- `workspaces/main/cnn_v2_weights.bin` - Trained weights