1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
|
# CNN v2 Debugging Tools
Tools for investigating CNN v2 mismatch between HTML tool and cnn_test.
---
## Identity Weight Generator
**Purpose:** Generate trivial .bin files with identity passthrough for debugging.
**Script:** `training/gen_identity_weights.py`
**Usage:**
```bash
# 1×1 identity (default)
./training/gen_identity_weights.py workspaces/main/weights/cnn_v2_identity.bin
# 3×3 identity
./training/gen_identity_weights.py workspaces/main/weights/cnn_v2_identity_3x3.bin --kernel-size 3
# Custom mip level
./training/gen_identity_weights.py output.bin --kernel-size 1 --mip-level 2
```
**Output:**
- Single layer, 12D→4D (4 input channels + 8 static features)
- Identity matrix: Output Ch{0,1,2,3} = Input Ch{0,1,2,3}
- Static features (Ch 4-11) are zeroed
- Minimal file size (~136 bytes for 1×1, ~904 bytes for 3×3)
**Validation:**
Load in HTML tool or cnn_test - output should match input (RGB only, ignoring static features).
---
## Composited Layer Visualization
**Purpose:** Save current layer view as single composited image (4 channels side-by-side, grayscale).
**Location:** HTML tool - "Layer Visualization" panel
**Usage:**
1. Load image + weights in HTML tool
2. Select layer to visualize (Static 0-3, Static 4-7, Layer 0, Layer 1, etc.)
3. Click "Save Composited" button
4. Downloads PNG: `composited_layer{N}_{W}x{H}.png`
**Output:**
- 4 channels stacked horizontally
- Grayscale representation
- Useful for comparing layer activations across tools
---
## Debugging Strategy
### Track a) Binary Conversion Chain
**Hypothesis:** Conversion error in .bin ↔ base64 ↔ Float32Array
**Test:**
1. Generate identity weights:
```bash
./training/gen_identity_weights.py workspaces/main/weights/test_identity.bin
```
2. Load in HTML tool - output should match input RGB
3. If mismatch:
- Check Python export: f16 packing in `export_cnn_v2_weights.py` line 105
- Check HTML parsing: `unpackF16()` in `index.html` line 805-815
- Check weight indexing: `get_weight()` shader function
**Key locations:**
- Python: `np.float16` → `view(np.uint32)` (line 105 of export script)
- JS: `DataView` → `unpackF16()` → manual f16 decode (line 773-803)
- WGSL: `unpack2x16float()` built-in (line 492 of shader)
### Track b) Layer Visualization
**Purpose:** Confirm layer outputs match between HTML and C++
**Method:**
1. Run identical input through both tools
2. Save composited layers from HTML tool
3. Compare with cnn_test output
4. Use identity weights to isolate weight loading from computation
### Track c) Trivial Test Case
**Use identity weights to test:**
- Weight loading (binary parsing)
- Feature generation (static features)
- Convolution (should be passthrough)
- Output packing
**Expected behavior:**
- Input RGB → Output RGB (exact match)
- Static features ignored (all zeros in identity matrix)
---
## Known Issues
**Current mismatch:** HTML tool and cnn_test produce different outputs for same input/weights.
**Suspects:**
1. F16 unpacking difference (CPU vs GPU vs JS)
2. Static feature generation (RGBD, UV, sin encoding)
3. Convolution kernel iteration order
4. Output packing/unpacking
**Next steps:**
1. Test with identity weights (eliminates weight loading)
2. Compare composited layer outputs
3. Add debug visualization for static features
4. Hex dump comparison (first 8 pixels) - use `--debug-hex` flag in cnn_test
---
## Related Documentation
- `doc/CNN_V2.md` - CNN v2 architecture
- `doc/CNN_V2_WEB_TOOL.md` - HTML tool documentation
- `doc/CNN_TEST_TOOL.md` - cnn_test CLI tool
- `training/export_cnn_v2_weights.py` - Binary export format
|
