1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
|
# CNN v2 Debugging Tools
Tools for investigating CNN v2 mismatch between HTML tool and cnn_test.
---
## Identity Weight Generator
**Purpose:** Generate trivial .bin files with identity passthrough for debugging.
**Script:** `training/gen_identity_weights.py`
**Usage:**
```bash
# 1×1 identity (default)
./training/gen_identity_weights.py workspaces/main/weights/cnn_v2_identity.bin
# 3×3 identity
./training/gen_identity_weights.py workspaces/main/weights/cnn_v2_identity_3x3.bin --kernel-size 3
# Mix mode: p0+p4, p1+p5, p2+p6, p3+p7
./training/gen_identity_weights.py output.bin --mix
# Custom mip level
./training/gen_identity_weights.py output.bin --kernel-size 1 --mip-level 2
```
**Output:**
- Single layer, 12D→4D (4 input channels + 8 static features)
- Identity mode: Output Ch{0,1,2,3} = Input Ch{0,1,2,3}
- Mix mode (--mix): Output Ch{i} = Input Ch{i} + Input Ch{i+4} (blends input with static features)
- Minimal file size (~136 bytes for 1×1, ~904 bytes for 3×3)
**Validation:**
Load in HTML tool or cnn_test - output should match input (RGB only, ignoring static features).
---
## Composited Layer Visualization
**Purpose:** Save current layer view as single composited image (4 channels side-by-side, grayscale).
**Location:** HTML tool - "Layer Visualization" panel
**Usage:**
1. Load image + weights in HTML tool
2. Select layer to visualize (Static 0-3, Static 4-7, Layer 0, Layer 1, etc.)
3. Click "Save Composited" button
4. Downloads PNG: `composited_layer{N}_{W}x{H}.png`
**Output:**
- 4 channels stacked horizontally
- Grayscale representation
- Useful for comparing layer activations across tools
---
## Debugging Strategy
### Track a) Binary Conversion Chain
**Hypothesis:** Conversion error in .bin ↔ base64 ↔ Float32Array
**Test:**
1. Generate identity weights:
```bash
./training/gen_identity_weights.py workspaces/main/weights/test_identity.bin
```
2. Load in HTML tool - output should match input RGB
3. If mismatch:
- Check Python export: f16 packing in `export_cnn_v2_weights.py` line 105
- Check HTML parsing: `unpackF16()` in `index.html` line 805-815
- Check weight indexing: `get_weight()` shader function
**Key locations:**
- Python: `np.float16` → `view(np.uint32)` (line 105 of export script)
- JS: `DataView` → `unpackF16()` → manual f16 decode (line 773-803)
- WGSL: `unpack2x16float()` built-in (line 492 of shader)
### Track b) Layer Visualization
**Purpose:** Confirm layer outputs match between HTML and C++
**Method:**
1. Run identical input through both tools
2. Save composited layers from HTML tool
3. Compare with cnn_test output
4. Use identity weights to isolate weight loading from computation
### Track c) Trivial Test Case
**Use identity weights to test:**
- Weight loading (binary parsing)
- Feature generation (static features)
- Convolution (should be passthrough)
- Output packing
**Expected behavior:**
- Input RGB → Output RGB (exact match)
- Static features ignored (all zeros in identity matrix)
---
## Known Issues
### ~~Layer 0 Visualization Scale~~ [FIXED]
**Issue:** Layer 0 output displayed at 0.5× brightness (divided by 2).
**Cause:** Line 1530 used `vizScale = 0.5` for all CNN layers, but Layer 0 is clamped [0,1] and doesn't need dimming.
**Fix:** Use scale 1.0 for Layer 0 output (layerIdx=1), 0.5 only for middle layers (ReLU, unbounded).
### Remaining Mismatch
**Current:** HTML tool and cnn_test produce different outputs for same input/weights.
**Suspects:**
1. F16 unpacking difference (CPU vs GPU vs JS)
2. Static feature generation (RGBD, UV, sin encoding)
3. Convolution kernel iteration order
4. Output packing/unpacking
**Next steps:**
1. Test with identity weights (eliminates weight loading)
2. Compare composited layer outputs
3. Add debug visualization for static features
4. Hex dump comparison (first 8 pixels) - use `--debug-hex` flag in cnn_test
---
## Related Documentation
- `doc/CNN_V2.md` - CNN v2 architecture
- `doc/CNN_V2_WEB_TOOL.md` - HTML tool documentation
- `doc/CNN_TEST_TOOL.md` - cnn_test CLI tool
- `training/export_cnn_v2_weights.py` - Binary export format
|
