diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-14 02:12:12 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-14 02:12:12 +0100 |
| commit | 043044ae7563c2f92760c428765e35b411da82ea (patch) | |
| tree | 0d640fec1517169d195747707b6c589c92fe7161 /workspaces/main | |
| parent | 4d119a1b6a6f460ca6d5a8ef85176c45663fd40a (diff) | |
Replace hard clamp with sigmoid activation in CNN v2
Fixes training collapse where p1/p2 channels saturate due to gradient
blocking at clamp boundaries. Sigmoid provides smooth [0,1] mapping
with continuous gradients.
Changes:
- Layer 0: clamp(x, 0, 1) → sigmoid(x)
- Final layer: clamp(x, 0, 1) → sigmoid(x)
- Middle layers: ReLU unchanged (already stable)
Updated files:
- training/train_cnn_v2.py: PyTorch model activations
- workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl: WGSL shader
- tools/cnn_v2_test/index.html: HTML validation tool
- doc/CNN_V2.md: Documentation
Validation:
- Build clean (no shader errors)
- 34/36 tests pass (2 unrelated script tests fail)
- 10-epoch training: loss 0.153 → 0.088 (good convergence)
- cnn_test processes images successfully
Breaking change: Old checkpoints trained with clamp() incompatible.
Retrain from scratch required.
handoff(Claude): CNN v2 sigmoid activation implemented and validated.
Diffstat (limited to 'workspaces/main')
| -rw-r--r-- | workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl | 8 |
1 files changed, 3 insertions, 5 deletions
diff --git a/workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl b/workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl index 4644003..cdbfd74 100644 --- a/workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl +++ b/workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl @@ -122,12 +122,10 @@ fn main(@builtin(global_invocation_id) id: vec3<u32>) { } // Activation (matches train_cnn_v2.py) - if (is_output) { - output[c] = clamp(sum, 0.0, 1.0); // Output layer: clamp [0,1] - } else if (params.is_layer_0 != 0u) { - output[c] = clamp(sum, 0.0, 1.0); // Layer 0: clamp [0,1] + if (is_output || params.is_layer_0 != 0u) { + output[c] = 1.0 / (1.0 + exp(-sum)); // Sigmoid [0,1] } else { - output[c] = max(0.0, sum); // Middle layers: ReLU + output[c] = max(0.0, sum); // ReLU } } |
