summaryrefslogtreecommitdiff
path: root/doc/CNN_EFFECT.md
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-10 21:11:05 +0100
committerskal <pascal.massimino@gmail.com>2026-02-10 21:11:05 +0100
commit7a05f4d33b611ba1e9b6c68e0d0bd67d6ea011ee (patch)
treea88109bee56197ffca8d7aacd07a878fae502d11 /doc/CNN_EFFECT.md
parent2fbfc406abe5a42f45face9b07a91ec64c0d4f78 (diff)
refactor: Optimize CNN grayscale computation
Compute gray once per fragment using dot() instead of per-layer. Pass gray as f32 parameter to conv functions instead of vec4 original. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc/CNN_EFFECT.md')
-rw-r--r--doc/CNN_EFFECT.md23
1 files changed, 14 insertions, 9 deletions
diff --git a/doc/CNN_EFFECT.md b/doc/CNN_EFFECT.md
index 4659fd3..22cf985 100644
--- a/doc/CNN_EFFECT.md
+++ b/doc/CNN_EFFECT.md
@@ -38,7 +38,7 @@ fn cnn_conv3x3_7to4(
samp: sampler,
uv: vec2<f32>,
resolution: vec2<f32>,
- original: vec4<f32>, # Original RGBD [-1,1]
+ gray: f32, # Grayscale [-1,1]
weights: array<array<f32, 8>, 36> # 9 pos × 4 out × (7 weights + bias)
) -> vec4<f32>
@@ -48,7 +48,7 @@ fn cnn_conv3x3_7to1(
samp: sampler,
uv: vec2<f32>,
resolution: vec2<f32>,
- original: vec4<f32>,
+ gray: f32,
weights: array<array<f32, 8>, 9> # 9 pos × (7 weights + bias)
) -> f32
```
@@ -56,7 +56,7 @@ fn cnn_conv3x3_7to1(
**Input normalization:**
- **fs_main** normalizes textures once: `(tex - 0.5) * 2` → [-1,1]
- **Conv functions** normalize UV coords: `(uv - 0.5) * 2` → [-1,1]
-- **Grayscale** computed from normalized RGBD: `0.2126*R + 0.7152*G + 0.0722*B`
+- **Grayscale** computed once in fs_main using dot product: `dot(original.rgb, vec3(0.2126, 0.7152, 0.0722))`
- **Inter-layer data** stays in [-1,1] (no denormalization)
- **Final output** denormalized for display: `(result + 1.0) * 0.5` → [0,1]
@@ -250,20 +250,25 @@ Expands to:
```wgsl
@fragment fn fs_main(@builtin(position) p: vec4<f32>) -> @location(0) vec4<f32> {
let uv = p.xy / uniforms.resolution;
- let input = textureSample(txt, smplr, uv); // Layer N-1 output
- let original = textureSample(original_input, smplr, uv); // Layer 0 input
-
+ let original_raw = textureSample(original_input, smplr, uv);
+ let original = (original_raw - 0.5) * 2.0; // Normalize to [-1,1]
+ let gray = dot(original.rgb, vec3<f32>(0.2126, 0.7152, 0.0722));
var result = vec4<f32>(0.0);
if (params.layer_index == 0) {
- result = cnn_conv3x3_with_coord(txt, smplr, uv, uniforms.resolution,
- rgba_weights_layer0, coord_weights_layer0, bias_layer0);
+ result = cnn_conv3x3_7to4_src(txt, smplr, uv, uniforms.resolution,
+ weights_layer0);
+ result = cnn_tanh(result);
+ }
+ else if (params.layer_index == 1) {
+ result = cnn_conv5x5_7to4(txt, smplr, uv, uniforms.resolution,
+ gray, weights_layer1);
result = cnn_tanh(result);
}
// ... other layers
// Blend with ORIGINAL input (not previous layer)
- return mix(original, result, params.blend_amount);
+ return mix(original_raw, result, params.blend_amount);
}
```