summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-10 21:11:05 +0100
committerskal <pascal.massimino@gmail.com>2026-02-10 21:11:05 +0100
commit7a05f4d33b611ba1e9b6c68e0d0bd67d6ea011ee (patch)
treea88109bee56197ffca8d7aacd07a878fae502d11 /doc
parent2fbfc406abe5a42f45face9b07a91ec64c0d4f78 (diff)
refactor: Optimize CNN grayscale computation
Compute gray once per fragment using dot() instead of per-layer. Pass gray as f32 parameter to conv functions instead of vec4 original. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc')
-rw-r--r--doc/CNN_EFFECT.md23
-rw-r--r--doc/CNN_RGBD_GRAYSCALE_SUMMARY.md10
2 files changed, 20 insertions, 13 deletions
diff --git a/doc/CNN_EFFECT.md b/doc/CNN_EFFECT.md
index 4659fd3..22cf985 100644
--- a/doc/CNN_EFFECT.md
+++ b/doc/CNN_EFFECT.md
@@ -38,7 +38,7 @@ fn cnn_conv3x3_7to4(
samp: sampler,
uv: vec2<f32>,
resolution: vec2<f32>,
- original: vec4<f32>, # Original RGBD [-1,1]
+ gray: f32, # Grayscale [-1,1]
weights: array<array<f32, 8>, 36> # 9 pos × 4 out × (7 weights + bias)
) -> vec4<f32>
@@ -48,7 +48,7 @@ fn cnn_conv3x3_7to1(
samp: sampler,
uv: vec2<f32>,
resolution: vec2<f32>,
- original: vec4<f32>,
+ gray: f32,
weights: array<array<f32, 8>, 9> # 9 pos × (7 weights + bias)
) -> f32
```
@@ -56,7 +56,7 @@ fn cnn_conv3x3_7to1(
**Input normalization:**
- **fs_main** normalizes textures once: `(tex - 0.5) * 2` → [-1,1]
- **Conv functions** normalize UV coords: `(uv - 0.5) * 2` → [-1,1]
-- **Grayscale** computed from normalized RGBD: `0.2126*R + 0.7152*G + 0.0722*B`
+- **Grayscale** computed once in fs_main using dot product: `dot(original.rgb, vec3(0.2126, 0.7152, 0.0722))`
- **Inter-layer data** stays in [-1,1] (no denormalization)
- **Final output** denormalized for display: `(result + 1.0) * 0.5` → [0,1]
@@ -250,20 +250,25 @@ Expands to:
```wgsl
@fragment fn fs_main(@builtin(position) p: vec4<f32>) -> @location(0) vec4<f32> {
let uv = p.xy / uniforms.resolution;
- let input = textureSample(txt, smplr, uv); // Layer N-1 output
- let original = textureSample(original_input, smplr, uv); // Layer 0 input
-
+ let original_raw = textureSample(original_input, smplr, uv);
+ let original = (original_raw - 0.5) * 2.0; // Normalize to [-1,1]
+ let gray = dot(original.rgb, vec3<f32>(0.2126, 0.7152, 0.0722));
var result = vec4<f32>(0.0);
if (params.layer_index == 0) {
- result = cnn_conv3x3_with_coord(txt, smplr, uv, uniforms.resolution,
- rgba_weights_layer0, coord_weights_layer0, bias_layer0);
+ result = cnn_conv3x3_7to4_src(txt, smplr, uv, uniforms.resolution,
+ weights_layer0);
+ result = cnn_tanh(result);
+ }
+ else if (params.layer_index == 1) {
+ result = cnn_conv5x5_7to4(txt, smplr, uv, uniforms.resolution,
+ gray, weights_layer1);
result = cnn_tanh(result);
}
// ... other layers
// Blend with ORIGINAL input (not previous layer)
- return mix(original, result, params.blend_amount);
+ return mix(original_raw, result, params.blend_amount);
}
```
diff --git a/doc/CNN_RGBD_GRAYSCALE_SUMMARY.md b/doc/CNN_RGBD_GRAYSCALE_SUMMARY.md
index 4c13693..3439f2c 100644
--- a/doc/CNN_RGBD_GRAYSCALE_SUMMARY.md
+++ b/doc/CNN_RGBD_GRAYSCALE_SUMMARY.md
@@ -20,7 +20,7 @@ Implemented CNN architecture upgrade: RGBD input → grayscale output with 7-cha
- **RGBD:** `(rgbd - 0.5) * 2`
- **UV coords:** `(uv - 0.5) * 2`
-- **Grayscale:** `(0.2126*R + 0.7152*G + 0.0722*B - 0.5) * 2`
+- **Grayscale:** `dot(original.rgb, vec3<f32>(0.2126, 0.7152, 0.0722))` (computed once, passed as parameter)
**Rationale:** Zero-centered inputs for tanh activation, better gradient flow.
@@ -48,13 +48,14 @@ Implemented CNN architecture upgrade: RGBD input → grayscale output with 7-cha
**Shaders (`/Users/skal/demo/workspaces/main/shaders/cnn/cnn_conv3x3.wgsl`):**
1. Added `cnn_conv3x3_7to4()`:
- - 7-channel input: [RGBD, uv_x, uv_y, gray]
+ - 7-channel input: [RGBD, uv_x, uv_y, gray] (gray passed as parameter)
- 4-channel output: RGBD
- Weights: `array<array<f32, 8>, 36>`
2. Added `cnn_conv3x3_7to1()`:
- - 7-channel input: [RGBD, uv_x, uv_y, gray]
+ - 7-channel input: [RGBD, uv_x, uv_y, gray] (gray passed as parameter)
- 1-channel output: grayscale
- Weights: `array<array<f32, 8>, 9>`
+3. Optimized: gray computed once in caller using `dot()`, not per-function
**Documentation (`/Users/skal/demo/doc/CNN_EFFECT.md`):**
1. Updated architecture section with RGBD→grayscale pipeline
@@ -71,7 +72,8 @@ CNNLayerParams and bind groups remain unchanged.
2. Each layer:
- Samples previous layer output (RGBD in [0,1])
- Normalizes RGBD to [-1,1]
- - Computes UV coords and grayscale, normalizes to [-1,1]
+ - Computes gray once using `dot()` (fs_main level)
+ - Normalizes UV coords to [-1,1] (inside conv functions)
- Concatenates 7-channel input
- Applies convolution with layer-specific weights
- Outputs RGBD (inner) or grayscale (final) in [-1,1]