From 1470dd240f48652d1fe97957fe44a49b0e1ee9a6 Mon Sep 17 00:00:00 2001 From: skal Date: Mon, 23 Mar 2026 00:26:52 +0100 Subject: wip(cnn_v3): shadow→dif intermediate + scene tweaks + migration plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - gbuf_shadow.wgsl: normal bias 0.05→0.02 - gbuf_pack.wgsl: compute dif=diffuse*shadow, drop shadow from t1.z, store dif in t1.w (INTERMEDIATE — incorrect packing, see migration plan) - gbuf_deferred.wgsl: read dif from t1.w.x (matches intermediate packing) - gbuf_view.wgsl: expand to 4×6 grid, show dif.r/g/b in row 5 (INTERMEDIATE — to be reverted to 4×5 with ch18=dif) - gbuffer_effect.cc: add small hovering sphere (r=0.6) above scene; swap cube/sphere positions; both spheres pulsate - docs/GBUF_DIF_MIGRATION.md: full migration plan with checklist handoff(Claude): intermediate commit — GBUF_DIF_MIGRATION.md §Current State describes what is wrong and the full implementation checklist (5 steps). --- cnn_v3/docs/GBUF_DIF_MIGRATION.md | 141 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 cnn_v3/docs/GBUF_DIF_MIGRATION.md (limited to 'cnn_v3/docs') diff --git a/cnn_v3/docs/GBUF_DIF_MIGRATION.md b/cnn_v3/docs/GBUF_DIF_MIGRATION.md new file mode 100644 index 0000000..f1a4551 --- /dev/null +++ b/cnn_v3/docs/GBUF_DIF_MIGRATION.md @@ -0,0 +1,141 @@ +// cnn_v3/docs/GBUF_DIF_MIGRATION.md +// Plan: replace G-buffer shadow channel with dif (diffuse × shadow) +// Status: IN PROGRESS — current commit is intermediate state, see §Current State + +# G-Buffer `shadow` → `dif` Migration Plan + +## Motivation + +The raw `shadow` channel (ch18) is less informative than `dif = max(0, dot(normal, light_dir)) * shadow` +because `shadow` alone ignores the diffuse Lambert term. The CNN learns better when it receives +the pre-multiplied occluded diffuse signal directly. `albedo` is already in ch0–2, so the CNN +can reconstruct the full shaded color as `albedo * (ambient + dif)`. + +## Design Decision + +**Replace ch18 (`shadow`) with ch18 (`dif`) in-place. Channel count stays 20.** + +- `dif` is a scalar: `max(0, dot(normal, KEY_LIGHT)) * shadow` +- KEY_LIGHT = normalize(1, 2, 1) = (0.408, 0.816, 0.408) — matches `gbuf_deferred.wgsl` +- Stored at the same position (t1.z byte 2) → no weight shape change +- `transp` stays at ch19 (t1.z byte 3) +- t1.w reverts to 0 (spare) + +### Feature layout (20 channels, unchanged count) + +| ch | name | type | range | source | +|----|----------|--------|----------|----------------| +| 0 | alb.r | f16 | [0,1] | feat_tex0.x lo | +| 1 | alb.g | f16 | [0,1] | feat_tex0.x hi | +| 2 | alb.b | f16 | [0,1] | feat_tex0.y lo | +| 3 | nrm.x | f16 | [-1,1] | feat_tex0.y hi | +| 4 | nrm.y | f16 | [-1,1] | feat_tex0.z lo | +| 5 | depth | f16 | [0,1] | feat_tex0.z hi | +| 6 | dzdx | f16 | (signed) | feat_tex0.w lo | +| 7 | dzdy | f16 | (signed) | feat_tex0.w hi | +| 8 | mat_id | u8 | [0,1] | feat_tex1.x[0] | +| 9 | prev.r | u8 | [0,1] | feat_tex1.x[1] | +| 10 | prev.g | u8 | [0,1] | feat_tex1.x[2] | +| 11 | prev.b | u8 | [0,1] | feat_tex1.x[3] | +| 12 | mip1.r | u8 | [0,1] | feat_tex1.y[0] | +| 13 | mip1.g | u8 | [0,1] | feat_tex1.y[1] | +| 14 | mip1.b | u8 | [0,1] | feat_tex1.y[2] | +| 15 | mip2.r | u8 | [0,1] | feat_tex1.y[3] | +| 16 | mip2.g | u8 | [0,1] | feat_tex1.z[0] | +| 17 | mip2.b | u8 | [0,1] | feat_tex1.z[1] | +| 18 | **dif** | u8 | [0,1] | feat_tex1.z[2] ← was shadow | +| 19 | transp | u8 | [0,1] | feat_tex1.z[3] | + +--- + +## Current State (intermediate — needs fixing) + +The commit tagged `wip(cnn_v3): shadow→dif intermediate` contains partial work. +The WGSL changes are **incorrect** — `dif` is redundantly stored in t1.w (3×) and +`shadow` was dropped from t1.z without putting `dif` in its place. + +### What is wrong + +| File | Problem | +|---|---| +| `gbuf_pack.wgsl` | t1.z = `mip2.g\|mip2.b\|transp\|spare` (shadow removed, dif not put there); t1.w = `dif\|dif\|dif\|spare` (redundant) | +| `gbuf_deferred.wgsl` | reads `dif` from `t1.w.x` — should be `t1.z.z` | +| `gbuf_view.wgsl` | expanded to 4×6 grid with ch20–22 as dif.rgb — should stay 4×5, ch18=dif | + +--- + +## Implementation Checklist + +### Step 1 — Fix WGSL (correct the in-place swap) + +- [ ] `cnn_v3/shaders/gbuf_pack.wgsl` + - t1.z: `pack4x8unorm(vec4f(mip2.g, mip2.b, dif, transp))` ← dif at byte 2 + - t1.w: `0u` ← revert to spare + - Remove comment line about t1.w dif + +- [ ] `cnn_v3/shaders/gbuf_deferred.wgsl` + - Read: `let dif = unpack4x8unorm(t1.z).z;` ← from t1.z byte 2 + +- [ ] `cnn_v3/shaders/gbuf_view.wgsl` + - Revert to 4×5 grid (ROWS = 5.0) + - Guard: `ch >= 20u` + - ch18 label: `dif` (4 chars: 0x64696600) + - ch19 label: `trns` (unchanged) + - Remove row-5 cases (20u, 21u, default→dif.b) + - Revert `else if (comp_idx == 2u)` → `else` (drop t1.w branch) + - Update header comment + +- [ ] `cnn_v3/shaders/cnn_v3_enc0.wgsl` + - Verify `load_feat()`: g = unpack4x8unorm(t1.z) → g.z = ch18 = dif ✓ (no change needed) + +### Step 2 — Python training + +- [ ] `cnn_v3/training/cnn_v3_utils.py` + - `assemble_features()`: ch18 = `dif` computed on-the-fly: + ```python + KEY_LIGHT = np.array([0.408, 0.816, 0.408]) + nor3 = oct_decode(normal) # (H,W,2) → (H,W,3) + diffuse = np.maximum(0, (nor3 * KEY_LIGHT).sum(-1)) + dif = diffuse * shadow # (H,W) + ``` + - Replace `shadow[..., None]` with `dif[..., None]` at index 18 + - `CONTEXT_CHANNELS = [8, 18, 19]` — same indices, update comment + +- [ ] `cnn_v3/training/pack_blender_sample.py` + - Optional: save `dif.png` (precomputed) alongside existing passes + - Not strictly required if utils.py computes on-the-fly + +### Step 3 — Web tool + +- [ ] `cnn_v3/tools/shaders.js` (FULL_PACK_SHADER) + - Add `oct_decode` inline (or inline the math) + - Compute `let dif = max(0., dot(oct_decode(nrm), vec3f(0.408, 0.816, 0.408))) * shd` + - Pack: t1.z = `pack4x8unorm(vec4f(m2.g, m2.b, dif, trp))` + - t1.w = `0u` + +### Step 4 — Test vectors + +- [ ] Re-run `cnn_v3/training/gen_test_vectors.py` to regenerate `test_vectors.h` + - ch18 value changes (dif ≠ shadow in general); old vectors are invalid + - Parity threshold (4.88e-4) should be unchanged + +### Step 5 — Docs + +- [ ] `cnn_v3/docs/CNN_V3.md` — update feature table (ch18 shadow → dif) +- [ ] `cnn_v3/docs/HOWTO.md` — §7 channel table, §3 pass-2 note +- [ ] This file: mark steps complete as they land + +--- + +## Architecture Impact + +| Dimension | Before | After | +|---|---|---| +| Channel count | 20 | 20 ✅ | +| Weight shapes | Conv(20→4, ...) | Conv(20→4, ...) ✅ | +| Total f16 weights | 1964 | 1964 ✅ | +| Training data regen | — | Not required ✅ | +| Parity test vectors | Valid | Must regenerate ❌ | +| Existing trained weights | Valid | Invalidated (ch18 distribution changes) ❌ | + +No real training pass has occurred yet, so weight invalidation is not a concern. -- cgit v1.2.3