From 13cf1438caa56b34529d4031ddf73d38286b70e5 Mon Sep 17 00:00:00 2001 From: skal Date: Mon, 23 Mar 2026 00:43:20 +0100 Subject: feat(cnn_v3): shadow→dif migration complete (ch18) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace raw shadow (ch18) with dif = max(0,dot(normal,KEY_LIGHT))*shadow across all layers. Channel count stays 20, weight shapes unchanged. - gbuf_pack.wgsl: t1.z = pack4x8unorm(mip2.g, mip2.b, dif, transp); t1.w = 0u - gbuf_deferred.wgsl: read dif from unpack4x8unorm(t1.z).z - gbuf_view.wgsl: revert to 4×5 grid, ch18=dif label, ch19=trns label - tools/shaders.js: FULL_PACK_SHADER adds oct_decode + computes dif - cnn_v3_utils.py: assemble_features() computes dif on-the-fly via oct_decode - docs: CNN_V3.md, HOWTO.md, HOW_TO_CNN.md, GBUF_DIF_MIGRATION.md updated handoff(Gemini): shadow→dif migration done, ready for first training pass --- cnn_v3/docs/HOW_TO_CNN.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) (limited to 'cnn_v3/docs/HOW_TO_CNN.md') diff --git a/cnn_v3/docs/HOW_TO_CNN.md b/cnn_v3/docs/HOW_TO_CNN.md index 458b68f..4966a61 100644 --- a/cnn_v3/docs/HOW_TO_CNN.md +++ b/cnn_v3/docs/HOW_TO_CNN.md @@ -97,7 +97,7 @@ It calls `pack_photo_sample.py` with both `--photo` and `--target` in a single s | `normal.png` | (128, 128, 0) uint8 | Neutral "no normal" → reconstructed (0,0,1) | | `depth.png` | All zeros uint16 | No depth data | | `matid.png` | All zeros uint8 | No material IDs | -| `shadow.png` | 255 everywhere uint8 | Assume fully lit | +| `shadow.png` | 255 everywhere uint8 | Assume fully lit (used to compute dif) | | `transp.png` | 1 − alpha uint8 | 0 = opaque | | `target.png` | Stylized target RGBA | Ground truth for training | @@ -134,7 +134,7 @@ done ### 1b. From Blender (Full G-Buffer) -Produces all 20 feature channels including normals, depth, mat IDs, and shadow. +Produces all 20 feature channels including normals, depth, mat IDs, and dif (diffuse×shadow). #### Blender requirements @@ -420,7 +420,7 @@ Applied per-sample to make the model robust to missing channels: | Channel group | Channels | Drop probability | |---------------|----------|-----------------| | Geometric | normal.xy, depth, depth_grad.xy [3,4,5,6,7] | `channel_dropout_p` (default 0.3) | -| Context | mat_id, shadow, transp [8,18,19] | `channel_dropout_p × 0.67` (~0.2) | +| Context | mat_id, dif, transp [8,18,19] | `channel_dropout_p × 0.67` (~0.2) | | Temporal | prev.rgb [9,10,11] | 0.5 (always) | This is why a model trained on Blender data also works on photos (geometry zeroed). @@ -781,7 +781,7 @@ Both produced by `export_cnn_v3_weights.py` (§3). | Texture | Format | Size | |---------|--------|------| | `feat_tex0` | rgba32uint | W × H (8 f16: albedo, normal, depth, depth_grad) | -| `feat_tex1` | rgba32uint | W × H (12 u8: mat_id, prev, mip1, mip2, shadow, transp) | +| `feat_tex1` | rgba32uint | W × H (12 u8: mat_id, prev, mip1, mip2, dif, transp) | | `enc0_tex` | rgba16float | W × H | | `enc1_tex` | rgba32uint | W/2 × H/2 (8 f16 packed) | | `bn_tex` | rgba32uint | W/4 × H/4 | @@ -790,7 +790,7 @@ Both produced by `export_cnn_v3_weights.py` (§3). ### Simple mode (photo input) -Albedo = image RGB, mip1/mip2 from GPU mipmaps, shadow = 1.0, transp = 1 − alpha, +Albedo = image RGB, mip1/mip2 from GPU mipmaps, dif = 1.0 (fully lit assumed), transp = 1 − alpha, all geometric channels (normal, depth, depth_grad, mat_id, prev) = 0. ### Browser requirements @@ -843,7 +843,7 @@ all geometric channels (normal, depth, depth_grad, mat_id, prev) = 0. | 9–11 | prev.rgb | previous frame output | zero during training | | 12–14 | mip1.rgb | pyrdown(albedo) | f32 [0,1] | | 15–17 | mip2.rgb | pyrdown(mip1) | f32 [0,1] | -| 18 | shadow | `shadow.png` | f32 [0,1] (1=lit) | +| 18 | dif | computed | f32 [0,1] max(0,dot(normal,KEY_LIGHT))×shadow | | 19 | transp | `transp.png` | f32 [0,1] (0=opaque) | **Feature texture packing** (`feat_tex0` / `feat_tex1`, both `rgba32uint`): @@ -858,6 +858,6 @@ feat_tex0 (4×u32 = 8 f16 channels via pack2x16float): feat_tex1 (4×u32 = 12 u8 channels + padding via pack4x8unorm): .x = pack4x8unorm(mat_id, prev.r, prev.g, prev.b) .y = pack4x8unorm(mip1.r, mip1.g, mip1.b, mip2.r) - .z = pack4x8unorm(mip2.g, mip2.b, shadow, transp) + .z = pack4x8unorm(mip2.g, mip2.b, dif, transp) .w = 0 (unused, 8 reserved channels) ``` -- cgit v1.2.3