diff options
| author | skal <pascal.massimino@gmail.com> | 2026-03-22 13:10:47 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-03-22 13:10:47 +0100 |
| commit | d181e145296a04384850e09be03b697458cd8439 (patch) | |
| tree | 5b3280253e96e3abcb9dc3889860f02ba8260c1b /TODO.md | |
| parent | 96b454743a2eac2ae1d4bdd38d7ca81cc4fd100b (diff) | |
fix(cnn_v3): fix texture format mismatches in cnn_v3_test sequence
- seq_compiler: add gbuf_albedo/gbuf_rgba32uint to NODE_TYPES
- timeline: declare gbuf_feat0/feat1 as gbuf_rgba32uint, route
CNNv3Effect output through cnn_v3_out (gbuf_albedo) + Passthrough
to sink (dec0 can't write directly to Rgba8Unorm sink)
- cnn_v3_effect: fix update_bind_groups using .set() instead of
.replace() causing FATAL assert on second frame
- TODO: add CNN v3 "2D mode" (G-buffer-free) future task
handoff(Gemini): CNNv3Effect now runs without crashes at --seek 48
Diffstat (limited to 'TODO.md')
| -rw-r--r-- | TODO.md | 17 |
1 files changed, 17 insertions, 0 deletions
@@ -84,6 +84,23 @@ PyTorch / HTML WebGPU / C++ WebGPU. - โ
`export_cnn_v3_weights.py` โ convert trained `.pth` โ `.bin` (f16) - See `cnn_v3/docs/HOWTO.md` ยง3 for training commands +## Future: CNN v3 "2D Mode" (G-buffer-free) + +Allow `CNNv3Effect` to run on a plain screen buffer / photo without a real G-buffer. +Fake the missing feature vectors (normals, depth, material IDs, shadow, transp) from +the RGB input alone: +- normals: approximate from local luminance gradient (Sobel) +- depth: constant (e.g. 0.5) or estimated from a simple heuristic +- material IDs / shadow / transp: neutral defaults (e.g. 0) + +This would let the effect be applied to any rendered frame (post-NTSC, post-Scratch, etc.) +without requiring a 3D G-buffer pass upstream, and enable training/inference on photos. + +Implementation sketch: +- New `CNNv3Effect2D` subclass (or a mode flag) that synthesizes `feat_tex0`/`feat_tex1` + internally from a single `rgba8unorm` input, then runs the same 5-pass U-Net. +- Separate `gbuf_pack_2d.wgsl` compute shader that fills feat0/feat1 from a photo buffer. + ## Future: CNN v2 8-bit Quantization Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB). |
