diff options
Diffstat (limited to 'TODO.md')
| -rw-r--r-- | TODO.md | 17 |
1 files changed, 17 insertions, 0 deletions
@@ -84,6 +84,23 @@ PyTorch / HTML WebGPU / C++ WebGPU. - โ
`export_cnn_v3_weights.py` โ convert trained `.pth` โ `.bin` (f16) - See `cnn_v3/docs/HOWTO.md` ยง3 for training commands +## Future: CNN v3 "2D Mode" (G-buffer-free) + +Allow `CNNv3Effect` to run on a plain screen buffer / photo without a real G-buffer. +Fake the missing feature vectors (normals, depth, material IDs, shadow, transp) from +the RGB input alone: +- normals: approximate from local luminance gradient (Sobel) +- depth: constant (e.g. 0.5) or estimated from a simple heuristic +- material IDs / shadow / transp: neutral defaults (e.g. 0) + +This would let the effect be applied to any rendered frame (post-NTSC, post-Scratch, etc.) +without requiring a 3D G-buffer pass upstream, and enable training/inference on photos. + +Implementation sketch: +- New `CNNv3Effect2D` subclass (or a mode flag) that synthesizes `feat_tex0`/`feat_tex1` + internally from a single `rgba8unorm` input, then runs the same 5-pass U-Net. +- Separate `gbuf_pack_2d.wgsl` compute shader that fills feat0/feat1 from a photo buffer. + ## Future: CNN v2 8-bit Quantization Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB). |
