From d181e145296a04384850e09be03b697458cd8439 Mon Sep 17 00:00:00 2001
From: skal <pascal.massimino@gmail.com>
Date: Sun, 22 Mar 2026 13:10:47 +0100
Subject: fix(cnn_v3): fix texture format mismatches in cnn_v3_test sequence

- seq_compiler: add gbuf_albedo/gbuf_rgba32uint to NODE_TYPES
- timeline: declare gbuf_feat0/feat1 as gbuf_rgba32uint, route
  CNNv3Effect output through cnn_v3_out (gbuf_albedo) + Passthrough
  to sink (dec0 can't write directly to Rgba8Unorm sink)
- cnn_v3_effect: fix update_bind_groups using .set() instead of
  .replace() causing FATAL assert on second frame
- TODO: add CNN v3 "2D mode" (G-buffer-free) future task

handoff(Gemini): CNNv3Effect now runs without crashes at --seek 48
---
 TODO.md | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

(limited to 'TODO.md')

diff --git a/TODO.md b/TODO.md
index 48518f2..4f38740 100644
--- a/TODO.md
+++ b/TODO.md
@@ -84,6 +84,23 @@ PyTorch / HTML WebGPU / C++ WebGPU.
 - ✅ `export_cnn_v3_weights.py` — convert trained `.pth` → `.bin` (f16)
 - See `cnn_v3/docs/HOWTO.md` §3 for training commands
 
+## Future: CNN v3 "2D Mode" (G-buffer-free)
+
+Allow `CNNv3Effect` to run on a plain screen buffer / photo without a real G-buffer.
+Fake the missing feature vectors (normals, depth, material IDs, shadow, transp) from
+the RGB input alone:
+- normals: approximate from local luminance gradient (Sobel)
+- depth: constant (e.g. 0.5) or estimated from a simple heuristic
+- material IDs / shadow / transp: neutral defaults (e.g. 0)
+
+This would let the effect be applied to any rendered frame (post-NTSC, post-Scratch, etc.)
+without requiring a 3D G-buffer pass upstream, and enable training/inference on photos.
+
+Implementation sketch:
+- New `CNNv3Effect2D` subclass (or a mode flag) that synthesizes `feat_tex0`/`feat_tex1`
+  internally from a single `rgba8unorm` input, then runs the same 5-pass U-Net.
+- Separate `gbuf_pack_2d.wgsl` compute shader that fills feat0/feat1 from a photo buffer.
+
 ## Future: CNN v2 8-bit Quantization
 
 Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB).
-- 
cgit v1.2.3