diff options
| -rw-r--r-- | cnn_v3/docs/CNN_V3.md | 4 | ||||
| -rw-r--r-- | cnn_v3/docs/HOWTO.md | 48 | ||||
| -rw-r--r-- | doc/SEQUENCE.md | 8 |
3 files changed, 36 insertions, 24 deletions
diff --git a/cnn_v3/docs/CNN_V3.md b/cnn_v3/docs/CNN_V3.md index f86aa5a..4d58811 100644 --- a/cnn_v3/docs/CNN_V3.md +++ b/cnn_v3/docs/CNN_V3.md @@ -836,7 +836,7 @@ FiLM γ/β computed JS-side from sliders (tiny MLP forward pass in JS), uploaded | `bn_tex` | W/2×H/2 | rgba32uint | 8 channels f16 (bottleneck output) | | `dec1_tex` | W×H | rgba32uint | 4 channels f16 (dec1 output) | | `dec0_tex` | W×H | rgba32uint | 4 channels f16 (dec0 output) | -| `prev_tex` | W×H | rgba8unorm | previous CNN output (temporal) | +| `prev_tex` | W×H | rgba16float | previous CNN output (temporal, `F16X8`) | Skip connections: enc0_tex and enc1_tex are **kept alive** across the full forward pass (not ping-ponged away). DEC1 and DEC0 read them directly. @@ -979,7 +979,7 @@ Reuse from existing shaders: - [ ] `cmake/DemoSourceLists.cmake` — add `cnn_v3_effect.cc` to COMMON_GPU_EFFECTS - [ ] `src/gpu/demo_effects.h` — add `#include "effects/cnn_v3_effect.h"` -- [ ] `workspaces/main/timeline.seq` — add `EFFECT + CNNv3Effect` +- [x] `workspaces/main/timeline.seq` — add `EFFECT + CNNv3Effect` (done: cnn_v3_debug sequence) --- diff --git a/cnn_v3/docs/HOWTO.md b/cnn_v3/docs/HOWTO.md index 48b5d68..5cfc371 100644 --- a/cnn_v3/docs/HOWTO.md +++ b/cnn_v3/docs/HOWTO.md @@ -95,20 +95,32 @@ outputs[1] → feat_tex1 (rgba32uint: mat_id, prev.rgb, mip1.rgb, mip2.rgb, d ### Temporal feedback (prev.rgb) -`GBufferEffect` owns a persistent internal node `<prefix>_prev` (rgba8unorm, `U8X4_NORM`). -Each frame it is GPU-copied from the CNN effect's output before Pass 1 runs, then bound as -`prev_cnn` in the pack shader (binding 6). +`GBufferEffect` owns a persistent internal node `<prefix>_prev` (`F16X8` = Rgba16Float, +`CopySrc|CopyDst`). Each frame it is GPU-copied from the CNN effect's output after all +effects render (`post_render`), then bound as `prev_cnn` in the pack shader (binding 6). -**To wire temporal feedback**, call once after constructing the effects: -```cpp -gbuf->set_cnn_output_node("cnn_v3_out"); // name of CNNv3Effect's output node +**Wiring is automatic** via `wire_dag()`, called by `Sequence::init_effect_nodes()`. +`GBufferEffect` scans the DAG for the first downstream consumer of its output nodes and +uses that effect's output as `cnn_output_node_`. No manual call needed. + +**Requirement**: the sequence must include `CNNv3Effect` downstream of `GBufferEffect`. +In `timeline.seq`, declare a `gbuf_albedo` output node and add the effect: + +```seq +NODE cnn_out gbuf_albedo +EFFECT + GBufferEffect source -> gbuf_feat0 gbuf_feat1 0 60 +EFFECT + CNNv3Effect gbuf_feat0 gbuf_feat1 -> cnn_out 0 60 ``` +If no CNN effect follows, `cnn_output_node_` stays empty and `post_render` is a no-op +(prev.rgb will be zero — correct for static/debug-only sequences). + Frame 0 behaviour: `_prev` is zeroed on allocation → `prev.rgb = 0`, matching the training convention (static frames use zero history). The copy uses `wgpuCommandEncoderCopyTextureToTexture` (no extra render pass overhead). -Both textures must be `rgba8unorm` — the CNN output sink (`U8X4_NORM`) satisfies this. +`node_prev_tex_` is `F16X8` (Rgba16Float) to match the `GBUF_ALBEDO` format of CNNv3Effect's +output — `CopyTextureToTexture` requires identical formats. --- @@ -320,10 +332,7 @@ SEQUENCE 0 0 "Scene with CNN v3" EFFECT + CNNv3Effect gbuf_feat0 gbuf_feat1 -> sink 0 60 ``` -After constructing the effects, wire temporal feedback: -```cpp -gbuf_effect->set_cnn_output_node("sink"); // or whichever node receives CNN output -``` +Temporal feedback is wired automatically by `wire_dag()` — no manual call needed. FiLM parameters uploaded each frame: ```cpp @@ -477,16 +486,15 @@ GBufViewEffect(const GpuContext& ctx, float start_time, float end_time) ``` -**Wiring example** (alongside GBufferEffect): +**Wiring example** — use `timeline.seq`, temporal feedback wires automatically: -```cpp -auto gbuf = std::make_shared<GBufferEffect>(ctx, - std::vector<std::string>{}, // no external inputs - std::vector<std::string>{"gbuf_feat0", "gbuf_feat1"}, 0.0f, 60.0f); -gbuf->set_cnn_output_node("cnn_out"); // wire temporal feedback after CNN is constructed -auto gview = std::make_shared<GBufViewEffect>(ctx, - std::vector<std::string>{"gbuf_feat0", "gbuf_feat1"}, - std::vector<std::string>{"gbuf_view_out"}, 0.0f, 60.0f); +```seq +NODE gbuf_feat0 gbuf_rgba32uint +NODE gbuf_feat1 gbuf_rgba32uint +NODE cnn_out gbuf_albedo +EFFECT + GBufferEffect source -> gbuf_feat0 gbuf_feat1 0 60 +EFFECT + CNNv3Effect gbuf_feat0 gbuf_feat1 -> cnn_out 0 60 +EFFECT + GBufViewEffect gbuf_feat0 gbuf_feat1 -> sink 0 60 ``` **Grid layout** (output resolution = input resolution, channel cells each 1/4 W × 1/5 H): diff --git a/doc/SEQUENCE.md b/doc/SEQUENCE.md index ca45756..3d7a6ce 100644 --- a/doc/SEQUENCE.md +++ b/doc/SEQUENCE.md @@ -132,7 +132,8 @@ class MyEffect : public Effect { void set_source_node(const std::string& n) { source_node_ = n; } void declare_nodes(NodeRegistry& reg) override { - reg.declare_node(node_prev_, NodeType::U8X4_NORM, -1, -1); + // Use a NodeType whose format matches source_node_ and has CopyDst. + reg.declare_node(node_prev_, NodeType::F16X8, -1, -1); } void render(...) override { @@ -221,7 +222,10 @@ if (!src_tex) return; // external view — no owned texture to copy - `GBUF_RGBA32UINT`: RGBA32Uint — packed feature textures (CNN v3 feat_tex0/1); `STORAGE_BINDING|TEXTURE_BINDING` **`COPY_SRC|COPY_DST`** is required on any node used with `wgpuCommandEncoderCopyTextureToTexture`. -`U8X4_NORM` has both; use it for temporal feedback dest nodes. +The `node_prev_` format **must match** the source texture format exactly — +`CopyTextureToTexture` requires identical formats. `F16X8` (Rgba16Float, +`CopySrc|CopyDst`) matches `GBUF_ALBEDO` (CNNv3Effect output). Use `U8X4_NORM` +only when the source is also Rgba8Unorm. **Aliasing**: Compiler detects ping-pong patterns (Effect i writes A reads B, Effect i+1 writes B reads A) and aliases nodes to same backing texture. |