diff options
| author | skal <pascal.massimino@gmail.com> | 2026-03-22 19:15:55 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-03-22 19:15:55 +0100 |
| commit | c5e66964c0463219019d0439ec20b79248637fa4 (patch) | |
| tree | 270895c85c2058ff53c2a8c3fa4242b16f2de0ff /cnn_v3/docs/HOWTO.md | |
| parent | 01df602ea6580edc418b70f121e521a8217f954c (diff) | |
feat(cnn_v3): GBufferEffect internal scene + GBufViewEffect debug wiring
GBufferEffect:
- set_scene() now owns Scene/Camera internally; no external pointers needed
- 20 randomly rotating cubes (xorshift32 seed, axis-angle animation)
- 4 pumping spheres (radius = base_r * (1 + audio_intensity * 0.8))
- Camera at (0,2.5,6) looking at origin; aspect updated per-frame
- GBufLightsUniforms: 2 directional lights (warm key + cool fill)
- object_type written to ObjectData.params.x (ready for SDF shadow)
- shadow/transp nodes cleared via zero-draw render passes (placeholder)
- bilinear sampler cached via create_linear_sampler() / sampler_.get()
- dead placeholder textures removed
GBufViewEffect:
- gbuf_view.wgsl: all channels now fully grayscale (removed color tint)
- seq_compiler.py: GBufViewEffect added to CLASS_TO_HEADER
- timeline.seq: cnn_v3_test uses GBufViewEffect -> sink for debug view
Docs: HOWTO.md §1 updated with set_scene() description + §1b implementation
plan for Pass 2 SDF shadow (shader spec, bind layout, C++ additions)
handoff(Gemini): GBufferEffect has internal scene, 36/36 tests green.
Next: implement Pass 2 shadow (gbuf_shadow.wgsl) per §1b plan in HOWTO.md.
Diffstat (limited to 'cnn_v3/docs/HOWTO.md')
| -rw-r--r-- | cnn_v3/docs/HOWTO.md | 136 |
1 files changed, 110 insertions, 26 deletions
diff --git a/cnn_v3/docs/HOWTO.md b/cnn_v3/docs/HOWTO.md index 08979e7..2d88019 100644 --- a/cnn_v3/docs/HOWTO.md +++ b/cnn_v3/docs/HOWTO.md @@ -22,57 +22,141 @@ It rasterizes proxy geometry to MRT G-buffer textures and packs them into two ### Adding to a Sequence -`GBufferEffect` does not exist in `seq_compiler.py` as a named effect yet -(no `.seq` syntax integration for Phase 1). Wire it directly in C++ alongside -your scene code, or add it to the timeline when the full CNNv3Effect is ready. +Both `GBufferEffect` and `GBufViewEffect` are registered in `seq_compiler.py` +(`CLASS_TO_HEADER`) and can be wired directly in `timeline.seq`. -**C++ wiring example** (e.g. inside a Sequence or main.cc): +**Debug view (G-buffer → sink)**: +```seq +SEQUENCE 12.00 0 "cnn_v3_test" + NODE gbuf_feat0 gbuf_rgba32uint + NODE gbuf_feat1 gbuf_rgba32uint + EFFECT + GBufferEffect source -> gbuf_feat0 gbuf_feat1 0.00 8.00 + EFFECT + GBufViewEffect gbuf_feat0 gbuf_feat1 -> sink 0.00 8.00 +``` -```cpp -#include "../../cnn_v3/src/gbuffer_effect.h" +**Full CNN pipeline**: +```seq +SEQUENCE 12.00 0 "cnn_v3_test" + NODE gbuf_feat0 gbuf_rgba32uint + NODE gbuf_feat1 gbuf_rgba32uint + NODE cnn_v3_out gbuf_albedo + EFFECT + GBufferEffect source -> gbuf_feat0 gbuf_feat1 0.00 8.00 + EFFECT + CNNv3Effect gbuf_feat0 gbuf_feat1 -> cnn_v3_out 0.00 8.00 + EFFECT + Passthrough cnn_v3_out -> sink 0.00 8.00 +``` -// Allocate once alongside your scene -auto gbuf = std::make_shared<GBufferEffect>( - ctx, /*inputs=*/{"prev_cnn"}, // or any dummy node - /*outputs=*/{"gbuf_feat0", "gbuf_feat1"}, - /*start=*/0.0f, /*end=*/60.0f); +### Internal scene -gbuf->set_scene(&my_scene, &my_camera); +Call `set_scene()` once before the first render to populate the built-in demo +scene. No external `Scene` or `Camera` pointer is required — the effect owns +them. -// In render loop, call before CNN pass: -gbuf->render(encoder, params, nodes); -``` +**What `set_scene()` creates:** +- **20 small cubes** — random positions in [-2,2]×[-1.5,1.5]³, scale 0.1–0.25, + random colors. Each has a random rotation axis and speed; animated each frame + via `quat::from_axis(axis, time * speed)`. +- **4 pumping spheres** — at fixed world positions, base radii 0.25–0.35. + Scale driven by `audio_intensity`: `r = base_r * (1 + audio_intensity * 0.8)`. +- **Camera** — position (0, 2.5, 6), target (0, 0, 0), 45° FOV. + Aspect ratio updated each frame from `params.aspect_ratio`. +- **Two directional lights** (uploaded to `lights_uniform_`, ready for shadow pass): + - Key: warm white (1.0, 0.92, 0.78), direction `normalize(1, 2, 1)` (upper-right-front) + - Fill: cool blue (0.4, 0.45, 0.8 × 0.4), direction `normalize(-1, 1, -1)` (upper-left-back) ### Internal passes Each frame, `GBufferEffect::render()` executes: -1. **Pass 1 — MRT rasterization** (`gbuf_raster.wgsl`) +1. **Pass 1 — MRT rasterization** (`gbuf_raster.wgsl`) ✅ - Proxy box (36 verts) × N objects, instanced - MRT outputs: `gbuf_albedo` (rgba16float), `gbuf_normal_mat` (rgba16float) - Depth test + write into `gbuf_depth` (depth32float) + - `obj.type` written to `ObjectData.params.x` for future SDF branching + +2. **Pass 2 — SDF shadow raymarching** (`gbuf_shadow.wgsl`) — TODO + - See implementation plan below. -2. **Pass 2/3 — SDF + Lighting** — TODO (placeholder: shadow=1, transp=0) +3. **Pass 3 — Transparency** — TODO (deferred; transp=0 for opaque scenes) -3. **Pass 4 — Pack compute** (`gbuf_pack.wgsl`) +4. **Pass 4 — Pack compute** (`gbuf_pack.wgsl`) ✅ - Reads all G-buffer textures + `prev_cnn` input - Writes `feat_tex0` + `feat_tex1` (rgba32uint, 20 channels, 32 bytes/pixel) + - Shadow / transp nodes cleared to 1.0 / 0.0 via zero-draw render passes + until Pass 2/3 are implemented. ### Output node names -By default the outputs are named from the `outputs` vector passed to the -constructor. Use these names when binding the CNN effect input: +Outputs are named from the `outputs` vector passed to the constructor: ``` outputs[0] → feat_tex0 (rgba32uint: albedo.rgb, normal.xy, depth, depth_grad.xy) outputs[1] → feat_tex1 (rgba32uint: mat_id, prev.rgb, mip1.rgb, mip2.rgb, shadow, transp) ``` -### Scene data +--- + +## 1b. GBufferEffect — Implementation Plan (Pass 2: SDF Shadow) + +### What remains + +| Item | Status | Notes | +|------|--------|-------| +| Pass 1: MRT raster | ✅ Done | proxy box, all object types | +| Pass 4: Pack compute | ✅ Done | 20 channels packed | +| Internal scene + animation | ✅ Done | cubes + spheres + 2 lights | +| Pass 2: SDF shadow | ❌ TODO | main missing piece | +| Pass 3: Transparency | ❌ TODO | low priority, opaque scenes only | +| Phase 4: type-aware SDF | ❌ TODO | optional refinement | + +### Pass 2: SDF shadow raymarching + +**New file: `cnn_v3/shaders/gbuf_shadow.wgsl`** — fullscreen render pass. + +Bind layout: + +| Binding | Type | Content | +|---------|------|---------| +| 0 | `uniform` | `GlobalUniforms` (`#include "common_uniforms"`) | +| 1 | `storage read` | `ObjectsBuffer` | +| 2 | `texture_depth_2d` | depth from Pass 1 | +| 3 | `sampler` (non-filtering) | depth load | +| 4 | `uniform` | `GBufLightsUniforms` (2 lights) | + +Algorithm per fragment: +1. Reconstruct world position from NDC depth + `globals.inv_view_proj` +2. For each object: `sdBox((inv_model * world_pos).xyz, vec3(1.0))` — proxy box in local space +3. For each light: offset ray origin by `0.02 * surface_normal`; march shadow ray toward `light.direction` +4. Soft shadow via `shadowWithStoredDistance()` from `render/raymarching_id` +5. Combine lights: `shadow = min(shadow_light0, shadow_light1)` +6. Discard fragments where depth == 1.0 (sky/background → shadow = 1.0) +7. Output shadow factor to RGBA8Unorm render target (`.r` = shadow) + +**C++ additions (`gbuffer_effect.h/.cc`):** +```cpp +RenderPipeline shadow_pipeline_; +void create_shadow_pipeline(); +``` +In `render()` between Pass 1 and the shadow/transp node clears: +- Build bind group (global_uniforms_buf_, objects_buf_, depth_view, sampler_, lights_uniform_) +- Run fullscreen triangle → `node_shadow_` color attachment +- Remove the `clear_node(node_shadow_, 1.0f)` placeholder once the pass is live + +**Register:** +- `cnn_v3/shaders/gbuf_shadow.wgsl` → `SHADER_GBUF_SHADOW` in `assets.txt` +- `extern const char* gbuf_shadow_wgsl;` in `gbuffer_effect.cc` + +### Phase 4: Object-type-aware SDF (optional) + +Branch on `obj.params.x` (populated since this commit) using `math/sdf_shapes`: + +| Type value | ObjectType | SDF | +|------------|-----------|-----| +| 0 | CUBE | `sdBox(local_p, vec3(1))` | +| 1 | SPHERE | `sdSphere(local_p, 1.0)` | +| 2 | PLANE | `sdPlane(local_p, vec3(0,1,0), obj.params.y)` | +| 3 | TORUS | `sdTorus(local_p, vec2(0.8, 0.2))` | -Call `set_scene(scene, camera)` before the first render. The effect uploads -`GlobalUniforms` (view-proj, camera pos, resolution) and `ObjectData` (model -matrix, color) to GPU storage buffers each frame. +Only worth adding after Pass 2 is validated visually. --- @@ -386,13 +470,13 @@ auto gview = std::make_shared<GBufViewEffect>(ctx, | Row | Col 0 | Col 1 | Col 2 | Col 3 | |-----|-------|-------|-------|-------| -| 0 | `alb.r` (red tint) | `alb.g` (green tint) | `alb.b` (blue tint) | `nrm.x` remap→[0,1] | +| 0 | `alb.r` | `alb.g` | `alb.b` | `nrm.x` remap→[0,1] | | 1 | `nrm.y` remap→[0,1] | `depth` (inverted) | `dzdx` ×20+0.5 | `dzdy` ×20+0.5 | | 2 | `mat_id` | `prev.r` | `prev.g` | `prev.b` | | 3 | `mip1.r` | `mip1.g` | `mip1.b` | `mip2.r` | | 4 | `mip2.g` | `mip2.b` | `shadow` | `transp` | -1-pixel gray grid lines separate cells. Dark background for out-of-range cells. +All channels displayed as grayscale. 1-pixel gray grid lines separate cells. Dark background for out-of-range cells. **Shader binding layout** (no sampler needed — integer texture): |
