diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-16 11:58:34 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-16 11:58:34 +0100 |
| commit | 7a383635525c9f9617965f3c79a9f2d6c86550cd (patch) | |
| tree | 84491cede72c416e2f0b8131854c849c4b1454a5 /doc | |
| parent | af5d0e4c3a6cb773a4fb51ac32f4c33a7f8d8224 (diff) | |
docs(sequence): update and compact v2 documentation
Documentation Changes:
- Rewrote SEQUENCE_v2.md: practical guide focused on actual implementation
- Removed design philosophy, added concrete examples
- Documented all implemented features and current limitations
- Added effect creation templates (standard post-process, 3D with depth)
- 130 lines → 222 lines (expanded with examples)
- Updated EFFECT_WORKFLOW.md for v2
- Changed from v1 Effect/PostProcessEffect to EffectV2
- Updated all steps for v2 workflow (6 steps instead of 8)
- Added complete templates with proper v2 signatures
- Documented common issues and solutions
- Removed v1-specific content
- Archived v1 documentation
- Moved doc/SEQUENCE.md → doc/archive/SEQUENCE_V1.md
- V1 system removed, documentation preserved for reference
Content Focus:
- Quick start examples (simple chain, multi-output, ping-pong)
- Timeline syntax reference with REQUIRED priority modifiers
- Architecture overview (SequenceV2, EffectV2, Node system)
- Compiler features (DAG validation, topological sort, ping-pong detection)
- Practical templates (copy-paste ready)
- Common issues section (build errors, runtime errors)
Status Documentation:
- ✅ Implemented: DAG validation, node aliasing, 7 effects ported
- ❌ Missing: Flatten mode, BPM handling, GetDemoDuration calculation
- TODO: Port remaining effects, implement flatten, update HTML editor
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/EFFECT_WORKFLOW.md | 332 | ||||
| -rw-r--r-- | doc/SEQUENCE_v2.md | 279 | ||||
| -rw-r--r-- | doc/archive/SEQUENCE_V1.md (renamed from doc/SEQUENCE.md) | 0 |
3 files changed, 355 insertions, 256 deletions
diff --git a/doc/EFFECT_WORKFLOW.md b/doc/EFFECT_WORKFLOW.md index 57cf904..bdec2b6 100644 --- a/doc/EFFECT_WORKFLOW.md +++ b/doc/EFFECT_WORKFLOW.md @@ -1,248 +1,256 @@ -# Effect Creation Workflow +# Effect Creation Workflow (v2) **Target Audience:** AI coding agents and developers -Automated checklist for adding new visual effects to the demo. +Checklist for adding visual effects using Sequence v2 system. --- ## Quick Reference -**For ShaderToy conversions:** Use `tools/shadertoy/convert_shadertoy.py` then follow steps 3-8 below. - -**For SDF/raymarching effects:** See `doc/SDF_EFFECT_GUIDE.md` for streamlined workflow using SDFEffect base class. - -**For custom effects:** Follow all steps 1-8. +**ShaderToy:** `tools/shadertoy/convert_shadertoy.py` then follow steps below +**SDF/Raymarching:** See `doc/SDF_EFFECT_GUIDE.md` +**Custom v2 effects:** Follow all steps 1-6 --- -## Step-by-Step Workflow +## Workflow ### 1. Create Effect Files -**Description:** Each visual effect must have its own dedicated header (`.h`) and implementation (`.cc`) file pair. - -**Location:** -- Header: `src/effects/<effect_name>_effect.h` -- Implementation: `src/effects/<effect_name>_effect.cc` -- Shader: `workspaces/main/shaders/<effect_name>.wgsl` +**Files** (v2 naming): +- Header: `src/effects/<name>_effect_v2.h` +- Implementation: `src/effects/<name>_effect_v2.cc` +- Shader: `workspaces/main/shaders/<name>_v2.wgsl` -**Naming Convention:** -- Class name: `<EffectName>Effect` (e.g., `TunnelEffect`, `PlasmaEffect`) -- Files: `<effect_name>_effect.*` (snake_case) +**Class name**: `<Name>EffectV2` (e.g., `TunnelEffectV2`) -**Base Class:** -- Post-process effects: inherit from `PostProcessEffect` -- Scene effects: inherit from `Effect` +**Base class**: `EffectV2` (all effects) -**Render Signature:** +**Constructor**: ```cpp -void render(WGPURenderPassEncoder pass, - const CommonPostProcessUniforms& uniforms) override; +MyEffectV2(const GpuContext& ctx, + const std::vector<std::string>& inputs, + const std::vector<std::string>& outputs); ``` -**Uniforms Available:** +**Required methods**: ```cpp -uniforms.time; // Physical seconds (constant speed) -uniforms.beat_time; // Musical beats (bar synchronization) -uniforms.beat_phase; // Fractional beat 0.0-1.0 (smooth oscillation) -uniforms.audio_intensity; // Audio peak for beat sync -uniforms.resolution; // Screen dimensions -uniforms.aspect_ratio; // Width/height ratio +void render(WGPUCommandEncoder encoder, + const UniformsSequenceParams& params, + NodeRegistry& nodes) override; + +// Optional: for effects needing temp nodes (depth buffers, intermediate textures) +void declare_nodes(NodeRegistry& registry) override; ``` -**Template:** See `tools/shadertoy/template.*` or use `convert_shadertoy.py` +**Uniforms**: +```cpp +params.time; // Physical seconds +params.beat_time; // Musical beats +params.beat_phase; // Fractional beat 0.0-1.0 +params.audio_intensity; // Audio peak +params.resolution; // vec2(width, height) +params.aspect_ratio; // width/height +``` ### 2. Add Shader to Assets -**File:** `workspaces/main/assets.txt` +**File**: `workspaces/main/assets.txt` -**Format:** ``` -SHADER_<UPPER_SNAKE_NAME>, NONE, shaders/<effect_name>.wgsl, "Effect description" +SHADER_<UPPER_NAME>, NONE, shaders/<name>_v2.wgsl, "Description" ``` -**Example:** -``` -SHADER_TUNNEL, NONE, shaders/tunnel.wgsl, "Tunnel effect shader" -``` - -**Asset ID:** Will be `AssetId::ASSET_SHADER_<UPPER_SNAKE_NAME>` in C++ +Asset ID: `AssetId::ASSET_SHADER_<UPPER_NAME>` ### 3. Add to CMakeLists.txt -**File:** `CMakeLists.txt` +**File**: `CMakeLists.txt` -**Action:** Add `src/effects/<effect_name>_effect.cc` to **BOTH** GPU_SOURCES sections: -- Headless mode section (around line 141-167) -- Normal mode section (around line 171-197) - -**Location:** After similar effects (post-process with post-process, scene with scene) - -**Example:** -```cmake -# In headless section (line ~152): - src/effects/solarize_effect.cc - src/effects/tunnel_effect.cc # <-- Add here - src/effects/chroma_aberration_effect.cc - -# In normal section (line ~183): - src/effects/solarize_effect.cc - src/effects/tunnel.cc # <-- Add here - src/effects/chroma_aberration_effect.cc -``` +Add `src/effects/<name>_effect_v2.cc` to **BOTH** GPU_SOURCES sections: +- Headless mode (around line 141-167) +- Normal mode (around line 171-197) ### 4. Include in demo_effects.h -**File:** `src/gpu/demo_effects.h` +**File**: `src/gpu/demo_effects.h` -**Action:** `src/gpu/demo_effects.h` now acts as a central include file. Add a single include directive for your new effect's header: ```cpp -#include "effects/<effect_name>_effect.h" +#include "effects/<name>_effect_v2.h" ``` -**Location:** Alphabetically with other effect includes - ### 5. Add to Timeline -**File:** `workspaces/main/timeline.seq` - -**Format:** -``` -SEQUENCE <start_time> <priority> - EFFECT <+|=|-> <EffectName>Effect <local_start> <local_end> [params...] -``` - -**Priority Modifiers (REQUIRED):** -- `+` : Increment priority -- `=` : Same priority as previous effect -- `-` : Decrement priority (for backgrounds) +**File**: `workspaces/main/timeline_v2.seq` -**Example:** ``` -SEQUENCE 0.0 0 - EFFECT + TunnelEffect 0.0 10.0 +SEQUENCE <start> <priority> "name" + EFFECT + MyEffectV2 source -> sink 0.0 4.0 ``` -**Common Mistake:** Missing priority modifier (`+`, `=`, `-`) after EFFECT keyword +**Priority modifiers** (REQUIRED): `+` (increment), `=` (same), `-` (decrement) -### 6. Update Tests - -**File:** `src/tests/gpu/test_demo_effects.cc` - -**Action:** Add effect to appropriate list: - -**Post-Process Effects (lines 80-93):** -```cpp -{"TunnelEffect", std::make_shared<TunnelEffect>(fixture.ctx())}, -``` - -**Scene Effects (lines 125-137):** -```cpp -{"TunnelEffect", std::make_shared<TunnelEffect>(fixture.ctx())}, -``` - -**3D Effects:** If requires Renderer3D, add to `requires_3d` check (line 148-151) - -### 7. Build and Test +### 6. Regenerate and Build ```bash -# Full build -cmake --build build -j4 +# Regenerate timeline.cc +python3 tools/seq_compiler_v2.py workspaces/main/timeline_v2.seq \ + --output src/generated/timeline.cc -# Run effect tests -cmake -S . -B build -DDEMO_BUILD_TESTS=ON -cmake --build build -j4 --target test_demo_effects -cd build && ./test_demo_effects +# Build +cmake --build build -j4 -# Run all tests -cd build && ctest +# Test +./build/demo64k ``` -### 8. Verify +--- -**Checklist:** -- [ ] Effect compiles without errors -- [ ] Effect appears in timeline -- [ ] test_demo_effects passes -- [ ] Effect renders correctly: `./build/demo64k` -- [ ] No shader compilation errors -- [ ] Follows naming conventions +## Templates ---- +### Standard Post-Process Effect -## Common Issues +```cpp +// my_effect_v2.h +#pragma once +#include "gpu/effect_v2.h" +#include "gpu/uniform_helper.h" -### Build Error: "no member named 'ASSET_..._SHADER'" +class MyEffectV2 : public EffectV2 { + public: + MyEffectV2(const GpuContext& ctx, + const std::vector<std::string>& inputs, + const std::vector<std::string>& outputs); + ~MyEffectV2() override; -**Cause:** Shader not in assets.txt or wrong asset ID name + void render(WGPUCommandEncoder encoder, + const UniformsSequenceParams& params, + NodeRegistry& nodes) override; -**Fix:** -1. Check `workspaces/main/assets.txt` has shader entry -2. Asset ID is `ASSET_` + uppercase entry name (e.g., `SHADER_TUNNEL` → `ASSET_SHADER_TUNNEL`) + private: + WGPURenderPipeline pipeline_; + WGPUBindGroup bind_group_; + UniformBuffer<UniformsSequenceParams> uniforms_buffer_; +}; +``` -### Build Error: "undefined symbol for architecture" +```cpp +// my_effect_v2.cc +#include "effects/my_effect_v2.h" +#include "gpu/post_process_helper.h" +#include "gpu/shaders.h" -**Cause:** Effect not in CMakeLists.txt GPU_SOURCES +MyEffectV2::MyEffectV2(const GpuContext& ctx, + const std::vector<std::string>& inputs, + const std::vector<std::string>& outputs) + : EffectV2(ctx, inputs, outputs), pipeline_(nullptr), bind_group_(nullptr) { + uniforms_buffer_.init(ctx_.device); + pipeline_ = create_post_process_pipeline(ctx_.device, + WGPUTextureFormat_RGBA8Unorm, + my_shader_v2_wgsl); +} -**Fix:** Add `.cc` file to BOTH sections (headless and normal mode) +MyEffectV2::~MyEffectV2() { + if (bind_group_) wgpuBindGroupRelease(bind_group_); + if (pipeline_) wgpuRenderPipelineRelease(pipeline_); +} -### Timeline Parse Error: "Expected '+', '=', or '-'" +void MyEffectV2::render(WGPUCommandEncoder encoder, + const UniformsSequenceParams& params, + NodeRegistry& nodes) { + WGPUTextureView input_view = nodes.get_view(input_nodes_[0]); + WGPUTextureView output_view = nodes.get_view(output_nodes_[0]); -**Cause:** Missing priority modifier after EFFECT keyword + uniforms_buffer_.update(ctx_.queue, params); + pp_update_bind_group(ctx_.device, pipeline_, &bind_group_, input_view, + uniforms_buffer_.get(), {nullptr, 0}); -**Fix:** Use `EFFECT +`, `EFFECT =`, or `EFFECT -` (never just `EFFECT`) + WGPURenderPassColorAttachment color_attachment = { + .view = output_view, +#if !defined(DEMO_CROSS_COMPILE_WIN32) + .depthSlice = WGPU_DEPTH_SLICE_UNDEFINED, +#endif + .loadOp = WGPULoadOp_Clear, + .storeOp = WGPUStoreOp_Store, + .clearValue = {0.0, 0.0, 0.0, 1.0} + }; -### Test Failure: Effect not in test list + WGPURenderPassDescriptor pass_desc = { + .colorAttachmentCount = 1, + .colorAttachments = &color_attachment + }; -**Cause:** Effect not added to test_demo_effects.cc + WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc); + wgpuRenderPassEncoderSetPipeline(pass, pipeline_); + wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr); + wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0); // Fullscreen triangle + wgpuRenderPassEncoderEnd(pass); + wgpuRenderPassEncoderRelease(pass); +} +``` -**Fix:** Add to `post_process_effects` or `scene_effects` list +### 3D Effect with Depth ---- +```cpp +class My3DEffectV2 : public EffectV2 { + std::string depth_node_; -## Automation Script Example + My3DEffectV2(const GpuContext& ctx, ...) + : EffectV2(ctx, inputs, outputs), + depth_node_(outputs[0] + "_depth") {} -```bash -#!/bin/bash -# Example automation for AI agents + void declare_nodes(NodeRegistry& registry) override { + registry.declare_node(depth_node_, NodeType::DEPTH24, -1, -1); + } -EFFECT_NAME="$1" # CamelCase (e.g., "Tunnel") -SNAKE_NAME=$(echo "$EFFECT_NAME" | sed 's/\([A-Z]\)/_\L\1/g' | sed 's/^_//') -UPPER_NAME=$(echo "$SNAKE_NAME" | tr '[:lower:]' '[:upper:]') + void render(WGPUCommandEncoder encoder, + const UniformsSequenceParams& params, + NodeRegistry& nodes) override { + WGPUTextureView color_view = nodes.get_view(output_nodes_[0]); + WGPUTextureView depth_view = nodes.get_view(depth_node_); -echo "Creating effect: $EFFECT_NAME" -echo " Snake case: $SNAKE_NAME" -echo " Upper case: $UPPER_NAME" + WGPURenderPassDepthStencilAttachment depth_attachment = { + .view = depth_view, + .depthLoadOp = WGPULoadOp_Clear, + .depthStoreOp = WGPUStoreOp_Discard, + .depthClearValue = 1.0f + }; -# 1. Generate files (if using ShaderToy) -# ./tools/shadertoy/convert_shadertoy.py shader.txt "$EFFECT_NAME" + WGPURenderPassDescriptor pass_desc = { + .colorAttachmentCount = 1, + .colorAttachments = &color_attachment, + .depthStencilAttachment = &depth_attachment + }; + // ... render 3D scene + } +}; +``` -# 2. Add to assets.txt -echo "SHADER_${UPPER_NAME}, NONE, shaders/${SNAKE_NAME}.wgsl, \"${EFFECT_NAME} effect\"" \ - >> workspaces/main/assets.txt +--- -# 3. Add to CMakeLists.txt (both sections) -# Use Edit tool to add to both GPU_SOURCES sections +## Common Issues -# 4. Add include to demo_effects.h -# Use Edit tool to add #include line +**Build Error: "no member named 'ASSET_..._SHADER'"** +- Shader not in `assets.txt` or wrong name +- Asset ID is `ASSET_` + uppercase entry name -# 5. Add to timeline.seq -# Use Edit tool to add EFFECT line with priority modifier +**Build Error: "undefined symbol"** +- Effect not in CMakeLists.txt GPU_SOURCES +- Must add to BOTH sections (headless + normal) -# 6. Add to test file -# Use Edit tool to add to appropriate test list +**Runtime Error: "Node not found"** +- Forgot `declare_nodes()` for temp nodes +- `init_effect_nodes()` not called (check generated timeline.cc) -# 7. Build -cmake --build build -j4 -``` +**Runtime Error: "invalid bind group"** +- Pipeline format doesn't match framebuffer (use RGBA8Unorm) +- Missing texture view or null resource --- ## See Also -- `tools/shadertoy/README.md` - ShaderToy conversion guide -- `doc/SEQUENCE.md` - Timeline format documentation -- `doc/CONTRIBUTING.md` - General contribution guidelines -- `src/effects/` - Existing effect examples +- `doc/SEQUENCE_v2.md` - Timeline syntax and architecture +- `tools/seq_compiler_v2.py` - Compiler implementation +- `src/effects/*_v2.{h,cc}` - Example effects diff --git a/doc/SEQUENCE_v2.md b/doc/SEQUENCE_v2.md index 0e9d5a9..7ce6efc 100644 --- a/doc/SEQUENCE_v2.md +++ b/doc/SEQUENCE_v2.md @@ -1,130 +1,221 @@ -# Sequence v2 System Documentation +# Sequence v2: DAG-based Effect Routing -**Status:** Phase 1 & 2 complete (Foundation + Compiler operational) +**Status:** ✅ Operational (Phase 4 complete, v1 removed) + +Explicit node system with DAG effect routing. Replaces v1 implicit framebuffer ping-pong. ## Quick Start ```bash -# Compile v2 timeline to C++ -python3 tools/seq_compiler_v2.py input.seq --output generated/timeline_v2.cc +# Compile timeline +python3 tools/seq_compiler_v2.py workspaces/main/timeline_v2.seq --output src/generated/timeline.cc -# Final mode (flattened, no vtables) -python3 tools/seq_compiler_v2.py input.seq --output timeline_v2.cc --flatten +# Flatten mode (future: inline effects, no vtables) +python3 tools/seq_compiler_v2.py timeline.seq --output timeline.cc --flatten ``` -## v2 Timeline Syntax +## Timeline Syntax ``` -# BPM 120 +# BPM 120 (optional, currently ignored) -SEQUENCE 0.0 0 "sequence_name" +SEQUENCE <start_time> <priority> ["name"] # Node declarations (optional, auto-inferred as u8x4_norm) - NODE gbuf_normal f16x8 + NODE temp1 u8x4_norm NODE depth depth24 + NODE gbuf_normal f16x8 - # Asset dependencies + # Asset dependencies (validated at compile-time) ASSET shader_blur - # Effect routing: inputs... -> outputs... - EFFECT + DeferredRender source depth -> gbuf_normal gbuf_pos 0.0 10.0 - EFFECT + Compose gbuf_normal gbuf_pos -> composed 0.0 10.0 - EFFECT + Blur composed -> sink 0.0 10.0 + # Effect routing with priority modifier + EFFECT <+|=|-> EffectClass input1 input2 -> output1 output2 start end [params] +``` + +**Priority modifiers** (REQUIRED): +- `+` : Increment priority (foreground) +- `=` : Same priority as previous +- `-` : Decrement priority (background) + +**Node types**: `u8x4_norm` (default), `f32x4`, `f16x8`, `depth24`, `compute_f32` + +**Reserved nodes**: `source` (input), `sink` (output) + +### Examples + +**Simple chain:** +``` +SEQUENCE 0.0 0 "basic" + EFFECT + HeptagonEffect source -> temp1 0.0 10.0 + EFFECT + GaussianBlur temp1 -> sink 0.0 10.0 +``` + +**Multi-output (G-buffer):** +``` +SEQUENCE 0.0 0 "deferred" + NODE gbuf_n f16x8 + NODE gbuf_p f32x4 + NODE depth depth24 + + EFFECT + DeferredRender source depth -> gbuf_n gbuf_p 0.0 10.0 + EFFECT + Compose gbuf_n gbuf_p -> sink 0.0 10.0 +``` + +**Ping-pong (auto-optimized):** +``` +SEQUENCE 0.0 0 "blur" + # Compiler detects alternating pattern, aliases temp2 -> temp1 + EFFECT + BlurH source -> temp1 0.0 10.0 + EFFECT + BlurV temp1 -> temp2 0.0 10.0 + EFFECT + Sharpen temp2 -> sink 0.0 10.0 +``` + +## Architecture + +### SequenceV2 Class + +```cpp +class SequenceV2 { + NodeRegistry nodes_; // Typed texture management + std::vector<EffectDAGNode> effect_dag_; // Topologically sorted + UniformsSequenceParams params_; // Per-frame uniforms + + virtual void preprocess(float time, float beat_time, float beat_phase, float audio_intensity); + virtual void render_effects(WGPUCommandEncoder encoder); +}; +``` + +### EffectV2 Class + +```cpp +class EffectV2 { + std::vector<std::string> input_nodes_; + std::vector<std::string> output_nodes_; + + virtual void declare_nodes(NodeRegistry& registry) {} // Optional temp nodes + virtual void render(WGPUCommandEncoder encoder, + const UniformsSequenceParams& params, + NodeRegistry& nodes) = 0; +}; ``` -**Node types:** `u8x4_norm`, `f32x4`, `f16x8`, `depth24`, `compute_f32` +### Node System -**Priority modifiers:** `+` (increment), `=` (same), `-` (decrement) +**Types**: Match WGSL texture formats +- `U8X4_NORM`: RGBA8Unorm (default for source/sink/intermediate) +- `F32X4`: RGBA32Float (HDR, compute outputs) +- `F16X8`: 8-channel float16 (G-buffer normals/vectors) +- `DEPTH24`: Depth24Plus (3D rendering) +- `COMPUTE_F32`: Storage buffer (non-texture compute data) ---- +**Aliasing**: Compiler detects ping-pong patterns (Effect i writes A reads B, Effect i+1 writes B reads A) and aliases nodes to same backing texture. -This document describes the high-level ideas for an improved Sequence + Effects system "version 2". +## Compiler Features -## Goal +**seq_compiler_v2.py** generates optimized C++ from `.seq`: -* more flexibility and less boilerplate -* fully configurable by text (.seq) -* no compatibility with previous version of sequences -* sequence can be stacked (with priority) and arranged in a timeline to create a demo +1. **DAG Validation**: Cycle detection, connectivity checks +2. **Topological Sort**: Execution order from dependencies +3. **Ping-pong Detection**: Automatic node aliasing +4. **Code Generation**: SequenceV2 subclasses with node declarations and effect DAG -## Structure +**Output**: Single `.cc` file with: +- Multiple `SequenceV2` subclasses (one per SEQUENCE) +- `InitializeV2Sequences()` - Registry initialization +- `GetActiveV2Sequence(float time)` - Active sequence lookup +- `RenderV2Timeline()` - Encoder-based and surface-based variants -### Sequence +## Creating Effects -A 'Sequence' unit consist of: - * A predeclared set of named assets and Effects that are used during the sequence - * a start-time, end-time - * a globally-visible set of internal time-dependent params (UniformsSequenceParams) derived from GlobalParams - * a globally-visible set of fixed params values (UniformsSequenceStaticParams) used to configure effects - * an input in RGBAu8 format ("Source"). Black by default (RGB=0,0,0,255). Buffer format: u8x4_norm (see below) - * an output in RGBAu8 format ("Sink"). This output goes to the next sequence in stack or to physical display - * the sink of a Sequence is the source of the next Sequence in the stack, or goes to screen if it's the last sequence in stack - * three programatic sections: 'Preprocess', 'EffectFlow' (time-ordered set of effects), 'Postprocess' - * a set of internal buffers (Nodes = compute buffers or textures, used as input or render targets by effects stack) - * this Nodes are visible to the final post-process effect - * Node persists during the whole sequence - * at compile time, one can detect which Node can actually be ping-pong buffers - * each Sequence's preprocess and postprocess is unique and attached to the sequence +**For standard post-process:** +```cpp +class MyEffectV2 : public EffectV2 { + WGPURenderPipeline pipeline_; + UniformBuffer<UniformsSequenceParams> uniforms_; -### Effects + MyEffectV2(const GpuContext& ctx, const std::vector<std::string>& inputs, + const std::vector<std::string>& outputs) + : EffectV2(ctx, inputs, outputs) { + uniforms_.init(ctx_.device); + pipeline_ = create_post_process_pipeline(ctx_.device, + WGPUTextureFormat_RGBA8Unorm, + my_shader_wgsl); + } -The effect unit consists in: - * An 'Effect' within the Sequence uses the UniformsSequenceParams and GlobalParams to update its internal state - * Receives updated InputNode from the previous effects (InputNode = buffer, textures, g-buffer, ...) - * processes them with a shader or c++ code (3d engine, etc.) - * fills its OutputNode, passed to the next effect or the final postprocessing step - * uses the sequence's assets and params only - * if needed an effect predefines an alias of one of its internal buffer as its declared 'postprocess_output' OutputNode - * by default, postprocess_output = sequence RGBA Source - * Effect's are not attached to a particular Sequence, and can be used in any Sequence + void render(WGPUCommandEncoder encoder, const UniformsSequenceParams& params, + NodeRegistry& nodes) override { + WGPUTextureView input = nodes.get_view(input_nodes_[0]); + WGPUTextureView output = nodes.get_view(output_nodes_[0]); + uniforms_.update(ctx_.queue, params); + // ... render pass + } +}; +``` + +**For 3D effects with depth:** +```cpp +class My3DEffectV2 : public EffectV2 { + std::string depth_node_; + + My3DEffectV2(...) : EffectV2(...), depth_node_(outputs[0] + "_depth") {} + + void declare_nodes(NodeRegistry& registry) override { + registry.declare_node(depth_node_, NodeType::DEPTH24, -1, -1); + } + + void render(...) { + WGPUTextureView color = nodes.get_view(output_nodes_[0]); + WGPUTextureView depth = nodes.get_view(depth_node_); + // ... render pass with depth attachment + } +}; +``` + +## Uniform Access + +```cpp +params.time; // Physical seconds (constant speed) +params.beat_time; // Musical beats (tempo-scaled) +params.beat_phase; // Fractional beat 0.0-1.0 +params.audio_intensity; // Audio peak for beat sync +params.resolution; // vec2(width, height) +params.aspect_ratio; // width/height +``` -### Preprocess: - * the preprocessing step sets up the local UniformsSequenceParams (prepare timers, objects' transforms, camera position, fills Uniform buffers, etc.) - * it's a single function attached to the Sequence unit (virtual method?) +## Status & Limitations -### Postprocess: - * the post-process unique (virtual) function is responsible for the final assembly into OuputNode - * it can take any internal Node as input (most likely the last effects' output) and produces its final Sink content +**Implemented** ✅: +- DAG validation, topological sort, ping-pong optimization +- Multi-input/multi-output effects +- Node aliasing (compile-time optimization) +- 7 effects ported (Passthrough, Placeholder, GaussianBlur, Heptagon, Particles, RotatingCube, Hybrid3D) -## How it works +**Missing/Future** ❌: +- Flatten mode (--flatten generates same code as dev mode) +- BPM handling (parsed but ignored) +- GetDemoDuration() calculation (hardcoded 40.0f) +- Sequence-relative time (uses global time) +- Asset validation (not checked against AssetId enum) +- Node type compatibility checking -### initialization +**TODO**: +- Port remaining effects (10+ effects, CNN effects) +- Implement flatten mode (inline effects, direct members) +- Update HTML timeline editor for v2 graph visualization -at initialization time, the sequence knows from compile-time: - * the list of assets it needs to be ready at start time - * the list of effects needed, their input nodes, their output node - * the list of internal textures, buffer, render target, depth-buffer it will need for its stack effect to work - * which intern Node buffers can actually be ping-pong buffers, to optimize resource - * generally speaking, the lifetime of a buffer during a frame is known at compile time and can be optimized +## Migration Notes -### Flow - * preprocess: prepare internal state at time t: UniformsSequenceParams - * for each effects: - ** bind the InputNode nodes (previous effect buffers, compute buffers, etc.) - ** runs the compute, vertex and fragment passes - ** fills its output Nodes - * process: turns its pre-declared input Nodes into its unique Sink node, ready for next sequence, or display +**V1 → V2 Differences**: +- V1: Implicit framebuffer ping-pong (framebuffer_a_ ↔ framebuffer_b_) +- V2: Explicit node declarations with routing -## Requirement +**Breaking Changes**: +- Effect base class changed (Effect → EffectV2) +- Constructor signature: `(GpuContext, inputs[], outputs[])` +- Render signature: Added `NodeRegistry& nodes` parameter +- No `is_post_process()` method (routing makes it explicit) - * unified code to flow the engine - * textual description of the sequence/effects arrangement (similar to the .seq) - * update the HTML editor tool - * unified description of buffer, texture and compute buffers at compilation time - ** example, of a sequence's internal Node declaration: - "NODE Name1 u8x16_norm" - "NODE Name2 f32x4" - "NODE Name3 f16x8" - "Node Name4 u8x4_norm" <- Source/Sink of a sequence, four bytes - * each effect - ** declares its requirement as input Node. - ** declares its output Node - ** can access the C++ version and WebGPU versions of GlobalParams, UniformsSequenceParams and UniformsSequenceStaticParams - ** are not attached to a particular sequence, but can be used in many - * validation and optimization at compile time (generating c++ code) by seq_compiler - * compilation can generate boilerplate preprocess() / postprocess() functions - * no backward compatibility needed. Sequence v1 can go. - * the HTML Editor overall code, look and functionalities need to be preserved though. Just adapted to v2. - ** basically the sequence box will need to show input Node and output Node and have an editable property panel for these - ** same for effects: ability to edit their input Node and output Node names - * a lot of v1 effects can be deleted (solarized, chroma aberration, etc.): they will be implemented ad-hoc in postprocess() with a single macro call within the final shader - * need a minimal migration plan for code. +**See Also**: +- `doc/EFFECT_WORKFLOW.md` - Step-by-step effect creation +- `tools/seq_compiler_v2.py` - Compiler implementation +- `workspaces/main/timeline_v2.seq` - Example timeline diff --git a/doc/SEQUENCE.md b/doc/archive/SEQUENCE_V1.md index 03d0c45..03d0c45 100644 --- a/doc/SEQUENCE.md +++ b/doc/archive/SEQUENCE_V1.md |
