# Sequence: DAG-based Effect Routing **Status:** ✅ Operational Explicit node system with DAG effect routing. ## Quick Start ```bash # Compile timeline python3 tools/seq_compiler.py workspaces/main/timeline.seq --output src/generated/timeline.cc # Flatten mode (future: inline effects, no vtables) python3 tools/seq_compiler.py timeline.seq --output timeline.cc --flatten ``` ## Timeline Syntax ``` # BPM 120 (optional, currently ignored) SEQUENCE ["name"] # Node declarations (optional, auto-inferred as u8x4_norm) NODE temp1 u8x4_norm NODE depth depth24 NODE gbuf_normal f16x8 # Asset dependencies (validated at compile-time) ASSET shader_blur # Effect routing with priority modifier EFFECT <+|=|-> EffectClass input1 input2 -> output1 output2 start end [params] ``` **Priority modifiers** (REQUIRED): - `+` : Increment priority (foreground) - `=` : Same priority as previous - `-` : Decrement priority (background) **Node types**: `u8x4_norm` (default), `f32x4`, `f16x8`, `depth24`, `compute_f32` **Reserved nodes**: `source` (input), `sink` (output) ### Examples **Simple chain:** ``` SEQUENCE 0.0 0 "basic" EFFECT + HeptagonEffect source -> temp1 0.0 10.0 EFFECT + GaussianBlur temp1 -> sink 0.0 10.0 ``` **Multi-output (G-buffer):** ``` SEQUENCE 0.0 0 "deferred" NODE gbuf_n f16x8 NODE gbuf_p f32x4 NODE depth depth24 EFFECT + DeferredRender source depth -> gbuf_n gbuf_p 0.0 10.0 EFFECT + Compose gbuf_n gbuf_p -> sink 0.0 10.0 ``` **Ping-pong (auto-optimized):** ``` SEQUENCE 0.0 0 "blur" # Compiler detects alternating pattern, aliases temp2 -> temp1 EFFECT + BlurH source -> temp1 0.0 10.0 EFFECT + BlurV temp1 -> temp2 0.0 10.0 EFFECT + Sharpen temp2 -> sink 0.0 10.0 ``` ## Architecture ### Sequence Class ```cpp class Sequence { NodeRegistry nodes_; // Typed texture management std::vector effect_dag_; // Topologically sorted UniformsSequenceParams params_; // Per-frame uniforms virtual void preprocess(float time, float beat_time, float beat_phase, float audio_intensity); virtual void render_effects(WGPUCommandEncoder encoder); }; ``` ### Effect Class ```cpp class Effect { std::vector input_nodes_; std::vector output_nodes_; virtual void declare_nodes(NodeRegistry& registry) {} // Optional temp nodes virtual void render(WGPUCommandEncoder encoder, const UniformsSequenceParams& params, NodeRegistry& nodes) = 0; }; ``` ### Node System **Types**: Match WGSL texture formats - `U8X4_NORM`: RGBA8Unorm (default for source/sink/intermediate) - `F32X4`: RGBA32Float (HDR, compute outputs) - `F16X8`: 8-channel float16 (G-buffer normals/vectors) - `DEPTH24`: Depth24Plus (3D rendering) - `COMPUTE_F32`: Storage buffer (non-texture compute data) **Aliasing**: Compiler detects ping-pong patterns (Effect i writes A reads B, Effect i+1 writes B reads A) and aliases nodes to same backing texture. ## Compiler Features **seq_compiler.py** generates optimized C++ from `.seq`: 1. **DAG Validation**: Cycle detection, connectivity checks 2. **Topological Sort**: Execution order from dependencies 3. **Ping-pong Detection**: Automatic node aliasing 4. **Code Generation**: Sequence subclasses with node declarations and effect DAG **Output**: Single `.cc` file with: - Multiple `Sequence` subclasses (one per SEQUENCE) - `InitializeSequences()` - Registry initialization - `GetActiveSequence(float time)` - Active sequence lookup - `RenderTimeline()` - Encoder-based and surface-based variants ## Creating Effects **For standard post-process:** ```cpp class MyEffect : public Effect { WGPURenderPipeline pipeline_; UniformBuffer uniforms_; MyEffect(const GpuContext& ctx, const std::vector& inputs, const std::vector& outputs); const std::vector& outputs) : Effect(ctx, inputs, outputs) { uniforms_.init(ctx_.device); pipeline_ = create_post_process_pipeline(ctx_.device, WGPUTextureFormat_RGBA8Unorm, my_shader_wgsl); } void render(WGPUCommandEncoder encoder, const UniformsSequenceParams& params, NodeRegistry& nodes) override { WGPUTextureView input = nodes.get_view(input_nodes_[0]); WGPUTextureView output = nodes.get_view(output_nodes_[0]); uniforms_.update(ctx_.queue, params); // ... render pass } }; ``` **For 3D effects with depth:** ```cpp class My3DEffect : public Effect { std::string depth_node_; My3DEffect(...) : Effect(...), depth_node_(outputs[0] + "_depth") {} void declare_nodes(NodeRegistry& registry) override { registry.declare_node(depth_node_, NodeType::DEPTH24, -1, -1); } void render(...) { WGPUTextureView color = nodes.get_view(output_nodes_[0]); WGPUTextureView depth = nodes.get_view(depth_node_); // ... render pass with depth attachment } }; ``` ## Uniform Access ```cpp params.time; // Physical seconds (constant speed) params.beat_time; // Musical beats (tempo-scaled) params.beat_phase; // Fractional beat 0.0-1.0 params.audio_intensity; // Audio peak for beat sync params.resolution; // vec2(width, height) params.aspect_ratio; // width/height ``` ## Status & Limitations **Implemented** ✅: - DAG validation, topological sort, ping-pong optimization - Multi-input/multi-output effects - Node aliasing (compile-time optimization) - 7 effects ported (Passthrough, Placeholder, GaussianBlur, Heptagon, Particles, RotatingCube, Hybrid3D) **Missing/Future** ❌: - Flatten mode (--flatten generates same code as dev mode) - BPM handling (parsed but ignored) - GetDemoDuration() calculation (hardcoded 40.0f) - Sequence-relative time (uses global time) - Asset validation (not checked against AssetId enum) - Node type compatibility checking **TODO**: - Port remaining effects (10+ effects, CNN effects) - Implement flatten mode (inline effects, direct members) - Update HTML timeline editor for graph visualization ## Migration Notes **Key Features**: - V1: Implicit framebuffer ping-pong (framebuffer_a_ ↔ framebuffer_b_) - Explicit node declarations with routing **Breaking Changes**: - Effect base class standardized - Constructor signature: `(GpuContext, inputs[], outputs[])` - Render signature: Added `NodeRegistry& nodes` parameter - No `is_post_process()` method (routing makes it explicit) **See Also**: - `doc/EFFECT_WORKFLOW.md` - Step-by-step effect creation - `tools/seq_compiler.py` - Compiler implementation - `workspaces/main/timeline.seq` - Example timeline