path: root/doc/SEQUENCE_v2.md

# Sequence v2: DAG-based Effect Routing

**Status:** ✅ Operational (Phase 4 complete, v1 removed)

Explicit node system with DAG effect routing. Replaces v1 implicit framebuffer ping-pong.

## Quick Start

```bash
# Compile timeline
python3 tools/seq_compiler_v2.py workspaces/main/timeline_v2.seq --output src/generated/timeline.cc

# Flatten mode (future: inline effects, no vtables)
python3 tools/seq_compiler_v2.py timeline.seq --output timeline.cc --flatten
```

## Timeline Syntax

```
# BPM 120 (optional, currently ignored)

SEQUENCE <start_time> <priority> ["name"]
  # Node declarations (optional, auto-inferred as u8x4_norm)
  NODE temp1 u8x4_norm
  NODE depth depth24
  NODE gbuf_normal f16x8

  # Asset dependencies (validated at compile-time)
  ASSET shader_blur

  # Effect routing with priority modifier
  EFFECT <+|=|-> EffectClass input1 input2 -> output1 output2 start end [params]
```

**Priority modifiers** (REQUIRED):
- `+` : Increment priority (foreground)
- `=` : Same priority as previous
- `-` : Decrement priority (background)

**Node types**: `u8x4_norm` (default), `f32x4`, `f16x8`, `depth24`, `compute_f32`

**Reserved nodes**: `source` (input), `sink` (output)

### Examples

**Simple chain:**
```
SEQUENCE 0.0 0 "basic"
  EFFECT + HeptagonEffect source -> temp1 0.0 10.0
  EFFECT + GaussianBlur temp1 -> sink 0.0 10.0
```

**Multi-output (G-buffer):**
```
SEQUENCE 0.0 0 "deferred"
  NODE gbuf_n f16x8
  NODE gbuf_p f32x4
  NODE depth depth24

  EFFECT + DeferredRender source depth -> gbuf_n gbuf_p 0.0 10.0
  EFFECT + Compose gbuf_n gbuf_p -> sink 0.0 10.0
```

**Ping-pong (auto-optimized):**
```
SEQUENCE 0.0 0 "blur"
  # Compiler detects alternating pattern, aliases temp2 -> temp1
  EFFECT + BlurH source -> temp1 0.0 10.0
  EFFECT + BlurV temp1 -> temp2 0.0 10.0
  EFFECT + Sharpen temp2 -> sink 0.0 10.0
```

## Architecture

### SequenceV2 Class

```cpp
class SequenceV2 {
  NodeRegistry nodes_;              // Typed texture management
  std::vector<EffectDAGNode> effect_dag_;  // Topologically sorted
  UniformsSequenceParams params_;   // Per-frame uniforms

  virtual void preprocess(float time, float beat_time, float beat_phase, float audio_intensity);
  virtual void render_effects(WGPUCommandEncoder encoder);
};
```

### EffectV2 Class

```cpp
class EffectV2 {
  std::vector<std::string> input_nodes_;
  std::vector<std::string> output_nodes_;

  virtual void declare_nodes(NodeRegistry& registry) {}  // Optional temp nodes
  virtual void render(WGPUCommandEncoder encoder,
                      const UniformsSequenceParams& params,
                      NodeRegistry& nodes) = 0;
};
```

### Node System

**Types**: Match WGSL texture formats
- `U8X4_NORM`: RGBA8Unorm (default for source/sink/intermediate)
- `F32X4`: RGBA32Float (HDR, compute outputs)
- `F16X8`: 8-channel float16 (G-buffer normals/vectors)
- `DEPTH24`: Depth24Plus (3D rendering)
- `COMPUTE_F32`: Storage buffer (non-texture compute data)

**Aliasing**: Compiler detects ping-pong patterns (Effect i writes A reads B, Effect i+1 writes B reads A) and aliases nodes to same backing texture.

## Compiler Features

**seq_compiler_v2.py** generates optimized C++ from `.seq`:

1. **DAG Validation**: Cycle detection, connectivity checks
2. **Topological Sort**: Execution order from dependencies
3. **Ping-pong Detection**: Automatic node aliasing
4. **Code Generation**: SequenceV2 subclasses with node declarations and effect DAG

**Output**: Single `.cc` file with:
- Multiple `SequenceV2` subclasses (one per SEQUENCE)
- `InitializeV2Sequences()` - Registry initialization
- `GetActiveV2Sequence(float time)` - Active sequence lookup
- `RenderV2Timeline()` - Encoder-based and surface-based variants

## Creating Effects

**For standard post-process:**
```cpp
class MyEffectV2 : public EffectV2 {
  WGPURenderPipeline pipeline_;
  UniformBuffer<UniformsSequenceParams> uniforms_;

  MyEffectV2(const GpuContext& ctx, const std::vector<std::string>& inputs,
             const std::vector<std::string>& outputs)
    : EffectV2(ctx, inputs, outputs) {
    uniforms_.init(ctx_.device);
    pipeline_ = create_post_process_pipeline(ctx_.device,
                                             WGPUTextureFormat_RGBA8Unorm,
                                             my_shader_wgsl);
  }

  void render(WGPUCommandEncoder encoder, const UniformsSequenceParams& params,
              NodeRegistry& nodes) override {
    WGPUTextureView input = nodes.get_view(input_nodes_[0]);
    WGPUTextureView output = nodes.get_view(output_nodes_[0]);
    uniforms_.update(ctx_.queue, params);
    // ... render pass
  }
};
```

**For 3D effects with depth:**
```cpp
class My3DEffectV2 : public EffectV2 {
  std::string depth_node_;

  My3DEffectV2(...) : EffectV2(...), depth_node_(outputs[0] + "_depth") {}

  void declare_nodes(NodeRegistry& registry) override {
    registry.declare_node(depth_node_, NodeType::DEPTH24, -1, -1);
  }

  void render(...) {
    WGPUTextureView color = nodes.get_view(output_nodes_[0]);
    WGPUTextureView depth = nodes.get_view(depth_node_);
    // ... render pass with depth attachment
  }
};
```

## Uniform Access

```cpp
params.time;            // Physical seconds (constant speed)
params.beat_time;       // Musical beats (tempo-scaled)
params.beat_phase;      // Fractional beat 0.0-1.0
params.audio_intensity; // Audio peak for beat sync
params.resolution;      // vec2(width, height)
params.aspect_ratio;    // width/height
```

## Status & Limitations

**Implemented** ✅:
- DAG validation, topological sort, ping-pong optimization
- Multi-input/multi-output effects
- Node aliasing (compile-time optimization)
- 7 effects ported (Passthrough, Placeholder, GaussianBlur, Heptagon, Particles, RotatingCube, Hybrid3D)

**Missing/Future** ❌:
- Flatten mode (--flatten generates same code as dev mode)
- BPM handling (parsed but ignored)
- GetDemoDuration() calculation (hardcoded 40.0f)
- Sequence-relative time (uses global time)
- Asset validation (not checked against AssetId enum)
- Node type compatibility checking

**TODO**:
- Port remaining effects (10+ effects, CNN effects)
- Implement flatten mode (inline effects, direct members)
- Update HTML timeline editor for v2 graph visualization

## Migration Notes

**V1 → V2 Differences**:
- V1: Implicit framebuffer ping-pong (framebuffer_a_ ↔ framebuffer_b_)
- V2: Explicit node declarations with routing

**Breaking Changes**:
- Effect base class changed (Effect → EffectV2)
- Constructor signature: `(GpuContext, inputs[], outputs[])`
- Render signature: Added `NodeRegistry& nodes` parameter
- No `is_post_process()` method (routing makes it explicit)

**See Also**:
- `doc/EFFECT_WORKFLOW.md` - Step-by-step effect creation
- `tools/seq_compiler_v2.py` - Compiler implementation
- `workspaces/main/timeline_v2.seq` - Example timeline