diff options
| -rw-r--r-- | README.md | 1 | ||||
| -rw-r--r-- | assets/demo.seq | 137 | ||||
| -rw-r--r-- | doc/SEQUENCE.md | 325 |
3 files changed, 336 insertions, 127 deletions
@@ -21,6 +21,7 @@ cmake --build build - `PHASE2_COMPRESSION.md`: Details on future asset compression strategies. - `PROCEDURAL.md`: Details on the procedural texture generation system. - `PROJECT_CONTEXT.md`: A comprehensive overview of the project's architecture, features, and current state. +- `SEQUENCE.md`: Complete reference for `.seq` file format and timeline system. - `SPEC_EDITOR.md`: Plans for a future web-based spectrogram editor tool. - `3D.md`: Overview of the 3D rendering pipeline and future enhancements. - `TODO.md`: A list of immediate, small-scale tasks. diff --git a/assets/demo.seq b/assets/demo.seq index 0d73374..35935a7 100644 --- a/assets/demo.seq +++ b/assets/demo.seq @@ -1,139 +1,22 @@ # ============================================================================ # DEMO SEQUENCE DEFINITION # ============================================================================ -# This file defines the timeline and layering of visual effects. +# Defines the timeline and layering of visual effects for the demo. # Compiled by seq_compiler into src/generated/timeline.cc at build time. # -# DEBUGGING & VALIDATION: -# Validate without generating code: -# ./build/seq_compiler assets/demo.seq +# DOCUMENTATION: See doc/SEQUENCE.md for complete syntax reference # -# Generate ASCII Gantt chart: -# ./build/seq_compiler assets/demo.seq --gantt=timeline.txt +# QUICK REFERENCE: +# SEQUENCE <start> <priority> [optional_end] +# EFFECT <ClassName> <start> <end> <priority> # -# Generate HTML/SVG Gantt chart (recommended): -# ./build/seq_compiler assets/demo.seq --gantt-html=timeline.html -# Open timeline.html in browser for interactive visualization +# Time notation: 0b (beats), 0.0 (seconds) +# Priority: Higher numbers render later (on top) # -# All modes can be combined: -# ./build/seq_compiler assets/demo.seq timeline.cc --gantt=t.txt --gantt-html=t.html +# VALIDATION & VISUALIZATION: +# ./build/seq_compiler assets/demo.seq # Validate only +# ./build/seq_compiler assets/demo.seq --gantt-html=t.html # HTML Gantt # -# BPM 120 -# -# -# SYNTAX: -# -# END_DEMO <time> -# Specifies when the demo should automatically exit (optional) -# If not specified, demo runs indefinitely until user closes window -# -# SEQUENCE <global_start> <priority> [optional_end] -# EFFECT <EffectClassName> <local_start> <local_end> <priority> [constructor_args...] -# -# Optional sequence end time: -# - Syntax: [time] (must be wrapped in brackets) -# - If specified, all effects in the sequence will be forcefully ended at this time -# - If not specified, effects run until their individual end times -# - Example: SEQUENCE 0 0 [30.0] ends the entire sequence at 30 seconds -# -# TIME NOTATION: -# - Beat numbers: "0", "64", "128" (no decimal point = beats) -# - Beat numbers with 'b': "0b", "64b", "128b" (explicit beat notation) -# - Seconds: "0.0", "32.0", "64.0" (decimal point = seconds) -# - Seconds with 's': "32.0s", "64.0s" (explicit seconds notation) -# Examples at 120 BPM: -# - Beat 0 = 0.0 seconds -# - Beat 64 = 32.0 seconds -# - Beat 120 = 60.0 seconds -# -# SEQUENCE: -# - global_start: When this sequence starts (beats or seconds) -# - priority: Render order between sequences (higher = rendered later/on top) -# Use 0-9 for scene effects, 10+ for post-processing -# - [optional_end]: Optional end time in brackets [time]. If specified, all effects -# in the sequence will be forcefully ended at this time (relative -# to the sequence start). If omitted, effects run until their -# individual end times. -# -# EFFECT: -# - EffectClassName: C++ class name (must be declared in demo_effects.h) -# - local_start: Start time relative to sequence start (beats or seconds) -# - local_end: End time relative to sequence start (beats or seconds) -# - priority: Render order within sequence (lower = drawn first/background) -# Negative priorities draw behind everything -# - constructor_args: Optional additional constructor parameters (rarely used) -# -# Effect Constructor Parameters: -# ALL effects automatically receive these standard parameters: -# - WGPUDevice device: WebGPU device handle -# - WGPUQueue queue: Command queue for GPU operations -# - WGPUTextureFormat format: Surface texture format -# -# Most effects only use these standard parameters and don't require additional args. -# If an effect needs custom initialization parameters, they are specified after -# the priority field. Example (hypothetical): -# EFFECT CustomEffect 0 10 0 1.5 "param" -# This would translate to: -# std::make_shared<CustomEffect>(device, queue, format, 1.5, "param") -# -# Runtime Parameters (passed to render() method): -# All effects receive these dynamic parameters every frame: -# - time: Global time in seconds (from demo start) -# - beat: Current beat fraction (0.0 to 1.0 within each beat) -# - intensity: Audio peak intensity (0.0 to 1.0, for beat detection) -# - aspect_ratio: Screen width/height ratio -# -# Currently available effects (all use standard constructor only): -# -# Scene Effects (render 3D geometry): -# - HeptagonEffect: Animated geometric heptagon shape -# Uses: time (animation), beat (rotation), aspect_ratio (projection) -# - ParticlesEffect: GPU-simulated particle system -# Uses: time (physics), intensity (emission rate) -# - Hybrid3DEffect: 3D objects with camera movement -# Uses: time (camera paths), beat (object animation), aspect_ratio -# - FlashCubeEffect: Large background cube with Perlin noise texture -# Uses: time (rotation), intensity (flash trigger on beat hits) -# -# Post-Process Effects (full-screen shaders, applied in priority order): -# - GaussianBlurEffect: Gaussian blur with beat pulsation -# Uses: intensity (blur size pulsates 0.5x-2.5x based on audio peak) -# - SolarizeEffect: Color inversion with alternating red/blue tints -# Uses: time (alternates every 2 seconds between color schemes) -# - ChromaAberrationEffect: RGB channel separation -# Uses: time (animation), intensity (separation amount) -# - ThemeModulationEffect: Brightness cycling (bright/dark alternation) -# Uses: time (cycles every 8 seconds: 1.0 bright to 0.35 dark) -# - FadeEffect: Fade to/from black -# Uses: time (fades in first 2s, fades out after 36s - hardcoded) -# - FlashEffect: White flash on strong beat hits -# Uses: intensity (triggers flash when > 0.7, exponential decay) -# -# TIPS: -# - Scene effects render to framebuffer with depth testing -# - Post-processing effects are full-screen shaders applied in order -# - Use negative priority for background elements (skybox, far objects) -# - Higher sequence priority = later in post-process chain -# -# EXAMPLES: -# # Basic sequence starting at beat 0 with priority 0 -# SEQUENCE 0 0 -# EFFECT FlashEffect 0.0 0.5 1 # Starts immediately, runs 0.5s, priority 1 -# EFFECT HeptagonEffect 0.2 10 0 # Starts at 0.2s, runs until 10s, priority 0 -# -# # Sequence with explicit end time - all effects terminated at 5 seconds -# SEQUENCE 8b 0 [5.0] -# EFFECT ParticlesEffect 0 120 1 # Would run 120s, but sequence ends at 5s -# -# # Post-processing chain (high priority renders last/on top) -# SEQUENCE 0 10 -# EFFECT GaussianBlurEffect 0 60 1 # Applied first -# EFFECT ChromaAberrationEffect 0 60 2 # Applied second -# EFFECT SolarizeEffect 0 60 3 # Applied last -# -# # Background element with negative priority (renders behind everything) -# SEQUENCE 0 0 -# EFFECT FlashCubeEffect 0 10 -1 # priority -1 = background layer # ============================================================================ SEQUENCE 0b 0 diff --git a/doc/SEQUENCE.md b/doc/SEQUENCE.md new file mode 100644 index 0000000..05cdc93 --- /dev/null +++ b/doc/SEQUENCE.md @@ -0,0 +1,325 @@ +# Sequence System Documentation + +This document describes the `.seq` file format used to define demo timelines. + +## Overview + +Sequence files (`.seq`) define the timeline and layering of visual effects in the demo. They are compiled by `seq_compiler` into C++ code at build time. + +## File Location + +- Demo sequence: `assets/demo.seq` +- Compiler: `tools/seq_compiler.cc` +- Generated output: `src/generated/timeline.cc` + +## Command Line Usage + +### Basic Compilation +```bash +./build/seq_compiler assets/demo.seq src/generated/timeline.cc +``` + +### Validation Only +```bash +./build/seq_compiler assets/demo.seq +``` +Validates syntax without generating code. + +### Gantt Chart Visualization + +**ASCII Gantt Chart:** +```bash +./build/seq_compiler assets/demo.seq --gantt=timeline.txt +``` + +**HTML/SVG Gantt Chart (Recommended):** +```bash +./build/seq_compiler assets/demo.seq --gantt-html=timeline.html +``` +Open `timeline.html` in browser for interactive visualization with hover tooltips. + +**Combined Modes:** +```bash +./build/seq_compiler assets/demo.seq timeline.cc --gantt=t.txt --gantt-html=t.html +``` + +--- + +## Syntax Reference + +### BPM Declaration +``` +# BPM 120 +``` +Specifies beats per minute for the demo. Used to convert beat notation to seconds. + +### END_DEMO Directive +``` +END_DEMO <time> +``` +Specifies when the demo should automatically exit (optional). +- If not specified, demo runs indefinitely until user closes window +- Supports beat notation (e.g., `64b`) or seconds (e.g., `32.0`) + +### SEQUENCE Declaration +``` +SEQUENCE <global_start> <priority> [optional_end] + EFFECT <EffectClassName> <local_start> <local_end> <priority> [constructor_args...] +``` + +**Parameters:** +- `global_start`: When this sequence starts (beats or seconds) +- `priority`: Render order between sequences (higher = rendered later/on top) + - Use 0-9 for scene effects + - Use 10+ for post-processing +- `[optional_end]`: Optional end time in brackets (e.g., `[30.0]`) + - If specified, all effects in the sequence will be forcefully ended at this time + - Time is relative to the sequence start + - If omitted, effects run until their individual end times + +### EFFECT Declaration +``` +EFFECT <EffectClassName> <local_start> <local_end> <priority> [constructor_args...] +``` + +**Parameters:** +- `EffectClassName`: C++ class name (must be declared in `demo_effects.h`) +- `local_start`: Start time relative to sequence start (beats or seconds) +- `local_end`: End time relative to sequence start (beats or seconds) +- `priority`: Render order within sequence (lower = drawn first/background) + - Negative priorities draw behind everything +- `constructor_args`: Optional additional parameters (rarely used) + +--- + +## Time Notation + +The sequence file supports flexible time notation: + +| Notation | Example | Description | +|----------|---------|-------------| +| Integer beats | `0`, `64`, `128` | No decimal point = beats | +| Explicit beats | `0b`, `64b`, `128b` | Suffix 'b' = beats | +| Decimal seconds | `0.0`, `32.0`, `64.0` | Decimal point = seconds | +| Explicit seconds | `32.0s`, `64.0s` | Suffix 's' = seconds | + +**Examples at 120 BPM:** +- Beat 0 = 0.0 seconds +- Beat 64 = 32.0 seconds +- Beat 120 = 60.0 seconds + +--- + +## Effect System + +### Constructor Parameters + +**Standard Parameters (ALL effects receive these automatically):** +- `WGPUDevice device`: WebGPU device handle +- `WGPUQueue queue`: Command queue for GPU operations +- `WGPUTextureFormat format`: Surface texture format + +Most effects only use these standard parameters. If an effect needs custom initialization, parameters are specified after the priority field. + +**Example (hypothetical custom effect):** +``` +EFFECT CustomEffect 0 10 0 1.5 "param" +``` +Translates to: +```cpp +std::make_shared<CustomEffect>(device, queue, format, 1.5, "param") +``` + +### Runtime Parameters + +All effects receive these dynamic parameters every frame in their `render()` method: + +| Parameter | Type | Description | +|-----------|------|-------------| +| `time` | float | Global time in seconds (from demo start) | +| `beat` | float | Current beat fraction (0.0 to 1.0 within each beat) | +| `intensity` | float | Audio peak intensity (0.0 to 1.0, for beat detection) | +| `aspect_ratio` | float | Screen width/height ratio | + +--- + +## Available Effects + +### Scene Effects +Render 3D geometry to the framebuffer with depth testing. + +#### HeptagonEffect +Animated geometric heptagon shape. +- **Uses:** `time` (animation), `beat` (rotation), `aspect_ratio` (projection) + +#### ParticlesEffect +GPU-simulated particle system. +- **Uses:** `time` (physics), `intensity` (emission rate) + +#### Hybrid3DEffect +3D objects with camera movement. +- **Uses:** `time` (camera paths), `beat` (object animation), `aspect_ratio` + +#### FlashCubeEffect +Large background cube with Perlin noise texture. +- **Uses:** `time` (rotation), `intensity` (flash trigger on beat hits) +- **Note:** Typically used with priority -1 for background layer + +### Post-Process Effects +Full-screen shaders applied in priority order after scene rendering. + +#### GaussianBlurEffect +Gaussian blur with beat pulsation. +- **Uses:** `intensity` (blur size pulsates 0.5x-2.5x based on audio peak) + +#### SolarizeEffect +Color inversion with alternating red/blue tints. +- **Uses:** `time` (alternates every 2 seconds between color schemes) + +#### ChromaAberrationEffect +RGB channel separation effect. +- **Uses:** `time` (animation), `intensity` (separation amount) + +#### ThemeModulationEffect +Brightness cycling (bright/dark alternation). +- **Uses:** `time` (cycles every 8 seconds: 1.0 bright to 0.35 dark) + +#### FadeEffect +Fade to/from black. +- **Uses:** `time` (fades in first 2s, fades out after 36s - hardcoded timing) + +#### FlashEffect +White flash on strong beat hits. +- **Uses:** `intensity` (triggers flash when > 0.7, exponential decay) + +--- + +## Tips & Best Practices + +### Layering +- Scene effects render to framebuffer with depth testing +- Post-processing effects are full-screen shaders applied in sequence priority order +- Use negative priority for background elements (skybox, far objects) +- Higher sequence priority = later in post-process chain (rendered on top) + +### Priority Guidelines +- **Scene Effects:** priority 0-9 + - Background: -1 (e.g., skybox, distant objects) + - Midground: 0-5 (main geometry) + - Foreground: 6-9 (overlays, particles) +- **Post-Process:** priority 10+ + - Apply in order: blur → distortion → color grading + +### Timing +- Use beat notation for music-synchronized effects +- Use seconds for absolute timing +- Keep effect durations reasonable (avoid very short < 0.1s unless intentional) +- Use sequence `[end_time]` to forcefully terminate long-running effects + +--- + +## Examples + +### Basic Sequence +``` +SEQUENCE 0 0 + EFFECT FlashEffect 0.0 0.5 1 # Starts immediately, runs 0.5s, priority 1 + EFFECT HeptagonEffect 0.2 10 0 # Starts at 0.2s, runs until 10s, priority 0 +``` + +### Sequence with Explicit End Time +``` +SEQUENCE 8b 0 [5.0] + EFFECT ParticlesEffect 0 120 1 # Would run 120s, but sequence ends at 5s +``` + +### Post-Processing Chain +``` +SEQUENCE 0 10 + EFFECT GaussianBlurEffect 0 60 1 # Applied first + EFFECT ChromaAberrationEffect 0 60 2 # Applied second + EFFECT SolarizeEffect 0 60 3 # Applied last +``` + +### Background Element +``` +SEQUENCE 0 0 + EFFECT FlashCubeEffect 0 10 -1 # priority -1 = background layer +``` + +### Music-Synchronized Effects +``` +# BPM 120 +SEQUENCE 0b 0 # Start at beat 0 (0.0s) + EFFECT FlashEffect 0b 1b 1 # Flash from beat 0 to beat 1 (0-0.5s) + EFFECT HeptagonEffect 4b 8b 0 # Main effect from beat 4 to 8 (2-4s) +``` + +--- + +## Debugging & Visualization + +### Gantt Charts + +Gantt charts help visualize the timeline structure: + +**ASCII Chart (Terminal-Friendly):** +``` +Time (s): 0 5 10 15 20 + |----|----|----|----|----| + +SEQ@0s [pri=0] + ████████████████████████ (0-20s) + FlashEffect [pri=1] + ▓▓·················· (0-1s) + HeptagonEffect [pri=0] + ▓▓▓▓▓▓▓▓▓▓▓▓········ (0-10s) +``` + +**HTML/SVG Chart (Interactive):** +- Color-coded sequences and effects +- Hover tooltips with full details +- Invalid time ranges highlighted in red +- Professional dark theme +- Zoom/pan support +- Easy to share or present + +### Validation + +Run validation without code generation to catch syntax errors: +```bash +./build/seq_compiler assets/demo.seq +# Output: "Validation successful: 14 sequences, explicit end time." +``` + +### Common Issues Caught by Validation +- Invalid time ranges (end < start) +- Missing effect classes +- Syntax errors +- Malformed time notation + +--- + +## Integration with Build System + +The sequence compiler is automatically invoked during build via CMake: + +```cmake +add_custom_command( + OUTPUT ${GENERATED_DIR}/timeline.cc + COMMAND seq_compiler assets/demo.seq ${GENERATED_DIR}/timeline.cc + DEPENDS assets/demo.seq + COMMENT "Compiling demo sequence..." +) +``` + +Editing `assets/demo.seq` triggers automatic recompilation and rebuilds affected targets. + +--- + +## See Also + +- `doc/HOWTO.md` - Building and running the demo +- `doc/CONTRIBUTING.md` - Coding guidelines +- `assets/demo.seq` - Current demo timeline +- `src/gpu/demo_effects.h` - Available effect classes |