# Beat-Based Timing System ## Summary **Timeline sequences now use musical beats as the primary time unit**, ensuring visual effects stay synchronized to music structure regardless of BPM changes. Variable tempo only affects audio sample triggering—visual effects run at constant physical time with optional beat-synchronized animation. **Key Change:** `CommonPostProcessUniforms` now provides both `time` (physical seconds) and `beat_time` (absolute beats) + `beat_phase` (fractional 0-1) for flexible animation. --- ## Changes Made ### 1. **Documentation Updated** - `doc/SEQUENCE.md`: Beat-based format as primary, updated runtime parameters - `tools/timeline_editor/README.md`: Beat notation as default ### 2. **Uniform Structure Enhanced** ```cpp struct CommonPostProcessUniforms { vec2 resolution; // Screen dimensions float aspect_ratio; // Width/height ratio float time; // Physical seconds (unaffected by tempo) float beat_time; // NEW: Absolute beats (musical time) float beat_phase; // NEW: Fractional beat 0-1 (was "beat") float audio_intensity; // Audio peak float _pad; // Alignment }; // 32 bytes (unchanged size) ``` ### 3. **Shader Updates** - All `common_uniforms.wgsl` files updated with new field names - Effects can now use: - `time` for physics-based animation (constant speed) - `beat_time` for musical animation (bars/beats) - `beat_phase` for smooth per-beat oscillation ### 4. **Seq Compiler** - `tools/seq_compiler.cc`: Beat notation as default parser behavior - Format: `5` = beats, `2.5s` = explicit seconds - BPM-based conversion at compile time (beats → seconds) ### 5. **Timeline Files Converted** - `workspaces/main/timeline.seq`: Added 's' suffix to preserve timing - `workspaces/test/timeline.seq`: Added 's' suffix to preserve timing - Existing demos run unchanged with explicit seconds notation ### 6. **Runtime Updates** - `main.cc`: Calculates `beat_time` and `beat_phase` from audio time - `gpu.cc`: Passes both physical time and beat time to effects - `effect.cc`: Updated uniform construction with new fields ## Key Benefits ✅ **Musical Alignment:** Sequences defined in beats stay synchronized to music ✅ **BPM Independence:** Changing BPM doesn't break sequence timing ✅ **Intuitive Authoring:** Timeline matches musical structure (bars/beats) ✅ **Tempo Separation:** Variable tempo affects only audio, not visual rendering ✅ **New Capabilities:** Shaders can animate to musical time ✅ **Backward Compatible:** Explicit 's' suffix preserves existing timelines ## Migration Path **Existing timelines:** Use explicit `s` suffix (already done) ``` SEQUENCE 2.50s 0 EFFECT + Flash 0.00s 1.00s ``` **New content:** Use beat notation (natural default) ``` # BPM 120 SEQUENCE 0 0 "Intro" # Beat 0 = bar 1 EFFECT + Flash 0 2 # Beats 0-2 (half bar) EFFECT + Fade 4 8 # Beats 4-8 (full bar) ``` ## Verification **Build:** ✅ Complete (100%) ```bash cmake --build build -j4 ``` **Tests:** ✅ 34/36 passing (94%) ```bash cd build && ctest ``` **Demo Run:** ✅ Verified ``` [GraphicsT=0.32, AudioT=0.13, Beat=0, Phase=0.26, Peak=1.00] [GraphicsT=0.84, AudioT=0.64, Beat=1, Phase=0.28, Peak=0.14] [GraphicsT=1.38, AudioT=1.15, Beat=2, Phase=0.30, Peak=0.92] ``` - Beat counting: ✅ Correct (0→1→2→3...) - Phase tracking: ✅ Correct (fractional 0.0-1.0) - Effect timing: ✅ Sequences start/end at correct times - Shader compilation: ✅ No errors **Commits:** - `89c4687` - feat: implement beat-based timing system - `641b5b6` - fix: update shader files to use beat_phase --- ## Usage Examples ### Timeline Authoring (Beat-Based) ```seq # BPM 120 SEQUENCE 0 0 "Intro (Bar 1)" EFFECT + Flash 0 2 # Beats 0-2 (half bar) EFFECT + Fade 2 4 # Beats 2-4 (second half) SEQUENCE 8 1 "Drop (Bar 3)" EFFECT + Heptagon 0 16 # Full 4 bars (16 beats) EFFECT + Particles 4 12 # Beats 4-12 (2 bars) ``` ### Shader Animation (Musical Time) ```wgsl // Pulse every 4 beats (one bar) let bar_pulse = sin(uniforms.beat_time * TAU / 4.0); // Smooth per-beat oscillation let beat_wave = sin(uniforms.beat_phase * TAU); // Physics-based (constant speed) let rotation = uniforms.time * TAU; ``` ### Legacy Timelines (Explicit Seconds) ```seq SEQUENCE 2.50s 0 EFFECT + Flash 0.00s 1.00s # Preserved timing ``` --- ## Architecture **Timing Separation:** ``` ┌─────────────────┐ │ Platform Clock │ (physical seconds) └────────┬────────┘ │ ┌────┴─────┬──────────────┐ ▼ ▼ ▼ Physical Audio Time Music Time Time (playback) (tempo-scaled) │ │ │ │ └──────┬───────┘ │ ▼ │ Beat Calculation │ (BPM conversion) │ │ └────────┬────────┘ ▼ Visual Effects Rendering (time + beat_time + beat_phase) ``` **Key Insight:** Variable tempo changes `music_time` for audio triggering, but visual effects receive constant `time` (physical) and derived `beat_time` (from audio playback, not music_time). --- ## Technical Details ### Uniform Size Maintained ```cpp // Before (32 bytes): struct { vec2 res; float _pad[2]; float aspect, time, beat, intensity; } // After (32 bytes): struct { vec2 res; float aspect, time, beat_time, beat_phase, intensity, _pad; } ``` ### Beat Calculation ```cpp // main.cc const float absolute_beat_time = current_audio_time * g_tracker_score.bpm / 60.0f; const float beat_phase = fmodf(absolute_beat_time, 1.0f); ``` ### Seq Compiler Logic ```cpp // Default: beats → seconds float beat = std::stof(value); float time = beat * 60.0f / bpm; // Explicit seconds: pass through if (value.back() == 's') return seconds; ``` --- ## Migration Guide **For New Content:** Use beat notation (recommended) ```seq # BPM 140 SEQUENCE 0 0 "Intro" EFFECT + Flash 0 4 # 4 beats = 1.71s @ 140 BPM ``` **For Existing Content:** Already migrated with 's' suffix ```seq SEQUENCE 2.50s 0 # Preserved exact timing EFFECT + Flash 0.00s 1.00s ``` **For Shader Effects:** - Use `uniforms.beat_phase` (not `uniforms.beat`) - Use `uniforms.beat_time` for bar-based animation - Use `uniforms.time` for constant-speed animation --- ## Files Modified **Core System:** - `src/gpu/effects/post_process_helper.h` - Uniform structure - `src/gpu/effect.{h,cc}` - Effect rendering signatures - `src/gpu/gpu.{h,cc}` - GPU draw interface - `src/main.cc`, `src/test_demo.cc` - Beat calculation **Shaders:** - `workspaces/{main,test}/shaders/common_uniforms.wgsl` - `assets/{common,final}/shaders/common_uniforms.wgsl` - All effect shaders using beat: `particle_spray_compute.wgsl`, `ellipse.wgsl` **Timeline Compiler:** - `tools/seq_compiler.cc` - Beat-as-default parser **Timelines:** - `workspaces/main/timeline.seq` - Explicit 's' suffix - `workspaces/test/timeline.seq` - Explicit 's' suffix **Documentation:** - `doc/SEQUENCE.md` - Beat notation format - `tools/timeline_editor/README.md` - Editor usage --- ## Future Enhancements 1. **Beat-Synced Effects:** Create effects that pulse/animate to bars 2. **Timeline Conversion:** Tool to convert explicit seconds → beats 3. **Editor Support:** Timeline editor beat grid visualization 4. **Shader Helpers:** WGSL functions for common beat patterns