diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-12 00:37:49 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-12 00:37:49 +0100 |
| commit | 87329c7cc7c779446d9f6ce56a9204462882fefc (patch) | |
| tree | f8b5626e3d0f29fefb2f59dd577d26cc91b276d7 | |
| parent | 641b5b692ddfce30cc1bcebe742b38e0bba39ce8 (diff) | |
docs: consolidate beat-based timing documentation
- Added comprehensive doc/BEAT_TIMING.md user guide
- Updated BEAT_TIMING_SUMMARY.md with verification results
- Updated PROJECT_CONTEXT.md to highlight timing system
- Updated README.md with doc links
- Included architecture diagrams and examples
- Added troubleshooting section
Complete reference for beat-based timeline authoring and shader
animation with musical timing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
| -rw-r--r-- | BEAT_TIMING_SUMMARY.md | 190 | ||||
| -rw-r--r-- | PROJECT_CONTEXT.md | 5 | ||||
| -rw-r--r-- | README.md | 7 | ||||
| -rw-r--r-- | doc/BEAT_TIMING.md | 272 |
4 files changed, 462 insertions, 12 deletions
diff --git a/BEAT_TIMING_SUMMARY.md b/BEAT_TIMING_SUMMARY.md index 6df8152..e593380 100644 --- a/BEAT_TIMING_SUMMARY.md +++ b/BEAT_TIMING_SUMMARY.md @@ -1,4 +1,12 @@ -# Beat-Based Timing Implementation Summary +# Beat-Based Timing System + +## Summary + +**Timeline sequences now use musical beats as the primary time unit**, ensuring visual effects stay synchronized to music structure regardless of BPM changes. Variable tempo only affects audio sample triggering—visual effects run at constant physical time with optional beat-synchronized animation. + +**Key Change:** `CommonPostProcessUniforms` now provides both `time` (physical seconds) and `beat_time` (absolute beats) + `beat_phase` (fractional 0-1) for flexible animation. + +--- ## Changes Made @@ -66,14 +74,178 @@ SEQUENCE 0 0 "Intro" # Beat 0 = bar 1 EFFECT + Fade 4 8 # Beats 4-8 (full bar) ``` -## Testing +## Verification + +**Build:** ✅ Complete (100%) +```bash +cmake --build build -j4 +``` + +**Tests:** ✅ 34/36 passing (94%) +```bash +cd build && ctest +``` + +**Demo Run:** ✅ Verified +``` +[GraphicsT=0.32, AudioT=0.13, Beat=0, Phase=0.26, Peak=1.00] +[GraphicsT=0.84, AudioT=0.64, Beat=1, Phase=0.28, Peak=0.14] +[GraphicsT=1.38, AudioT=1.15, Beat=2, Phase=0.30, Peak=0.92] +``` +- Beat counting: ✅ Correct (0→1→2→3...) +- Phase tracking: ✅ Correct (fractional 0.0-1.0) +- Effect timing: ✅ Sequences start/end at correct times +- Shader compilation: ✅ No errors + +**Commits:** +- `89c4687` - feat: implement beat-based timing system +- `641b5b6` - fix: update shader files to use beat_phase + +--- + +## Usage Examples + +### Timeline Authoring (Beat-Based) +```seq +# BPM 120 +SEQUENCE 0 0 "Intro (Bar 1)" + EFFECT + Flash 0 2 # Beats 0-2 (half bar) + EFFECT + Fade 2 4 # Beats 2-4 (second half) + +SEQUENCE 8 1 "Drop (Bar 3)" + EFFECT + Heptagon 0 16 # Full 4 bars (16 beats) + EFFECT + Particles 4 12 # Beats 4-12 (2 bars) +``` + +### Shader Animation (Musical Time) +```wgsl +// Pulse every 4 beats (one bar) +let bar_pulse = sin(uniforms.beat_time * TAU / 4.0); + +// Smooth per-beat oscillation +let beat_wave = sin(uniforms.beat_phase * TAU); + +// Physics-based (constant speed) +let rotation = uniforms.time * TAU; +``` + +### Legacy Timelines (Explicit Seconds) +```seq +SEQUENCE 2.50s 0 + EFFECT + Flash 0.00s 1.00s # Preserved timing +``` + +--- + +## Architecture + +**Timing Separation:** +``` +┌─────────────────┐ +│ Platform Clock │ (physical seconds) +└────────┬────────┘ + │ + ┌────┴─────┬──────────────┐ + ▼ ▼ ▼ +Physical Audio Time Music Time + Time (playback) (tempo-scaled) + │ │ │ + │ └──────┬───────┘ + │ ▼ + │ Beat Calculation + │ (BPM conversion) + │ │ + └────────┬────────┘ + ▼ + Visual Effects Rendering + (time + beat_time + beat_phase) +``` + +**Key Insight:** Variable tempo changes `music_time` for audio triggering, but visual effects receive constant `time` (physical) and derived `beat_time` (from audio playback, not music_time). + +--- + +## Technical Details + +### Uniform Size Maintained +```cpp +// Before (32 bytes): +struct { vec2 res; float _pad[2]; float aspect, time, beat, intensity; } + +// After (32 bytes): +struct { vec2 res; float aspect, time, beat_time, beat_phase, intensity, _pad; } +``` + +### Beat Calculation +```cpp +// main.cc +const float absolute_beat_time = current_audio_time * g_tracker_score.bpm / 60.0f; +const float beat_phase = fmodf(absolute_beat_time, 1.0f); +``` + +### Seq Compiler Logic +```cpp +// Default: beats → seconds +float beat = std::stof(value); +float time = beat * 60.0f / bpm; + +// Explicit seconds: pass through +if (value.back() == 's') return seconds; +``` + +--- + +## Migration Guide + +**For New Content:** Use beat notation (recommended) +```seq +# BPM 140 +SEQUENCE 0 0 "Intro" + EFFECT + Flash 0 4 # 4 beats = 1.71s @ 140 BPM +``` + +**For Existing Content:** Already migrated with 's' suffix +```seq +SEQUENCE 2.50s 0 # Preserved exact timing + EFFECT + Flash 0.00s 1.00s +``` + +**For Shader Effects:** +- Use `uniforms.beat_phase` (not `uniforms.beat`) +- Use `uniforms.beat_time` for bar-based animation +- Use `uniforms.time` for constant-speed animation + +--- + +## Files Modified + +**Core System:** +- `src/gpu/effects/post_process_helper.h` - Uniform structure +- `src/gpu/effect.{h,cc}` - Effect rendering signatures +- `src/gpu/gpu.{h,cc}` - GPU draw interface +- `src/main.cc`, `src/test_demo.cc` - Beat calculation + +**Shaders:** +- `workspaces/{main,test}/shaders/common_uniforms.wgsl` +- `assets/{common,final}/shaders/common_uniforms.wgsl` +- All effect shaders using beat: `particle_spray_compute.wgsl`, `ellipse.wgsl` + +**Timeline Compiler:** +- `tools/seq_compiler.cc` - Beat-as-default parser + +**Timelines:** +- `workspaces/main/timeline.seq` - Explicit 's' suffix +- `workspaces/test/timeline.seq` - Explicit 's' suffix + +**Documentation:** +- `doc/SEQUENCE.md` - Beat notation format +- `tools/timeline_editor/README.md` - Editor usage -- **Build:** ✅ Complete (100%) -- **Tests:** ✅ 34/36 passing (94%) -- **Demo:** Ready to run with `./build/demo64k` +--- -## Next Steps +## Future Enhancements -1. Test visual effects with new beat_time parameter -2. Create beat-synchronized shader animations -3. Consider converting existing timelines to beat notation +1. **Beat-Synced Effects:** Create effects that pulse/animate to bars +2. **Timeline Conversion:** Tool to convert explicit seconds → beats +3. **Editor Support:** Timeline editor beat grid visualization +4. **Shader Helpers:** WGSL functions for common beat patterns diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 7e8107e..e57763e 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -31,14 +31,15 @@ ## Current Status +- **Timing System:** **Beat-based timelines** for musical synchronization. Sequences defined in beats, converted to seconds at runtime. Effects receive both physical time (constant) and beat time (musical). Variable tempo affects audio only. See `BEAT_TIMING_SUMMARY.md`. - **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=<name>`. Shared common assets. - **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. Comprehensive tests. -- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Modular WGSL composition. +- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition. - **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline. - **Effects:** CNN post-processing foundation (3-layer architecture, modular snippets). CNNEffect validated in demo. - **Tools:** CNN test tool (readback works, output incorrect - under investigation). Texture readback utility functional. - **Build:** Asset dependency tracking. Size measurement. Hot-reload (debug-only). -- **Testing:** **36/36 passing (100%)** +- **Testing:** **34/36 passing (94%)** --- @@ -12,10 +12,15 @@ cmake --build build -j4 ## Documentation +**Quick Start:** - **PROJECT_CONTEXT.md** - Current status and architecture overview - **TODO.md** - Active tasks and priorities - **doc/HOWTO.md** - Common operations (building, testing, assets) -- **doc/CONTRIBUTING.md** - Development guidelines and protocols - **doc/EFFECT_WORKFLOW.md** - Step-by-step guide for adding visual effects +**Key Features:** +- **BEAT_TIMING_SUMMARY.md** - Beat-based timing system (NEW) +- **doc/BEAT_TIMING.md** - Timeline authoring guide +- **doc/CONTRIBUTING.md** - Development guidelines and protocols + See `doc/` for detailed technical documentation. diff --git a/doc/BEAT_TIMING.md b/doc/BEAT_TIMING.md new file mode 100644 index 0000000..cf7f377 --- /dev/null +++ b/doc/BEAT_TIMING.md @@ -0,0 +1,272 @@ +# Beat-Based Timing System + +## Overview + +The demo uses **beat-based timing** for visual effect sequences, ensuring musical synchronization regardless of BPM changes. All timeline sequences are authored in beats (musical time) and converted to physical seconds at runtime. + +**Key Principle:** Variable tempo only affects audio sample triggering. Visual effects run at constant physical time with optional beat-synchronized animation. + +--- + +## Quick Start + +### Timeline Authoring +```seq +# BPM 120 +SEQUENCE 0 0 "Intro" # Beat 0 (bar 1) + EFFECT + Flash 0 2 # Beats 0-2 (half bar) + EFFECT + Fade 4 8 # Beats 4-8 (full bar) + +SEQUENCE 16 1 "Drop" # Beat 16 (bar 5) + EFFECT + Heptagon 0 16 # 4 bars +``` + +**Conversion:** At 120 BPM, 1 beat = 0.5 seconds, 4 beats = 2 seconds + +### Shader Animation +```wgsl +@group(0) @binding(2) var<uniform> uniforms: CommonUniforms; + +// Use beat_time for musical animation +let bar_cycle = uniforms.beat_time / 4.0; // Bars +let pulse = sin(bar_cycle * TAU); + +// Use beat_phase for smooth per-beat effects +let wave = sin(uniforms.beat_phase * TAU); + +// Use time for constant-speed physics +let rotation = uniforms.time * TAU; +``` + +--- + +## Uniform Structure + +All effects receive `CommonPostProcessUniforms` with timing data: + +```cpp +struct CommonPostProcessUniforms { + vec2 resolution; // Screen dimensions (pixels) + float aspect_ratio; // Width/height ratio + float time; // Physical seconds (constant, unaffected by tempo) + float beat_time; // Absolute beats (musical time from audio clock) + float beat_phase; // Fractional beat 0.0-1.0 (smooth oscillation) + float audio_intensity; // Audio peak for beat sync + float _pad; // Alignment padding +}; // 32 bytes +``` + +**Use Cases:** +- `time`: Physics animation, constant-speed rotation/movement +- `beat_time`: Bar-based patterns, musical synchronization +- `beat_phase`: Smooth per-beat pulse/wave effects + +--- + +## Timeline Format + +### Time Notation + +**Default:** Beats (no suffix) +```seq +SEQUENCE 0 0 # Beat 0 + EFFECT + Flash 0 4 # Beats 0-4 +``` + +**Explicit Seconds:** Use `s` suffix (rare) +```seq +SEQUENCE 2.5s 0 # 2.5 physical seconds + EFFECT + Flash 0 4 # Still uses beats for duration +``` + +**Explicit Beats:** Use `b` suffix (optional clarity) +```seq +SEQUENCE 8b 0 # Same as "8" + EFFECT + Flash 0b 4b # Same as "0 4" +``` + +### BPM Declaration + +**Required** in all timeline files: +```seq +# BPM 120 +``` + +Specifies beats per minute for runtime conversion to seconds. + +--- + +## Architecture + +### Timing Flow + +``` +Platform Clock (physical seconds) + │ + ├──► Physical Time ────────┐ + │ (constant) │ + │ │ + └──► Audio Time ────┐ │ + (playback) │ │ + ▼ │ + Beat Calculation │ + (BPM * 60) │ + │ │ + ▼ ▼ + Visual Effects Rendering + (time + beat_time + beat_phase) +``` + +### Key Insight + +**Variable tempo changes `music_time`** (used for audio event triggering), but **visual effects receive `time` (physical)** and **`beat_time` (from audio playback clock)**, not from tempo-scaled music time. + +This separation ensures: +- ✅ Visual effects run at constant frame rate +- ✅ Beat-synced animations track actual audio playback +- ✅ Tempo changes don't cause visual stuttering + +--- + +## Implementation + +### Beat Calculation (Runtime) + +```cpp +// main.cc - Calculate from audio playback time +const float absolute_beat_time = current_audio_time * g_tracker_score.bpm / 60.0f; +const float beat_phase = fmodf(absolute_beat_time, 1.0f); + +// Pass to GPU rendering +gpu_draw(visual_peak, aspect_ratio, physical_time, absolute_beat_time, beat_phase); +``` + +### Timeline Compilation + +```cpp +// seq_compiler.cc - Convert beats to seconds at compile time +std::string convert_to_time(const std::string& value, float bpm) { + if (value.back() == 's') return explicit_seconds; // Pass through + + // Default: treat as beats + float beat = std::stof(value); + float time = beat * 60.0f / bpm; + return time; +} +``` + +**Result:** Generated `timeline.cc` contains physical seconds for effect activation. + +--- + +## Migration + +### Existing Timelines + +Already migrated with explicit `s` suffix to preserve timing: +```seq +SEQUENCE 2.50s 0 # Physical seconds preserved + EFFECT + Flash 0.00s 1.00s +``` + +### New Content + +Use beat notation (recommended): +```seq +# BPM 140 +SEQUENCE 0 0 "Intro" + EFFECT + Flash 0 4 # 4 beats = 1.71s @ 140 BPM + EFFECT + Fade 4 8 # 4 beats = 1.71s +``` + +**Benefits:** +- Natural musical alignment (bars/beats) +- BPM changes don't break timing +- Easier to author to music + +--- + +## Examples + +### Four-Bar Pattern +```seq +# BPM 120 +SEQUENCE 0 0 "Verse 1" + EFFECT - Background 0 16 # 4 bars background + EFFECT + Flash 0 1 # First beat flash + EFFECT + Pulse 4 5 # Second bar pulse + EFFECT + Fade 15 16 # Final beat fade +``` + +### Multi-Bar Sequence +```seq +SEQUENCE 16 0 "Chorus" # Bar 5 + EFFECT + Heptagon 0 32 # 8 bars (full chorus) + EFFECT + Particles 8 24 # Bars 7-11 (middle) +``` + +### Beat-Synced Shader +```wgsl +fn fragment_main(@location(0) uv: vec2<f32>) -> @location(0) vec4<f32> { + // Pulse every bar (4 beats) + let bar_phase = fract(uniforms.beat_time / 4.0); + let bar_pulse = smoothstep(0.0, 0.1, bar_phase) * + (1.0 - smoothstep(0.9, 1.0, bar_phase)); + + // Smooth per-beat wave + let beat_wave = sin(uniforms.beat_phase * TAU); + + // Combine + let intensity = bar_pulse * 0.5 + beat_wave * 0.3; + return vec4<f32>(color * intensity, 1.0); +} +``` + +--- + +## Troubleshooting + +### Shader Compilation Error: "invalid accessor 'beat'" + +**Cause:** Old shader using `uniforms.beat` (deprecated field) + +**Fix:** Use `uniforms.beat_phase` or `uniforms.beat_time` + +```wgsl +// OLD (error): +let x = uniforms.beat; + +// NEW: +let x = uniforms.beat_phase; // For 0-1 fractional +let y = uniforms.beat_time; // For absolute beats +``` + +### Timeline Parse Error + +**Cause:** Missing BPM declaration + +**Fix:** Add BPM at top of file +```seq +# BPM 120 ← Required +SEQUENCE 0 0 +``` + +### Effects Start at Wrong Time + +**Cause:** Mixing beats and seconds without explicit suffixes + +**Fix:** Be explicit +```seq +SEQUENCE 8 0 # 8 beats (not 8 seconds) +SEQUENCE 8s 0 # 8 seconds (explicit) +SEQUENCE 8b 0 # 8 beats (explicit, same as first) +``` + +--- + +## See Also + +- **Format Reference:** `doc/SEQUENCE.md` - Complete .seq syntax +- **Implementation:** `BEAT_TIMING_SUMMARY.md` - Technical details +- **Effect Creation:** `doc/EFFECT_WORKFLOW.md` - Adding new effects +- **Timeline Editor:** `tools/timeline_editor/README.md` - Visual editing |
