summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-12 00:37:49 +0100
committerskal <pascal.massimino@gmail.com>2026-02-12 00:37:49 +0100
commit87329c7cc7c779446d9f6ce56a9204462882fefc (patch)
treef8b5626e3d0f29fefb2f59dd577d26cc91b276d7
parent641b5b692ddfce30cc1bcebe742b38e0bba39ce8 (diff)
docs: consolidate beat-based timing documentation
- Added comprehensive doc/BEAT_TIMING.md user guide - Updated BEAT_TIMING_SUMMARY.md with verification results - Updated PROJECT_CONTEXT.md to highlight timing system - Updated README.md with doc links - Included architecture diagrams and examples - Added troubleshooting section Complete reference for beat-based timeline authoring and shader animation with musical timing. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
-rw-r--r--BEAT_TIMING_SUMMARY.md190
-rw-r--r--PROJECT_CONTEXT.md5
-rw-r--r--README.md7
-rw-r--r--doc/BEAT_TIMING.md272
4 files changed, 462 insertions, 12 deletions
diff --git a/BEAT_TIMING_SUMMARY.md b/BEAT_TIMING_SUMMARY.md
index 6df8152..e593380 100644
--- a/BEAT_TIMING_SUMMARY.md
+++ b/BEAT_TIMING_SUMMARY.md
@@ -1,4 +1,12 @@
-# Beat-Based Timing Implementation Summary
+# Beat-Based Timing System
+
+## Summary
+
+**Timeline sequences now use musical beats as the primary time unit**, ensuring visual effects stay synchronized to music structure regardless of BPM changes. Variable tempo only affects audio sample triggering—visual effects run at constant physical time with optional beat-synchronized animation.
+
+**Key Change:** `CommonPostProcessUniforms` now provides both `time` (physical seconds) and `beat_time` (absolute beats) + `beat_phase` (fractional 0-1) for flexible animation.
+
+---
## Changes Made
@@ -66,14 +74,178 @@ SEQUENCE 0 0 "Intro" # Beat 0 = bar 1
EFFECT + Fade 4 8 # Beats 4-8 (full bar)
```
-## Testing
+## Verification
+
+**Build:** ✅ Complete (100%)
+```bash
+cmake --build build -j4
+```
+
+**Tests:** ✅ 34/36 passing (94%)
+```bash
+cd build && ctest
+```
+
+**Demo Run:** ✅ Verified
+```
+[GraphicsT=0.32, AudioT=0.13, Beat=0, Phase=0.26, Peak=1.00]
+[GraphicsT=0.84, AudioT=0.64, Beat=1, Phase=0.28, Peak=0.14]
+[GraphicsT=1.38, AudioT=1.15, Beat=2, Phase=0.30, Peak=0.92]
+```
+- Beat counting: ✅ Correct (0→1→2→3...)
+- Phase tracking: ✅ Correct (fractional 0.0-1.0)
+- Effect timing: ✅ Sequences start/end at correct times
+- Shader compilation: ✅ No errors
+
+**Commits:**
+- `89c4687` - feat: implement beat-based timing system
+- `641b5b6` - fix: update shader files to use beat_phase
+
+---
+
+## Usage Examples
+
+### Timeline Authoring (Beat-Based)
+```seq
+# BPM 120
+SEQUENCE 0 0 "Intro (Bar 1)"
+ EFFECT + Flash 0 2 # Beats 0-2 (half bar)
+ EFFECT + Fade 2 4 # Beats 2-4 (second half)
+
+SEQUENCE 8 1 "Drop (Bar 3)"
+ EFFECT + Heptagon 0 16 # Full 4 bars (16 beats)
+ EFFECT + Particles 4 12 # Beats 4-12 (2 bars)
+```
+
+### Shader Animation (Musical Time)
+```wgsl
+// Pulse every 4 beats (one bar)
+let bar_pulse = sin(uniforms.beat_time * TAU / 4.0);
+
+// Smooth per-beat oscillation
+let beat_wave = sin(uniforms.beat_phase * TAU);
+
+// Physics-based (constant speed)
+let rotation = uniforms.time * TAU;
+```
+
+### Legacy Timelines (Explicit Seconds)
+```seq
+SEQUENCE 2.50s 0
+ EFFECT + Flash 0.00s 1.00s # Preserved timing
+```
+
+---
+
+## Architecture
+
+**Timing Separation:**
+```
+┌─────────────────┐
+│ Platform Clock │ (physical seconds)
+└────────┬────────┘
+ │
+ ┌────┴─────┬──────────────┐
+ ▼ ▼ ▼
+Physical Audio Time Music Time
+ Time (playback) (tempo-scaled)
+ │ │ │
+ │ └──────┬───────┘
+ │ ▼
+ │ Beat Calculation
+ │ (BPM conversion)
+ │ │
+ └────────┬────────┘
+ ▼
+ Visual Effects Rendering
+ (time + beat_time + beat_phase)
+```
+
+**Key Insight:** Variable tempo changes `music_time` for audio triggering, but visual effects receive constant `time` (physical) and derived `beat_time` (from audio playback, not music_time).
+
+---
+
+## Technical Details
+
+### Uniform Size Maintained
+```cpp
+// Before (32 bytes):
+struct { vec2 res; float _pad[2]; float aspect, time, beat, intensity; }
+
+// After (32 bytes):
+struct { vec2 res; float aspect, time, beat_time, beat_phase, intensity, _pad; }
+```
+
+### Beat Calculation
+```cpp
+// main.cc
+const float absolute_beat_time = current_audio_time * g_tracker_score.bpm / 60.0f;
+const float beat_phase = fmodf(absolute_beat_time, 1.0f);
+```
+
+### Seq Compiler Logic
+```cpp
+// Default: beats → seconds
+float beat = std::stof(value);
+float time = beat * 60.0f / bpm;
+
+// Explicit seconds: pass through
+if (value.back() == 's') return seconds;
+```
+
+---
+
+## Migration Guide
+
+**For New Content:** Use beat notation (recommended)
+```seq
+# BPM 140
+SEQUENCE 0 0 "Intro"
+ EFFECT + Flash 0 4 # 4 beats = 1.71s @ 140 BPM
+```
+
+**For Existing Content:** Already migrated with 's' suffix
+```seq
+SEQUENCE 2.50s 0 # Preserved exact timing
+ EFFECT + Flash 0.00s 1.00s
+```
+
+**For Shader Effects:**
+- Use `uniforms.beat_phase` (not `uniforms.beat`)
+- Use `uniforms.beat_time` for bar-based animation
+- Use `uniforms.time` for constant-speed animation
+
+---
+
+## Files Modified
+
+**Core System:**
+- `src/gpu/effects/post_process_helper.h` - Uniform structure
+- `src/gpu/effect.{h,cc}` - Effect rendering signatures
+- `src/gpu/gpu.{h,cc}` - GPU draw interface
+- `src/main.cc`, `src/test_demo.cc` - Beat calculation
+
+**Shaders:**
+- `workspaces/{main,test}/shaders/common_uniforms.wgsl`
+- `assets/{common,final}/shaders/common_uniforms.wgsl`
+- All effect shaders using beat: `particle_spray_compute.wgsl`, `ellipse.wgsl`
+
+**Timeline Compiler:**
+- `tools/seq_compiler.cc` - Beat-as-default parser
+
+**Timelines:**
+- `workspaces/main/timeline.seq` - Explicit 's' suffix
+- `workspaces/test/timeline.seq` - Explicit 's' suffix
+
+**Documentation:**
+- `doc/SEQUENCE.md` - Beat notation format
+- `tools/timeline_editor/README.md` - Editor usage
-- **Build:** ✅ Complete (100%)
-- **Tests:** ✅ 34/36 passing (94%)
-- **Demo:** Ready to run with `./build/demo64k`
+---
-## Next Steps
+## Future Enhancements
-1. Test visual effects with new beat_time parameter
-2. Create beat-synchronized shader animations
-3. Consider converting existing timelines to beat notation
+1. **Beat-Synced Effects:** Create effects that pulse/animate to bars
+2. **Timeline Conversion:** Tool to convert explicit seconds → beats
+3. **Editor Support:** Timeline editor beat grid visualization
+4. **Shader Helpers:** WGSL functions for common beat patterns
diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md
index 7e8107e..e57763e 100644
--- a/PROJECT_CONTEXT.md
+++ b/PROJECT_CONTEXT.md
@@ -31,14 +31,15 @@
## Current Status
+- **Timing System:** **Beat-based timelines** for musical synchronization. Sequences defined in beats, converted to seconds at runtime. Effects receive both physical time (constant) and beat time (musical). Variable tempo affects audio only. See `BEAT_TIMING_SUMMARY.md`.
- **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=<name>`. Shared common assets.
- **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. Comprehensive tests.
-- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Modular WGSL composition.
+- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition.
- **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline.
- **Effects:** CNN post-processing foundation (3-layer architecture, modular snippets). CNNEffect validated in demo.
- **Tools:** CNN test tool (readback works, output incorrect - under investigation). Texture readback utility functional.
- **Build:** Asset dependency tracking. Size measurement. Hot-reload (debug-only).
-- **Testing:** **36/36 passing (100%)**
+- **Testing:** **34/36 passing (94%)**
---
diff --git a/README.md b/README.md
index ac723db..2f74e54 100644
--- a/README.md
+++ b/README.md
@@ -12,10 +12,15 @@ cmake --build build -j4
## Documentation
+**Quick Start:**
- **PROJECT_CONTEXT.md** - Current status and architecture overview
- **TODO.md** - Active tasks and priorities
- **doc/HOWTO.md** - Common operations (building, testing, assets)
-- **doc/CONTRIBUTING.md** - Development guidelines and protocols
- **doc/EFFECT_WORKFLOW.md** - Step-by-step guide for adding visual effects
+**Key Features:**
+- **BEAT_TIMING_SUMMARY.md** - Beat-based timing system (NEW)
+- **doc/BEAT_TIMING.md** - Timeline authoring guide
+- **doc/CONTRIBUTING.md** - Development guidelines and protocols
+
See `doc/` for detailed technical documentation.
diff --git a/doc/BEAT_TIMING.md b/doc/BEAT_TIMING.md
new file mode 100644
index 0000000..cf7f377
--- /dev/null
+++ b/doc/BEAT_TIMING.md
@@ -0,0 +1,272 @@
+# Beat-Based Timing System
+
+## Overview
+
+The demo uses **beat-based timing** for visual effect sequences, ensuring musical synchronization regardless of BPM changes. All timeline sequences are authored in beats (musical time) and converted to physical seconds at runtime.
+
+**Key Principle:** Variable tempo only affects audio sample triggering. Visual effects run at constant physical time with optional beat-synchronized animation.
+
+---
+
+## Quick Start
+
+### Timeline Authoring
+```seq
+# BPM 120
+SEQUENCE 0 0 "Intro" # Beat 0 (bar 1)
+ EFFECT + Flash 0 2 # Beats 0-2 (half bar)
+ EFFECT + Fade 4 8 # Beats 4-8 (full bar)
+
+SEQUENCE 16 1 "Drop" # Beat 16 (bar 5)
+ EFFECT + Heptagon 0 16 # 4 bars
+```
+
+**Conversion:** At 120 BPM, 1 beat = 0.5 seconds, 4 beats = 2 seconds
+
+### Shader Animation
+```wgsl
+@group(0) @binding(2) var<uniform> uniforms: CommonUniforms;
+
+// Use beat_time for musical animation
+let bar_cycle = uniforms.beat_time / 4.0; // Bars
+let pulse = sin(bar_cycle * TAU);
+
+// Use beat_phase for smooth per-beat effects
+let wave = sin(uniforms.beat_phase * TAU);
+
+// Use time for constant-speed physics
+let rotation = uniforms.time * TAU;
+```
+
+---
+
+## Uniform Structure
+
+All effects receive `CommonPostProcessUniforms` with timing data:
+
+```cpp
+struct CommonPostProcessUniforms {
+ vec2 resolution; // Screen dimensions (pixels)
+ float aspect_ratio; // Width/height ratio
+ float time; // Physical seconds (constant, unaffected by tempo)
+ float beat_time; // Absolute beats (musical time from audio clock)
+ float beat_phase; // Fractional beat 0.0-1.0 (smooth oscillation)
+ float audio_intensity; // Audio peak for beat sync
+ float _pad; // Alignment padding
+}; // 32 bytes
+```
+
+**Use Cases:**
+- `time`: Physics animation, constant-speed rotation/movement
+- `beat_time`: Bar-based patterns, musical synchronization
+- `beat_phase`: Smooth per-beat pulse/wave effects
+
+---
+
+## Timeline Format
+
+### Time Notation
+
+**Default:** Beats (no suffix)
+```seq
+SEQUENCE 0 0 # Beat 0
+ EFFECT + Flash 0 4 # Beats 0-4
+```
+
+**Explicit Seconds:** Use `s` suffix (rare)
+```seq
+SEQUENCE 2.5s 0 # 2.5 physical seconds
+ EFFECT + Flash 0 4 # Still uses beats for duration
+```
+
+**Explicit Beats:** Use `b` suffix (optional clarity)
+```seq
+SEQUENCE 8b 0 # Same as "8"
+ EFFECT + Flash 0b 4b # Same as "0 4"
+```
+
+### BPM Declaration
+
+**Required** in all timeline files:
+```seq
+# BPM 120
+```
+
+Specifies beats per minute for runtime conversion to seconds.
+
+---
+
+## Architecture
+
+### Timing Flow
+
+```
+Platform Clock (physical seconds)
+ │
+ ├──► Physical Time ────────┐
+ │ (constant) │
+ │ │
+ └──► Audio Time ────┐ │
+ (playback) │ │
+ ▼ │
+ Beat Calculation │
+ (BPM * 60) │
+ │ │
+ ▼ ▼
+ Visual Effects Rendering
+ (time + beat_time + beat_phase)
+```
+
+### Key Insight
+
+**Variable tempo changes `music_time`** (used for audio event triggering), but **visual effects receive `time` (physical)** and **`beat_time` (from audio playback clock)**, not from tempo-scaled music time.
+
+This separation ensures:
+- ✅ Visual effects run at constant frame rate
+- ✅ Beat-synced animations track actual audio playback
+- ✅ Tempo changes don't cause visual stuttering
+
+---
+
+## Implementation
+
+### Beat Calculation (Runtime)
+
+```cpp
+// main.cc - Calculate from audio playback time
+const float absolute_beat_time = current_audio_time * g_tracker_score.bpm / 60.0f;
+const float beat_phase = fmodf(absolute_beat_time, 1.0f);
+
+// Pass to GPU rendering
+gpu_draw(visual_peak, aspect_ratio, physical_time, absolute_beat_time, beat_phase);
+```
+
+### Timeline Compilation
+
+```cpp
+// seq_compiler.cc - Convert beats to seconds at compile time
+std::string convert_to_time(const std::string& value, float bpm) {
+ if (value.back() == 's') return explicit_seconds; // Pass through
+
+ // Default: treat as beats
+ float beat = std::stof(value);
+ float time = beat * 60.0f / bpm;
+ return time;
+}
+```
+
+**Result:** Generated `timeline.cc` contains physical seconds for effect activation.
+
+---
+
+## Migration
+
+### Existing Timelines
+
+Already migrated with explicit `s` suffix to preserve timing:
+```seq
+SEQUENCE 2.50s 0 # Physical seconds preserved
+ EFFECT + Flash 0.00s 1.00s
+```
+
+### New Content
+
+Use beat notation (recommended):
+```seq
+# BPM 140
+SEQUENCE 0 0 "Intro"
+ EFFECT + Flash 0 4 # 4 beats = 1.71s @ 140 BPM
+ EFFECT + Fade 4 8 # 4 beats = 1.71s
+```
+
+**Benefits:**
+- Natural musical alignment (bars/beats)
+- BPM changes don't break timing
+- Easier to author to music
+
+---
+
+## Examples
+
+### Four-Bar Pattern
+```seq
+# BPM 120
+SEQUENCE 0 0 "Verse 1"
+ EFFECT - Background 0 16 # 4 bars background
+ EFFECT + Flash 0 1 # First beat flash
+ EFFECT + Pulse 4 5 # Second bar pulse
+ EFFECT + Fade 15 16 # Final beat fade
+```
+
+### Multi-Bar Sequence
+```seq
+SEQUENCE 16 0 "Chorus" # Bar 5
+ EFFECT + Heptagon 0 32 # 8 bars (full chorus)
+ EFFECT + Particles 8 24 # Bars 7-11 (middle)
+```
+
+### Beat-Synced Shader
+```wgsl
+fn fragment_main(@location(0) uv: vec2<f32>) -> @location(0) vec4<f32> {
+ // Pulse every bar (4 beats)
+ let bar_phase = fract(uniforms.beat_time / 4.0);
+ let bar_pulse = smoothstep(0.0, 0.1, bar_phase) *
+ (1.0 - smoothstep(0.9, 1.0, bar_phase));
+
+ // Smooth per-beat wave
+ let beat_wave = sin(uniforms.beat_phase * TAU);
+
+ // Combine
+ let intensity = bar_pulse * 0.5 + beat_wave * 0.3;
+ return vec4<f32>(color * intensity, 1.0);
+}
+```
+
+---
+
+## Troubleshooting
+
+### Shader Compilation Error: "invalid accessor 'beat'"
+
+**Cause:** Old shader using `uniforms.beat` (deprecated field)
+
+**Fix:** Use `uniforms.beat_phase` or `uniforms.beat_time`
+
+```wgsl
+// OLD (error):
+let x = uniforms.beat;
+
+// NEW:
+let x = uniforms.beat_phase; // For 0-1 fractional
+let y = uniforms.beat_time; // For absolute beats
+```
+
+### Timeline Parse Error
+
+**Cause:** Missing BPM declaration
+
+**Fix:** Add BPM at top of file
+```seq
+# BPM 120 ← Required
+SEQUENCE 0 0
+```
+
+### Effects Start at Wrong Time
+
+**Cause:** Mixing beats and seconds without explicit suffixes
+
+**Fix:** Be explicit
+```seq
+SEQUENCE 8 0 # 8 beats (not 8 seconds)
+SEQUENCE 8s 0 # 8 seconds (explicit)
+SEQUENCE 8b 0 # 8 beats (explicit, same as first)
+```
+
+---
+
+## See Also
+
+- **Format Reference:** `doc/SEQUENCE.md` - Complete .seq syntax
+- **Implementation:** `BEAT_TIMING_SUMMARY.md` - Technical details
+- **Effect Creation:** `doc/EFFECT_WORKFLOW.md` - Adding new effects
+- **Timeline Editor:** `tools/timeline_editor/README.md` - Visual editing