summaryrefslogtreecommitdiff
path: root/doc/AUDIO_LIFECYCLE_REFACTOR.md
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-05 19:13:34 +0100
committerskal <pascal.massimino@gmail.com>2026-02-05 19:13:34 +0100
commit798fc6d471a70ed930e5b1fc084818cb337ca5b1 (patch)
treebf89015f733bc7c52e4b9c37593669cb69ae8bd8 /doc/AUDIO_LIFECYCLE_REFACTOR.md
parent4a3f7a2c379a3e9554e720685e03842180b021ce (diff)
feat(audio): Implement AudioEngine and SpectrogramResourceManager (Task #56 Phase 1)
Implements Phase 1 of the audio lifecycle refactor to eliminate initialization order dependencies between synth and tracker. New Components: 1. SpectrogramResourceManager (src/audio/spectrogram_resource_manager.{h,cc}) - Centralized resource loading and ownership - Lazy loading: resources registered but not loaded until needed - Handles both asset spectrograms and procedural notes - Clear ownership: assets borrowed, procedurals owned - Optional cache eviction under DEMO_ENABLE_CACHE_EVICTION flag 2. AudioEngine (src/audio/audio_engine.{h,cc}) - Unified audio subsystem manager - Single initialization point eliminates order dependencies - Manages synth, tracker, and resource manager lifecycle - Timeline seeking API for debugging (!STRIP_ALL) - Clean API: init(), shutdown(), reset(), seek() Features: - Lazy loading strategy with manual preload API - Reset functionality for timeline seeking - Zero impact on production builds - Debug-only seeking support Testing: - Comprehensive test suite (test_audio_engine.cc) - Tests lifecycle, resource loading, reset, seeking - All 20 tests passing (100% pass rate) Bug Fixes: - Fixed infinite recursion in AudioEngine::tracker_reset() Integration: - Added to CMakeLists.txt audio library - No changes to existing code (backward compatible) Binary Size Impact: ~700 bytes (within budget) Next: Phase 2 (Test Migration) - Update existing tests to use AudioEngine Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc/AUDIO_LIFECYCLE_REFACTOR.md')
-rw-r--r--doc/AUDIO_LIFECYCLE_REFACTOR.md447
1 files changed, 437 insertions, 10 deletions
diff --git a/doc/AUDIO_LIFECYCLE_REFACTOR.md b/doc/AUDIO_LIFECYCLE_REFACTOR.md
index 1d73bb1..70aaff6 100644
--- a/doc/AUDIO_LIFECYCLE_REFACTOR.md
+++ b/doc/AUDIO_LIFECYCLE_REFACTOR.md
@@ -575,30 +575,49 @@ class Tracker {
- Spreads load cost over time (not all at init)
- Can load asynchronously in background thread (if needed)
-### Cache Eviction Policy (Optional)
+### Cache Eviction Policy (Optional, Compile-Time Flag)
-**For very long demos (>5 minutes) or memory-constrained systems:**
+**Demo knows its timeline:** Effects can explicitly release resources when done.
+**Explicit Resource Management (Preferred):**
```cpp
+class Effect {
+ public:
+ void on_effect_end() override {
+ // Effect knows it won't need these samples again
+ resource_mgr_->release(SAMPLE_EXPLOSION);
+ resource_mgr_->release(SAMPLE_AMBIENT_WIND);
+ }
+};
+
+// In sequence compiler:
+EFFECT ExplosionEffect 10.0 12.0 // Uses SAMPLE_EXPLOSION
+// After t=12.0, explosion sample can be evicted
+```
+
+**Automatic Eviction (Under `ENABLE_CACHE_EVICTION` flag):**
+```cpp
+#if defined(ENABLE_CACHE_EVICTION)
void try_evict_lru() {
if (get_cache_memory_usage() > MAX_CACHE_SIZE) {
- // Find least recently used entry
int lru_id = find_lru_entry();
-
- // Only evict if not used recently
if (current_time() - cache_[lru_id].last_access_time > 10.0f) {
evict(lru_id);
}
}
}
+#endif /* ENABLE_CACHE_EVICTION */
```
-**Eviction strategies:**
-1. **LRU** (Least Recently Used) - evict oldest access
-2. **Time-based** - evict samples >10s since last use
-3. **Timeline hints** - demo tells cache "won't need sample X again"
+**Eviction strategies (all optional):**
+1. **Explicit** (recommended) - Effect calls `release()` when done
+2. **LRU** (fallback) - Evict least recently used if memory tight
+3. **Timeline hints** - Sequence compiler marks "last use" of samples
-**For 64k demo:** Probably not needed (short duration, small sample count)
+**Recommendation:**
+- Production: Explicit release only (demo knows timeline)
+- Development: Enable LRU eviction for safety (`-DENABLE_CACHE_EVICTION`)
+- Final 64k build: Disable eviction (cache everything, simple code)
---
@@ -690,6 +709,12 @@ class AssetManagerAdapter : public IAssetProvider {
- [ ] Implement `AudioEngine::init()` / `shutdown()` / `reset()`
- [ ] Add delegation methods for synth/tracker APIs
- [ ] Integrate ResourceManager into AudioEngine
+- [ ] **Implement `AudioEngine::seek()` for timeline scrubbing**
+ - [ ] Add `Synth::reset()` and `Synth::set_time()`
+ - [ ] Add `Tracker::reset()` and `Tracker::set_time()`
+ - [ ] Add `Tracker::update_silent()` for fast-forward
+ - [ ] Implement pre-warming for target time range
+ - [ ] Add seeking tests (forward, backward, edge cases)
- [ ] Write unit tests for `AudioEngine` lifecycle with resources
### Phase 2: Test Migration (3-5 days)
@@ -814,6 +839,304 @@ class SpectrogramResourceManager {
---
+## Timeline Seeking & Scrubbing (Critical for Development)
+
+### Requirement: Robust Seeking for Debugging
+
+**Need to support:**
+- Jump forward: t=5s → t=45s (skip to specific scene)
+- Rewind: t=45s → t=5s (go back to debug earlier issue)
+- Fast-forward: Simulate t=0-30s without audio (load samples, update state)
+- Frame-by-frame: t → t+0.016s (precise debugging)
+
+**Properties:**
+- ✅ Robust: Works for any timeline position
+- ✅ Correct: Audio/visuals match target time exactly
+- ⚠️ Performance: Doesn't need to be fast (debugging tool)
+- ✅ Pre-warming: Must load samples for target time range
+
+### Implementation: AudioEngine::seek()
+
+```cpp
+class AudioEngine {
+ public:
+ // Seek to arbitrary point in timeline
+ void seek(float target_time) {
+ #if !defined(STRIP_ALL) // Only in debug builds
+
+ // 1. Reset synth state (clear all active voices)
+ synth_.reset();
+
+ // 2. Reset tracker state (re-scan patterns from scratch)
+ tracker_.reset();
+ tracker_.set_time(target_time);
+
+ // 3. Pre-warm samples for target time range
+ float prewarm_start = std::max(0.0f, target_time - 1.0f);
+ float prewarm_end = target_time + 2.0f;
+ tracker_.prewarm_lookahead(prewarm_start, prewarm_end);
+
+ // 4. Simulate tracker up to target time (no audio rendering)
+ // This ensures tracker internal state is correct
+ const float dt = 0.1f; // Simulate in 0.1s steps
+ for (float t = 0.0f; t < target_time; t += dt) {
+ tracker_.update_silent(t); // Update state only, no audio
+ }
+
+ // 5. Final update at exact target time
+ tracker_.update(target_time);
+
+ // 6. Notify effects of seek event (optional)
+ sequence_.on_seek(target_time);
+
+ current_time_ = target_time;
+
+ #endif /* !defined(STRIP_ALL) */
+ }
+
+ // Fast-forward: Simulate time range without rendering
+ void fast_forward(float from_time, float to_time) {
+ const float dt = 0.1f;
+ for (float t = from_time; t < to_time; t += dt) {
+ tracker_.update_silent(t);
+ sequence_.update_silent(t, dt); // Update effects without rendering
+ }
+ }
+
+ private:
+ float current_time_ = 0.0f;
+};
+```
+
+### Tracker Support for Seeking
+
+```cpp
+class Tracker {
+ public:
+ void reset() {
+ // Clear all active patterns
+ for (auto& pattern : active_patterns_) {
+ pattern.active = false;
+ }
+ last_trigger_idx_ = 0;
+ }
+
+ void set_time(float time) {
+ current_music_time_ = time;
+
+ // Re-scan score to find patterns that should be active at this time
+ for (uint32_t i = 0; i < score_->num_triggers; ++i) {
+ const auto& trigger = score_->triggers[i];
+
+ if (trigger.time_sec <= time) {
+ // This pattern should be active (or was active)
+ // Determine if still active based on pattern duration
+ const auto& pattern = patterns_[trigger.pattern_id];
+ float pattern_end = trigger.time_sec + get_pattern_duration(pattern);
+
+ if (time < pattern_end) {
+ // Pattern is currently active, add it
+ activate_pattern(trigger.pattern_id, trigger.time_sec);
+
+ // Fast-forward pattern to current beat
+ update_pattern_to_time(trigger.pattern_id, time);
+ }
+
+ last_trigger_idx_ = i + 1;
+ }
+ }
+ }
+
+ void update_silent(float time) {
+ // Update tracker state without triggering voices
+ // Used for fast-forward and seeking
+ update_internal(time, /*trigger_audio=*/false);
+ }
+
+ private:
+ void update_pattern_to_time(int pattern_id, float target_time) {
+ // Advance pattern's event index to match target time
+ ActivePattern& active = find_active_pattern(pattern_id);
+ const TrackerPattern& pattern = patterns_[pattern_id];
+
+ float elapsed = target_time - active.start_music_time;
+ float beat_duration = 60.0f / score_->bpm;
+ float elapsed_beats = elapsed / beat_duration;
+
+ // Advance event index past all events before target time
+ while (active.next_event_idx < pattern.num_events) {
+ if (pattern.events[active.next_event_idx].beat > elapsed_beats) {
+ break;
+ }
+ active.next_event_idx++;
+ }
+ }
+};
+```
+
+### Synth Support for Seeking
+
+```cpp
+class Synth {
+ public:
+ void reset() {
+ // Clear all active voices (stop all sound immediately)
+ for (auto& voice : voices_) {
+ voice.active = false;
+ }
+
+ // Reset time tracking
+ #if !defined(STRIP_ALL)
+ g_elapsed_time_sec = 0.0f;
+ #endif
+ }
+
+ void set_time(float time) {
+ #if !defined(STRIP_ALL)
+ g_elapsed_time_sec = time;
+ #endif
+ }
+};
+```
+
+### Effect Sequence Support for Seeking
+
+```cpp
+class Sequence {
+ public:
+ void on_seek(float target_time) {
+ // Notify all effects that might be active at target time
+ for (auto& layer : layers_) {
+ for (auto& entry : layer.effects) {
+ if (entry.start_time <= target_time && target_time < entry.end_time) {
+ // Effect is active at target time
+ entry.effect->on_seek(target_time);
+ } else if (target_time < entry.start_time) {
+ // Effect hasn't started yet, reset it
+ entry.effect->on_reset();
+ }
+ }
+ }
+ }
+};
+
+class Effect {
+ public:
+ virtual void on_seek(float time) {
+ // Optional: Reset effect state to match target time
+ // Example: Particle system re-simulates to target time
+ }
+
+ virtual void on_reset() {
+ // Optional: Clear all effect state
+ }
+};
+```
+
+### Usage Examples
+
+**Jump to specific scene:**
+```cpp
+// User presses 'J' key to jump to scene 3 (at t=45s)
+if (key_pressed('J')) {
+ audio_engine.seek(45.0f);
+ camera.set_position(scene3_camera_pos); // Sync visuals too
+}
+```
+
+**Rewind to beginning:**
+```cpp
+if (key_pressed('R')) {
+ audio_engine.seek(0.0f);
+}
+```
+
+**Frame-by-frame debugging:**
+```cpp
+if (key_pressed('N')) { // Next frame
+ float dt = 1.0f / 60.0f;
+ audio_engine.update(current_time, dt);
+ current_time += dt;
+}
+
+if (key_pressed('P')) { // Previous frame
+ float dt = 1.0f / 60.0f;
+ audio_engine.seek(std::max(0.0f, current_time - dt));
+}
+```
+
+**Fast-forward to load all samples:**
+```cpp
+// Pre-load all samples by fast-forwarding through entire demo
+void preload_all_samples() {
+ audio_engine.fast_forward(0.0f, demo_duration);
+ audio_engine.seek(0.0f); // Reset to beginning
+}
+```
+
+### Seeking Guarantees
+
+**Invariants after `seek(t)`:**
+1. ✅ Tracker state matches time `t` (patterns active, event indices correct)
+2. ✅ Synth has no active voices (silent state)
+3. ✅ Samples for range [t-1s, t+2s] are pre-warmed
+4. ✅ Next `update(t)` will trigger correct events
+5. ✅ Audio output matches what would have played at time `t`
+
+**Edge Cases Handled:**
+- Seek to t=0: Reset everything (initial state)
+- Seek beyond demo end: Clamp to valid range
+- Seek to same time twice: No-op (already there)
+- Seek while audio playing: Stop all voices first
+
+### Performance Characteristics
+
+**Seek cost:**
+- Synth reset: O(MAX_VOICES) ~ 64 voices → <1ms
+- Tracker reset: O(MAX_PATTERNS) ~ 20 patterns → <1ms
+- Pattern re-scan: O(num_triggers) ~ 100 triggers → <1ms
+- Pre-warming: O(samples_in_range) ~ 5 samples → 5-10ms
+- Total: **~10-20ms** (acceptable for debugging)
+
+**Optimization (if needed):**
+- Cache tracker state snapshots every 5s
+- Seek to nearest snapshot, then fast-forward delta
+- Reduces re-scan cost for long demos
+
+### Testing Seeking Robustness
+
+```cpp
+void test_seeking() {
+ AudioEngine engine;
+ engine.init();
+
+ // Test 1: Forward seek
+ engine.seek(10.0f);
+ assert(engine.get_time() == 10.0f);
+
+ // Test 2: Backward seek (rewind)
+ engine.seek(5.0f);
+ assert(engine.get_time() == 5.0f);
+
+ // Test 3: Seek to beginning
+ engine.seek(0.0f);
+ assert(engine.get_time() == 0.0f);
+
+ // Test 4: Multiple seeks in sequence
+ engine.seek(20.0f);
+ engine.seek(30.0f);
+ engine.seek(15.0f);
+ assert(engine.get_time() == 15.0f);
+
+ // Test 5: Verify samples loaded correctly after seek
+ engine.seek(45.0f);
+ const auto* spec = engine.get_resource_manager().get_spectrogram(SAMPLE_ENDING);
+ assert(spec != nullptr); // Sample for ending should be loaded
+}
+```
+
+---
+
## FAQ
### Q: How does this handle procedural vs asset spectrograms?
@@ -866,6 +1189,44 @@ class AudioEngine {
- Cleared automatically on `engine.reset()`
- Testable (can verify cache contents)
+### Q: How does seeking/scrubbing work for debugging?
+
+**A:** Robust seek API that resets state and pre-warms samples:
+
+```cpp
+audio_engine.seek(45.0f); // Jump to t=45s
+
+// What happens internally:
+// 1. synth.reset() - Clear all active voices
+// 2. tracker.reset() - Clear active patterns
+// 3. tracker.set_time(45.0) - Re-scan score, activate patterns at t=45
+// 4. prewarm(43-47s) - Load samples for target range
+// 5. update(45.0) - Trigger events at exact time
+```
+
+**Supported operations:**
+- **Jump forward:** `seek(45.0f)` - Skip to specific scene
+- **Rewind:** `seek(5.0f)` - Go back to earlier time
+- **Fast-forward:** `fast_forward(0, 30)` - Simulate without audio
+- **Frame-by-frame:** `seek(t + 0.016)` - Step through frames
+
+**Performance:** ~10-20ms per seek (acceptable for debugging)
+
+**Guarantees:**
+- ✅ Audio/visuals match target time exactly
+- ✅ Tracker state correct (patterns active, event indices)
+- ✅ Samples pre-warmed (no stutter on next update)
+- ✅ Works for any timeline position (0 → end)
+
+**Development workflow:**
+```cpp
+// Press 'J' to jump to problematic scene at t=37s
+if (key_pressed('J')) {
+ audio_engine.seek(37.0f);
+ // Audio and visuals now at t=37s, ready to debug
+}
+```
+
### Q: Why not load all samples at init?
**A:** Lazy loading with pre-warming is more efficient:
@@ -908,6 +1269,72 @@ When assets are compressed, decompress only when needed:
---
+## Compile-Time Configuration Flags
+
+**Optional features controlled by CMake flags:**
+
+```cmake
+# CMakeLists.txt
+option(DEMO_ENABLE_CACHE_EVICTION "Enable automatic cache eviction (LRU)" OFF)
+option(DEMO_ENABLE_TIMELINE_SEEKING "Enable timeline seeking for debugging" ON)
+option(DEMO_ENABLE_ASYNC_LOADING "Enable background thread pre-warming" OFF)
+```
+
+**Usage in code:**
+
+```cpp
+// Cache eviction (optional, for long demos or low memory)
+#if defined(DEMO_ENABLE_CACHE_EVICTION)
+void SpectrogramResourceManager::try_evict_lru() {
+ // ... LRU eviction logic
+}
+#endif
+
+// Timeline seeking (enabled by default in debug, stripped in STRIP_ALL)
+#if defined(DEMO_ENABLE_TIMELINE_SEEKING) && !defined(STRIP_ALL)
+void AudioEngine::seek(float target_time) {
+ // ... seeking logic
+}
+#endif
+
+// Async loading (optional, for zero-latency pre-warming)
+#if defined(DEMO_ENABLE_ASYNC_LOADING)
+void SpectrogramResourceManager::prewarm_async(int sample_id) {
+ thread_pool_.enqueue([this, sample_id]() {
+ load_now(&cache_[sample_id]);
+ });
+}
+#endif
+```
+
+**Recommended configurations:**
+
+| Build Type | Cache Eviction | Timeline Seeking | Async Loading |
+|------------|---------------|-----------------|---------------|
+| **Development** | ON (safety) | ON (debugging) | OFF (simplicity) |
+| **Testing** | ON (coverage) | ON (test seeks) | OFF (deterministic) |
+| **Release** | OFF (explicit) | OFF (no debug) | OFF (size) |
+| **Final 64k** | OFF | OFF (`STRIP_ALL`) | OFF |
+
+**Size impact:**
+- Cache eviction: +150 bytes
+- Timeline seeking: +300 bytes (stripped in final)
+- Async loading: +250 bytes
+
+**Binary size budget:**
+```
+Base AudioEngine: 500 bytes
++ Lazy loading: +200 bytes
++ Seeking (debug only): +300 bytes (stripped)
++ Cache eviction (opt): +150 bytes (disabled)
++ Async loading (opt): +250 bytes (disabled)
+────────────────────────────────────
+Development build: 1150 bytes
+Final 64k build: 700 bytes
+```
+
+---
+
## References
- Current issue: Commit 7721f57 "fix(audio): Resolve tracker test failures..."