diff options
| -rw-r--r-- | ANALYSIS_VARIABLE_TEMPO.md | 372 |
1 files changed, 372 insertions, 0 deletions
diff --git a/ANALYSIS_VARIABLE_TEMPO.md b/ANALYSIS_VARIABLE_TEMPO.md new file mode 100644 index 0000000..f09996b --- /dev/null +++ b/ANALYSIS_VARIABLE_TEMPO.md @@ -0,0 +1,372 @@ +# Analysis: Variable Tempo / Unified Music Time + +## Current Architecture Analysis + +### 1. BPM Dependencies (Problems Found) + +#### Location 1: `TrackerScore` (tracker.h:32) +```cpp +struct TrackerScore { + const TrackerPatternTrigger* triggers; + uint32_t num_triggers; + float bpm; // ❌ FIXED BPM - Cannot change dynamically +}; +``` + +#### Location 2: `tracker_update()` (tracker.cc:57) +```cpp +float beat_to_sec = 60.0f / g_tracker_score.bpm; // ❌ Used to convert beats→seconds +``` + +**Critical Issue (tracker.cc:84):** +```cpp +int frame_offset = (int)(event.beat * beat_to_sec * 32000.0f / DCT_SIZE); +``` + +**Problem:** The BPM is **baked into the spectrogram** at pattern trigger time! +- Events within a pattern are composited once with fixed spacing +- The resulting spectrogram frames encode the tempo +- Once triggered, the pattern plays at the encoded tempo +- **Cannot speed up/slow down dynamically** + +#### Location 3: `main.cc` (lines 99, 166) +```cpp +// Bass pattern timing +if (t - last_beat_time > (60.0f / g_tracker_score.bpm) / 2.0) { // ❌ BPM dependency + +// Visual beat calculation +float beat = fmodf((float)current_time * g_tracker_score.bpm / 60.0f, 1.0f); // ❌ BPM dependency +``` + +--- + +## Current Time Flow + +``` +Physical Time (hardware clock) + ↓ +platform_state.time (real seconds elapsed) + ↓ +current_time = platform_state.time + seek_offset + ↓ +tracker_update(current_time) ← checks if patterns should trigger + ↓ +Pattern compositing: event.beat * (60.0 / BPM) * 32kHz + ↓ +Spectrogram generated with FIXED frame spacing + ↓ +synth_trigger_voice() → registers spectrogram + ↓ +Audio callback @ 32kHz pulls samples via synth_render() + ↓ +Voice advances through spectral frames at FIXED rate (one frame per DCT_SIZE samples) +``` + +**The Fundamental Problem:** +- Music time = Physical time (1:1 mapping) +- Patterns are pre-rendered at fixed tempo +- Synth playback rate is locked to 32kHz sample clock + +--- + +## What Needs to Change + +### Architecture Requirements + +1. **Separate Music Time from Physical Time** + - Music time advances independently: `music_time += dt * tempo_scale` + - `tempo_scale = 1.0` → normal speed + - `tempo_scale = 2.0` → double speed + - `tempo_scale = 0.5` → half speed + +2. **Remove BPM from Pattern Compositing** + - Events should use abstract "beat" units, not seconds + - Pattern generation should NOT convert beats→frames using BPM + - Keep events in "music time" space + +3. **Variable Playback Rate in Synth** + - Synth must support dynamic playback speed + - As tempo changes, synth advances through spectral frames faster/slower + +--- + +## Proposed Solution + +### Phase 1: Introduce Unified Music Time + +#### Change 1: Update `TrackerScore` +```cpp +struct TrackerScore { + const TrackerPatternTrigger* triggers; + uint32_t num_triggers; + // REMOVE: float bpm; +}; +``` + +#### Change 2: Update `TrackerPatternTrigger` +```cpp +struct TrackerPatternTrigger { + float music_time; // Renamed from time_sec, now abstract units + uint16_t pattern_id; +}; +``` + +#### Change 3: Update `TrackerEvent` +```cpp +struct TrackerEvent { + float beat; // Keep as-is, already abstract + uint16_t sample_id; + float volume; + float pan; +}; +``` + +#### Change 4: Update `tracker_update()` signature +```cpp +// OLD: +void tracker_update(float time_sec); + +// NEW: +void tracker_update(float music_time); +``` + +**Note:** Pattern compositing still needs tempo for now (see Phase 2). + +--- + +### Phase 2: Variable Tempo Synth Playback + +The hard part: **Dynamic playback speed without pitch shifting.** + +#### Option A: Naive Approach (Pitch Shifts - NOT GOOD) +```cpp +// Simply advance through frames faster/slower +v.current_spectral_frame += playback_speed; // ❌ Changes pitch! +``` + +**Problem:** Advancing through spectral frames faster/slower changes pitch. +- 2x speed → 2x frequency (octave up) +- 0.5x speed → 0.5x frequency (octave down) + +#### Option B: Resample Spectrograms (Complex) +- Generate spectrograms at "reference tempo" (e.g., 1.0) +- At playback time, resample to match current tempo +- **Pros:** Preserves pitch +- **Cons:** CPU-intensive, requires interpolation + +#### Option C: Time-Stretching (Best Quality) +- Use phase vocoder or similar algorithm +- Stretches/compresses time without changing pitch +- **Pros:** High quality +- **Cons:** Very complex, may exceed 64k budget + +#### Option D: Pre-render at Reference Tempo (Recommended for 64k) +**Key Insight:** For a 64k demo, pre-render everything at a fixed internal tempo. + +```cpp +// At pattern trigger time: +// 1. Composite pattern at "reference tempo" (e.g., 120 BPM baseline) +int frame_offset = (int)(event.beat * REFERENCE_BEAT_DURATION * 32000.0f / DCT_SIZE); + +// 2. Store tempo multiplier with voice +struct Voice { + float playback_speed; // NEW: 1.0 = normal, 2.0 = double speed, etc. + // ... existing fields +}; + +// 3. In synth_render(), advance at variable rate: +void synth_render(float* output_buffer, int num_frames) { + for each voice { + // Advance through spectral frames at tempo-adjusted rate + float frame_advance = playback_speed / DCT_SIZE; + + // Use fractional frame index + float spectral_frame_pos; // NEW: float instead of int + + // Interpolate between frames + int frame0 = (int)spectral_frame_pos; + int frame1 = frame0 + 1; + float frac = spectral_frame_pos - frame0; + + // Blend spectral data from frame0 and frame1 + for (int j = 0; j < DCT_SIZE; ++j) { + windowed_frame[j] = lerp(spectral[frame0][j], spectral[frame1][j], frac); + } + } +} +``` + +**Trade-off:** This changes pitch! But for a demo, this might be acceptable. + +--- + +## Sync with Physical Audio Device + +### Current System +``` +Audio Callback (32kHz fixed) + ↓ +synth_render(buffer, num_frames) ← pulls samples + ↓ +Voices advance: ++buffer_pos, ++current_spectral_frame + ↓ +Fixed playback rate (one spectral frame = DCT_SIZE samples) +``` + +### With Variable Tempo +``` +Audio Callback (32kHz fixed - CANNOT CHANGE) + ↓ +synth_render(buffer, num_frames, current_playback_speed) + ↓ +Voices advance: buffer_pos += playback_speed + ↓ +Variable playback rate (playback_speed can be > 1.0 or < 1.0) +``` + +**Key Point:** Hardware audio rate (32kHz) is FIXED. We cannot change it. + +**Solution:** Adjust how fast we consume spectrogram data: +- `playback_speed = 2.0` → consume frames 2x faster → music plays 2x speed +- `playback_speed = 0.5` → consume frames 0.5x slower → music plays 0.5x speed + +--- + +## Implementation Roadmap + +### Step 1: Remove BPM from TrackerScore ✅ Ready +- Remove `float bpm` field +- Update tracker_compiler to not generate BPM +- Update tests + +### Step 2: Add Global Tempo State +```cpp +// In audio/tracker.h or main.cc +float g_current_tempo = 1.0f; // Multiplier: 1.0 = normal, 2.0 = double speed + +void tracker_set_tempo(float tempo_multiplier); +float tracker_get_tempo(); +``` + +### Step 3: Update tracker_update() +```cpp +void tracker_update(float music_time) { + // music_time is now abstract, not physical seconds + // Patterns trigger when music_time >= trigger.music_time +} +``` + +### Step 4: Update Pattern Compositing (Keep Reference Tempo) +```cpp +// Use fixed reference tempo for compositing (e.g., 120 BPM) +const float REFERENCE_BPM = 120.0f; +const float reference_beat_to_sec = 60.0f / REFERENCE_BPM; + +int frame_offset = (int)(event.beat * reference_beat_to_sec * 32000.0f / DCT_SIZE); +``` + +### Step 5: Variable Playback in Synth (Advanced) +```cpp +// Store playback speed with each voice +struct Voice { + float playback_speed; // Set when voice is triggered + float spectral_frame_pos; // Change to float for interpolation + // ... +}; + +// In synth_trigger_voice(): +v.playback_speed = g_current_tempo; + +// In synth_render(): +// Advance with fractional frame increment +v.spectral_frame_pos += v.playback_speed / DCT_SIZE; + +// Interpolate spectral data (required for smooth tempo changes) +``` + +### Step 6: Update Main Loop +```cpp +// In main.cc +static float g_music_time = 0.0f; +static float g_tempo_scale = 1.0f; // Can be animated! + +void update_game_logic(double physical_time) { + float dt = get_delta_time(); + + // Music time advances at variable rate + g_music_time += dt * g_tempo_scale; + + // Animate tempo (example) + g_tempo_scale = 1.0f + 0.5f * sinf(physical_time * 0.1f); // Oscillate 0.5x - 1.5x + + tracker_set_tempo(g_tempo_scale); + tracker_update(g_music_time); +} +``` + +--- + +## Open Questions + +1. **Pitch Shifting Acceptable?** + - If tempo changes, should pitch change too? + - For demoscene effect: probably YES (classic effect) + - For "realistic" music: NO (need time-stretching) + +2. **Tempo Curve Continuity** + - Should tempo changes be smoothed (acceleration curves)? + - Or instant jumps (may cause audio glitches)? + +3. **Spectral Frame Interpolation** + - Linear interpolation sufficient? + - Or need cubic/sinc for quality? + +4. **Tracker Compilation** + - Should tracker_compiler still know about tempo? + - Or output pure "beat" units? + +--- + +## Recommendation + +**For a 64k demo, I recommend:** + +### Minimal Change Approach (Easiest) +1. Keep BPM for compositing (as "reference tempo") +2. Add `g_current_tempo` multiplier +3. Make synth advance through frames at `tempo * playback_speed` +4. Accept pitch shifting as intentional effect + +**Changes needed:** +- Add `float g_current_tempo` global +- Update `synth_render()` to use `playback_speed` +- Main loop: `music_time += dt * g_current_tempo` +- Tracker: rename `time_sec` → `music_time` (conceptual only) + +**Size impact:** ~100 bytes +**Complexity:** Low +**Enables:** Full variable tempo with pitch shift effect + +### Advanced Approach (Better Quality) +- Implement spectral frame interpolation +- Add time-stretching (phase vocoder lite) +- Size impact: +2-3 KB + +--- + +## Conclusion + +**Is the code ready?** ❌ NO + +**What needs to change?** +1. BPM must become a "reference tempo" for compositing only +2. Add global `tempo_scale` variable +3. Synth must support variable playback speed +4. Main loop must track `music_time` separately from physical time + +**Sync with audio device:** +- Hardware rate (32kHz) is FIXED +- We control playback speed by advancing through spectral frames faster/slower +- This inherently changes pitch (unless we add time-stretching) + +**Recommended next step:** Implement minimal change approach first, then iterate. |