diff options
Diffstat (limited to 'ANALYSIS_VARIABLE_TEMPO.md')
| -rw-r--r-- | ANALYSIS_VARIABLE_TEMPO.md | 372 |
1 files changed, 0 insertions, 372 deletions
diff --git a/ANALYSIS_VARIABLE_TEMPO.md b/ANALYSIS_VARIABLE_TEMPO.md deleted file mode 100644 index f09996b..0000000 --- a/ANALYSIS_VARIABLE_TEMPO.md +++ /dev/null @@ -1,372 +0,0 @@ -# Analysis: Variable Tempo / Unified Music Time - -## Current Architecture Analysis - -### 1. BPM Dependencies (Problems Found) - -#### Location 1: `TrackerScore` (tracker.h:32) -```cpp -struct TrackerScore { - const TrackerPatternTrigger* triggers; - uint32_t num_triggers; - float bpm; // ❌ FIXED BPM - Cannot change dynamically -}; -``` - -#### Location 2: `tracker_update()` (tracker.cc:57) -```cpp -float beat_to_sec = 60.0f / g_tracker_score.bpm; // ❌ Used to convert beats→seconds -``` - -**Critical Issue (tracker.cc:84):** -```cpp -int frame_offset = (int)(event.beat * beat_to_sec * 32000.0f / DCT_SIZE); -``` - -**Problem:** The BPM is **baked into the spectrogram** at pattern trigger time! -- Events within a pattern are composited once with fixed spacing -- The resulting spectrogram frames encode the tempo -- Once triggered, the pattern plays at the encoded tempo -- **Cannot speed up/slow down dynamically** - -#### Location 3: `main.cc` (lines 99, 166) -```cpp -// Bass pattern timing -if (t - last_beat_time > (60.0f / g_tracker_score.bpm) / 2.0) { // ❌ BPM dependency - -// Visual beat calculation -float beat = fmodf((float)current_time * g_tracker_score.bpm / 60.0f, 1.0f); // ❌ BPM dependency -``` - ---- - -## Current Time Flow - -``` -Physical Time (hardware clock) - ↓ -platform_state.time (real seconds elapsed) - ↓ -current_time = platform_state.time + seek_offset - ↓ -tracker_update(current_time) ← checks if patterns should trigger - ↓ -Pattern compositing: event.beat * (60.0 / BPM) * 32kHz - ↓ -Spectrogram generated with FIXED frame spacing - ↓ -synth_trigger_voice() → registers spectrogram - ↓ -Audio callback @ 32kHz pulls samples via synth_render() - ↓ -Voice advances through spectral frames at FIXED rate (one frame per DCT_SIZE samples) -``` - -**The Fundamental Problem:** -- Music time = Physical time (1:1 mapping) -- Patterns are pre-rendered at fixed tempo -- Synth playback rate is locked to 32kHz sample clock - ---- - -## What Needs to Change - -### Architecture Requirements - -1. **Separate Music Time from Physical Time** - - Music time advances independently: `music_time += dt * tempo_scale` - - `tempo_scale = 1.0` → normal speed - - `tempo_scale = 2.0` → double speed - - `tempo_scale = 0.5` → half speed - -2. **Remove BPM from Pattern Compositing** - - Events should use abstract "beat" units, not seconds - - Pattern generation should NOT convert beats→frames using BPM - - Keep events in "music time" space - -3. **Variable Playback Rate in Synth** - - Synth must support dynamic playback speed - - As tempo changes, synth advances through spectral frames faster/slower - ---- - -## Proposed Solution - -### Phase 1: Introduce Unified Music Time - -#### Change 1: Update `TrackerScore` -```cpp -struct TrackerScore { - const TrackerPatternTrigger* triggers; - uint32_t num_triggers; - // REMOVE: float bpm; -}; -``` - -#### Change 2: Update `TrackerPatternTrigger` -```cpp -struct TrackerPatternTrigger { - float music_time; // Renamed from time_sec, now abstract units - uint16_t pattern_id; -}; -``` - -#### Change 3: Update `TrackerEvent` -```cpp -struct TrackerEvent { - float beat; // Keep as-is, already abstract - uint16_t sample_id; - float volume; - float pan; -}; -``` - -#### Change 4: Update `tracker_update()` signature -```cpp -// OLD: -void tracker_update(float time_sec); - -// NEW: -void tracker_update(float music_time); -``` - -**Note:** Pattern compositing still needs tempo for now (see Phase 2). - ---- - -### Phase 2: Variable Tempo Synth Playback - -The hard part: **Dynamic playback speed without pitch shifting.** - -#### Option A: Naive Approach (Pitch Shifts - NOT GOOD) -```cpp -// Simply advance through frames faster/slower -v.current_spectral_frame += playback_speed; // ❌ Changes pitch! -``` - -**Problem:** Advancing through spectral frames faster/slower changes pitch. -- 2x speed → 2x frequency (octave up) -- 0.5x speed → 0.5x frequency (octave down) - -#### Option B: Resample Spectrograms (Complex) -- Generate spectrograms at "reference tempo" (e.g., 1.0) -- At playback time, resample to match current tempo -- **Pros:** Preserves pitch -- **Cons:** CPU-intensive, requires interpolation - -#### Option C: Time-Stretching (Best Quality) -- Use phase vocoder or similar algorithm -- Stretches/compresses time without changing pitch -- **Pros:** High quality -- **Cons:** Very complex, may exceed 64k budget - -#### Option D: Pre-render at Reference Tempo (Recommended for 64k) -**Key Insight:** For a 64k demo, pre-render everything at a fixed internal tempo. - -```cpp -// At pattern trigger time: -// 1. Composite pattern at "reference tempo" (e.g., 120 BPM baseline) -int frame_offset = (int)(event.beat * REFERENCE_BEAT_DURATION * 32000.0f / DCT_SIZE); - -// 2. Store tempo multiplier with voice -struct Voice { - float playback_speed; // NEW: 1.0 = normal, 2.0 = double speed, etc. - // ... existing fields -}; - -// 3. In synth_render(), advance at variable rate: -void synth_render(float* output_buffer, int num_frames) { - for each voice { - // Advance through spectral frames at tempo-adjusted rate - float frame_advance = playback_speed / DCT_SIZE; - - // Use fractional frame index - float spectral_frame_pos; // NEW: float instead of int - - // Interpolate between frames - int frame0 = (int)spectral_frame_pos; - int frame1 = frame0 + 1; - float frac = spectral_frame_pos - frame0; - - // Blend spectral data from frame0 and frame1 - for (int j = 0; j < DCT_SIZE; ++j) { - windowed_frame[j] = lerp(spectral[frame0][j], spectral[frame1][j], frac); - } - } -} -``` - -**Trade-off:** This changes pitch! But for a demo, this might be acceptable. - ---- - -## Sync with Physical Audio Device - -### Current System -``` -Audio Callback (32kHz fixed) - ↓ -synth_render(buffer, num_frames) ← pulls samples - ↓ -Voices advance: ++buffer_pos, ++current_spectral_frame - ↓ -Fixed playback rate (one spectral frame = DCT_SIZE samples) -``` - -### With Variable Tempo -``` -Audio Callback (32kHz fixed - CANNOT CHANGE) - ↓ -synth_render(buffer, num_frames, current_playback_speed) - ↓ -Voices advance: buffer_pos += playback_speed - ↓ -Variable playback rate (playback_speed can be > 1.0 or < 1.0) -``` - -**Key Point:** Hardware audio rate (32kHz) is FIXED. We cannot change it. - -**Solution:** Adjust how fast we consume spectrogram data: -- `playback_speed = 2.0` → consume frames 2x faster → music plays 2x speed -- `playback_speed = 0.5` → consume frames 0.5x slower → music plays 0.5x speed - ---- - -## Implementation Roadmap - -### Step 1: Remove BPM from TrackerScore ✅ Ready -- Remove `float bpm` field -- Update tracker_compiler to not generate BPM -- Update tests - -### Step 2: Add Global Tempo State -```cpp -// In audio/tracker.h or main.cc -float g_current_tempo = 1.0f; // Multiplier: 1.0 = normal, 2.0 = double speed - -void tracker_set_tempo(float tempo_multiplier); -float tracker_get_tempo(); -``` - -### Step 3: Update tracker_update() -```cpp -void tracker_update(float music_time) { - // music_time is now abstract, not physical seconds - // Patterns trigger when music_time >= trigger.music_time -} -``` - -### Step 4: Update Pattern Compositing (Keep Reference Tempo) -```cpp -// Use fixed reference tempo for compositing (e.g., 120 BPM) -const float REFERENCE_BPM = 120.0f; -const float reference_beat_to_sec = 60.0f / REFERENCE_BPM; - -int frame_offset = (int)(event.beat * reference_beat_to_sec * 32000.0f / DCT_SIZE); -``` - -### Step 5: Variable Playback in Synth (Advanced) -```cpp -// Store playback speed with each voice -struct Voice { - float playback_speed; // Set when voice is triggered - float spectral_frame_pos; // Change to float for interpolation - // ... -}; - -// In synth_trigger_voice(): -v.playback_speed = g_current_tempo; - -// In synth_render(): -// Advance with fractional frame increment -v.spectral_frame_pos += v.playback_speed / DCT_SIZE; - -// Interpolate spectral data (required for smooth tempo changes) -``` - -### Step 6: Update Main Loop -```cpp -// In main.cc -static float g_music_time = 0.0f; -static float g_tempo_scale = 1.0f; // Can be animated! - -void update_game_logic(double physical_time) { - float dt = get_delta_time(); - - // Music time advances at variable rate - g_music_time += dt * g_tempo_scale; - - // Animate tempo (example) - g_tempo_scale = 1.0f + 0.5f * sinf(physical_time * 0.1f); // Oscillate 0.5x - 1.5x - - tracker_set_tempo(g_tempo_scale); - tracker_update(g_music_time); -} -``` - ---- - -## Open Questions - -1. **Pitch Shifting Acceptable?** - - If tempo changes, should pitch change too? - - For demoscene effect: probably YES (classic effect) - - For "realistic" music: NO (need time-stretching) - -2. **Tempo Curve Continuity** - - Should tempo changes be smoothed (acceleration curves)? - - Or instant jumps (may cause audio glitches)? - -3. **Spectral Frame Interpolation** - - Linear interpolation sufficient? - - Or need cubic/sinc for quality? - -4. **Tracker Compilation** - - Should tracker_compiler still know about tempo? - - Or output pure "beat" units? - ---- - -## Recommendation - -**For a 64k demo, I recommend:** - -### Minimal Change Approach (Easiest) -1. Keep BPM for compositing (as "reference tempo") -2. Add `g_current_tempo` multiplier -3. Make synth advance through frames at `tempo * playback_speed` -4. Accept pitch shifting as intentional effect - -**Changes needed:** -- Add `float g_current_tempo` global -- Update `synth_render()` to use `playback_speed` -- Main loop: `music_time += dt * g_current_tempo` -- Tracker: rename `time_sec` → `music_time` (conceptual only) - -**Size impact:** ~100 bytes -**Complexity:** Low -**Enables:** Full variable tempo with pitch shift effect - -### Advanced Approach (Better Quality) -- Implement spectral frame interpolation -- Add time-stretching (phase vocoder lite) -- Size impact: +2-3 KB - ---- - -## Conclusion - -**Is the code ready?** ❌ NO - -**What needs to change?** -1. BPM must become a "reference tempo" for compositing only -2. Add global `tempo_scale` variable -3. Synth must support variable playback speed -4. Main loop must track `music_time` separately from physical time - -**Sync with audio device:** -- Hardware rate (32kHz) is FIXED -- We control playback speed by advancing through spectral frames faster/slower -- This inherently changes pitch (unless we add time-stretching) - -**Recommended next step:** Implement minimal change approach first, then iterate. |
