path: root/ANALYSIS_VARIABLE_TEMPO.md

# Analysis: Variable Tempo / Unified Music Time

## Current Architecture Analysis

### 1. BPM Dependencies (Problems Found)

#### Location 1: `TrackerScore` (tracker.h:32)
```cpp
struct TrackerScore {
  const TrackerPatternTrigger* triggers;
  uint32_t num_triggers;
  float bpm;  // ❌ FIXED BPM - Cannot change dynamically
};
```

#### Location 2: `tracker_update()` (tracker.cc:57)
```cpp
float beat_to_sec = 60.0f / g_tracker_score.bpm;  // ❌ Used to convert beats→seconds
```

**Critical Issue (tracker.cc:84):**
```cpp
int frame_offset = (int)(event.beat * beat_to_sec * 32000.0f / DCT_SIZE);
```

**Problem:** The BPM is **baked into the spectrogram** at pattern trigger time!
- Events within a pattern are composited once with fixed spacing
- The resulting spectrogram frames encode the tempo
- Once triggered, the pattern plays at the encoded tempo
- **Cannot speed up/slow down dynamically**

#### Location 3: `main.cc` (lines 99, 166)
```cpp
// Bass pattern timing
if (t - last_beat_time > (60.0f / g_tracker_score.bpm) / 2.0) {  // ❌ BPM dependency

// Visual beat calculation
float beat = fmodf((float)current_time * g_tracker_score.bpm / 60.0f, 1.0f);  // ❌ BPM dependency
```

---

## Current Time Flow

```
Physical Time (hardware clock)
    ↓
platform_state.time (real seconds elapsed)
    ↓
current_time = platform_state.time + seek_offset
    ↓
tracker_update(current_time)  ← checks if patterns should trigger
    ↓
Pattern compositing: event.beat * (60.0 / BPM) * 32kHz
    ↓
Spectrogram generated with FIXED frame spacing
    ↓
synth_trigger_voice() → registers spectrogram
    ↓
Audio callback @ 32kHz pulls samples via synth_render()
    ↓
Voice advances through spectral frames at FIXED rate (one frame per DCT_SIZE samples)
```

**The Fundamental Problem:**
- Music time = Physical time (1:1 mapping)
- Patterns are pre-rendered at fixed tempo
- Synth playback rate is locked to 32kHz sample clock

---

## What Needs to Change

### Architecture Requirements

1. **Separate Music Time from Physical Time**
   - Music time advances independently: `music_time += dt * tempo_scale`
   - `tempo_scale = 1.0` → normal speed
   - `tempo_scale = 2.0` → double speed
   - `tempo_scale = 0.5` → half speed

2. **Remove BPM from Pattern Compositing**
   - Events should use abstract "beat" units, not seconds
   - Pattern generation should NOT convert beats→frames using BPM
   - Keep events in "music time" space

3. **Variable Playback Rate in Synth**
   - Synth must support dynamic playback speed
   - As tempo changes, synth advances through spectral frames faster/slower

---

## Proposed Solution

### Phase 1: Introduce Unified Music Time

#### Change 1: Update `TrackerScore`
```cpp
struct TrackerScore {
  const TrackerPatternTrigger* triggers;
  uint32_t num_triggers;
  // REMOVE: float bpm;
};
```

#### Change 2: Update `TrackerPatternTrigger`
```cpp
struct TrackerPatternTrigger {
  float music_time;  // Renamed from time_sec, now abstract units
  uint16_t pattern_id;
};
```

#### Change 3: Update `TrackerEvent`
```cpp
struct TrackerEvent {
  float beat;  // Keep as-is, already abstract
  uint16_t sample_id;
  float volume;
  float pan;
};
```

#### Change 4: Update `tracker_update()` signature
```cpp
// OLD:
void tracker_update(float time_sec);

// NEW:
void tracker_update(float music_time);
```

**Note:** Pattern compositing still needs tempo for now (see Phase 2).

---

### Phase 2: Variable Tempo Synth Playback

The hard part: **Dynamic playback speed without pitch shifting.**

#### Option A: Naive Approach (Pitch Shifts - NOT GOOD)
```cpp
// Simply advance through frames faster/slower
v.current_spectral_frame += playback_speed;  // ❌ Changes pitch!
```

**Problem:** Advancing through spectral frames faster/slower changes pitch.
- 2x speed → 2x frequency (octave up)
- 0.5x speed → 0.5x frequency (octave down)

#### Option B: Resample Spectrograms (Complex)
- Generate spectrograms at "reference tempo" (e.g., 1.0)
- At playback time, resample to match current tempo
- **Pros:** Preserves pitch
- **Cons:** CPU-intensive, requires interpolation

#### Option C: Time-Stretching (Best Quality)
- Use phase vocoder or similar algorithm
- Stretches/compresses time without changing pitch
- **Pros:** High quality
- **Cons:** Very complex, may exceed 64k budget

#### Option D: Pre-render at Reference Tempo (Recommended for 64k)
**Key Insight:** For a 64k demo, pre-render everything at a fixed internal tempo.

```cpp
// At pattern trigger time:
// 1. Composite pattern at "reference tempo" (e.g., 120 BPM baseline)
int frame_offset = (int)(event.beat * REFERENCE_BEAT_DURATION * 32000.0f / DCT_SIZE);

// 2. Store tempo multiplier with voice
struct Voice {
  float playback_speed;  // NEW: 1.0 = normal, 2.0 = double speed, etc.
  // ... existing fields
};

// 3. In synth_render(), advance at variable rate:
void synth_render(float* output_buffer, int num_frames) {
  for each voice {
    // Advance through spectral frames at tempo-adjusted rate
    float frame_advance = playback_speed / DCT_SIZE;

    // Use fractional frame index
    float spectral_frame_pos;  // NEW: float instead of int

    // Interpolate between frames
    int frame0 = (int)spectral_frame_pos;
    int frame1 = frame0 + 1;
    float frac = spectral_frame_pos - frame0;

    // Blend spectral data from frame0 and frame1
    for (int j = 0; j < DCT_SIZE; ++j) {
      windowed_frame[j] = lerp(spectral[frame0][j], spectral[frame1][j], frac);
    }
  }
}
```

**Trade-off:** This changes pitch! But for a demo, this might be acceptable.

---

## Sync with Physical Audio Device

### Current System
```
Audio Callback (32kHz fixed)
    ↓
synth_render(buffer, num_frames) ← pulls samples
    ↓
Voices advance: ++buffer_pos, ++current_spectral_frame
    ↓
Fixed playback rate (one spectral frame = DCT_SIZE samples)
```

### With Variable Tempo
```
Audio Callback (32kHz fixed - CANNOT CHANGE)
    ↓
synth_render(buffer, num_frames, current_playback_speed)
    ↓
Voices advance: buffer_pos += playback_speed
    ↓
Variable playback rate (playback_speed can be > 1.0 or < 1.0)
```

**Key Point:** Hardware audio rate (32kHz) is FIXED. We cannot change it.

**Solution:** Adjust how fast we consume spectrogram data:
- `playback_speed = 2.0` → consume frames 2x faster → music plays 2x speed
- `playback_speed = 0.5` → consume frames 0.5x slower → music plays 0.5x speed

---

## Implementation Roadmap

### Step 1: Remove BPM from TrackerScore ✅ Ready
- Remove `float bpm` field
- Update tracker_compiler to not generate BPM
- Update tests

### Step 2: Add Global Tempo State
```cpp
// In audio/tracker.h or main.cc
float g_current_tempo = 1.0f;  // Multiplier: 1.0 = normal, 2.0 = double speed

void tracker_set_tempo(float tempo_multiplier);
float tracker_get_tempo();
```

### Step 3: Update tracker_update()
```cpp
void tracker_update(float music_time) {
  // music_time is now abstract, not physical seconds
  // Patterns trigger when music_time >= trigger.music_time
}
```

### Step 4: Update Pattern Compositing (Keep Reference Tempo)
```cpp
// Use fixed reference tempo for compositing (e.g., 120 BPM)
const float REFERENCE_BPM = 120.0f;
const float reference_beat_to_sec = 60.0f / REFERENCE_BPM;

int frame_offset = (int)(event.beat * reference_beat_to_sec * 32000.0f / DCT_SIZE);
```

### Step 5: Variable Playback in Synth (Advanced)
```cpp
// Store playback speed with each voice
struct Voice {
  float playback_speed;  // Set when voice is triggered
  float spectral_frame_pos;  // Change to float for interpolation
  // ...
};

// In synth_trigger_voice():
v.playback_speed = g_current_tempo;

// In synth_render():
// Advance with fractional frame increment
v.spectral_frame_pos += v.playback_speed / DCT_SIZE;

// Interpolate spectral data (required for smooth tempo changes)
```

### Step 6: Update Main Loop
```cpp
// In main.cc
static float g_music_time = 0.0f;
static float g_tempo_scale = 1.0f;  // Can be animated!

void update_game_logic(double physical_time) {
  float dt = get_delta_time();

  // Music time advances at variable rate
  g_music_time += dt * g_tempo_scale;

  // Animate tempo (example)
  g_tempo_scale = 1.0f + 0.5f * sinf(physical_time * 0.1f);  // Oscillate 0.5x - 1.5x

  tracker_set_tempo(g_tempo_scale);
  tracker_update(g_music_time);
}
```

---

## Open Questions

1. **Pitch Shifting Acceptable?**
   - If tempo changes, should pitch change too?
   - For demoscene effect: probably YES (classic effect)
   - For "realistic" music: NO (need time-stretching)

2. **Tempo Curve Continuity**
   - Should tempo changes be smoothed (acceleration curves)?
   - Or instant jumps (may cause audio glitches)?

3. **Spectral Frame Interpolation**
   - Linear interpolation sufficient?
   - Or need cubic/sinc for quality?

4. **Tracker Compilation**
   - Should tracker_compiler still know about tempo?
   - Or output pure "beat" units?

---

## Recommendation

**For a 64k demo, I recommend:**

### Minimal Change Approach (Easiest)
1. Keep BPM for compositing (as "reference tempo")
2. Add `g_current_tempo` multiplier
3. Make synth advance through frames at `tempo * playback_speed`
4. Accept pitch shifting as intentional effect

**Changes needed:**
- Add `float g_current_tempo` global
- Update `synth_render()` to use `playback_speed`
- Main loop: `music_time += dt * g_current_tempo`
- Tracker: rename `time_sec` → `music_time` (conceptual only)

**Size impact:** ~100 bytes
**Complexity:** Low
**Enables:** Full variable tempo with pitch shift effect

### Advanced Approach (Better Quality)
- Implement spectral frame interpolation
- Add time-stretching (phase vocoder lite)
- Size impact: +2-3 KB

---

## Conclusion

**Is the code ready?** ❌ NO

**What needs to change?**
1. BPM must become a "reference tempo" for compositing only
2. Add global `tempo_scale` variable
3. Synth must support variable playback speed
4. Main loop must track `music_time` separately from physical time

**Sync with audio device:**
- Hardware rate (32kHz) is FIXED
- We control playback speed by advancing through spectral frames faster/slower
- This inherently changes pitch (unless we add time-stretching)

**Recommended next step:** Implement minimal change approach first, then iterate.