path: root/doc/AUDIO_TIMING_ARCHITECTURE.md

# Audio Timing Architecture - Proper Solution (February 7, 2026)

## Problem Statement

**Original Issue:** "demo is still flashing a lot" due to audio-visual timing mismatch.

**Root Causes:**
1. Multiple time sources with no clear hierarchy
2. Hardcoded latency constants (400ms) in solution proposals
3. Beat calculation using wrong time source
4. Peak decay rate not matched to music tempo

---

## Correct Architecture ✅

### Single Source of Truth: Physical Clock

```cpp
platform_get_time() → ONE authoritative wall clock from OS
```

**Everything else derives from this:**

```
Physical Time (platform_get_time())
    ↓
┌────────────────────────────────────────────────┐
│ Audio System tracks its own state:            │
│  • audio_get_playback_time()                  │
│    → Based on ring buffer samples consumed    │
│    → Automatically accounts for buffering     │
│    → NO hardcoded constants!                  │
└────────────────────────────────────────────────┘
    ↓
┌────────────────────────────────────────────────┐
│ Music Time (tracker time)                     │
│  • Derived from audio playback time           │
│  • Scaled by tempo_scale                      │
│  • Used by tracker for event triggering       │
└────────────────────────────────────────────────┘
```

### Time Sources Summary

| Time Source | Purpose | How to Get | Use For |
|-------------|---------|------------|---------|
| **Physical Time** | Wall clock, frame deltas | `platform_get_time()` | dt calculations, physics |
| **Audio Playback Time** | What's being HEARD | `audio_get_playback_time()` | Audio-visual sync, beat display |
| **Music Time** | Tracker time (tempo-scaled) | `g_music_time` | Tracker event triggering |

---

## Implementation: test_demo.cc

### Before (Wrong ❌)

```cpp
const double current_time = platform_state.time;  // Physical time

// Beat calculation based on physical time
const float beat_time = (float)current_time * 120.0f / 60.0f;

// But peak is measured at audio playback time (400ms behind!)
const float raw_peak = audio_get_realtime_peak();

// MISMATCH: beat and peak are from different time sources!
```

**Problem:** Visual beat shows beat 2 (physical time), but peak shows beat 1 (audio playback time).

### After (Correct ✅)

```cpp
const double physical_time = platform_state.time;  // For dt calculations

// Audio playback time: what's being HEARD right now
const float audio_time = audio_get_playback_time();

// Beat calculation uses AUDIO TIME (matches peak measurement)
const float beat_time = audio_time * 120.0f / 60.0f;

// Peak is measured at audio playback time
const float raw_peak = audio_get_realtime_peak();

// SYNCHRONIZED: beat and peak are from same time source!
```

**Result:** Visual beat shows beat 1, peak shows beat 1 → synchronized! ✅

---

## How audio_get_playback_time() Works

**Implementation** (audio.cc:169-173):
```cpp
float audio_get_playback_time() {
  const int64_t total_samples = g_ring_buffer.get_total_read();
  return (float)total_samples / (RING_BUFFER_SAMPLE_RATE * RING_BUFFER_CHANNELS);
}
```

**Key Points:**
1. **Tracks samples consumed** by audio callback (not samples rendered)
2. **Automatically accounts for ring buffer latency** (no hardcoded constants!)
3. **Self-consistent** with `audio_get_realtime_peak()` (measured at same moment)

**Example Timeline:**
```
T(physical) = 1.00s:
  → Ring buffer has 400ms of lookahead
  → Audio callback is playing samples rendered at T=0.60s (music time)
  → total_read counter reflects 0.60s worth of samples
  → audio_get_playback_time() returns 0.60s
  → audio_get_realtime_peak() measured from samples at 0.60s
  → Beat calculation: 0.60s * 2 = 1.2 → beat 1
  → SYNCHRONIZED! ✅
```

---

## Remaining Issues: Data-Driven Configuration

### Issue #1: Hardcoded Decay Rate

**Current** (miniaudio_backend.cc:166):
```cpp
realtime_peak_ *= 0.5f;  // Hardcoded: 50% per callback
```

**Problem:** Decay rate should match music tempo, not be hardcoded!

**Proposed Solution:**
```cpp
// AudioBackend should query decay rate from audio system:
float decay_rate = audio_get_peak_decay_rate();  // Returns BPM-adjusted rate
realtime_peak_ *= decay_rate;
```

**How to calculate:**
```cpp
// In audio system (based on current BPM):
float audio_get_peak_decay_rate() {
  const float bpm = tracker_get_bpm();  // e.g., 120
  const float beat_interval = 60.0f / bpm;  // e.g., 0.5s
  const float callback_interval = 0.128f;  // Measured from device

  // Decay to 10% within one beat:
  // decay_rate^(beat_interval / callback_interval) = 0.1
  // decay_rate = 0.1^(callback_interval / beat_interval)

  const float num_callbacks_per_beat = beat_interval / callback_interval;
  return powf(0.1f, 1.0f / num_callbacks_per_beat);
}
```

**Result:** At 120 BPM, decay to 10% in 0.5s (1 beat). At 60 BPM, decay to 10% in 1.0s (1 beat). Adapts automatically!

---

### Issue #2: Hardcoded BPM

**Current** (test_demo.cc:305):
```cpp
const float beat_time = audio_time * 120.0f / 60.0f;  // Hardcoded BPM
```

**Problem:** BPM should come from tracker/music data!

**Proposed Solution:**
```cpp
// Tracker should expose BPM:
const float bpm = tracker_get_bpm();  // From TrackerScore
const float beat_time = audio_time * bpm / 60.0f;

// Or even better, tracker calculates beat directly:
const float beat = tracker_get_current_beat(audio_time);
```

**Implementation:**
```cpp
// In tracker.h/cc:
float tracker_get_bpm() {
  return g_tracker_score.bpm;  // From parsed .track file
}

float tracker_get_current_beat(float audio_time) {
  return audio_time * (g_tracker_score.bpm / 60.0f);
}
```

**Result:** Change BPM in `.track` file → everything updates automatically!

---

### Issue #3: No API for "What time is it in sequence world?"

**User's Suggestion:**
> "ask the AudioSystem or demo system (MainSequence?) what 'time' it is in the sequence world"

**Current Approach:** Each system tracks its own time independently
- test_demo.cc: Uses `audio_get_playback_time()` directly
- main.cc: Uses `platform_state.time + seek_time`
- MainSequence: Uses `global_time` parameter passed to `render_frame()`

**Problem:** No central "what time should I use?" API

**Proposed API:**
```cpp
// In MainSequence or AudioEngine:
class TimeProvider {
 public:
  // Returns: Current time in "sequence world" (accounting for all latencies)
  float get_current_time() const {
    return audio_get_playback_time();  // Use audio playback time
  }

  // Returns: Current beat (fractional)
  float get_current_beat() const {
    return get_current_time() * (bpm_ / 60.0f);
  }

  // Returns: Current peak (synchronized with current time)
  float get_current_peak() const {
    return audio_get_realtime_peak();  // Already synchronized
  }
};

// Usage in test_demo.cc or main.cc:
const float time = g_time_provider.get_current_time();
const float beat = g_time_provider.get_current_beat();
const float peak = g_time_provider.get_current_peak();

// All guaranteed to be synchronized!
```

**Benefits:**
- Single point of query for all timing
- Hides implementation details (ring buffer, latency, etc.)
- Easy to change timing strategy globally
- Clear contract: "This is the time to use for audio-visual sync"

---

## Next Steps

### Completed ✅
1. ✅ Use `audio_get_playback_time()` instead of physical time for beat calculation
2. ✅ Faster decay rate (0.5 instead of 0.7) to prevent constant flashing
3. ✅ Peak meter visualization to verify timing visually
4. ✅ No hardcoded latency constants (system queries its own state)

### Future Work (Deferred)

#### Task: Add tracker_get_bpm() API
**Purpose:** Read BPM from `.track` file instead of hardcoding in test_demo.cc/main.cc

**Implementation:**
```cpp
// In tracker.h:
float tracker_get_bpm();  // Returns g_tracker_score.bpm

// In tracker.cc:
float tracker_get_bpm() {
  return g_tracker_score.bpm;
}

// Usage in test_demo.cc/main.cc:
const float bpm = tracker_get_bpm();  // Instead of hardcoded 120.0f
const float beat_time = audio_time * (bpm / 60.0f);
```

**Benefits:**
- Change BPM in `.track` file → everything updates automatically
- No hardcoded BPM values in demo code
- Supports variable BPM (future enhancement)

---

#### Task: BPM-Aware Peak Decay Rate
**Purpose:** Calculate decay rate based on current BPM to match beat interval

**Implementation:**
```cpp
// In audio.h:
float audio_get_peak_decay_rate();  // BPM-adjusted decay

// In audio.cc:
float audio_get_peak_decay_rate() {
  const float bpm = tracker_get_bpm();
  const float beat_interval = 60.0f / bpm;  // e.g., 0.5s at 120 BPM
  const float callback_interval = 0.128f;   // Measured from device

  // Decay to 10% within one beat:
  const float n = beat_interval / callback_interval;
  return powf(0.1f, 1.0f / n);
}

// In miniaudio_backend.cc:
realtime_peak_ *= audio_get_peak_decay_rate();  // Instead of hardcoded 0.5f
```

**Benefits:**
- Peak decays in exactly 1 beat (regardless of BPM)
- At 120 BPM: decay = 0.5 (500ms fade)
- At 60 BPM: decay = 0.7 (1000ms fade)
- Adapts automatically to tempo changes

---

#### Task: TimeProvider Class (Architectural)
**Purpose:** Centralize all timing queries with single source of truth

**Design:**
```cpp
// In audio/time_provider.h:
class TimeProvider {
 public:
  TimeProvider();

  // Returns: Current time in "sequence world" (what's being heard)
  float get_current_time() const {
    return audio_get_playback_time();
  }

  // Returns: Current beat (fractional, BPM-aware)
  float get_current_beat() const {
    const float bpm = tracker_get_bpm();
    return get_current_time() * (bpm / 60.0f);
  }

  // Returns: Current peak (synchronized with current time)
  float get_current_peak() const {
    return audio_get_realtime_peak();
  }

  // Returns: Current BPM
  float get_bpm() const {
    return tracker_get_bpm();
  }
};

// Usage in test_demo.cc, main.cc, effects:
extern TimeProvider g_time_provider;  // Global or MainSequence member

const float time = g_time_provider.get_current_time();
const float beat = g_time_provider.get_current_beat();
const float peak = g_time_provider.get_current_peak();

// All guaranteed to be synchronized!
```

**Integration with MainSequence:**
```cpp
class MainSequence {
 public:
  TimeProvider time_provider;

  void render_frame(float global_time, float beat, float peak,
                    float aspect_ratio, WGPUSurface surface) {
    // Effects can query: time_provider.get_current_time() etc.
  }
};
```

**Benefits:**
- Single point of query for all timing
- Hides implementation details (ring buffer, latency)
- Easy to change timing strategy globally
- Clear contract: "This is the time for audio-visual sync"
- No more passing time parameters everywhere

**Migration Path:**
1. Create TimeProvider class
2. Expose as global or MainSequence member
3. Gradually migrate test_demo.cc, main.cc, effects to use it
4. Remove time/beat/peak parameters from render functions
5. Everything queries TimeProvider directly

---

### Design Principles Established

1. ✅ **Single physical clock:** `platform_get_time()` is the only wall clock
2. ✅ **Systems expose their state:** `audio_get_playback_time()` knows its latency
3. ✅ **No hardcoded constants:** System queries its own state dynamically
4. ✅ **Data-driven configuration:** BPM from tracker, decay from BPM (future)
5. ✅ **Synchronized time sources:** Beat and peak from same moment

---

## Testing Verification

### With Peak Meter Visualization

Run `./build/test_demo` and observe:
- ✅ Red bar extends when kicks hit (beats 0, 2, 4, ...)
- ✅ Bar width matches FlashEffect intensity
- ✅ Bar decays before next beat (no constant red bar)
- ✅ Snares show narrower bar width (~50-70%)

### With Peak Logging

Run `./build/test_demo --log-peaks peaks.txt` and verify:
```bash
# Expected pattern (120 BPM, kicks every 1s):
Beat 0 (T=0.0s): High peak (kick)
Beat 1 (T=0.5s): Medium peak (snare)
Beat 2 (T=1.0s): High peak (kick)
Beat 3 (T=1.5s): Medium peak (snare)
...
```

### Console Output

Should show:
```
[AudioT=0.06, Beat=0, Frac=0.13, Peak=1.00]  ← Kick
[AudioT=0.58, Beat=1, Frac=0.15, Peak=0.62]  ← Snare (quieter)
[AudioT=1.09, Beat=2, Frac=0.18, Peak=0.16]  ← Decayed (between beats)
[AudioT=2.62, Beat=5, Frac=0.25, Peak=1.00]  ← Kick
```

**No more constant Peak=1.00 from beat 15 onward!**

---

## Summary

### What We Fixed
1. ✅ **Use audio playback time** instead of physical time for beat calculation
2. ✅ **Faster decay** (0.5 instead of 0.7) to match beat interval
3. ✅ **No hardcoded latency** - system queries its own state

### What Still Needs Improvement
1. ⚠️ **BPM should come from tracker** (not hardcoded 120)
2. ⚠️ **Decay rate should be calculated from BPM** (not hardcoded 0.5)
3. ⚠️ **Centralized TimeProvider** for all timing queries

### Key Insight (User's Contribution)
> "There should be a unique tick-source somewhere, that is the real physical_time. Then, we shouldn't hardcode the constants like 400ms, but really ask the AudioSystem or demo system (MainSequence?) what 'time' it is in the sequence world."

**This is the correct architectural principle!** ✅
- ONE physical clock (platform_get_time)
- Systems expose their own state (audio_get_playback_time)
- No hardcoded constants - query the system
- Data-driven configuration (BPM from tracker)

---

*Created: February 7, 2026*
*Architectural discussion and implementation complete*