1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
|
# To-Do List
This file tracks prioritized tasks with detailed attack plans.
## Recently Completed (February 6, 2026)
- [x] **Critical Shader Bug Fixes & Test Infrastructure** (February 6, 2026)
- [x] **Shader Validation Errors**: Fixed three critical WGSL bugs causing demo64k and test_3d_render crashes:
- Removed dead code using non-existent `inverse()` function in renderer_3d.wgsl
- Fixed `get_normal_basic()` signature mismatch in sdf_utils.wgsl and lighting.wgsl (obj_type: f32 → obj_params: vec4<f32>)
- Fixed scene_query_linear.wgsl incorrectly declaring binding 2 (BVH buffer) - was identical to BVH version due to copy-paste error
- [x] **Root Cause**: Linear shader mode expected no binding 2, but shader declared it, causing pipeline/shader mismatch
- [x] **New Test Coverage**: Created `test_shader_compilation.cc` that compiles all production shaders through WebGPU, tests both BVH and Linear composition modes, validates WGSL syntax/bindings/types
- [x] **Test Gap Analysis**: Existing test_shader_assets only checked keywords, not actual compilation. New test would have caught all three bugs.
- **Result**: demo64k runs without WebGPU errors, test_3d_render no longer crashes, 22/23 tests pass (FftTest unrelated), comprehensive regression prevention
- [x] **Task C: Build System Optimization** (February 6, 2026)
- [x] **Header Split**: Refactored `asset_manager.h` into `asset_manager_dcl.h` (forward declarations), `asset_manager.h` (core API), and `asset_manager_utils.h` (typed helpers for TextureAsset/MeshAsset).
- [x] **Asset Dependency Tracking**: Added file-level dependencies for all assets (.wgsl shaders, .spec audio, .obj meshes). CMake now tracks 42 demo assets + 17 test assets individually.
- [x] **Performance Impact**: Editing TextureAsset/MeshAsset helpers: 4.82s → 2.01s (58% faster). Editing shaders now triggers correct rebuild (was broken before - stale code bug fixed).
- [x] **Developer Workflow**: No more manual `touch demo_assets.txt` workaround needed. Edit any asset file and rebuild works correctly.
- **Result**: Critical correctness bug fixed (shader changes were not triggering rebuilds). Build system now has proper dependency tracking for all source and asset files.
## Recently Completed (February 5, 2026)
- [x] **Audio Lifecycle Refactor (Task #56)**:
- [x] **Phase 1: Design & Prototype**: Created `AudioEngine` class and `SpectrogramResourceManager` to manage audio subsystem initialization and resource loading with lazy loading strategy.
- [x] **Phase 2: Test Migration**: Migrated all tracker-related tests (`test_tracker.cc`, `test_tracker_timing.cc`, `test_variable_tempo.cc`, `test_wav_dump.cc`) to use AudioEngine instead of direct synth/tracker initialization.
- [x] **Phase 3: Production Integration**: Updated `main.cc` to use AudioEngine, eliminating initialization order fragility in production code. Fixed pre-existing demo crash (procedural texture loading).
- [x] **Phase 4: Cleanup & Documentation**: Removed backwards compatibility (synth_init() from audio_init()), updated HOWTO.md and CONTRIBUTING.md with AudioEngine usage patterns and best practices.
- **Result**: Initialization order dependency eliminated. All 20 tests pass. Binary size impact <500 bytes. Demo runs successfully.
- [x] **Physics & Collision (Task #49)**:
- [x] **CPU-Side SDF Library**: Implemented `sdSphere`, `sdBox`, `sdTorus`, `sdPlane` in `src/3d/sdf_cpu.h` using `mini_math.h`. Added `calc_normal` for numerical gradients.
- [x] **BVH Construction**: Implemented `BVHNode` and `BVHBuilder` in `src/3d/bvh.h/cc`. Optimized broad-phase collision queries with `BVH::Query`.
- [x] **Physics System**: Created `PhysicsSystem` in `src/3d/physics.h/cc` implementing semi-implicit Euler integration, broad-phase BVH culling, and narrow-phase SDF proxy probing.
- [x] **Integration & Visualization**: Added `velocity`, `mass`, `is_static` to `Object3D`. Integrated physics loop into `test_3d_render.cc` and added BVH wireframe visualization to `VisualDebug`.
- [x] **Audio Playback Debugging & Core Audio Optimization**:
- [x] **Core Audio Timing Fix**: Resolved stop-and-go audio glitches caused by timing mismatch. Core Audio optimized for 44.1kHz (10ms periods), but our 32kHz system expected uniform ~13.78ms callbacks, causing resampling jitter. Fix: Added `allowNominalSampleRateChange = TRUE` to force OS-level 32kHz native and `performanceProfile = conservative` for 4096-frame buffers (128ms). Result: Stable ~128ms callbacks, <1ms jitter, zero underruns.
- [x] **Ring Buffer Capacity**: Increased from 200ms to 400ms (25,600 samples) to handle tempo scaling headroom. Added comprehensive bounds checking with abort() on violations.
- [x] **Tempo-Scaled Buffer Fill**: Fixed critical bug where buffer pre-fill didn't scale dt by tempo (`audio_render_ahead(g_music_time, dt * g_tempo_scale)`). Buffer now maintains 400ms fullness during 2.0x acceleration.
- [x] **Extensive Diagnostics**: Added high-resolution timing tracking (clock_gettime), callback interval measurement, buffer level monitoring, underrun detection. All under conditional compilation for future debugging.
- [x] **NOTE_ Parsing Fix & Sample Caching**:
- [x] **Parser Bug Fix**: Fixed `is_note_name()` checking only first letter (A-G), causing ASSET_KICK_1 → A0 (27.5 Hz) false positives. Required "NOTE_" prefix to distinguish notes from assets. Updated music.track to use NOTE_E2, NOTE_G4 format.
- [x] **Resource Exhaustion Discovery**: Found every event created NEW spectrogram (14 unique samples → 228 registrations). MAX_SPECTROGRAMS=16 insufficient, causing spectrogram_id=-1 errors.
- [x] **Comprehensive Caching**: Implemented cache in `tracker_init()` pre-registering all samples. Assets: loaded once from AssetManager. Generated notes: created once, stored in persistent pool. All cached synth_ids reused.
- [x] **Memory Reduction**: MAX_SPECTROGRAMS reduced from 256 (temporary workaround) to 32 (2.3x headroom over 14 actual). 88% memory savings vs uncached approach.
- [x] **Resource Analysis Tool**: Enhanced `tracker_compiler` to report required/recommended pool sizes, cache potential, memory usage. Analysis showed 152/228 required without caching, 14 with caching.
- [x] **Debug Logging Infrastructure**:
- [x] **Central Debug Header**: Created `src/util/debug.h` with 7 category macros (DEBUG_LOG_AUDIO, DEBUG_LOG_RING_BUFFER, DEBUG_LOG_TRACKER, DEBUG_LOG_SYNTH, DEBUG_LOG_3D, DEBUG_LOG_ASSETS, DEBUG_LOG_GPU).
- [x] **CMake Integration**: Added `DEMO_ENABLE_DEBUG_LOGS` option defining `DEBUG_LOG_ALL` to enable all categories. Individual categories can be enabled selectively.
- [x] **Source Code Conversion**: Updated miniaudio_backend.cc (timing, device config), ring_buffer.cc (underruns), tracker.cc (validation), synth.cc (parameter checks) to use category macros.
- [x] **Zero Runtime Cost**: Default build: macros compile to `((void)0)`. Debug build: comprehensive logging preserved for troubleshooting.
- [x] **Pre-Commit Policy**: Updated CONTRIBUTING.md requiring debug build verification before significant commits to ensure diagnostic code remains maintainable.
- [x] **Verification**: Both default and debug builds compile without errors. Audio playback works correctly in both modes.
- [x] **Event-Based Tracker for Tempo Scaling**:
- [x] **Problem Identified**: Notes within patterns didn't accelerate with tempo changes. Pattern events were pre-composited into single spectrograms at fixed positions.
- [x] **Refactored Architecture**: Changed from pattern compositing to individual event triggering. Each TrackerEvent now triggers as separate voice.
- [x] **Dynamic Beat Calculation**: `elapsed_beats = (music_time - start_time) / beat_duration` allows notes to respect tempo scaling.
- [x] **ActivePattern Tracking**: Added structure to track pattern_id, start_music_time, and next_event_idx for each active pattern instance.
- [x] **Removed Compositing Logic**: Deleted paste_spectrogram approach that baked events at fixed frame offsets.
- [x] **Full Tempo Scaling**: At 2.0x tempo, both pattern triggering AND note spacing play 2x faster. At 0.5x tempo, both play 2x slower.
- [x] **Testing**: Updated test_tracker.cc to verify individual event triggers at specific beat times. All 17 tests pass.
- [x] **WAV Verification**: Confirmed with WAV dump showing 61.24s music time in 60s physical time during tempo transitions.
- [x] **WAV Dump Backend for Debugging**:
- [x] **Implementation**: Created WavDumpBackend implementing AudioBackend interface to render audio offline to .wav files.
- [x] **Command-Line Option**: Added `--dump_wav output.wav` flag to enable offline rendering instead of live playback.
- [x] **Stereo Bug Fix**: Fixed critical mono/stereo mismatch. Synth outputs STEREO (interleaved L/R) but initial implementation wrote MONO.
- [x] **Correct Allocation**: Changed to allocate `frames * 2` samples and write stereo format (num_channels = 2).
- [x] **Format Matching**: WAV header now correctly specifies 16-bit PCM, stereo, 32kHz (matches live audio exactly).
- [x] **Regression Test**: Added test_wav_dump.cc with critical assertion `assert(header.num_channels == 2)` to prevent future mismatches.
- [x] **Tempo Simulation**: WAV dump backend simulates tempo scaling matching main.cc logic for accurate offline rendering.
- [x] **All Tests Pass**: 17/17 tests pass including WAV format verification.
- [x] **Variable Tempo System**:
- [x] **Music Time Abstraction**: Implemented unified music time in `main.cc` that advances at `tempo_scale` rate, decoupling from physical time.
- [x] **Tempo Control**: Added `g_tempo_scale` (default 1.0) allowing future dynamic tempo changes without pitch shifting.
- [x] **Reset Tricks**: Comprehensive tests verify 2x speed-up and 2x slow-down reset techniques work correctly.
- [x] **Test Suite**: Created `test_variable_tempo.cc` with 6 test scenarios: basic scaling, speed-up/slow-down resets, pattern density swap, continuous acceleration, oscillating tempo.
- [x] **Perfect Verification**: All tests pass, confirming music_time advances correctly at variable rates (e.g., 2.0x tempo → 2x faster triggering).
- [x] **Zero Pitch Shift**: Spectrograms unchanged, only trigger timing affected (as designed).
- [x] **Documentation**: Created `ANALYSIS_VARIABLE_TEMPO_V2.md` explaining simplified trigger-timing approach.
- [x] **Task #51.3 & #51.4: Tracker Test Suite & Build Integration**:
- [x] **Comprehensive Test Suite**: Created `test_tracker_timing.cc` with 7 test scenarios using MockAudioBackend.
- [x] **Simultaneous Trigger Verification**: Confirmed multiple patterns at same time trigger with **0.000ms delta** (perfect sync).
- [x] **Test Coverage**: Basic recording, progressive triggering, simultaneous triggers, monotonicity, seek/fast-forward, timestamp clustering, render integration.
- [x] **Real Music Data**: Tests use generated tracker music data for realistic validation.
- [x] **Build Integration**: Added to CMake with proper dependencies on generated music data.
- [x] **All Tests Pass**: 15/15 tests passing (100% success rate).
- [x] **Task #51.2: Mock Audio Backend**:
- [x] **Event Recording**: Created `VoiceTriggerEvent` structure to capture timestamp, spectrogram_id, volume, and pan.
- [x] **MockAudioBackend Class**: Implemented test-only backend with event recording and time tracking capabilities.
- [x] **Time Management**: Added `advance_time()`, `set_time()`, and `get_current_time()` for deterministic testing.
- [x] **Frame Rendering Hook**: Implemented `on_frames_rendered()` to automatically update time based on audio frames (32kHz).
- [x] **Synth Integration**: Verified mock backend correctly captures voice triggers from synth engine.
- [x] **Comprehensive Tests**: Created `test_mock_backend.cc` with 6 test scenarios covering all mock functionality.
- [x] **Build Integration**: Added mock backend to test builds, all 14 tests pass.
- [x] **Task #51.1: Audio Backend Abstraction**:
- [x] **Interface Created**: Defined `AudioBackend` interface in `src/audio/audio_backend.h` with hooks for voice triggering and frame rendering.
- [x] **Production Backend**: Moved miniaudio implementation from `audio.cc` to `MiniaudioBackend` class, maintaining backward compatibility.
- [x] **Audio Refactoring**: Updated `audio.cc` to use backend abstraction with automatic fallback to `MiniaudioBackend`.
- [x] **Event Hooks**: Added time tracking to `synth.cc` with `on_voice_triggered()` callbacks (guarded by `!STRIP_ALL`).
- [x] **Verification Test**: Created `test_audio_backend.cc` to verify backend injection and event recording work correctly.
- [x] **Build Integration**: Updated CMakeLists.txt to include new backend files and link audio tests properly with util/procedural dependencies.
- [x] **Zero Size Impact**: All test infrastructure under `#if !defined(STRIP_ALL)`, production path unchanged.
- [x] **Task #50: WGSL Modularization**:
- [x] **Recursive Composition**: Updated `ShaderComposer` to support recursive `#include "snippet_name"` directives with cycle detection.
- [x] **Granular SDF Library**: Extracted `math/sdf_shapes.wgsl`, `math/sdf_utils.wgsl`, `render/shadows.wgsl`, `render/scene_query.wgsl`, and `render/lighting_utils.wgsl`.
- [x] **Pipeline Update**: Refactored `Renderer3D` and `renderer_3d.wgsl` to use the new modular system, reducing C++-side dependency management.
- [x] **Platform Fix**: Resolved `WGPUShaderSourceWGSL` usage on macOS to ensure compatibility with composed shader strings.
- [x] **Task #48: Improve Audio Coverage**:
- [x] **New Tests**: Added `test_dct` (100% coverage for transforms) and `test_audio_gen` (94% coverage for procedural audio).
- [x] **Enhanced Tests**: Updated `test_synth` to cover rendering loop, double-buffering, and resource exhaustion.
- [x] **Coverage Boost**: Increased `src/audio/` coverage from ~42% to 93%.
- [x] **Task #47: Improve Asset Manager Coverage**:
- [x] **New Tests**: Added tests for unknown procedural functions, generation failures, and edge cases in `src/tests/test_assets.cc`.
- [x] **Tooling Update**: Downgraded `asset_packer` validation error to warning to allow testing invalid assets.
- [x] **Coverage Boost**: Increased `src/util/asset_manager.cc` coverage from 71% to 88%.
- [x] **Task #46: Enhance Coverage Script**: Updated `scripts/gen_coverage_report.sh` to accept an optional directory argument for targeted coverage reports (e.g., `src/procedural`).
- [x] **Task #45: Improve Procedural Generation Coverage**:
- [x] **Unit Tests:** Implemented comprehensive tests for `gen_perlin`, `make_periodic`, and default parameter handling in `src/tests/test_procedural.cc`.
- [x] **Coverage Boost:** Increased `src/procedural/generator.cc` coverage from 38% to 96%.
- [x] **Task #44: Developer Tooling (Coverage)**:
- [x] **Implement Code Coverage:** Added `DEMO_ENABLE_COVERAGE` CMake option and created `scripts/gen_coverage_report.sh` to generate HTML coverage reports using `lcov` on macOS.
- [x] **Documentation:** Updated `doc/HOWTO.md` with usage instructions.
- [x] **Skybox & Two-pass Rendering Stability**:
- [x] **Fixed Two-pass Rendering:** Implemented mandatory clear operations for color and depth when the skybox is absent, preventing black screens and depth validation errors.
- [x] **Implemented Rotating Skybox:** Added `inv_view_proj` to `GlobalUniforms` and updated the skybox shader to perform world-space ray unprojection (`inv_view_proj`), enabling correct rotation with the camera.
- [x] **Enhanced Procedural Noise:** Implemented a multi-octave Value Noise generator for higher-quality skybox textures.
- [x] **Scene Integrity:** Restored proper object indexing and removed redundant geometry, ensuring the floor grid and objects render correctly.
- [x] **Task #57: Interactive Timeline Editor (Phase 1 Complete)** 🎉
- [x] **Core Parser & Renderer**: Implemented demo.seq parser with BPM, beat notation, priority modifiers. Gantt-style timeline rendering with dynamic sequence/effect positioning.
- [x] **Drag & Drop**: Sequences and effects draggable along timeline with proper offset calculation. Fixed critical e.target vs e.currentTarget bug preventing erratic jumping.
- [x] **Resize Handles**: Left/right handles on selected effects. Allow negative relative times (effects extend before sequence start).
- [x] **Snap-to-Beat**: Checkbox toggle with beat markers. Automatic snapping when dragging in beat mode.
- [x] **Properties Panel**: Floating, collapsible panel with auto-apply (no Apply button). Properties update on input change.
- [x] **Stack-Order Priority**: Up/Down buttons to reorder effects within sequence. Stack position determines rendering priority (higher index = rendered later = on top).
- [x] **Priority Modifiers**: Toggle button for "Same as Above" (= modifier) vs "Increment" (+ modifier). Visual feedback in button text.
- [x] **Diagonal Scroll**: Mouse wheel navigation with 10% viewport slack. Smooth following to time-ordered sequence cascade. Flash animation on active sequence change.
- [x] **Dynamic Bounds**: Sequence visual bounds calculated from min/max effect times. Cumulative Y positioning prevents overlap.
- [x] **Waveform Visualization**: Load WAV files (via Web Audio API) and display waveform above timeline. Scales with zoom (pixels per second). Documented integration with `--dump_wav` flag.
- [x] **UI Polish**: Hoverable sequence names (large centered text, fades on hover). No scrollbars (hidden CSS). Clean, minimal interface.
- [x] **File I/O**: Load/save demo.seq files with proper serialization. BPM parsing and display. Re-order sequences by time.
- [x] **Delete & Add**: Delete sequences/effects. Add new sequences (at time 0, priority 0).
- **Result**: Fully functional timeline editor ready for production use. Phase 1.1 complete (basic editing, snap-to-beat, waveform). Phase 1.2 (Add Effect button) and Phase 2.5 (music.track visualization) documented in ROADMAP.md.
- **Files**: `tools/timeline_editor/index.html` (~1200 lines), `README.md`, `ROADMAP.md` (3 phases, 117-161 hour estimate).
## Critical Fixes
- [ ] **Task A: Fix Test Mesh** (Partially Complete)
- [x] Fixed mesh normal transformation bug (double-transpose in shaders)
- [x] Fixed quaternion rotation stretching (non-normalized axis in quat::from_axis)
- [x] Fixed mesh shadow scaling (excluded meshes from SDF scale factor)
- [x] Fixed floor rendering artifacts (changed from PLANE to BOX)
- [ ] **Task A.1**: Investigate and fix missing shadows in test_mesh
- [ ] **Task A.2**: Investigate and fix ObjectType::PLANE with non-uniform scaling
- **Issue**: PLANE with extreme non-uniform scaling (e.g., vec3(20, 0.01, 20)) causes incorrect SDF distance calculations in shadows
- **Workaround**: Changed test_mesh floor to use ObjectType::BOX instead
- **Root Cause**: sdPlane distance calculation doesn't account for non-uniform scaling when transformed to local space
- **Investigation needed**: Determine if PLANE should handle non-uniform scaling or if it's unsupported by design
- [ ] **Task B: Move platform-specific conditional code into a single header location**
- [ ] Abstract out `#if defined(DEMO_CROSS_COMPILE_WIN32)` statements from core `.cc` and `.h` sources.
- [ ] Centralize platform-specific code and `#ifdef`s into `platform.h` or equivalent.
- [ ] Address `str_view()` calls that cause compilation breaks.
## Priority 1: Spectral Brush Editor (Task #5) [IN PROGRESS]
**Goal:** Create a web-based tool for procedurally tracing audio spectrograms. Replaces large `.spec` binary assets with tiny procedural C++ code (50-100× compression).
**Design Document:** See `doc/SPECTRAL_BRUSH_EDITOR.md` for complete architecture.
**Core Concept: "Spectral Brush"**
- **Central Curve** (Bezier): Traces time-frequency path through spectrogram
- **Vertical Profile**: Shapes "brush stroke" around curve (Gaussian, Decaying Sinusoid, Noise)
**Workflow:**
```
.wav → Load in editor → Trace with Bezier curves → Export procedural_params.txt + C++ code
```
### Phase 1: C++ Runtime (Foundation)
- [ ] **Files:** `src/audio/spectral_brush.h`, `src/audio/spectral_brush.cc`
- [ ] Define API (`ProfileType`, `draw_bezier_curve()`, `evaluate_profile()`)
- [ ] Implement linear Bezier interpolation
- [ ] Implement Gaussian profile evaluation
- [ ] Implement home-brew deterministic RNG (for future noise support)
- [ ] Add unit tests (`src/tests/test_spectral_brush.cc`)
- [ ] **Deliverable:** Compiles, tests pass
### Phase 2: Editor Core
- [ ] **Files:** `tools/spectral_editor/index.html`, `script.js`, `style.css`, `dct.js` (reuse from old editor)
- [ ] HTML structure (canvas, controls, file input)
- [ ] Canvas rendering (dual-layer: reference + procedural)
- [ ] Bezier curve editor (click to place, drag to adjust, delete control points)
- [ ] Profile controls (Gaussian sigma slider)
- [ ] Real-time spectrogram rendering
- [ ] Audio playback (IDCT → Web Audio API)
- [ ] Undo/Redo system (action history with snapshots)
- [ ] **Keyboard shortcuts:**
- Key '1': Play procedural sound
- Key '2': Play original .wav
- Space: Play/pause
- Ctrl+Z: Undo
- Ctrl+Shift+Z: Redo
- Delete: Remove control point
- [ ] **Deliverable:** Interactive editor, can trace .wav files
### Phase 3: File I/O
- [ ] Load .wav (decode, FFT/STFT → spectrogram)
- [ ] Load .spec (binary format parser)
- [ ] Save procedural_params.txt (human-readable, re-editable)
- [ ] Generate C++ code (ready to compile)
- [ ] Load procedural_params.txt (re-editing workflow)
- [ ] **Deliverable:** Full save/load cycle works
### Phase 4: Future Extensions (Post-MVP)
- [ ] Cubic Bezier interpolation (smoother curves)
- [ ] Decaying sinusoid profile (metallic sounds)
- [ ] Noise profile (textured sounds)
- [ ] Composite profiles (add/subtract/multiply)
- [ ] Multi-dimensional Bezier ({freq, amplitude, decay, ...})
- [ ] Frequency snapping (snap to musical notes)
- [ ] Generic `gen_from_params()` code generation
**Design Decisions:**
- Linear Bezier interpolation (Phase 1), cubic later
- Soft parameter limits in UI (not enforced)
- Home-brew RNG (small, deterministic)
- Single function per sound (generic loader later)
- Start with Bezier + Gaussian only
**Size Impact:** 50-100× compression (5 KB .spec → ~100 bytes C++ code)
---
## Priority 2: 3D System Enhancements (Task #18)
**Goal:** Establish a pipeline for importing complex 3D scenes to replace hardcoded geometry.
- [x] **Task #18.0: Basic OBJ Asset Pipeline** (New)
- [x] Define `ASSET_MESH` type in `asset_manager`.
- [x] Update `asset_packer` to parse simple `.obj` files (positions, normals, UVs) and serialize them.
- [x] Update `Renderer3D` to handle `ObjectType::MESH` in the rasterization path.
- [ ] **Task #36: Blender Exporter:** Create a Python script (`tools/blender_export.py`) to export meshes/cameras/lights to a binary asset format. (Deprioritized)
- [ ] **Task #37: Asset Ingestion:** Update `asset_packer` to handle the new 3D binary format.
- [ ] **Task #38: Runtime Loader:** Implement a minimal C++ parser to load the scene data into the ECS/Renderer.
- [x] **Task #18-B: GPU BVH & Shadows** (Optimization)
- [x] **Upload BVH:** Create a storage buffer for `BVHNode` data and upload the CPU-built BVH every frame in `Renderer3D`.
- [x] **Shader Traversal:** Implement stack-based BVH traversal in `scene_query.wgsl` to replace the linear loop in `map_scene`.
- [x] **Shadow Optimization:** Use the BVH traversal for `calc_shadow` to skip occluded objects efficiently.
- [x] **Compile-Time Switch:** Implemented dual-pipeline architecture (BVH vs Linear) using `ShaderComposer` substitution to avoid runtime branching cost.
## Priority 3: WGSL Modularization (Task #50) [RECURRENT]
**Goal**: Refactor `ShaderComposer` and WGSL assets to support granular, reusable snippets and `#include` directives. This is an ongoing task to maintain shader code hygiene as new features are added.
## Priority 4: Developer Tooling & CI
**Goal**: Improve developer workflows, code quality, and release processes.
- [ ] **Task #51: Tracker Timing Verification**
- [x] **Task #51.1: Audio Backend Abstraction**: Create an interface to separate audio output from synth logic, enabling testable backends.
- [x] **Task #51.2: Mock Audio Backend**: Implement a test backend that records voice trigger events with precise timestamps.
- [x] **Task #51.3: Tracker Test Suite**: Create `test_tracker.cc` to verify pattern triggering, timing accuracy, and synchronization.
- [x] **Task #51.4: Integration with Build**: Wire up tests to CMake and ensure they run in CI.
## Phase 2: Size Optimization (Final Goal)
- [ ] **Task #34: Full STL Removal**: Replace all remaining `std::vector`, `std::map`, and `std::string` usage with custom minimal containers or C-style arrays to allow for CRT replacement. (Minimal Priority - deferred to end).
- [ ] **Task #22: Windows Native Platform**: Replace GLFW with direct Win32 API calls for the final 64k push.
- [ ] **Task #28: Spectrogram Quantization**: Research optimal frequency bin distribution and implement quantization.
- [ ] **Task #35: CRT Replacement**: investigation and implementation of CRT-free entry point.
## Future Goals & Ideas (Untriaged)
- [ ] **Task #52: Procedural SDF Font**: Minimal bezier/spline set for [A-Z, 0-9] and SDF rendering.
- [ ] **Task #53: Particles Shader Polish**: Improve visual quality of particles.
- [ ] **Task #55: SDF Random Planes Intersection**: Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection.
- [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling.
- [ ] **Task #58: Advanced Shader Factorization**: Further factorize WGSL code into smaller, reusable snippets.
- [ ] **Task #59: Comprehensive RNG Library**: Add WGSL snippets for float/vec2/vec3 noise (Perlin, Gyroid, etc.) and random number generators.
- [ ] **Task #60: OOP Refactoring**: Investigate if more C++ code can be made object-oriented without size penalty (vs functional style).
- [ ] **Task #61: GPU Procedural Generation**: Implement system to generate procedural data (textures, geometry) on GPU and read back to CPU.
- [ ] **Task #62: Physics Engine Enhancements (PBD & Rotation)**:
- [ ] **Task #62.1: Quaternion Rotation**: Implement quaternion-based rotation for `Object3D` and incorporate angular momentum into physics.
- [ ] **Task #62.2: Position Based Dynamics (PBD)**: Refactor solver to re-evaluate velocity after resolving all collisions and constraints.
- [ ] **Task #63: Refactor large files**: Split `src/3d/renderer.cc` (currently > 500 lines) into sub-functionalities.
---
## Task #51: Tracker Timing Verification - Detailed Attack Plan
**Problem Statement**: The tracker and synthesizer have audio sync issues. There's no robust way to verify that tracker patterns trigger at the correct timestamps without running the full audio hardware stack.
**Goal**: Implement a testable audio backend abstraction with event recording capabilities to verify tracker timing accuracy.
### Task #51.1: Audio Backend Abstraction
**Objective**: Decouple audio output from synthesis logic to enable testing without hardware.
**Implementation Steps**:
- [ ] **Create `src/audio/audio_backend.h`**:
- Define `AudioBackend` interface with pure virtual methods:
- `init()`: Initialize backend resources
- `start()`: Start audio playback/recording
- `shutdown()`: Clean up resources
- `on_voice_triggered(timestamp, spec_id, volume, pan)`: Hook for voice events
- Add `#if !defined(STRIP_ALL)` guards around test-only methods
- [ ] **Create `src/audio/miniaudio_backend.h` and `.cc`**:
- Move current miniaudio implementation from `audio.cc` to `MiniaudioBackend` class
- Implement `AudioBackend` interface
- Keep production behavior identical (no regressions)
- This backend does NOT record events (production path)
- [ ] **Refactor `src/audio/audio.cc`**:
- Add global `AudioBackend* g_audio_backend` pointer
- Add `void audio_set_backend(AudioBackend* backend)` function (under `!STRIP_ALL`)
- Default to `MiniaudioBackend` if no backend is set
- Replace direct miniaudio calls with backend interface calls
- [ ] **Update `src/audio/synth.cc`**:
- Add external hook: `extern AudioBackend* g_audio_backend_for_events`
- In `synth_trigger_voice()`, call `backend->on_voice_triggered()` if backend exists
- Ensure hook is `#if !defined(STRIP_ALL)` guarded
**Size Impact**: Zero (test code stripped in final build).
**Validation**: Existing tests (`test_synth.cc`) must pass unchanged.
---
### Task #51.2: Mock Audio Backend
**Objective**: Create a test-only backend that records all audio events with timestamps.
**Implementation Steps**:
- [ ] **Create `src/audio/mock_audio_backend.h` and `.cc`** (under `#if !defined(STRIP_ALL)`):
- Define `struct VoiceTriggerEvent`:
```cpp
struct VoiceTriggerEvent {
float timestamp_sec;
int spectrogram_id;
float volume;
float pan;
};
```
- Implement `MockAudioBackend` class:
- Maintain `std::vector<VoiceTriggerEvent> recorded_events`
- Override `on_voice_triggered()` to record events with current time
- Add `const std::vector<VoiceTriggerEvent>& get_events() const`
- Add `void clear_events()`
- Add `void advance_time(float delta_sec)` to simulate time progression
- Implement `init()`, `start()`, `shutdown()` as no-ops
- [ ] **Time Tracking**:
- Add `float current_time_sec` member to `MockAudioBackend`
- Increment `current_time_sec` in `advance_time()`
- Use `current_time_sec` as timestamp when recording events
- [ ] **Synth Integration**:
- When `synth_render()` is called, calculate frames rendered
- Notify backend of time elapsed: `frames / sample_rate`
- Update mock's internal clock accordingly
**Testing**: Create minimal unit test in `test_tracker.cc` to verify event recording works.
---
### Task #51.3: Tracker Test Suite
**Objective**: Comprehensive tests for tracker pattern triggering and timing accuracy.
**Implementation Steps**:
- [ ] **Create `src/tests/test_tracker.cc`**:
- **Test 1: Single Pattern Trigger**
- Define a minimal `TrackerScore` with 1 pattern at `t=1.0s`
- Set up mock backend
- Call `tracker_update(0.5)` → verify no events
- Call `tracker_update(1.0)` → verify 1 event recorded
- Validate event timestamp matches expected trigger time
- **Test 2: Multiple Pattern Triggers**
- Score with 3 patterns at `t=0.5s, 1.0s, 2.0s`
- Progressively call `tracker_update()` with increasing times
- Verify each pattern triggers exactly once at correct time
- **Test 3: Event Timing Accuracy**
- Pattern with multiple events at different beat offsets
- Verify each event's timestamp matches: `pattern_start_time + (beat * beat_duration)`
- Use tolerance: `±1 frame` (1/32000 sec ≈ 31.25µs)
- **Test 4: BPM Scaling**
- Same pattern tested at different BPMs (60, 120, 180)
- Verify beat-to-time conversion is accurate: `beat_sec = 60.0 / bpm`
- **Test 5: Pattern Overlap**
- Two patterns with overlapping time ranges
- Verify both trigger correctly without interference
- **Test 6: Asset vs Procedural Samples**
- Pattern using both asset-based spectrograms and procedural notes
- Verify both types render and trigger correctly
- **Test 7: Seek/Fast-Forward Simulation**
- Simulate `audio_render_silent()` behavior
- Start at `t=0`, fast-forward to `t=10.0s`
- Verify all patterns in range [0, 10] triggered correctly
- [ ] **Helper Functions**:
```cpp
void assert_event_at_time(const std::vector<VoiceTriggerEvent>& events,
float expected_time, float tolerance = 0.001f);
void assert_event_count(const std::vector<VoiceTriggerEvent>& events,
int expected_count);
TrackerScore create_test_score(const std::vector<float>& trigger_times,
float bpm = 120.0f);
```
**Coverage Target**: 95%+ for `src/audio/tracker.cc`.
---
### Task #51.4: Integration with Build
**Objective**: Wire up tests to CMake and ensure they run automatically.
**Implementation Steps**:
- [ ] **Update `src/CMakeLists.txt`**:
- Add `mock_audio_backend.cc` to test-only sources (under `DEMO_BUILD_TESTS`)
- Link `test_tracker` executable with audio subsystem library
- [ ] **Add test to CTest**:
```cmake
if(DEMO_BUILD_TESTS)
add_executable(test_tracker tests/test_tracker.cc audio/mock_audio_backend.cc)
target_link_libraries(test_tracker PRIVATE audio_lib util_lib)
add_test(NAME TrackerTest COMMAND test_tracker)
endif()
```
- [ ] **Verify in CI**:
- Run `cmake --build build && cd build && ctest`
- Ensure `test_tracker` runs and passes
- Check coverage report includes tracker.cc
**Validation**: `ctest` output shows `TrackerTest: PASSED`.
---
## Implementation Layout Summary
### New Files
```
src/audio/audio_backend.h # Interface definition
src/audio/miniaudio_backend.h # Production backend (header)
src/audio/miniaudio_backend.cc # Production backend (impl)
src/audio/mock_audio_backend.h # Test backend (header, !STRIP_ALL)
src/audio/mock_audio_backend.cc # Test backend (impl, !STRIP_ALL)
src/tests/test_tracker.cc # Comprehensive tracker tests
```
### Modified Files
```
src/audio/audio.cc # Backend abstraction layer
src/audio/synth.cc # Add event hooks
src/CMakeLists.txt # Add new files and tests
```
### File Structure
```
src/audio/
├── audio_backend.h [NEW] Interface (50 lines)
├── miniaudio_backend.h [NEW] Header (30 lines)
├── miniaudio_backend.cc [NEW] Production impl (~100 lines, moved from audio.cc)
├── mock_audio_backend.h [NEW] Test header (60 lines)
├── mock_audio_backend.cc [NEW] Test impl (~120 lines)
├── audio.h [MODIFIED] Add backend setter
├── audio.cc [MODIFIED] Use backend abstraction (~30 lines changed)
├── synth.cc [MODIFIED] Add event hook (~10 lines)
└── tracker.cc [NO CHANGE]
src/tests/
└── test_tracker.cc [NEW] Test suite (~400 lines)
```
### Code Organization Principles
1. **Zero Size Impact**: All test infrastructure under `#if !defined(STRIP_ALL)`
2. **Backward Compatible**: Production path unchanged, existing tests pass
3. **Clean Separation**: Interface-based design, easy to add more backends later
4. **Testable**: Mock backend has minimal dependencies (no hardware/threads)
---
## Low Priority: Developer Tools & Editors
### Task #57: Interactive Timeline Editor (Gantt Chart Evolution)
**Goal:** Create a web-based interactive editor for `demo.seq` files based on the existing Gantt chart HTML output.
**Features:**
- **Load/Save**: Parse and serialize `demo.seq` files
- **Visual Timeline**: Interactive Gantt-style view of sequences and effects
- **Drag & Drop**: Move sequences and effects along the timeline
- **Edit Timing**: Adjust start/end times, priorities
- **Overlap Detection**: Visual feedback for sequence/effect conflicts
- **Zoom/Pan**: Navigate long timelines easily
- **Effect Properties**: Edit constructor arguments inline
**Technical Scope:**
- Pure HTML/JavaScript tool (no backend needed)
- Loads `demo.seq` text file via file input
- Interactive visualization using D3.js or similar
- Saves modified `demo.seq` back to disk
- **Does NOT**: Interact with C++ code, preview rendering, compile timeline
**File Structure:**
```
tools/timeline_editor/
├── index.html # Main editor UI
├── editor.js # Core editing logic
├── parser.js # demo.seq parser
├── serializer.js # demo.seq writer
├── timeline.js # Gantt chart renderer
└── README.md # Usage instructions
```
**Use Cases:**
- Quickly adjust demo timing without recompiling
- Visualize sequence overlaps and priorities
- Experiment with different timing arrangements
- Export modified timeline for integration
**Priority:** Very Low (quality of life improvement, not critical)
**Effort:** 1-2 weeks for basic functionality
**Dependencies:** None (standalone tool)
---
## Future Goals
|