From e2c3c3e95b6a9e53b4631b271640bb9914f8c95e Mon Sep 17 00:00:00 2001 From: skal Date: Thu, 5 Mar 2026 10:03:32 +0100 Subject: fix(audio): OLA encoder never ran; version never propagated to decoder MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two bugs kept the v2 OLA path permanently disabled: 1. SpectrogramResourceManager::load_asset() never set spec.version from SpecHeader::version — all .spec assets loaded with version=0, so ola_mode was always false in the voice. 2. spectool analyze_audio() used non-overlapping chunks (stride=DCT_SIZE), hamming_window_512, and hardcoded header.version=1 — OLA analysis was never implemented in the encoder. Fixes: propagate header->version in load_asset(); switch spectool to OLA_HOP_SIZE stride, hann_window_512, and SPEC_VERSION_V2_OLA. Regenerated all .spec files. handoff(Gemini): OLA enc/dec chain now correct end-to-end. .spec files are v2 (50% overlap, Hann). No API changes; 33/34 tests pass (WavDumpBackendTest pre-existing failure unrelated). Co-Authored-By: Claude Sonnet 4.6 --- PROJECT_CONTEXT.md | 4 ++-- TODO.md | 9 +-------- doc/COMPLETED.md | 6 +++--- src/audio/spectrogram_resource_manager.cc | 1 + tools/spectool.cc | 15 +++++++++------ workspaces/main/music/BASS_GUITAR_FEEL.spec | Bin 51216 -> 98320 bytes workspaces/main/music/BASS_SYNTH_1.spec | Bin 32784 -> 61456 bytes workspaces/main/music/CLARINET_COUGH.spec | Bin 387088 -> 768016 bytes workspaces/main/music/CRASH_DMX.spec | Bin 217104 -> 432144 bytes workspaces/main/music/HIHAT_CLOSED_DMX.spec | Bin 20496 -> 36880 bytes workspaces/main/music/HIHAT_CLOSED_DUFF.spec | Bin 12304 -> 22544 bytes workspaces/main/music/HIHAT_CLOSED_ER_1.spec | Bin 16400 -> 28688 bytes workspaces/main/music/KICK_606.spec | Bin 139280 -> 272400 bytes workspaces/main/music/KICK_90S_2.spec | Bin 30736 -> 57360 bytes workspaces/main/music/RIDE_CUP_1.spec | Bin 3577872 -> 7151632 bytes workspaces/main/music/SNARE_808.spec | Bin 120848 -> 235536 bytes workspaces/main/music/SNARE_909_TUNE_8.spec | Bin 43024 -> 79888 bytes workspaces/main/music/SNARE_BLUE_ROOM.spec | Bin 43024 -> 81936 bytes workspaces/main/music/SPLASH_GROUNDED.spec | Bin 57360 -> 110608 bytes workspaces/main/music/SYNTH_BASS_DISTORT.spec | Bin 32784 -> 61456 bytes 20 files changed, 16 insertions(+), 19 deletions(-) diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 29395e7..d9e37e8 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -33,7 +33,7 @@ - **Timing System:** **Beat-based timelines** for musical synchronization. Sequences defined in beats, converted to seconds at runtime. Effects receive both physical time (constant) and beat time (musical). Variable tempo affects audio only. See `doc/BEAT_TIMING.md`. - **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=`. Organized structure: `music/`, `weights/`, `obj/`, `shaders/`. Shared common shaders in `src/shaders/`. See `doc/WORKSPACE_SYSTEM.md`. -- **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. OLA-IDCT synthesis (v2 .spec): Hann window, 50% overlap, click-free. V1 (raw DCT-512) preserved for compatibility. MP3→spec encoder updated to match. Existing .spec files need regen to activate v2. +- **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. OLA-IDCT synthesis (v2 .spec): Hann window, 50% overlap, click-free. V1 (raw DCT-512) preserved for generated notes. .spec files regenerated as v2. - **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition with ShaderComposer. 24 shared common shaders (math, render, compute). - **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline. - **Effects:** CNN post-processing: CNNEffect (v1) and CNNv2Effect operational. CNN v2: sigmoid activation, storage buffer weights (~3.2 KB), 7D static features, dynamic layers. Training stable, convergence validated. @@ -46,7 +46,7 @@ ## Next Up -**Active:** Spectral Brush Editor (procedural compression), CNN v2 quantization, .spec v2 regen (OLA) +**Active:** Spectral Brush Editor (procedural compression), CNN v2 quantization **Ongoing:** Test infrastructure maintenance (34/34 passing) **Future:** Size optimization (64k target), 3D enhancements diff --git a/TODO.md b/TODO.md index c0f8bb1..45623e8 100644 --- a/TODO.md +++ b/TODO.md @@ -21,14 +21,7 @@ Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB). --- -## Priority 3: Regenerate .spec files as v2 [REQUIRED] - -Existing `.spec` files in `workspaces/main/music/` were encoded with v1 (no overlap). -Rebuild with the MP3 assets to produce v2 (OLA, Hann, hop=256) — click-free output. - ---- - -## Priority 4: Test Infrastructure Maintenance [ONGOING] +## Priority 3: Test Infrastructure Maintenance [ONGOING] **Status:** 34/34 tests passing diff --git a/doc/COMPLETED.md b/doc/COMPLETED.md index e37ddbf..9b9f549 100644 --- a/doc/COMPLETED.md +++ b/doc/COMPLETED.md @@ -33,10 +33,10 @@ Use `read @doc/archive/FILENAME.md` to access archived documents. - [x] **OLA-IDCT Synthesis — click-free .spec decoding** - **Goal**: Eliminate frame-boundary clicks in spectrogram→PCM synthesis. - - **Implementation**: Added v2 spectrogram format (`SPEC_VERSION_V2_OLA`). Synthesis uses Hann-windowed IDCT with 50% overlap-add (hop=256, overlap=256). Per-voice `overlap_buf[256]` accumulates the tail from the previous IDCT frame. V1 path (raw DCT-512) preserved for generated notes and old `.spec` files. Hann window precomputed at `synth_init()`. MP3 encoder switched from Hamming to Hann, now slides a 512-sample analysis window by 256 samples per frame (OLA analysis), emitting ~2× as many frames. `SpecHeader.version` field propagated through `Spectrogram.version` to the voice's `ola_mode` flag at trigger time. - - **Files**: `src/audio/dct.h`, `src/audio/synth.h`, `src/audio/synth.cc`, `src/audio/window.h`, `src/audio/window.cc`, `src/audio/tracker.cc` + - **Implementation**: Added v2 spectrogram format (`SPEC_VERSION_V2_OLA`). Synthesis uses Hann-windowed IDCT with 50% overlap-add (hop=256, overlap=256). Per-voice `overlap_buf[256]` accumulates the tail from the previous IDCT frame. V1 path (raw DCT-512) preserved for generated notes. Hann window precomputed at `synth_init()`. `SpecHeader.version` field propagated through `Spectrogram.version` to the voice's `ola_mode` flag at trigger time. + - **Bug fix (2026-03-05)**: Two bugs prevented OLA from ever activating: (1) `SpectrogramResourceManager::load_asset()` never set `resource->spec.version` from `header->version` — all loaded assets got `version=0`, OLA path silently skipped. (2) `spectool analyze_audio()` used non-overlapping frames (`chunk_idx * DCT_SIZE` stride), `hamming_window_512`, and hardcoded `header.version = 1` — the OLA encoder was never implemented. Fixed both; `.spec` files regenerated. + - **Files**: `src/audio/spectrogram_resource_manager.cc`, `tools/spectool.cc`, `workspaces/main/music/*.spec` - **Tests**: 34/34 passing - - **Pending**: Regenerate `.spec` files from MP3 assets to activate v2 encoding. ## Recently Completed (February 21, 2026) diff --git a/src/audio/spectrogram_resource_manager.cc b/src/audio/spectrogram_resource_manager.cc index 30939e0..c500528 100644 --- a/src/audio/spectrogram_resource_manager.cc +++ b/src/audio/spectrogram_resource_manager.cc @@ -199,6 +199,7 @@ void SpectrogramResourceManager::load_asset(Resource* resource) { resource->spec.spectral_data_a = spectral_data; resource->spec.spectral_data_b = spectral_data; resource->spec.num_frames = header->num_frames; + resource->spec.version = header->version; resource->owned_data = nullptr; // Asset data is not owned #if defined(DEBUG_LOG_ASSETS) diff --git a/tools/spectool.cc b/tools/spectool.cc index c3aebb2..70bcae2 100644 --- a/tools/spectool.cc +++ b/tools/spectool.cc @@ -110,15 +110,18 @@ int analyze_audio(const char* in_path, const char* out_path, bool normalize, } } - // Second pass: Windowing + DCT + // Second pass: Windowing + DCT (OLA v2: Hann window, 50% overlap) std::vector spec_data; float window[WINDOW_SIZE]; - hamming_window_512(window); + hann_window_512(window); - // Process PCM data in DCT_SIZE chunks - const size_t num_chunks = (pcm_data.size() + DCT_SIZE - 1) / DCT_SIZE; + // Process PCM data with OLA_HOP_SIZE stride (50% overlap) + const size_t hop = OLA_HOP_SIZE; + const size_t num_chunks = (pcm_data.size() > DCT_SIZE) + ? (pcm_data.size() - DCT_SIZE) / hop + 1 + : 1; for (size_t chunk_idx = 0; chunk_idx < num_chunks; ++chunk_idx) { - const size_t chunk_start = chunk_idx * DCT_SIZE; + const size_t chunk_start = chunk_idx * hop; const size_t chunk_end = (chunk_start + DCT_SIZE < pcm_data.size()) ? chunk_start + DCT_SIZE : pcm_data.size(); @@ -202,7 +205,7 @@ int analyze_audio(const char* in_path, const char* out_path, bool normalize, SpecHeader header; memcpy(header.magic, "SPEC", 4); - header.version = 1; + header.version = SPEC_VERSION_V2_OLA; header.dct_size = DCT_SIZE; header.num_frames = trimmed_data.size() / DCT_SIZE; diff --git a/workspaces/main/music/BASS_GUITAR_FEEL.spec b/workspaces/main/music/BASS_GUITAR_FEEL.spec index 54f49a6..2521025 100644 Binary files a/workspaces/main/music/BASS_GUITAR_FEEL.spec and b/workspaces/main/music/BASS_GUITAR_FEEL.spec differ diff --git a/workspaces/main/music/BASS_SYNTH_1.spec b/workspaces/main/music/BASS_SYNTH_1.spec index 33bc0a0..9a3f585 100644 Binary files a/workspaces/main/music/BASS_SYNTH_1.spec and b/workspaces/main/music/BASS_SYNTH_1.spec differ diff --git a/workspaces/main/music/CLARINET_COUGH.spec b/workspaces/main/music/CLARINET_COUGH.spec index a2b1657..f41ff4a 100644 Binary files a/workspaces/main/music/CLARINET_COUGH.spec and b/workspaces/main/music/CLARINET_COUGH.spec differ diff --git a/workspaces/main/music/CRASH_DMX.spec b/workspaces/main/music/CRASH_DMX.spec index 45ee52d..fc36694 100644 Binary files a/workspaces/main/music/CRASH_DMX.spec and b/workspaces/main/music/CRASH_DMX.spec differ diff --git a/workspaces/main/music/HIHAT_CLOSED_DMX.spec b/workspaces/main/music/HIHAT_CLOSED_DMX.spec index 8fce1d2..655b0f9 100644 Binary files a/workspaces/main/music/HIHAT_CLOSED_DMX.spec and b/workspaces/main/music/HIHAT_CLOSED_DMX.spec differ diff --git a/workspaces/main/music/HIHAT_CLOSED_DUFF.spec b/workspaces/main/music/HIHAT_CLOSED_DUFF.spec index f738271..3042a42 100644 Binary files a/workspaces/main/music/HIHAT_CLOSED_DUFF.spec and b/workspaces/main/music/HIHAT_CLOSED_DUFF.spec differ diff --git a/workspaces/main/music/HIHAT_CLOSED_ER_1.spec b/workspaces/main/music/HIHAT_CLOSED_ER_1.spec index bb03f5e..ac780f2 100644 Binary files a/workspaces/main/music/HIHAT_CLOSED_ER_1.spec and b/workspaces/main/music/HIHAT_CLOSED_ER_1.spec differ diff --git a/workspaces/main/music/KICK_606.spec b/workspaces/main/music/KICK_606.spec index 10af84a..68f5f74 100644 Binary files a/workspaces/main/music/KICK_606.spec and b/workspaces/main/music/KICK_606.spec differ diff --git a/workspaces/main/music/KICK_90S_2.spec b/workspaces/main/music/KICK_90S_2.spec index 126409e..91a9c34 100644 Binary files a/workspaces/main/music/KICK_90S_2.spec and b/workspaces/main/music/KICK_90S_2.spec differ diff --git a/workspaces/main/music/RIDE_CUP_1.spec b/workspaces/main/music/RIDE_CUP_1.spec index 78867c1..b0fed88 100644 Binary files a/workspaces/main/music/RIDE_CUP_1.spec and b/workspaces/main/music/RIDE_CUP_1.spec differ diff --git a/workspaces/main/music/SNARE_808.spec b/workspaces/main/music/SNARE_808.spec index 2923c3c..bc77d85 100644 Binary files a/workspaces/main/music/SNARE_808.spec and b/workspaces/main/music/SNARE_808.spec differ diff --git a/workspaces/main/music/SNARE_909_TUNE_8.spec b/workspaces/main/music/SNARE_909_TUNE_8.spec index 4693c05..3f38ca6 100644 Binary files a/workspaces/main/music/SNARE_909_TUNE_8.spec and b/workspaces/main/music/SNARE_909_TUNE_8.spec differ diff --git a/workspaces/main/music/SNARE_BLUE_ROOM.spec b/workspaces/main/music/SNARE_BLUE_ROOM.spec index b24baa7..3d489dd 100644 Binary files a/workspaces/main/music/SNARE_BLUE_ROOM.spec and b/workspaces/main/music/SNARE_BLUE_ROOM.spec differ diff --git a/workspaces/main/music/SPLASH_GROUNDED.spec b/workspaces/main/music/SPLASH_GROUNDED.spec index a919be4..210a784 100644 Binary files a/workspaces/main/music/SPLASH_GROUNDED.spec and b/workspaces/main/music/SPLASH_GROUNDED.spec differ diff --git a/workspaces/main/music/SYNTH_BASS_DISTORT.spec b/workspaces/main/music/SYNTH_BASS_DISTORT.spec index 33bc0a0..9a3f585 100644 Binary files a/workspaces/main/music/SYNTH_BASS_DISTORT.spec and b/workspaces/main/music/SYNTH_BASS_DISTORT.spec differ -- cgit v1.2.3