diff options
20 files changed, 16 insertions, 19 deletions
diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 29395e7..d9e37e8 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -33,7 +33,7 @@ - **Timing System:** **Beat-based timelines** for musical synchronization. Sequences defined in beats, converted to seconds at runtime. Effects receive both physical time (constant) and beat time (musical). Variable tempo affects audio only. See `doc/BEAT_TIMING.md`. - **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=<name>`. Organized structure: `music/`, `weights/`, `obj/`, `shaders/`. Shared common shaders in `src/shaders/`. See `doc/WORKSPACE_SYSTEM.md`. -- **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. OLA-IDCT synthesis (v2 .spec): Hann window, 50% overlap, click-free. V1 (raw DCT-512) preserved for compatibility. MP3→spec encoder updated to match. Existing .spec files need regen to activate v2. +- **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. OLA-IDCT synthesis (v2 .spec): Hann window, 50% overlap, click-free. V1 (raw DCT-512) preserved for generated notes. .spec files regenerated as v2. - **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition with ShaderComposer. 24 shared common shaders (math, render, compute). - **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline. - **Effects:** CNN post-processing: CNNEffect (v1) and CNNv2Effect operational. CNN v2: sigmoid activation, storage buffer weights (~3.2 KB), 7D static features, dynamic layers. Training stable, convergence validated. @@ -46,7 +46,7 @@ ## Next Up -**Active:** Spectral Brush Editor (procedural compression), CNN v2 quantization, .spec v2 regen (OLA) +**Active:** Spectral Brush Editor (procedural compression), CNN v2 quantization **Ongoing:** Test infrastructure maintenance (34/34 passing) **Future:** Size optimization (64k target), 3D enhancements @@ -21,14 +21,7 @@ Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB). --- -## Priority 3: Regenerate .spec files as v2 [REQUIRED] - -Existing `.spec` files in `workspaces/main/music/` were encoded with v1 (no overlap). -Rebuild with the MP3 assets to produce v2 (OLA, Hann, hop=256) — click-free output. - ---- - -## Priority 4: Test Infrastructure Maintenance [ONGOING] +## Priority 3: Test Infrastructure Maintenance [ONGOING] **Status:** 34/34 tests passing diff --git a/doc/COMPLETED.md b/doc/COMPLETED.md index e37ddbf..9b9f549 100644 --- a/doc/COMPLETED.md +++ b/doc/COMPLETED.md @@ -33,10 +33,10 @@ Use `read @doc/archive/FILENAME.md` to access archived documents. - [x] **OLA-IDCT Synthesis — click-free .spec decoding** - **Goal**: Eliminate frame-boundary clicks in spectrogram→PCM synthesis. - - **Implementation**: Added v2 spectrogram format (`SPEC_VERSION_V2_OLA`). Synthesis uses Hann-windowed IDCT with 50% overlap-add (hop=256, overlap=256). Per-voice `overlap_buf[256]` accumulates the tail from the previous IDCT frame. V1 path (raw DCT-512) preserved for generated notes and old `.spec` files. Hann window precomputed at `synth_init()`. MP3 encoder switched from Hamming to Hann, now slides a 512-sample analysis window by 256 samples per frame (OLA analysis), emitting ~2× as many frames. `SpecHeader.version` field propagated through `Spectrogram.version` to the voice's `ola_mode` flag at trigger time. - - **Files**: `src/audio/dct.h`, `src/audio/synth.h`, `src/audio/synth.cc`, `src/audio/window.h`, `src/audio/window.cc`, `src/audio/tracker.cc` + - **Implementation**: Added v2 spectrogram format (`SPEC_VERSION_V2_OLA`). Synthesis uses Hann-windowed IDCT with 50% overlap-add (hop=256, overlap=256). Per-voice `overlap_buf[256]` accumulates the tail from the previous IDCT frame. V1 path (raw DCT-512) preserved for generated notes. Hann window precomputed at `synth_init()`. `SpecHeader.version` field propagated through `Spectrogram.version` to the voice's `ola_mode` flag at trigger time. + - **Bug fix (2026-03-05)**: Two bugs prevented OLA from ever activating: (1) `SpectrogramResourceManager::load_asset()` never set `resource->spec.version` from `header->version` — all loaded assets got `version=0`, OLA path silently skipped. (2) `spectool analyze_audio()` used non-overlapping frames (`chunk_idx * DCT_SIZE` stride), `hamming_window_512`, and hardcoded `header.version = 1` — the OLA encoder was never implemented. Fixed both; `.spec` files regenerated. + - **Files**: `src/audio/spectrogram_resource_manager.cc`, `tools/spectool.cc`, `workspaces/main/music/*.spec` - **Tests**: 34/34 passing - - **Pending**: Regenerate `.spec` files from MP3 assets to activate v2 encoding. ## Recently Completed (February 21, 2026) diff --git a/src/audio/spectrogram_resource_manager.cc b/src/audio/spectrogram_resource_manager.cc index 30939e0..c500528 100644 --- a/src/audio/spectrogram_resource_manager.cc +++ b/src/audio/spectrogram_resource_manager.cc @@ -199,6 +199,7 @@ void SpectrogramResourceManager::load_asset(Resource* resource) { resource->spec.spectral_data_a = spectral_data; resource->spec.spectral_data_b = spectral_data; resource->spec.num_frames = header->num_frames; + resource->spec.version = header->version; resource->owned_data = nullptr; // Asset data is not owned #if defined(DEBUG_LOG_ASSETS) diff --git a/tools/spectool.cc b/tools/spectool.cc index c3aebb2..70bcae2 100644 --- a/tools/spectool.cc +++ b/tools/spectool.cc @@ -110,15 +110,18 @@ int analyze_audio(const char* in_path, const char* out_path, bool normalize, } } - // Second pass: Windowing + DCT + // Second pass: Windowing + DCT (OLA v2: Hann window, 50% overlap) std::vector<float> spec_data; float window[WINDOW_SIZE]; - hamming_window_512(window); + hann_window_512(window); - // Process PCM data in DCT_SIZE chunks - const size_t num_chunks = (pcm_data.size() + DCT_SIZE - 1) / DCT_SIZE; + // Process PCM data with OLA_HOP_SIZE stride (50% overlap) + const size_t hop = OLA_HOP_SIZE; + const size_t num_chunks = (pcm_data.size() > DCT_SIZE) + ? (pcm_data.size() - DCT_SIZE) / hop + 1 + : 1; for (size_t chunk_idx = 0; chunk_idx < num_chunks; ++chunk_idx) { - const size_t chunk_start = chunk_idx * DCT_SIZE; + const size_t chunk_start = chunk_idx * hop; const size_t chunk_end = (chunk_start + DCT_SIZE < pcm_data.size()) ? chunk_start + DCT_SIZE : pcm_data.size(); @@ -202,7 +205,7 @@ int analyze_audio(const char* in_path, const char* out_path, bool normalize, SpecHeader header; memcpy(header.magic, "SPEC", 4); - header.version = 1; + header.version = SPEC_VERSION_V2_OLA; header.dct_size = DCT_SIZE; header.num_frames = trimmed_data.size() / DCT_SIZE; diff --git a/workspaces/main/music/BASS_GUITAR_FEEL.spec b/workspaces/main/music/BASS_GUITAR_FEEL.spec Binary files differindex 54f49a6..2521025 100644 --- a/workspaces/main/music/BASS_GUITAR_FEEL.spec +++ b/workspaces/main/music/BASS_GUITAR_FEEL.spec diff --git a/workspaces/main/music/BASS_SYNTH_1.spec b/workspaces/main/music/BASS_SYNTH_1.spec Binary files differindex 33bc0a0..9a3f585 100644 --- a/workspaces/main/music/BASS_SYNTH_1.spec +++ b/workspaces/main/music/BASS_SYNTH_1.spec diff --git a/workspaces/main/music/CLARINET_COUGH.spec b/workspaces/main/music/CLARINET_COUGH.spec Binary files differindex a2b1657..f41ff4a 100644 --- a/workspaces/main/music/CLARINET_COUGH.spec +++ b/workspaces/main/music/CLARINET_COUGH.spec diff --git a/workspaces/main/music/CRASH_DMX.spec b/workspaces/main/music/CRASH_DMX.spec Binary files differindex 45ee52d..fc36694 100644 --- a/workspaces/main/music/CRASH_DMX.spec +++ b/workspaces/main/music/CRASH_DMX.spec diff --git a/workspaces/main/music/HIHAT_CLOSED_DMX.spec b/workspaces/main/music/HIHAT_CLOSED_DMX.spec Binary files differindex 8fce1d2..655b0f9 100644 --- a/workspaces/main/music/HIHAT_CLOSED_DMX.spec +++ b/workspaces/main/music/HIHAT_CLOSED_DMX.spec diff --git a/workspaces/main/music/HIHAT_CLOSED_DUFF.spec b/workspaces/main/music/HIHAT_CLOSED_DUFF.spec Binary files differindex f738271..3042a42 100644 --- a/workspaces/main/music/HIHAT_CLOSED_DUFF.spec +++ b/workspaces/main/music/HIHAT_CLOSED_DUFF.spec diff --git a/workspaces/main/music/HIHAT_CLOSED_ER_1.spec b/workspaces/main/music/HIHAT_CLOSED_ER_1.spec Binary files differindex bb03f5e..ac780f2 100644 --- a/workspaces/main/music/HIHAT_CLOSED_ER_1.spec +++ b/workspaces/main/music/HIHAT_CLOSED_ER_1.spec diff --git a/workspaces/main/music/KICK_606.spec b/workspaces/main/music/KICK_606.spec Binary files differindex 10af84a..68f5f74 100644 --- a/workspaces/main/music/KICK_606.spec +++ b/workspaces/main/music/KICK_606.spec diff --git a/workspaces/main/music/KICK_90S_2.spec b/workspaces/main/music/KICK_90S_2.spec Binary files differindex 126409e..91a9c34 100644 --- a/workspaces/main/music/KICK_90S_2.spec +++ b/workspaces/main/music/KICK_90S_2.spec diff --git a/workspaces/main/music/RIDE_CUP_1.spec b/workspaces/main/music/RIDE_CUP_1.spec Binary files differindex 78867c1..b0fed88 100644 --- a/workspaces/main/music/RIDE_CUP_1.spec +++ b/workspaces/main/music/RIDE_CUP_1.spec diff --git a/workspaces/main/music/SNARE_808.spec b/workspaces/main/music/SNARE_808.spec Binary files differindex 2923c3c..bc77d85 100644 --- a/workspaces/main/music/SNARE_808.spec +++ b/workspaces/main/music/SNARE_808.spec diff --git a/workspaces/main/music/SNARE_909_TUNE_8.spec b/workspaces/main/music/SNARE_909_TUNE_8.spec Binary files differindex 4693c05..3f38ca6 100644 --- a/workspaces/main/music/SNARE_909_TUNE_8.spec +++ b/workspaces/main/music/SNARE_909_TUNE_8.spec diff --git a/workspaces/main/music/SNARE_BLUE_ROOM.spec b/workspaces/main/music/SNARE_BLUE_ROOM.spec Binary files differindex b24baa7..3d489dd 100644 --- a/workspaces/main/music/SNARE_BLUE_ROOM.spec +++ b/workspaces/main/music/SNARE_BLUE_ROOM.spec diff --git a/workspaces/main/music/SPLASH_GROUNDED.spec b/workspaces/main/music/SPLASH_GROUNDED.spec Binary files differindex a919be4..210a784 100644 --- a/workspaces/main/music/SPLASH_GROUNDED.spec +++ b/workspaces/main/music/SPLASH_GROUNDED.spec diff --git a/workspaces/main/music/SYNTH_BASS_DISTORT.spec b/workspaces/main/music/SYNTH_BASS_DISTORT.spec Binary files differindex 33bc0a0..9a3f585 100644 --- a/workspaces/main/music/SYNTH_BASS_DISTORT.spec +++ b/workspaces/main/music/SYNTH_BASS_DISTORT.spec |