From 0bd2d4eae458d9017de4e2c4e04c1c1cc5315520 Mon Sep 17 00:00:00 2001 From: skal Date: Tue, 3 Feb 2026 12:47:27 +0100 Subject: feat(audio): Fix tracker bugs and implement rock demo track Critical Bug Fixes: - Fixed pool exhaustion: Tracker slots never freed after use, music stopped after 8 patterns. Implemented round-robin allocation with cleanup. - Fixed note name parsing: Added automatic note-to-frequency conversion in tracker_compiler. Bass and melody now play correctly. - Fixed timing mismatch: Patterns are 2 seconds but triggered every 4 seconds, causing silence gaps. Updated SCORE to trigger every 2 seconds. Improvements: - Implemented dynamic resource sizing in tracker_compiler: Analyzes score to determine optimal MAX_VOICES/MAX_SPECTROGRAMS values. - Created comprehensive rock track: 11 patterns with drums, bass, power chords, and lead melody over 25 seconds. - Added 213 lines of asset system documentation with 8 prioritized tasks. Known Issues for next session: - Audio quality could be improved (some artifacts remain) - Note synthesis uses default parameters, needs tuning - Pattern overlaps might cause voice exhaustion under heavy load Files Changed: - src/audio/tracker.cc: Round-robin pool allocation, cleanup logic - tools/tracker_compiler.cc: Note name parser, resource usage analysis - src/audio/synth.h: Increased limits to 16 based on analysis - assets/music.track: 230-line rock arrangement - doc/ASSET_SYSTEM.md: Comprehensive documentation + 8 tasks - TODO.md: Updated with recent completions and known issues handoff(Gemini): Music system now functional but needs quality improvements. Audio artifacts and synthesis tuning remain. See TODO.md for details. Co-Authored-By: Claude Sonnet 4.5 --- doc/ASSET_SYSTEM.md | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 213 insertions(+) (limited to 'doc') diff --git a/doc/ASSET_SYSTEM.md b/doc/ASSET_SYSTEM.md index 3493936..9cf15ba 100644 --- a/doc/ASSET_SYSTEM.md +++ b/doc/ASSET_SYSTEM.md @@ -73,3 +73,216 @@ Shader code (WGSL) can also be managed as assets. construct a `std::string` or `std::string_view` using the returned pointer and size. 4. Register them with the `ShaderComposer`. + +--- + +# Current Implementation Status + +## What's Working ✅ +- **Build-Time Packer** (`tools/asset_packer.cc`): Parses `demo_assets.txt`, embeds binary files as C++ arrays +- **Runtime Manager** (`src/util/asset_manager.{h,cc}`): O(1) retrieval with caching, SIOF fix via singleton +- **Asset Types**: Static binaries (`.spec`, `.wgsl`), procedural textures (`PROC(func, params)`) +- **Safety Guarantees**: 16-byte alignment, null-terminator for strings, size reporting +- **Testing**: `test_assets.cc` validates retrieval, caching, procedural generation +- **Current Manifest**: 33 assets (15 audio samples, 17 WGSL shaders, 1 procedural texture) + +## Design Strengths +- Enum-based type safety (no magic numbers or strings at runtime) +- Zero runtime cost for static assets (direct pointer return) +- Clean separation of build-time and runtime concerns +- Procedural generation support (saves binary size) +- Thread-safe for immutable assets (cache is write-once) + +## Design Weaknesses +- **No compression**: Only `NONE` supported, critical blocker for 64k goal +- **STL dependencies**: `std::map`, `std::string` in runtime code (conflicts with CRT replacement) +- **Hardcoded procedural dimensions**: Assumes 256×256 RGBA8 (lines 85-86 in `asset_manager.cc`) +- **No integrity checks**: No CRC/hash validation for asset correctness +- **Limited asset types**: No mesh/scene format yet (needed for Task #18) + +--- + +# Asset System Improvement Tasks + +## High Priority (Critical for 64k Goal) + +### Task #27: Asset Compression Layer +**Goal**: Implement runtime decompression to reduce binary size by 30-50%. + +**Attack Plan**: +- [ ] **27.1**: Add compression type enum (`NONE`, `ZLIB`, `RLE`, `CUSTOM`) +- [ ] **27.2**: Update `asset_packer.cc` to compress assets during build + - Integrate a lightweight compression library (e.g., miniz for zlib) + - Store compressed size + original size in `AssetRecord` +- [ ] **27.3**: Implement runtime decompressor in `asset_manager.cc` + - Allocate decompressed buffer on first `GetAsset()` call + - Cache decompressed data, allow `DropAsset()` to free memory +- [ ] **27.4**: Update `assets.txt` format to specify compression per-asset + - Example: `KICK_1, ZLIB, kick1.spec, "Compressed kick sample"` +- [ ] **27.5**: Add tests for compressed asset retrieval and DropAsset + +**Size Impact**: Estimated 30-50% reduction on asset blob (shaders compress well, spectrograms moderately). + +**Dependency**: None (standalone improvement). + +--- + +### Task #28: Spectrogram Quantization +**Goal**: Compress audio assets from `float32` to `uint16_t` with logarithmic frequency binning. + +**Attack Plan**: +- [ ] **28.1**: Research optimal frequency bin distribution (logarithmic vs linear) +- [ ] **28.2**: Update `spectool` to export quantized `.spec` format + - Add `--quantize` flag to `spectool analyze` + - Map spectral values to `uint16_t` range with configurable dynamic range +- [ ] **28.3**: Update audio synthesis engine to decode quantized spectrograms + - Add dequantization step in `src/audio/synth.cc` +- [ ] **28.4**: Re-generate all `.spec` assets with quantization enabled +- [ ] **28.5**: Add tests to verify audio quality degradation is acceptable + +**Size Impact**: 2-4× reduction on `.spec` files (currently 15 audio assets in manifest). + +**Dependency**: Task #27 (compression) for maximum size savings. + +--- + +### Task #29: WGSL Shader Minification +**Goal**: Minify WGSL shaders to reduce text asset size. + +**Attack Plan**: +- [ ] **29.1**: Implement WGSL minifier in `tools/wgsl_minify.py` + - Remove comments, unnecessary whitespace + - Rename variables to single-letter identifiers (preserve entry points) + - Strip debug labels and unused code +- [ ] **29.2**: Integrate minifier into build pipeline + - Add `MINIFY` compression type to `asset_packer.cc` + - Auto-minify all `SHADER_*` assets during build +- [ ] **29.3**: Validate minified shaders compile correctly + - Add shader compilation test to CI/CD + +**Size Impact**: 40-60% reduction on shader text (currently 17 WGSL shaders in manifest). + +**Dependency**: None (standalone improvement). + +--- + +## Medium Priority (Code Hygiene & Maintainability) + +### Task #20.1: Remove STL from Asset Manager (Part of Task #20) +**Goal**: Eliminate `std::map` and `std::string` to prepare for CRT replacement. + +**Attack Plan**: +- [ ] **20.1.1**: Replace `kAssetManagerProcGenFuncMap` (line 21 in `asset_manager.cc`) + - Switch to static array lookup or compile-time switch-case + - Example: + ```cpp + static const struct { const char* name; ProcGenFunc func; } kProcGenFuncs[] = { + {"gen_noise", procedural::gen_noise}, + {"gen_grid", procedural::gen_grid}, + {"make_periodic", procedural::make_periodic}, + }; + ``` +- [ ] **20.1.2**: Replace `std::map` in `asset_packer.cc` (build-time tool, lower priority) +- [ ] **20.1.3**: Verify no STL usage remains in `src/util/asset_manager.cc` + +**Size Impact**: Minor (few KB), but critical for future CRT replacement. + +**Dependency**: None. + +--- + +### Task #30: Procedural Asset Generalization +**Goal**: Support variable dimensions and formats for procedural textures. + +**Attack Plan**: +- [ ] **30.1**: Update `assets.txt` format to encode dimensions in `proc_params` + - Example: `NOISE_TEX, PROC(gen_noise, 256, 256, 4, 1234, 16), _, "Width, Height, Channels, Seed, Freq"` +- [ ] **30.2**: Update `asset_packer.cc` to parse dimension parameters +- [ ] **30.3**: Update `GetAsset()` to read dimensions from `proc_params` + - Allocate buffer dynamically based on `width * height * channels` +- [ ] **30.4**: Add support for grayscale (1 channel) and RGB (3 channels) formats + +**Size Impact**: Negligible (unlocks new use cases, no binary size change). + +**Dependency**: None. + +--- + +### Task #31: Asset Integrity Validation +**Goal**: Add CRC32/hash checks to detect corrupted or incorrect assets. + +**Attack Plan**: +- [ ] **31.1**: Compute CRC32 for each asset during build (`asset_packer.cc`) + - Store CRC in `AssetRecord` struct +- [ ] **31.2**: Add optional validation in `GetAsset()` (debug builds only) + - Compare computed CRC against stored value + - Log warnings if mismatch detected +- [ ] **31.3**: Add `--verify-assets` flag to test suite + - Run full integrity check on all assets + +**Size Impact**: +4 bytes per asset (CRC32), negligible overhead. + +**Dependency**: None. + +--- + +## Lower Priority (Future Enhancements) + +### Task #18.1: 3D Asset Format Support (Part of Task #18) +**Goal**: Extend asset system to handle binary mesh/scene data from Blender. + +**Attack Plan**: +- [ ] **18.1.1**: Define binary mesh format (vertices, indices, normals, UVs) + - Consider alignment requirements for GPU upload +- [ ] **18.1.2**: Update `asset_packer.cc` to handle `.mesh` files + - Add `MESH3D` asset type +- [ ] **18.1.3**: Implement runtime parser in `src/3d/mesh_loader.cc` + - Parse binary blob into `Mesh` struct +- [ ] **18.1.4**: Create Blender export script (`tools/blender_export.py`) + +**Size Impact**: Depends on scene complexity (meshes can be large, consider compression). + +**Dependency**: Task #27 (compression) for mesh data. + +--- + +### Task #32: Asset Hot-Reloading (Development Only) +**Goal**: Enable live asset updates for faster iteration (under `!STRIP_ALL`). + +**Attack Plan**: +- [ ] **32.1**: Add file watcher for `assets/final/` directory + - Monitor `.spec`, `.wgsl` files for changes +- [ ] **32.2**: Trigger rebuild of `assets_data.cc` on file change + - Invoke `asset_packer` and recompile +- [ ] **32.3**: Add API to reload specific assets at runtime + - `ReloadAsset(AssetId)` for development use + +**Size Impact**: None (stripped from final build). + +**Dependency**: None (development-only feature). + +--- + +### Task #33: Asset Usage Tracking & Dead Asset Elimination +**Goal**: Identify and remove unused assets to reduce binary size. + +**Attack Plan**: +- [ ] **33.1**: Add instrumentation to track `GetAsset()` calls + - Log which assets are accessed during a full demo run +- [ ] **33.2**: Generate report of unused assets +- [ ] **33.3**: Remove unused entries from `demo_assets.txt` + +**Size Impact**: Depends on dead assets (could save 5-15% if unused assets exist). + +**Dependency**: None. + +--- + +# Priority Recommendation + +For the **64k goal**, prioritize in this order: +1. **Task #27** (Compression) - Biggest size win +2. **Task #29** (Shader Minification) - Easy win, low risk +3. **Task #28** (Spectrogram Quantization) - Medium effort, high impact +4. **Task #20.1** (Remove STL) - Required for future CRT replacement +5. **Task #18.1** (3D Assets) - Once size budget allows -- cgit v1.2.3