path: root/doc/ASSET_SYSTEM.md

# compact asset system for the 64k demo

This file describe the features of a compact asset system used in the demo.

The idea is the following:

# run-time asset retrieval:
 All assets are const byte arrays. We need a 'const uint8_t* GetAsset(uint16 asset_id)'
 The asset ID is defined in a 'assets.h' header, generated during the final
 compile by our own assembling tool. assets.h needs to be re-generated
 infrequently.

# assembling the assets:

## description file 'assets.txt'
 All assets are just files in the assets/final/ directory
 This directory needs a assets.txt text file to describe the asset files.
 Each line of the assets.txt file contain, comma-separated:
   * the name of the asset (that will be used by the #define in assets.h),
   * the name of the file associated
   * the compression to use for this asset (default NONE. More options later)
   * and optionally the type of assets.

## example For instance, a line in assets.txt will read:

SAMPLE_142, sample_142.spec, NONE, "this is a drum kick sample"

This instructs the final assembled file assets.h to have a code line:
  #define ASSET_SAMPLE_142 6323

(6323 is just an associated id)

(or an enum instead of #define's)

so that we can call
```
#include "asset.h"
const uint8_t* mysample = GetAsset(ASSET_SAMPLE_142);
...etc
```

(if we use enums, GetAssert() signature needs to be changed)

### Lazy decompression
to save memory some assets can be decompressed 'at retrieval time' but kept
compressed in memory until then.
This means that we need a 'void DropAsset(uint16 asset_id, const uint8* asset)'
method to handle memory disallocation depending on the asset type.

### assembling tool

we need a simple tool that:
  * takes the assets.txt file and parse it
  * generates the assets.h file with asset enums
  * generates the assets_data.cc file with all the data
  * put these in the source tree
  * this process needs a script for automation

## Technical Details

To support diverse usage (binary data, strings, typed structs), the asset packer enforces the following:
1. **Alignment**: All static asset arrays are declared with `alignas(16)`. This ensures that `reinterpret_cast` to types like `float*` or SIMD vectors is safe (provided the data layout matches).
2. **String Safety**: Every static asset is appended with a null-terminator (`0x00`). This allows the raw pointer to be safely used as a `const char*` C-string.
3. **Size Reporting**: The `size` field in `AssetRecord` reflects the *original* file size. However, the underlying buffer is guaranteed to be at least `size + 1` bytes long (due to the null terminator).

# Shader Snippets

Shader code (WGSL) can also be managed as assets.
1. Create `.wgsl` files in `assets/final/shaders/` (or similar).
2. Add them to `assets/final/demo_assets.txt` with compression `NONE`.
   Example: `SHADER_COMMON, shaders/common.wgsl, NONE`
3. In C++, retrieve them using `GetAsset()`. Since they are binary blobs,
   construct a `std::string` or `std::string_view` using the returned pointer
   and size.
4. Register them with the `ShaderComposer`.

---

# Current Implementation Status

## What's Working ✅
- **Build-Time Packer** (`tools/asset_packer.cc`): Parses `demo_assets.txt`, embeds binary files as C++ arrays
- **Runtime Manager** (`src/util/asset_manager.{h,cc}`): O(1) retrieval with caching, SIOF fix via singleton
- **Asset Types**: Static binaries (`.spec`, `.wgsl`), procedural textures (`PROC(func, params)`)
- **Safety Guarantees**: 16-byte alignment, null-terminator for strings, size reporting
- **Testing**: `test_assets.cc` validates retrieval, caching, procedural generation
- **Current Manifest**: 33 assets (15 audio samples, 17 WGSL shaders, 1 procedural texture)

## Design Strengths
- Enum-based type safety (no magic numbers or strings at runtime)
- Zero runtime cost for static assets (direct pointer return)
- Clean separation of build-time and runtime concerns
- Procedural generation support (saves binary size)
- Thread-safe for immutable assets (cache is write-once)

## Design Weaknesses
- **No compression**: Only `NONE` supported, critical blocker for 64k goal
- **STL dependencies**: `std::map`, `std::string` in runtime code (conflicts with CRT replacement)
- **Hardcoded procedural dimensions**: Assumes 256×256 RGBA8 (lines 85-86 in `asset_manager.cc`)
- **No integrity checks**: No CRC/hash validation for asset correctness
- **Limited asset types**: No mesh/scene format yet (needed for Task #18)

---

# Asset System Improvement Tasks

## High Priority (Critical for 64k Goal)

### Task #27: Asset Compression Layer
**Goal**: Implement runtime decompression to reduce binary size by 30-50%.

**Attack Plan**:
- [ ] **27.1**: Add compression type enum (`NONE`, `ZLIB`, `RLE`, `CUSTOM`)
- [ ] **27.2**: Update `asset_packer.cc` to compress assets during build
  - Integrate a lightweight compression library (e.g., miniz for zlib)
  - Store compressed size + original size in `AssetRecord`
- [ ] **27.3**: Implement runtime decompressor in `asset_manager.cc`
  - Allocate decompressed buffer on first `GetAsset()` call
  - Cache decompressed data, allow `DropAsset()` to free memory
- [ ] **27.4**: Update `assets.txt` format to specify compression per-asset
  - Example: `KICK_1, ZLIB, kick1.spec, "Compressed kick sample"`
- [ ] **27.5**: Add tests for compressed asset retrieval and DropAsset

**Size Impact**: Estimated 30-50% reduction on asset blob (shaders compress well, spectrograms moderately).

**Dependency**: None (standalone improvement).

---

### Task #28: Spectrogram Quantization
**Goal**: Compress audio assets from `float32` to `uint16_t` with logarithmic frequency binning.

**Attack Plan**:
- [ ] **28.1**: Research optimal frequency bin distribution (logarithmic vs linear)
- [ ] **28.2**: Update `spectool` to export quantized `.spec` format
  - Add `--quantize` flag to `spectool analyze`
  - Map spectral values to `uint16_t` range with configurable dynamic range
- [ ] **28.3**: Update audio synthesis engine to decode quantized spectrograms
  - Add dequantization step in `src/audio/synth.cc`
- [ ] **28.4**: Re-generate all `.spec` assets with quantization enabled
- [ ] **28.5**: Add tests to verify audio quality degradation is acceptable

**Size Impact**: 2-4× reduction on `.spec` files (currently 15 audio assets in manifest).

**Dependency**: Task #27 (compression) for maximum size savings.

---

### Task #29: WGSL Shader Minification
**Goal**: Minify WGSL shaders to reduce text asset size.

**Attack Plan**:
- [ ] **29.1**: Implement WGSL minifier in `tools/wgsl_minify.py`
  - Remove comments, unnecessary whitespace
  - Rename variables to single-letter identifiers (preserve entry points)
  - Strip debug labels and unused code
- [ ] **29.2**: Integrate minifier into build pipeline
  - Add `MINIFY` compression type to `asset_packer.cc`
  - Auto-minify all `SHADER_*` assets during build
- [ ] **29.3**: Validate minified shaders compile correctly
  - Add shader compilation test to CI/CD

**Size Impact**: 40-60% reduction on shader text (currently 17 WGSL shaders in manifest).

**Dependency**: None (standalone improvement).

---

## Medium Priority (Code Hygiene & Maintainability)

### Task #20.1: Remove STL from Asset Manager (Part of Task #20)
**Goal**: Eliminate `std::map` and `std::string` to prepare for CRT replacement.

**Attack Plan**:
- [ ] **20.1.1**: Replace `kAssetManagerProcGenFuncMap` (line 21 in `asset_manager.cc`)
  - Switch to static array lookup or compile-time switch-case
  - Example:
    ```cpp
    static const struct { const char* name; ProcGenFunc func; } kProcGenFuncs[] = {
      {"gen_noise", procedural::gen_noise},
      {"gen_grid", procedural::gen_grid},
      {"make_periodic", procedural::make_periodic},
    };
    ```
- [ ] **20.1.2**: Replace `std::map` in `asset_packer.cc` (build-time tool, lower priority)
- [ ] **20.1.3**: Verify no STL usage remains in `src/util/asset_manager.cc`

**Size Impact**: Minor (few KB), but critical for future CRT replacement.

**Dependency**: None.

---

### Task #30: Procedural Asset Generalization
**Goal**: Support variable dimensions and formats for procedural textures.

**Attack Plan**:
- [ ] **30.1**: Update `assets.txt` format to encode dimensions in `proc_params`
  - Example: `NOISE_TEX, PROC(gen_noise, 256, 256, 4, 1234, 16), _, "Width, Height, Channels, Seed, Freq"`
- [ ] **30.2**: Update `asset_packer.cc` to parse dimension parameters
- [ ] **30.3**: Update `GetAsset()` to read dimensions from `proc_params`
  - Allocate buffer dynamically based on `width * height * channels`
- [ ] **30.4**: Add support for grayscale (1 channel) and RGB (3 channels) formats

**Size Impact**: Negligible (unlocks new use cases, no binary size change).

**Dependency**: None.

---

### Task #31: Asset Integrity Validation
**Goal**: Add CRC32/hash checks to detect corrupted or incorrect assets.

**Attack Plan**:
- [ ] **31.1**: Compute CRC32 for each asset during build (`asset_packer.cc`)
  - Store CRC in `AssetRecord` struct
- [ ] **31.2**: Add optional validation in `GetAsset()` (debug builds only)
  - Compare computed CRC against stored value
  - Log warnings if mismatch detected
- [ ] **31.3**: Add `--verify-assets` flag to test suite
  - Run full integrity check on all assets

**Size Impact**: +4 bytes per asset (CRC32), negligible overhead.

**Dependency**: None.

---

## Lower Priority (Future Enhancements)

### Task #18.1: 3D Asset Format Support (Part of Task #18)
**Goal**: Extend asset system to handle binary mesh/scene data from Blender.

**Attack Plan**:
- [ ] **18.1.1**: Define binary mesh format (vertices, indices, normals, UVs)
  - Consider alignment requirements for GPU upload
- [ ] **18.1.2**: Update `asset_packer.cc` to handle `.mesh` files
  - Add `MESH3D` asset type
- [ ] **18.1.3**: Implement runtime parser in `src/3d/mesh_loader.cc`
  - Parse binary blob into `Mesh` struct
- [ ] **18.1.4**: Create Blender export script (`tools/blender_export.py`)

**Size Impact**: Depends on scene complexity (meshes can be large, consider compression).

**Dependency**: Task #27 (compression) for mesh data.

---

### Task #32: Asset Hot-Reloading (Development Only)
**Goal**: Enable live asset updates for faster iteration (under `!STRIP_ALL`).

**Attack Plan**:
- [ ] **32.1**: Add file watcher for `assets/final/` directory
  - Monitor `.spec`, `.wgsl` files for changes
- [ ] **32.2**: Trigger rebuild of `assets_data.cc` on file change
  - Invoke `asset_packer` and recompile
- [ ] **32.3**: Add API to reload specific assets at runtime
  - `ReloadAsset(AssetId)` for development use

**Size Impact**: None (stripped from final build).

**Dependency**: None (development-only feature).

---

### Task #33: Asset Usage Tracking & Dead Asset Elimination
**Goal**: Identify and remove unused assets to reduce binary size.

**Attack Plan**:
- [ ] **33.1**: Add instrumentation to track `GetAsset()` calls
  - Log which assets are accessed during a full demo run
- [ ] **33.2**: Generate report of unused assets
- [ ] **33.3**: Remove unused entries from `demo_assets.txt`

**Size Impact**: Depends on dead assets (could save 5-15% if unused assets exist).

**Dependency**: None.

---

# Priority Recommendation

For the **64k goal**, prioritize in this order:
1. **Task #27** (Compression) - Biggest size win
2. **Task #29** (Shader Minification) - Easy win, low risk
3. **Task #28** (Spectrogram Quantization) - Medium effort, high impact
4. **Task #20.1** (Remove STL) - Required for future CRT replacement
5. **Task #18.1** (3D Assets) - Once size budget allows