diff options
| -rw-r--r-- | PROJECT_CONTEXT.md | 21 | ||||
| -rw-r--r-- | doc/3D.md | 223 | ||||
| -rw-r--r-- | doc/ASSET_SYSTEM.md | 317 | ||||
| -rw-r--r-- | doc/BUILD.md | 105 | ||||
| -rw-r--r-- | doc/CONTEXT_MAINTENANCE.md | 356 | ||||
| -rw-r--r-- | doc/CONTRIBUTING.md | 470 | ||||
| -rw-r--r-- | doc/FETCH_DEPS.md | 68 | ||||
| -rw-r--r-- | doc/HOWTO.md | 468 | ||||
| -rw-r--r-- | doc/MASKING_SYSTEM.md | 247 | ||||
| -rw-r--r-- | doc/PROCEDURAL.md | 24 | ||||
| -rw-r--r-- | doc/SCENE_FORMAT.md | 70 | ||||
| -rw-r--r-- | doc/SEQUENCE.md | 284 | ||||
| -rw-r--r-- | doc/SPECTRAL_BRUSH_EDITOR.md | 474 | ||||
| -rw-r--r-- | doc/TRACKER.md | 74 | ||||
| -rw-r--r-- | doc/test_demo_README.md | 275 |
15 files changed, 847 insertions, 2629 deletions
diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 3afab58..636f339 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -65,17 +65,26 @@ Style: - **Tooling & Optimization** - [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling. - [x] **Task #39: Visual Debugging System**: Implemented wireframe primitives (Sphere, Cone, Cross, Trajectory) for debugging. - - [ ] **Task #53: Particles Shader Polish**: Improve visual quality of particles. - - [ ] **Task #55: SDF Random Planes Intersection**: Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection. -- **Tooling & Optimization** - - [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling. +--- +## Design Docs Quick Reference + +For detailed documentation, use Read tool to load specific docs: + +- **doc/TRACKER.md**: Audio pattern system with unit-less timing (1 unit = 4 beats). Text-based music score compiled to C++ runtime. +- **doc/3D.md**: Hybrid SDF raymarching with BVH acceleration and Position Based Dynamics physics. +- **doc/ASSET_SYSTEM.md**: Build-time asset packer with 16-byte alignment, enum-based O(1) retrieval, procedural generation support. +- **doc/BUILD.md**: Multi-platform builds (Debug/STRIP_ALL/FINAL_STRIP), cross-compilation, size reporting. +- **doc/SPECTRAL_BRUSH_EDITOR.md**: Web tool for tracing spectrograms with Bezier curves (50-100× compression). +- **doc/SEQUENCE.md**: .seq timeline format with BPM notation, priority modifiers, Gantt visualization. +- **doc/MASKING_SYSTEM.md**: Auxiliary texture registry for inter-effect screen-space partitioning. +- **doc/SCENE_FORMAT.md**: Binary scene format (SCN1) with object transforms, physics, mesh references. +- **doc/test_demo_README.md**: 16s audio/visual sync test tool with tempo variation and peak logging. +- **doc/CONTEXT_MAINTENANCE.md**: Context hygiene protocol (archive to COMPLETED.md monthly, keep Tier 1 files lean). --- ## Future Goals - **Task #36: Blender Exporter**: Create script to export scenes to internal binary format. (Deprioritized) -- **Task #5: Implement Spectrogram Editor** - - [ ] Develop a web-based tool (`tools/editor`) for creating and editing `.spec` files visually. - **Task #21: Shader Optimization** - [ ] Use macros or code generation to factorize common WGSL code (normals, bump, lighting). - [ ] Implement Tri-planar mapping for better procedural textures. @@ -1,197 +1,118 @@ -# 3D system and rendering pipeline +# 3D System and Rendering Pipeline -This sub-project describe how the 3D worlds are going to be rendered. -We want a camera to move around moving and dynamic objects. -These objects can be need a physics and collision system. +## Core Concept -## the idea +Hybrid SDF/rasterization pipeline with physics and collision detection. -Each object has: - * a bounding box or capsule - * a BVH is maintained while these move around physically (or not) - * to render a frame we cull these bounding volumes - * at rendering time, the bounding box or sphere is turned into a quad or triangle - fan and a shader associated with the object is called (after proper world-object-camera transformations) - * each object can be queries for: - a) ray-object intersection ("return the distance from the object at this point P in this direction D") - b) Signed Distance Field ("what is the minimum distance to the object from this point P?") - -So in the end, we'll - a) move the camera and lights along paths - b) transform the bounding volumes and cull for visible boxes - c) project them - d) call the objects' respective shaders for rendering - -We want to use shadow maps, so multi-passes is expected. - -## debugging features - -The assist the visual debugging, we need a 'visual_debug' mode (code to be -removed with STRIP_ALL) that: +## Object Representation - a) draws a wireframe around the bounding volumes (box, etc.) - b) draw the trajectories (camera, objects, ...) - c) show the collision points - d) displays the ray/object intersection interactively - e) show the lights (direction, cone, ...), display their shadow-map in 3d and on-screen (2d). +Each object has: +- Bounding volume (box/capsule) +- SDF function (signed distance field) +- Ray-intersection query support +- Material/shader association -This must be captured and tracked as **Task #39: Visual Debugging System**. +## Render Pipeline +1. Move camera and lights along paths +2. Transform and cull bounding volumes (BVH) +3. Project visible volumes to screen +4. Raymarch SDF inside proxy geometry (fragment shader) +5. Multi-pass for shadow maps -## Global Acceleration Structure (BVH) +## Visual Debugging (Task #39) -To support efficient global queries (e.g., for soft shadows via raymarching, or physics), we will implement a dynamic Bounding Volume Hierarchy (BVH). +Debug mode (`!STRIP_ALL`) provides: +- Wireframe bounding volumes +- Camera/object trajectories +- Collision point visualization +- Ray-intersection display +- Light visualization (direction, cone, shadow maps) -### Data Structures +## BVH Acceleration Structure -**CPU/GPU Common Node Layout:** -The node structure is designed to be 32-byte aligned for efficient GPU access. +### Purpose +Efficient spatial queries for rendering (GPU) and physics (CPU). +### Node Layout ```cpp struct BVHNode { - // 16 bytes float min_x, min_y, min_z; - int32_t left_idx; // If < 0, this is a leaf node. - - // 16 bytes + int32_t left_idx; // < 0 = leaf node float max_x, max_y, max_z; - int32_t right_idx; // If leaf, this holds the object_index. + int32_t right_idx; // leaf: object_index }; ``` -**WGSL Representation:** +WGSL: ```wgsl struct BVHNode { - min: vec3<f32>, - left_idx: i32, - max: vec3<f32>, - obj_idx_or_right: i32, + min: vec3<f32>, left_idx: i32, + max: vec3<f32>, obj_idx_or_right: i32, }; @group(0) @binding(2) var<storage, read> bvh_nodes: array<BVHNode>; ``` ### Construction (CPU) +Rebuild every frame (< 100 objects): +1. Calculate AABBs for all objects +2. Recursive midpoint split (largest variance axis) +3. Linearize tree for GPU upload -Since the scene is small (< 100 objects) and dynamic, we will **rebuild** the BVH from scratch every frame on the CPU. This avoids the complexity of refitting and balancing. The BVH is purely a spatial acceleration structure and does not contain physics logic (velocity, mass, etc.), allowing it to be used for both rendering (GPU) and physics (CPU). - -**Algorithm: Recursive Midpoint Split** -1. **Calculate Centroids**: Compute the center of the AABB for all active objects. -2. **Split**: - * Find the axis with the largest variance in centroid positions. - * Sort objects along that axis (or just partition based on the median). - * Create a node, assign `left` and `right` children recursively. - * Stop when a node has 1 object (Leaf). -3. **Linearize**: Flatten the tree into a `std::vector<BVHNode>` for GPU upload. - -### Traversal (GPU/WGSL) - -Since WGSL does not support recursion, we use a fixed-size stack. - +### Traversal (GPU) +Fixed-size stack (no recursion in WGSL): ```wgsl fn map_scene(pos: vec3<f32>) -> f32 { var min_dist = 10000.0; var stack: array<i32, 32>; - var stack_ptr = 0; - - // Push root - stack[stack_ptr] = 0; - stack_ptr++; - - while (stack_ptr > 0) { - stack_ptr--; - let node_idx = stack[stack_ptr]; - let node = bvh_nodes[node_idx]; - - // Check AABB intersection - if (aabb_sdf(pos, node.min, node.max) < min_dist) { - if (node.left_idx < 0) { - // Leaf: Evaluate Object SDF - let obj_idx = node.obj_idx_or_right; - let d = object_sdf(pos, obj_idx); - min_dist = min(min_dist, d); - } else { - // Internal: Push children - // Optimization: Push furthest child first so we pop closest first - stack[stack_ptr] = node.left_idx; - stack_ptr++; - stack[stack_ptr] = node.obj_idx_or_right; - stack_ptr++; - } - } - } + // Push root, iterate until stack empty + // Check AABB, evaluate leaf SDFs return min_dist; } ``` ## Physics & Collision System -We need a lightweight physics engine that leverages our SDF representation. - ### Components -We will extend `Object3D` or add a `PhysicsBody` struct: -* **Mass/InverseMass**: For dynamic response. -* **Velocity**: Linear velocity `vec3`. -* **Restitution**: Bounciness (0.0 - 1.0). -* **Friction**: Damping factor. - -### Broad Phase: BVH Re-use -We reuse the same BVH constructed for rendering to accelerate CPU-side collision detection. -* **Query**: `QueryBVH(AABB query_box)` returns a list of potential candidates. -* This avoids $O(N^2)$ checks. +Extend `Object3D` with: +- Mass/InverseMass +- Velocity (linear) +- Restitution (bounciness) +- Friction -### Narrow Phase: SDF-Based Collision -Since we don't have explicit meshes for collision, we use the analytical SDFs. +### Broad Phase +Reuse BVH for O(N log N) collision detection. -**Algorithm: Proxy Point Probing** -Instead of full SDF-on-SDF intersection (which is expensive), we approximate dynamic objects as a set of "Probe Points" (e.g., bounding box corners, center). +### Narrow Phase +**Proxy Point Probing** (SDF-based): +1. Transform probe points into neighbor's local space +2. Evaluate SDF at probe points +3. If `d < 0`: collision detected + - Penetration depth: `p = -d` + - Normal: `∇SDF` via central differences + - Contact point: `P_world - N * p` -1. **Transform**: For a dynamic object $A$ and candidate neighbor $B$: - * Transform $A$'s probe points into $B$'s local space: $P_{local} = InverseModel_B * P_{world}$. -2. **Evaluate**: Calculate $d = SDF_B(P_{local})$. -3. **Collision**: If $d < 0$: - * **Penetration Depth**: $p = -d$. - * **Normal**: Compute gradient $N = \nabla SDF_B(P_{local})$ via central differences. Transform $N$ back to world space. - * **Contact Point**: $P_{contact} = P_{world} - N * p$. - -### Narrow Phase & Resolution: -* If collision detected (depth $p > 0$): - * **Positional Correction**: Move object by $N * p$ (to resolve penetration). - * **Velocity Response (PBD)**: In Position Based Dynamics (PBD), the velocity is re-evaluated *after* all collisions and constraints have been resolved: `v = (p_new - p_old) / dt`. - * **Friction**: Apply tangential damping. +### Collision Response (Position Based Dynamics) +1. Integrate position: `p_pred = p + v * dt` +2. Broad phase: Find collision pairs (BVH) +3. Narrow phase: Resolve penetrations on `p_pred` +4. Update velocity: `v = (p_pred - p) / dt` +5. Finalize: `p = p_pred` ### Rotation & Angular Momentum -Objects should support rotation via **Quaternions**. -* **State**: Add `quat orientation` and `vec3 angular_velocity` to `Object3D`. -* **Integration**: Update orientation using angular velocity: `q = q + 0.5 * dt * [0, w] * q`, then normalize. -* **Narrow Phase**: Proxy points must be rotated by the quaternion: `P_local = Inverse(Q) * (P_world - Position)`. -* **Collision Response**: Collision impulses should affect both linear and angular momentum. - -### Solver Loop (Position Based Dynamics) -1. **Integrate Position (Predicted)**: `p_pred = p + v * dt`. -2. **Broad Phase**: Find pairs using BVH. -3. **Narrow Phase & Resolution**: - * Resolve penetrations and constraints on `p_pred`. -4. **Update Velocity**: `v = (p_pred - p) / dt`. -5. **Finalize Position**: `p = p_pred`. - -### Code Integration Plan - -1. **CPU-Side SDF Library**: - * Create `src/3d/sdf_cpu.h` implementing the same primitives (`sdSphere`, `sdBox`, `sdTorus`) as the WGSL shaders, using `src/util/mini_math.h`. -2. **Physics System Class**: - * Create `src/3d/physics.h/cc`. - * `PhysicsSystem::Update(Scene& scene, float dt)`. - * `PhysicsSystem::ResolveCollision(Object3D& a, Object3D& b)`. -3. **Main Loop Integration**: - * Call `PhysicsSystem::Update` before `Renderer3D::render`. +- Quaternion-based orientation +- Angular velocity `vec3` +- Integration: `q = q + 0.5 * dt * [0, w] * q` +- Collision impulses affect both linear and angular momentum -## future step +## Integration Plan -Have an exporter from Blender modelling software. That would be a converter -from simple blender-exported files to our internal format (as an asset for -our AssetManager or as c++ code directly) - **Task #36: Blender Exporter**. +1. **CPU-Side SDF Library**: `src/3d/sdf_cpu.h` (sdSphere, sdBox, sdTorus) +2. **Physics System**: `src/3d/physics.h/cc` (Update, ResolveCollision) +3. **Main Loop**: Call `PhysicsSystem::Update` before `Renderer3D::render` -## latter improvement +## Future Tasks -How to handle transparency? Multi-Ray-casting? -We need to think about the lighting strategy. - **Task #40: Advanced Lighting & Transparency**.
\ No newline at end of file +- **Task #36**: Blender exporter (convert scenes to binary format) +- **Task #40**: Advanced lighting & transparency (multi-ray-casting) +- **Task #39**: Visual debugging system (wireframes, trajectories) diff --git a/doc/ASSET_SYSTEM.md b/doc/ASSET_SYSTEM.md index aea99a2..4e34378 100644 --- a/doc/ASSET_SYSTEM.md +++ b/doc/ASSET_SYSTEM.md @@ -1,271 +1,108 @@ -# compact asset system for the 64k demo +# Compact Asset System for the 64k Demo -This file describe the features of a compact asset system used in the demo. +This file describes the asset system architecture and runtime API. -The idea is the following: +## Core Concept -# run-time asset retrieval: - All assets are const byte arrays. We need a 'const uint8_t* GetAsset(uint16 asset_id)' - The asset ID is defined in a 'assets.h' header, generated during the final - compile by our own assembling tool. assets.h needs to be re-generated - infrequently. +All assets are const byte arrays embedded in the binary. Runtime retrieval via `GetAsset(AssetId)` returns direct pointers with O(1) lookup. -# assembling the assets: +## Runtime Asset Retrieval -## description file 'assets.txt' - All assets are just files in the assets/final/ directory - This directory needs a assets.txt text file to describe the asset files. - Each line of the assets.txt file contain, comma-separated: - * the name of the asset (that will be used by the #define in assets.h), - * the name of the file associated - * the compression to use for this asset (default NONE. More options later) - * and optionally the type of assets. - -## example For instance, a line in assets.txt will read: - -SAMPLE_142, sample_142.spec, NONE, "this is a drum kick sample" - -This instructs the final assembled file assets.h to have a code line: - #define ASSET_SAMPLE_142 6323 +```cpp +#include "assets.h" +const uint8_t* mysample = GetAsset(ASSET_SAMPLE_142, &size); +// ... use asset +DropAsset(ASSET_SAMPLE_142, mysample); // For lazy decompression cleanup +``` -(6323 is just an associated id) +## Asset Manifest Format -(or an enum instead of #define's) +File: `assets/final/demo_assets.txt` -so that we can call +Format (CSV): ``` -#include "asset.h" -const uint8_t* mysample = GetAsset(ASSET_SAMPLE_142); -...etc +ASSET_NAME, filename.ext, COMPRESSION, "Description" ``` -(if we use enums, GetAssert() signature needs to be changed) - -### Lazy decompression -to save memory some assets can be decompressed 'at retrieval time' but kept -compressed in memory until then. -This means that we need a 'void DropAsset(uint16 asset_id, const uint8* asset)' -method to handle memory disallocation depending on the asset type. - -### assembling tool - -we need a simple tool that: - * takes the assets.txt file and parse it - * generates the assets.h file with asset enums - * generates the assets_data.cc file with all the data - * put these in the source tree - * this process needs a script for automation - -## Technical Details - -To support diverse usage (binary data, strings, typed structs), the asset packer enforces the following: -1. **Alignment**: All static asset arrays are declared with `alignas(16)`. This ensures that `reinterpret_cast` to types like `float*` or SIMD vectors is safe (provided the data layout matches). -2. **String Safety**: Every static asset is appended with a null-terminator (`0x00`). This allows the raw pointer to be safely used as a `const char*` C-string. -3. **Size Reporting**: The `size` field in `AssetRecord` reflects the *original* file size. However, the underlying buffer is guaranteed to be at least `size + 1` bytes long (due to the null terminator). - -# Shader Snippets - -Shader code (WGSL) can also be managed as assets. -1. Create `.wgsl` files in `assets/final/shaders/` (or similar). -2. Add them to `assets/final/demo_assets.txt` with compression `NONE`. - Example: `SHADER_COMMON, shaders/common.wgsl, NONE` -3. In C++, retrieve them using `GetAsset()`. Since they are binary blobs, - construct a `std::string` or `std::string_view` using the returned pointer - and size. -4. Register them with the `ShaderComposer`. - ---- - -# Current Implementation Status - -## What's Working ✅ -- **Build-Time Packer** (`tools/asset_packer.cc`): Parses `demo_assets.txt`, embeds binary files as C++ arrays -- **Runtime Manager** (`src/util/asset_manager.{h,cc}`): O(1) retrieval with caching, SIOF fix via singleton -- **Asset Types**: Static binaries (`.spec`, `.wgsl`), procedural textures (`PROC(func, params)`) -- **Safety Guarantees**: 16-byte alignment, null-terminator for strings, size reporting -- **Testing**: `test_assets.cc` validates retrieval, caching, procedural generation -- **Current Manifest**: 33 assets (15 audio samples, 17 WGSL shaders, 1 procedural texture) - -## Design Strengths -- Enum-based type safety (no magic numbers or strings at runtime) -- Zero runtime cost for static assets (direct pointer return) -- Clean separation of build-time and runtime concerns -- Procedural generation support (saves binary size) -- Thread-safe for immutable assets (cache is write-once) - -## Design Weaknesses -- **No compression**: Only `NONE` supported, critical blocker for 64k goal -- **STL dependencies**: `std::map`, `std::string` in runtime code (conflicts with CRT replacement) -- **Hardcoded procedural dimensions**: Assumes 256×256 RGBA8 (lines 85-86 in `asset_manager.cc`) -- **No integrity checks**: No CRC/hash validation for asset correctness -- **Limited asset types**: No mesh/scene format yet (needed for Task #18) - ---- - -# Asset System Improvement Tasks - -## High Priority (Critical for 64k Goal) - -### Task #27: Asset Compression Layer -**Goal**: Implement runtime decompression to reduce binary size by 30-50%. - -**Attack Plan**: -- [ ] **27.1**: Add compression type enum (`NONE`, `ZLIB`, `RLE`, `CUSTOM`) -- [ ] **27.2**: Update `asset_packer.cc` to compress assets during build - - Integrate a lightweight compression library (e.g., miniz for zlib) - - Store compressed size + original size in `AssetRecord` -- [ ] **27.3**: Implement runtime decompressor in `asset_manager.cc` - - Allocate decompressed buffer on first `GetAsset()` call - - Cache decompressed data, allow `DropAsset()` to free memory -- [ ] **27.4**: Update `assets.txt` format to specify compression per-asset - - Example: `KICK_1, ZLIB, kick1.spec, "Compressed kick sample"` -- [ ] **27.5**: Add tests for compressed asset retrieval and DropAsset - -**Size Impact**: Estimated 30-50% reduction on asset blob (shaders compress well, spectrograms moderately). - -**Dependency**: None (standalone improvement). - ---- - -### Task #28: Spectrogram Quantization -**Goal**: Compress audio assets from `float32` to `uint16_t` with logarithmic frequency binning. - -**Attack Plan**: -- [ ] **28.1**: Research optimal frequency bin distribution (logarithmic vs linear) -- [ ] **28.2**: Update `spectool` to export quantized `.spec` format - - Add `--quantize` flag to `spectool analyze` - - Map spectral values to `uint16_t` range with configurable dynamic range -- [ ] **28.3**: Update audio synthesis engine to decode quantized spectrograms - - Add dequantization step in `src/audio/synth.cc` -- [ ] **28.4**: Re-generate all `.spec` assets with quantization enabled -- [ ] **28.5**: Add tests to verify audio quality degradation is acceptable - -**Size Impact**: 2-4× reduction on `.spec` files (currently 15 audio assets in manifest). - -**Dependency**: Task #27 (compression) for maximum size savings. - ---- - -### Task #29: WGSL Shader Minification -**Goal**: Minify WGSL shaders to reduce text asset size. - -**Attack Plan**: -- [ ] **29.1**: Implement WGSL minifier in `tools/wgsl_minify.py` - - Remove comments, unnecessary whitespace - - Rename variables to single-letter identifiers (preserve entry points) - - Strip debug labels and unused code -- [ ] **29.2**: Integrate minifier into build pipeline - - Add `MINIFY` compression type to `asset_packer.cc` - - Auto-minify all `SHADER_*` assets during build -- [ ] **29.3**: Validate minified shaders compile correctly - - Add shader compilation test to CI/CD - -**Size Impact**: 40-60% reduction on shader text (currently 17 WGSL shaders in manifest). - -**Dependency**: None (standalone improvement). - ---- - -## Medium Priority (Code Hygiene & Maintainability) - -### Task #20.1: Remove STL from Asset Manager (Part of Task #20) - COMPLETED -**Goal**: Eliminate `std::map` and `std::string` to prepare for CRT replacement. -**Status**: Initial removal for `AssetManager` hot paths completed. Full removal across the codebase deferred to Phase 2. - ---- - -### Task #30: Procedural Asset Generalization -**Goal**: Support variable dimensions and formats for procedural textures. - -**Attack Plan**: -- [ ] **30.1**: Update `assets.txt` format to encode dimensions in `proc_params` - - Example: `NOISE_TEX, PROC(gen_noise, 256, 256, 4, 1234, 16), _, "Width, Height, Channels, Seed, Freq"` -- [ ] **30.2**: Update `asset_packer.cc` to parse dimension parameters -- [ ] **30.3**: Update `GetAsset()` to read dimensions from `proc_params` - - Allocate buffer dynamically based on `width * height * channels` -- [ ] **30.4**: Add support for grayscale (1 channel) and RGB (3 channels) formats - -**Size Impact**: Negligible (unlocks new use cases, no binary size change). - -**Dependency**: None. - ---- - -### Task #31: Asset Integrity Validation -**Goal**: Add CRC32/hash checks to detect corrupted or incorrect assets. - -**Attack Plan**: -- [ ] **31.1**: Compute CRC32 for each asset during build (`asset_packer.cc`) - - Store CRC in `AssetRecord` struct -- [ ] **31.2**: Add optional validation in `GetAsset()` (debug builds only) - - Compare computed CRC against stored value - - Log warnings if mismatch detected -- [ ] **31.3**: Add `--verify-assets` flag to test suite - - Run full integrity check on all assets - -**Size Impact**: +4 bytes per asset (CRC32), negligible overhead. +Example: +``` +SAMPLE_142, sample_142.spec, NONE, "This is a drum kick sample" +SHADER_COMMON, shaders/common.wgsl, NONE, "Common utilities" +``` -**Dependency**: None. +This generates: +```cpp +enum class AssetId : uint16_t { + ASSET_SAMPLE_142 = 6323, + SHADER_COMMON = 6324, + // ... +}; +``` ---- +## Build Pipeline -## Lower Priority (Future Enhancements) +Tool: `tools/asset_packer.cc` -### Task #18.1: 3D Asset Format Support (Part of Task #18) -**Goal**: Extend asset system to handle binary mesh/scene data from Blender. +1. Parse `demo_assets.txt` +2. Read binary files from `assets/final/` +3. Generate `assets.h` (enum definitions) +4. Generate `assets_data.cc` (embedded byte arrays) +5. Auto-triggered by CMake on manifest changes -**Attack Plan**: -- [ ] **18.1.1**: Define binary mesh format (vertices, indices, normals, UVs) - - Consider alignment requirements for GPU upload -- [ ] **18.1.2**: Update `asset_packer.cc` to handle `.mesh` files - - Add `MESH3D` asset type -- [ ] **18.1.3**: Implement runtime parser in `src/3d/mesh_loader.cc` - - Parse binary blob into `Mesh` struct -- [ ] **18.1.4**: Create Blender export script (`tools/blender_export.py`) +## Technical Guarantees -**Size Impact**: Depends on scene complexity (meshes can be large, consider compression). +1. **Alignment**: All arrays declared `alignas(16)` for safe `reinterpret_cast` +2. **String Safety**: Null-terminator appended (safe as C-string) +3. **Size Reporting**: `size` reflects original file size (buffer is `size + 1`) -**Dependency**: Task #27 (compression) for mesh data. +## Shader Asset Usage ---- +```cpp +size_t size; +const uint8_t* shader_src = GetAsset(AssetId::SHADER_COMMON, &size); +std::string_view code(reinterpret_cast<const char*>(shader_src), size); +ShaderComposer::register_snippet("common", code); +``` -### Task #32: Asset Hot-Reloading (Development Only) -**Goal**: Enable live asset updates for faster iteration (under `!STRIP_ALL`). +## Procedural Assets -**Attack Plan**: -- [ ] **32.1**: Add file watcher for `assets/final/` directory - - Monitor `.spec`, `.wgsl` files for changes -- [ ] **32.2**: Trigger rebuild of `assets_data.cc` on file change - - Invoke `asset_packer` and recompile -- [ ] **32.3**: Add API to reload specific assets at runtime - - `ReloadAsset(AssetId)` for development use +Format: `PROC(function_name, params...)` -**Size Impact**: None (stripped from final build). +Example: +``` +NOISE_TEX, PROC(gen_noise, 1234, 16), NONE, "Noise texture seed=1234 freq=16" +``` -**Dependency**: None (development-only feature). +Runtime: First `GetAsset()` call invokes generator, caches result. ---- +## Design Strengths -### Task #33: Asset Usage Tracking & Dead Asset Elimination -**Goal**: Identify and remove unused assets to reduce binary size. +- Enum-based type safety (no magic numbers) +- Zero runtime cost for static assets +- Procedural generation support (size savings) +- Thread-safe for immutable assets -**Attack Plan**: -- [ ] **33.1**: Add instrumentation to track `GetAsset()` calls - - Log which assets are accessed during a full demo run -- [ ] **33.2**: Generate report of unused assets -- [ ] **33.3**: Remove unused entries from `demo_assets.txt` +## Design Weaknesses -**Size Impact**: Depends on dead assets (could save 5-15% if unused assets exist). +- **No compression**: Only `NONE` supported (critical blocker for 64k) +- **STL dependencies**: `std::map` in runtime code (conflicts with CRT removal) +- **Hardcoded procedural dimensions**: Assumes 256×256 RGBA8 +- **No integrity checks**: No CRC/hash validation -**Dependency**: None. +## Planned Improvements ---- +See **TODO.md** for detailed task breakdown: +- **Task #27**: Asset compression (zlib, RLE) +- **Task #28**: Spectrogram quantization (float32 → uint16_t) +- **Task #29**: WGSL shader minification +- **Task #30**: Procedural asset generalization +- **Task #31**: Asset integrity validation (CRC32) +- **Task #32**: Hot-reloading (dev builds) +- **Task #33**: Dead asset elimination -# Priority Recommendation +## Current Manifest -For the **64k goal**, prioritize in this order: -1. **Task #27** (Compression) - Biggest size win -2. **Task #29** (Shader Minification) - Easy win, low risk -3. **Task #28** (Spectrogram Quantization) - Medium effort, high impact -4. **Task #18.1** (3D Assets) - Once size budget allows -5. **Task #34: Full STL Removal** - Deferred to final optimization phase. +33 assets total: +- 15 audio samples (`.spec`) +- 17 WGSL shaders +- 1 procedural texture diff --git a/doc/BUILD.md b/doc/BUILD.md index e78aa91..1f65122 100644 --- a/doc/BUILD.md +++ b/doc/BUILD.md @@ -1,82 +1,69 @@ # Build Instructions -Debug build: +## Quick Start + +```bash +# Development build cmake -S . -B build -cmake --build build +cmake --build build -j4 -Size-optimized build: +# Size-optimized build cmake -S . -B build -DDEMO_SIZE_OPT=ON -cmake --build build - -## Windows Cross-Compilation (from macOS) - -Requires `mingw-w64` and `wine-stable` (for testing). +cmake --build build -j4 -1. Fetch Windows binaries: - ```bash - ./scripts/fetch_win_deps.sh - ``` - -2. Build for Windows: - ```bash - ./scripts/build_win.sh - ``` - This will produce `build_win/demo64k_packed.exe`. +# Final stripped build +cmake -S . -B build_final -DDEMO_FINAL_STRIP=ON +cmake --build build_final -j4 +``` -3. Run with Wine: - ```bash - ./scripts/run_win.sh - ``` +## Build Modes -# Building the Project with Xcode +| Mode | Flags | Use Case | +|------|-------|----------| +| Debug | `-DCMAKE_BUILD_TYPE=Debug` | Development, full error checking + debug features | +| STRIP_ALL | `-DDEMO_STRIP_ALL=ON` | Release candidate, full error checking, no debug features (~64k target) | +| FINAL_STRIP | `-DDEMO_FINAL_STRIP=ON` | Final release, no error checking, absolute minimum size | -This document provides instructions for building the 64k Demo project using Xcode on macOS. +## Dependencies -## Prerequisites +Install via Homebrew (macOS): +```bash +brew install wgpu-native +``` -- **Xcode:** Ensure you have Xcode installed. You can download it from the Mac App Store. -- **CMake:** This project uses CMake as its build system. Ensure CMake is installed on your system. +Or use project init script: +```bash +./scripts/project_init.sh +``` -## Build Steps +Required libraries: +- **miniaudio** (single header, auto-fetched by init script) +- **wgpu-native** (WebGPU implementation) +- **glfw3webgpu** (GLFW surface helper, auto-fetched) +- **UPX** (executable packer, optional for Linux/Windows) -1. **Open Terminal:** Navigate to the root directory of the project (`/Users/skal/demo`). +## Windows Cross-Compilation (macOS) -2. **Create Build Directory:** It is recommended to build out-of-source. Create a build directory: - ```bash - mkdir build - cd build - ``` +Requires `mingw-w64` and `wine-stable`. -3. **Configure with CMake for Xcode:** Generate the Xcode project files using CMake. For Xcode, you typically specify the generator as `Xcode`. - ```bash - cmake .. -G "Xcode" - ``` - This command will configure the project and create an `.xcodeproj` file within the `build` directory. +```bash +# Fetch Windows dependencies +./scripts/fetch_win_deps.sh -4. **Open Xcode Project:** Open the generated `.xcodeproj` file in Xcode: - ```bash - open YourProjectName.xcodeproj - ``` - (Replace `YourProjectName.xcodeproj` with the actual name of the generated project file, usually `demo.xcodeproj` or similar). +# Build for Windows +./scripts/build_win.sh -5. **Build and Run in Xcode:** - * Select your desired target (e.g., `demo64k`, `test_3d_render`). - * Choose a build scheme (e.g., `Debug` or `Release`) - * Click the "Run" button (or press `Cmd+R`) to build and launch the application. +# Test with Wine +./scripts/run_win.sh +``` -## GPU Performance Capture (Future Task) +Produces `build_win/demo64k_packed.exe`. -To enable future GPU performance analysis, a conditional compilation macro `ENABLE_GPU_PERF_CAPTURE` can be defined. This can be controlled via CMake build types or specific flags when configuring the project: +## Xcode Build (macOS) ```bash -# Example of defining the macro via CMake (add to CMakeLists.txt) -# if (MY_GPU_PERF_BUILD) -# target_compile_definitions(demo64k PRIVATE ENABLE_GPU_PERF_CAPTURE=1) -# endif +cmake -S . -B build -G "Xcode" +open build/demo.xcodeproj ``` -This macro can then be used within the code (e.g., in `gpu.h` or `platform.h`) to conditionally compile performance monitoring or capture code. - -## Shader Performance Analysis Task - -For detailed shader performance analysis on macOS, **Xcode's Metal debugger** is the recommended tool. A task has been added to `TODO.md` to integrate this into our workflow for future optimization efforts. +Use Xcode Metal debugger for shader performance analysis. diff --git a/doc/CONTEXT_MAINTENANCE.md b/doc/CONTEXT_MAINTENANCE.md index c228cd7..5dbfc84 100644 --- a/doc/CONTEXT_MAINTENANCE.md +++ b/doc/CONTEXT_MAINTENANCE.md @@ -1,332 +1,200 @@ # Context Maintenance Guide -This document describes how to maintain clean, efficient context for AI agents (Claude, Gemini) working on this project. +Keep AI context focused on **actionable, current information** while archiving **historical details** for reference. -## Philosophy - -**Goal**: Keep AI context focused on **actionable, current information** while archiving **historical details** for reference. - -**Principle**: The context should answer "What am I working on **now**?" not "What did we do 6 months ago?" +**Principle**: Answer "What am I working on **now**?" not "What did we do 6 months ago?" --- ## File Organization Hierarchy ### Tier 1: Critical (Always Loaded) -These files are essential for every session and should remain lean: +Essential for every session, remain lean: -- **CLAUDE.md** / **GEMINI.md** - Agent configuration (keep to ~30 lines) -- **PROJECT_CONTEXT.md** - High-level project overview (keep to ~200 lines) - - Current status summary (not detailed milestones) - - Architectural overview (brief) - - Next priorities (not detailed plans) -- **TODO.md** - Active tasks only (keep to ~300 lines) - - Current and next tasks - - Brief attack plans (not 200-line implementation details) - - Mark completed tasks and reference COMPLETED.md +- **CLAUDE.md** / **GEMINI.md** - Agent configuration (~30 lines) +- **PROJECT_CONTEXT.md** - High-level overview (~200 lines) +- **TODO.md** - Active tasks only (~300 lines) - **README.md** - Quick start guide ### Tier 2: Technical Reference (Always Loaded) Essential for daily work: -- **doc/HOWTO.md** - Usage instructions, common commands -- **doc/CONTRIBUTING.md** - Coding guidelines, commit policies -- **doc/AI_RULES.md** - Agent-specific rules +- **doc/HOWTO.md** - Usage instructions +- **doc/CONTRIBUTING.md** - Coding guidelines +- **doc/AI_RULES.md** - Agent rules ### Tier 3: Design Docs (Load On-Demand) -Load these only when working on specific subsystems: +Load when working on specific subsystems: -- **doc/ASSET_SYSTEM.md** - Asset pipeline details -- **doc/BUILD.md** - Build system details -- **doc/3D.md** - 3D rendering architecture -- **doc/SPEC_EDITOR.md** - Spectral editor design -- **doc/TRACKER.md** - Audio tracker system -- **doc/PROCEDURAL.md** - Procedural generation -- **doc/ANALYSIS_VARIABLE_TEMPO_V2.md** - Tempo system analysis +- **doc/ASSET_SYSTEM.md**, **doc/BUILD.md**, **doc/3D.md** +- **doc/SPEC_EDITOR.md**, **doc/TRACKER.md**, **doc/PROCEDURAL.md** ### Tier 4: Historical Archive (Load Rarely) -Reference material for understanding past decisions: +Reference for past decisions: -- **doc/COMPLETED.md** - Detailed completion history -- **doc/HANDOFF*.md** - Agent handoff documents -- **doc/*_ANALYSIS.md** - Technical investigations -- **doc/*_SUMMARY.md** - Task/feature summaries +- **doc/COMPLETED.md**, **doc/HANDOFF*.md** +- **doc/*_ANALYSIS.md**, **doc/*_SUMMARY.md** --- ## Maintenance Schedule -### After Major Milestone (Immediately) -**Trigger**: Completed a multi-week task or significant feature - -**Actions**: -1. ✅ Move detailed completion notes from TODO.md to COMPLETED.md -2. ✅ Update PROJECT_CONTEXT.md "Current Status" (remove old "Recently Completed") -3. ✅ Archive detailed implementation plans from TODO.md -4. ✅ Update "Next Up" section in PROJECT_CONTEXT.md - -**Time**: 10-15 minutes - -### Monthly Review (1st of Month) -**Trigger**: Calendar-based - -**Actions**: -1. ✅ Archive tasks completed >30 days ago from TODO.md → COMPLETED.md -2. ✅ Review PROJECT_CONTEXT.md for outdated "Recently Completed" items -3. ✅ Check for temporary analysis docs in root directory → move to doc/ -4. ✅ Verify CLAUDE.md/GEMINI.md only load Tier 1-2 files by default - -**Time**: 15-20 minutes +### After Major Milestone +1. Move completion notes from TODO.md to COMPLETED.md +2. Update PROJECT_CONTEXT.md "Current Status" +3. Archive detailed plans from TODO.md +4. Update "Next Up" section -### On-Demand (When Context Feels Slow) -**Trigger**: AI responses taking longer, context bloat suspected +### Monthly Review +1. Archive tasks completed >30 days ago +2. Remove old "Recently Completed" items +3. Move temp analysis docs to doc/ +4. Verify agent configs load only Tier 1-2 -**Actions**: -1. ✅ Run token analysis: `wc -w CLAUDE.md PROJECT_CONTEXT.md TODO.md doc/*.md | sort -rn` -2. ✅ Check for oversized files (>500 lines in Tier 1-2) -3. ✅ Aggressively archive verbose sections -4. ✅ Consider splitting large design docs - -**Time**: 30-45 minutes +### On-Demand (Context Bloat) +1. Run: `wc -w CLAUDE.md PROJECT_CONTEXT.md TODO.md` +2. Check for oversized files (>500 lines in Tier 1-2) +3. Archive verbose sections +4. Split large design docs --- -## What to Archive - -### ✅ ALWAYS Archive: -- Detailed implementation plans for completed tasks (>100 lines) -- Verbose milestone descriptions with step-by-step breakdowns -- Analysis documents for resolved issues -- Deprecated design docs (keep note in main doc pointing to archive) -- Old handoff documents (>1 month old) +## Decision Matrix: What to Archive? -### ⚠️ SUMMARIZE Before Archiving: -- Major architectural decisions (keep 2-3 line summary) -- Critical bug resolutions (keep root cause + solution) -- Performance optimizations (keep before/after metrics) +| Content Type | Action | Keep in Main Docs | +|--------------|--------|------------------| +| Completed tasks (>30 days) | ✅ Archive | 2-line summary | +| Detailed plans (>100 lines) | ✅ Archive | Goal + approach | +| Analysis of resolved issues | ✅ Archive | Root cause + solution | +| Deprecated designs | ✅ Archive | Pointer to archive | +| Major architectural decisions | ⚠️ Summarize | 3-line summary | +| Performance optimizations | ⚠️ Summarize | Before/after metrics | +| Current priorities | ❌ Keep | Full details | +| Coding guidelines | ❌ Keep | Full details | +| Active constraints | ❌ Keep | Full details | -### ❌ NEVER Archive: -- Current task priorities -- Essential coding guidelines (CONTRIBUTING.md, AI_RULES.md) -- Quick-start instructions (README.md, HOWTO.md) -- Active architectural constraints +**Quick rule**: If >30 days old AND >50 lines of detail → Archive with 2-line summary. --- -## Red Flags (Time to Clean Up) +## Red Flags (Clean Up Now) -### File Size Red Flags: -- PROJECT_CONTEXT.md >500 lines -- TODO.md >800 lines +### File Size +- PROJECT_CONTEXT.md >500 lines (critical: >800) +- TODO.md >800 lines (critical: >1200) - Any Tier 1-2 file >1000 lines -### Content Red Flags: -- "Recently Completed" section >100 lines in PROJECT_CONTEXT.md -- TODO.md contains tasks marked `[DONE]` for >30 days -- Detailed implementation plans (>50 lines) for completed tasks in TODO.md -- Analysis documents in root directory (should be in doc/) +### Content +- "Recently Completed" >100 lines in PROJECT_CONTEXT.md +- TODO.md has tasks marked `[DONE]` >30 days ago +- Detailed plans (>50 lines) for completed tasks +- Analysis docs in root directory (move to doc/) -### Context Red Flags: +### Performance - AI responses noticeably slower - Token usage >50K on initial load -- Repeated "let me read..." for basic project info +- Repeated "let me read..." for basic info --- ## Archiving Workflow -### 1. Identify Content to Archive -```bash -# Find large files -find . -name "*.md" -type f -exec wc -l {} + | sort -rn | head -10 - -# Find old completed tasks in TODO.md -grep -n "\[x\].*202[0-9]-[0-9][0-9]-[0-9][0-9]" TODO.md -``` - -### 2. Move to COMPLETED.md -```markdown -## Recently Completed (YYYY-MM-DD) - -- [x] **Task Name**: Brief summary - - Key points (2-3 bullets max) - - Result: One-line outcome -``` - -### 3. Update Source Files -- Replace verbose sections with: `**Note**: Detailed history in doc/COMPLETED.md` -- Update cross-references -- Keep high-level summaries - -### 4. Update Agent Configs -Update CLAUDE.md and GEMINI.md if files moved: -```markdown -# Design docs (load on-demand when needed) -# Use: "read @doc/COMPLETED.md" for detailed history -``` - -### 5. Verify -```bash -# Check context size -cat CLAUDE.md PROJECT_CONTEXT.md TODO.md | wc -w -# Target: <10,000 words for Tier 1 files - -# Verify no broken references -grep -r "@doc/" CLAUDE.md GEMINI.md PROJECT_CONTEXT.md TODO.md -``` - ---- - -## Common Mistakes to Avoid - -### ❌ Don't: -- Archive current priorities or active tasks -- Remove essential architectural context -- Break cross-references without updating -- Move files without updating CLAUDE.md/GEMINI.md -- Delete content (archive instead) -- Keep outdated "Recently Completed" sections >3 months - -### ✅ Do: -- Preserve key technical insights (in summaries) -- Update file references after moving docs -- Keep audit trail (note where content moved) -- Maintain searchable keywords (for finding archives) -- Regular small cleanups (better than big purges) - ---- - -## Quick Reference: File Moves - -When moving files from root to doc/: +**Quick steps:** +1. Find content: `grep -n "\[x\].*202[0-9]" TODO.md` +2. Move to COMPLETED.md with format: + ```markdown + - [x] **Task Name**: Brief summary (2-3 bullets max, one-line result) + ``` +3. Replace verbose sections: `**Note**: See doc/COMPLETED.md` +4. Update agent configs if files moved +5. Verify: `grep -r "@doc/" CLAUDE.md GEMINI.md` +**File moves:** ```bash -# 1. Move file (use git mv if tracked) git mv OLDFILE.md doc/OLDFILE.md - -# 2. Update CLAUDE.md -# Change: @OLDFILE.md -# To: # Use: "read @doc/OLDFILE.md" when needed - -# 3. Update GEMINI.md (same as above) - -# 4. Update cross-references in other files -grep -r "OLDFILE.md" *.md doc/*.md - -# 5. Commit with clear message -git commit -m "docs: Move OLDFILE.md to doc/ for better organization" +# Update CLAUDE.md/GEMINI.md references +# Update cross-references: grep -r "OLDFILE.md" *.md doc/*.md +git commit -m "docs: Move OLDFILE.md to doc/" ``` --- ## Context Size Targets -### Optimal Targets: -- **CLAUDE.md**: <30 lines -- **PROJECT_CONTEXT.md**: <300 lines -- **TODO.md**: <500 lines -- **Total Tier 1-2**: <10,000 words (~40K tokens) - -### Warning Thresholds: -- **PROJECT_CONTEXT.md**: >500 lines -- **TODO.md**: >800 lines -- **Total Tier 1-2**: >15,000 words (~60K tokens) - -### Critical (Clean Up Immediately): -- **PROJECT_CONTEXT.md**: >800 lines -- **TODO.md**: >1200 lines -- **Total Tier 1-2**: >20,000 words (~80K tokens) +| File | Optimal | Warning | Critical | +|------|---------|---------|----------| +| CLAUDE.md | <30 lines | - | - | +| PROJECT_CONTEXT.md | <300 lines | >500 | >800 | +| TODO.md | <500 lines | >800 | >1200 | +| Total Tier 1-2 | <10K words | >15K | >20K | --- ## Examples -### Good: Lean TODO.md Entry +### Good: Lean TODO.md ```markdown -- [ ] **Task #72: Implement Audio Compression** - - Goal: Reduce .spec file sizes by 50% - - Approach: Quantize to uint16_t, use RLE encoding - - See: doc/AUDIO_COMPRESSION_PLAN.md for details +- [ ] **Task #72: Audio Compression** + - Goal: Reduce .spec sizes by 50% + - Approach: uint16_t quantization, RLE + - See: doc/AUDIO_COMPRESSION_PLAN.md ``` -### Bad: Verbose TODO.md Entry +### Bad: Verbose TODO.md ```markdown -- [ ] **Task #72: Implement Audio Compression** - - Goal: Reduce .spec file sizes by 50% - - Background: Currently using float32 which is wasteful... - - Research: Investigated 5 compression algorithms... - - Step 1: Create compression interface - - Define CompressorBase class - - Add compress() and decompress() methods - - Implement factory pattern for algorithm selection - - Step 2: Implement RLE compressor - - [100+ more lines of detailed implementation] +- [ ] **Task #72: Audio Compression** + - Goal: Reduce .spec sizes by 50% + - Background: float32 wasteful... [20 lines] + - Step 1: Create interface + - Define CompressorBase [100+ lines] ``` -### Good: Lean PROJECT_CONTEXT.md Status +### Good: Lean PROJECT_CONTEXT.md ```markdown ### Current Status -- Audio: Stable, 93% test coverage, variable tempo support -- 3D: Hybrid SDF/raster, BVH acceleration, physics system -- Shaders: Modular WGSL, comprehensive compilation tests +- Audio: Stable, 93% test coverage, variable tempo +- 3D: Hybrid SDF/raster, BVH acceleration ``` -### Bad: Verbose PROJECT_CONTEXT.md Status +### Bad: Verbose PROJECT_CONTEXT.md ```markdown ### Recently Completed - -#### Audio Peak Measurement (February 7, 2026) -- **Real-Time Peak Fix**: Fixed critical bug where... [20 lines] -- **Peak Decay Optimization**: Changed decay from 0.95 to 0.7... [15 lines] -- **SilentBackend**: Created test backend with... [25 lines] -- [200+ more lines of detailed milestones] +#### Audio Peak Fix (Feb 7) +- Real-Time Peak: Fixed bug where... [20 lines] +- Decay Optimization: Changed 0.95 to 0.7... [15 lines] +[200+ lines of milestones] ``` --- -## Questions? - -If unsure whether to archive something, ask: +## Common Mistakes -1. **Is this about work completed >30 days ago?** → Archive -2. **Is this a detailed implementation plan?** → Archive (keep 2-line summary) -3. **Is this needed for next week's work?** → Keep -4. **Is this >50 lines of historical detail?** → Archive -5. **Would a new contributor need this immediately?** → Keep if yes +### ❌ Don't: +- Archive active tasks or priorities +- Remove essential architectural context +- Break cross-references without updating +- Delete content (archive instead) +- Keep "Recently Completed" >3 months -When in doubt, **archive with a pointer**. Better to have clean context and occasionally load an archive than to bloat every session. +### ✅ Do: +- Preserve key insights in summaries +- Update file references after moves +- Keep audit trail (note where moved) +- Regular small cleanups (not big purges) --- -## Maintenance Checklist +## Quick Decision Test -Copy this for monthly reviews: - -```markdown -## Context Maintenance - [Month YYYY] +If unsure whether to archive: -### File Sizes (before) -- [ ] PROJECT_CONTEXT.md: ___ lines -- [ ] TODO.md: ___ lines -- [ ] Tier 1-2 total: ___ words +1. Completed >30 days ago? → Archive +2. Detailed plan (>50 lines)? → Archive (keep 2-line summary) +3. Needed next week? → Keep +4. New contributor needs immediately? → Keep -### Actions Taken -- [ ] Archived tasks >30 days old to COMPLETED.md -- [ ] Moved temp analysis docs to doc/ -- [ ] Updated PROJECT_CONTEXT.md current status -- [ ] Simplified TODO.md task descriptions -- [ ] Verified CLAUDE.md/GEMINI.md config -- [ ] Updated cross-references - -### File Sizes (after) -- [ ] PROJECT_CONTEXT.md: ___ lines (target: <300) -- [ ] TODO.md: ___ lines (target: <500) -- [ ] Tier 1-2 total: ___ words (target: <10K) - -### Notes -[Any issues encountered, files moved, etc.] -``` +When in doubt, **archive with pointer**. Better clean context than bloat. --- -*Last updated: February 7, 2026* +*Last updated: February 9, 2026* diff --git a/doc/CONTRIBUTING.md b/doc/CONTRIBUTING.md index d7ea448..3a09dbc 100644 --- a/doc/CONTRIBUTING.md +++ b/doc/CONTRIBUTING.md @@ -1,451 +1,173 @@ # Contributing Guidelines -This document outlines the conventions to follow when contributing to this project. - ## Commit Policy -### Verify Build and Tests Before Committing - -Before preparing or proposing a commit, you **must** perform the following verifications to prevent regressions: +### Verify Before Committing -**Automated Verification (Recommended):** +**Automated (Recommended):** ```bash ./scripts/check_all.sh ``` -This script automatically: -- Builds with tests and tools enabled -- Runs the full test suite (26 tests) -- Verifies all tools compile (spectool, specview, specplay) -- Cross-compiles for Windows (if mingw-w64 is installed) - -**Manual Verification:** - -1. **MacOS / Linux (Native)**: - * Build the project with tests and tools enabled: - ```bash - cmake -S . -B build -DDEMO_BUILD_TESTS=ON -DDEMO_BUILD_TOOLS=ON - cmake --build build -j8 - ``` - * Run the entire test suite: - ```bash - cd build && ctest --output-on-failure - ``` - * Verify tools compile: - ```bash - cmake --build build --target spectool specview specplay - ``` - * Ensure all tests pass and all tools build successfully. - -2. **Windows (Cross-Compilation)**: - * If `mingw-w64` is installed on your system, you **must** also verify the Windows build. - * Run `./scripts/build_win.sh`. - * Ensure the build succeeds and produces the `demo64k_packed.exe` binary. - * Check the size report to ensure no unexpected bloat. +Runs tests, builds tools, cross-compiles Windows. -Refer to the "Testing" and "Windows Cross-Compilation" sections in `HOWTO.md` for detailed instructions. - -### Verify Debug Logging Compiles Before Committing +**Manual:** +```bash +# 1. Tests +cmake -S . -B build -DDEMO_BUILD_TESTS=ON -DDEMO_BUILD_TOOLS=ON +cmake --build build -j4 +cd build && ctest --output-on-failure -Before any significant commit (especially those modifying audio, rendering, or asset systems), **verify that the debug logging code compiles** without errors. This ensures that the diagnostic code remains maintainable even when not actively used. +# 2. Windows (if mingw-w64 installed) +./scripts/build_win.sh +``` -To enable all debug logging and verify compilation: +### Debug Logging Verification ```bash cmake -S . -B build_debug_check -DDEMO_ENABLE_DEBUG_LOGS=ON cmake --build build_debug_check -j4 ``` +Must compile without errors. -The build **must** succeed without errors or warnings. Debug logging is stripped from release builds but preserved for development and troubleshooting. - -#### Debug Logging Categories +**Debug macros** (`src/util/debug.h`): +- `DEBUG_LOG_AUDIO`, `DEBUG_LOG_RING_BUFFER`, `DEBUG_LOG_TRACKER` +- `DEBUG_LOG_SYNTH`, `DEBUG_LOG_3D`, `DEBUG_LOG_ASSETS`, `DEBUG_LOG_GPU` -The project uses conditional debug logging macros defined in `src/util/debug.h`: -- `DEBUG_LOG_AUDIO`: Audio backend timing, callbacks, device configuration -- `DEBUG_LOG_RING_BUFFER`: Ring buffer state, underruns, bounds checks -- `DEBUG_LOG_TRACKER`: Music pattern triggers, sample caching -- `DEBUG_LOG_SYNTH`: Voice management, spectrogram registration -- `DEBUG_LOG_3D`: 3D rendering, camera, scene queries -- `DEBUG_LOG_ASSETS`: Asset loading, procedural generation -- `DEBUG_LOG_GPU`: WebGPU commands, pipeline state - -Use these macros instead of direct `fprintf(stderr, ...)` calls in new diagnostic code. - -**Example:** +Example: ```cpp -#include "util/debug.h" - #if defined(DEBUG_LOG_AUDIO) - static int callback_count = 0; - DEBUG_AUDIO("[CALLBACK #%d] Processing %d frames\n", ++callback_count, frame_count); -#endif /* defined(DEBUG_LOG_AUDIO) */ + DEBUG_AUDIO("[CALLBACK #%d] frames=%d\n", ++count, frames); +#endif ``` -### Format Code Before Committing - -All code **must** be formatted using `clang-format` before committing. This ensures a consistent coding style across the entire codebase. - -**Warning**: Never apply formatting or make any manual edits to files in the `third_party/` directory. These are external libraries and must remain unmodified. - -To format your code, run the following command from the project root: +### Code Formatting ```bash clang-format -i $(git ls-files | grep -E '\.(h|cc)$' | grep -vE '^(assets|archive|third_party)/') ``` +Never format `third_party/`. -Refer to the `.clang-format` file in the project root for the specific style rules. - -### Ensure Newline at End of File - -All source files (`.h`, `.cc`, `.cpp`, etc.) must end with a newline character. This prevents "No newline at end of file" errors from linters and ensures consistent file handling. - -### Source File Headers - -Every source file (`.h`, `.cc`) must begin with a concise 3-line comment header describing its purpose. - -Example: -```cpp -// This file is part of the 64k demo project. -// It implements the core audio synthesis engine. -// This is not a user-facing header, but an internal one. -``` - -### Function and method comments - -Functions and methods, especially if they are internal non user-facing, -should at least have a 1-line comment describing what they do or their -how/when they should be called. Except if they are just 1-line function -or very very short, obvious ones. - -### '#endif' directive - -The closing #endif directive must recall the corresponding opening #ifdef -clause they are closing - -Example: -```cpp -#ifdef MY_TAG -...some code -#endif /* MY TAG */ -``` - -We must also prefer '#if defined(MY_QUITE_LONG_TAG)' over '#ifdef MY_QUITE_LONG_TAG' -especially if there's a risk of having later something like: -```cpp -#if defined(MY_TAG_1) && !defined(MY_TAG_2) -``` +### File Requirements +- Newline at end of file +- 3-line header comment +- Max 500 lines (split if larger) -### use and abuse 'const' directives +## Coding Style -Especially for local variable, use 'const' qualification as much as -possible. +### Core Rules +- `const` everywhere possible +- `const T* name` not `const T *name` +- Pre-increment `++x` not `x++` +- Spaces around operators: `x = (a + b) * c;` +- No trailing whitespace +- No `auto` (except complex iterators) +- No C++ casts (`static_cast`, `reinterpret_cast`) -As an example, don't use: +### Preprocessor ```cpp -StructA variable_name = StructA(...); +#if defined(MY_TAG) + ... +#endif /* defined(MY_TAG) */ ``` -but prefer instead: +### Struct Initialization ```cpp -const StructA variable_name = StructA(...); -``` - -if variable_name is not mutated afterward. - -Also: pass parameter as "const ref" as much as possible -(```const Struct& param``` instead of pointers or non-const refs) - -In summary: use 'const variable = ...;` as much as possible. - -### put spaces around code and operators (cosmetics) - -Don't compact the code to much horizontally, and prefer adding extra -spaces around code and operators. - -Example: -```cpp -const bool v = my_variable && (my_function() / 3. > (1. / x)); -const y = function_call(3, x, 2.); -for (int x = 0; x < 24; ++x) { ... } -``` - -instead of -```cpp -const bool v=my_variable&&my_function()/3.>(1./x); -const y = function_call(3,x,2); -for(int x=0;x<24;++x){ ... } -``` - -### prefer prefixed incrementation over suffixed - -Use pre-incrementation: -```cpp -++x -``` - -instead of post-incrementation: - -```cpp -x++ -``` - -### use extra () for boolean operations - -Even if they are not strictly needed due to operator precedence rules, -prefer adding extra ()'s around tests for clarity, with parcimony. - -### c++ cast - -don't use reinterpret_cast<>, static_cast<> or const_cast<>. - -### pointer declaration - -prefer ```const T* name``` to ```const T *name```. - -### 'auto' type - -Don't use 'auto' as type, unless for complex iterators or very complex -types. - -### don't use trailing whitespaces - -don't. - -### no missing \newline at the end of file - -make sure each file has a final \n newline. - -### c++ keyword indentation: - -The keyword 'public', 'protected', 'private' should be intended 1 character -less than the methods. +// Good +const WGPUDescriptor desc = { + .format = g_format, + .dimension = WGPUTextureViewDimension_2D, +}; -Example: -```cpp - private: - int field_; +// Bad +WGPUDescriptor desc = {}; +desc.format = g_format; +desc.dimension = WGPUTextureViewDimension_2D; ``` -instead of: +### Class Keywords ```cpp -private: + private: // 1 space indent int field_; ``` -### try to use per-field initialized const struct - -Use the `.field = ...,` initialization pattern instead for `var.field = -...;`, especially if it means you can have the variable be declared 'const' -that way. - -Example, instead of: -```cpp -WGPUTextureViewDescriptor view_desc = {}; -view_desc.format = g_format; -view_desc.dimension = WGPUTextureViewDimension_2D; -view_desc.mipLevelCount = 1; -view_desc.arrayLayerCount = 1; -``` - -use: -```cpp -const WGPUTextureViewDescriptor view_desc = { - .format = g_format, - .dimension = WGPUTextureViewDimension_2D, - .mipLevelCount = 1, - .arrayLayerCount = 1, -}; -``` - -Make sure the `.field = ...,` initialization pattern is compatible with the compiler / c++ version used. - -### vertical space - -keep the code compact vertically. That includes shader code, too. -Use only one statement per line. - -### File size limit - -Any file larger than 500 lines should ideally be split into sub-functionalities in separate files. (Exceptions: platform code or generated files). - -### finally - -Make sure everything is reflected in clang-format. +### Comments +- 1-line comment for non-obvious functions +- 3-line header for all source files ## Development Protocols -### Adding a New Visual Effect - -1. **Implement**: Create or update a class in `src/gpu/demo_effects.cc` (and declare in `demo_effects.h`) that inherits from `Effect`. - - Implement `init()` for one-time resource setup (e.g., using the asset system). - - Implement `compute()` if you need GPU-side physics or state updates. - - Implement `render()` to record WebGPU draw commands. -2. **Register**: Add an `EFFECT` entry to `assets/demo.seq` specifying the class name, start/end times, and any constructor arguments. -3. **Update Tests** (REQUIRED): Add your effect to `src/tests/test_demo_effects.cc`: - - Add effect to the appropriate test list (`test_post_process_effects()` or `test_scene_effects()`) - - Increment `EXPECTED_POST_PROCESS_COUNT` or `EXPECTED_SCENE_COUNT` at the top of the file - - If your effect requires `Renderer3D` with full shader setup, add it to the `requires_3d` check - - The test will fail with a clear error message if you forget this step -4. **Verify**: Build with `DEMO_ALL_OPTIONS=ON` and run tests: - ```bash - cmake -S . -B build -DDEMO_BUILD_TESTS=ON - cmake --build build --target test_demo_effects - cd build && ./test_demo_effects - ``` - Then test your effect at its specific timestamp with `--seek`. - -### Audio Subsystem Initialization - -The audio subsystem uses `AudioEngine` to manage initialization order and lifecycle. - -**In production code (`main.cc` or similar):** +### Adding Visual Effect +1. Implement `Effect` subclass in `src/gpu/demo_effects.cc` +2. Add to `assets/demo.seq` +3. **Update `test_demo_effects.cc`**: + - Add to test list + - Increment `EXPECTED_*_COUNT` +4. Verify: +```bash +cmake -S . -B build -DDEMO_BUILD_TESTS=ON +cmake --build build -j4 --target test_demo_effects +cd build && ./test_demo_effects +``` +### Audio Initialization +**Production:** ```cpp -#include "audio/audio_engine.h" - -// 1. Initialize audio backend audio_init(); - -// 2. Initialize audio engine (manages synth + tracker) static AudioEngine g_audio_engine; g_audio_engine.init(); - -// 3. In main loop g_audio_engine.update(music_time); ``` -**In tests:** - +**Tests:** ```cpp -#include "audio/audio_engine.h" - -void test_audio_feature() { - AudioEngine engine; - engine.init(); - - // Test logic here - engine.update(1.0f); - - engine.shutdown(); // Always cleanup at end -} +AudioEngine engine; +engine.init(); +engine.update(1.0f); +engine.shutdown(); ``` -**Low-level usage (when AudioEngine is not needed):** - -For tests that only need synth or tracker (not both), you can still call `synth_init()` or `tracker_init()` directly. However, if you need both, always use `AudioEngine` to ensure correct initialization order. - -**Direct synth API calls are valid:** - -- `synth_register_spectrogram()` - Register spectrograms -- `synth_trigger_voice()` - Trigger audio playback -- `synth_get_output_peak()` - Get audio levels for visualization -- `synth_render()` - Low-level rendering - -These are performance-critical APIs and should be called directly, not wrapped by AudioEngine. +Direct synth APIs (`synth_register_spectrogram`, `synth_trigger_voice`, etc.) are valid for performance-critical code. ### Fatal Error Checking - -The project uses fatal error checking macros that can be stripped for final builds. These are defined in `src/util/fatal_error.h`. - -**When to use fatal error checks:** - -Use fatal error checking for conditions that indicate programming errors or corrupt state that cannot be recovered from: -- Bounds checking (array/buffer overflows) -- Null pointer checks -- Invariant violations -- Invalid state transitions - -**DO NOT use for:** -- Expected runtime errors (file not found, network errors) -- Recoverable conditions -- Input validation (use return codes instead) - -**Macro Reference:** +Use `src/util/fatal_error.h` for programming errors only: ```cpp -#include "util/fatal_error.h" +// Most common (90%) +FATAL_CHECK(pos < capacity, "overflow: %d >= %d\n", pos, capacity); -// Conditional check (most common - 90% of cases) -FATAL_CHECK(write_pos >= capacity, - "write_pos out of bounds: %d >= %d\n", - write_pos, capacity); - -// Unconditional error (should-never-happen) +// Unconditional FATAL_ERROR("Invalid state: %d\n", state); -// Unreachable code marker (switch defaults) +// Unreachable switch (type) { - case TYPE_A: return handle_a(); - case TYPE_B: return handle_b(); + case A: return handle_a(); default: FATAL_UNREACHABLE(); } -// Assertion-style (documenting invariants) -FATAL_ASSERT(buffer != nullptr); -FATAL_ASSERT(count > 0 && count < MAX_COUNT); +// Assertion +FATAL_ASSERT(ptr != nullptr); -// Complex validation blocks +// Complex validation FATAL_CODE_BEGIN - static int callback_depth = 0; - ++callback_depth; - if (callback_depth > 1) { - FATAL_ERROR("Callback re-entered! depth=%d", callback_depth); - } + if (depth > MAX) FATAL_ERROR("depth=%d\n", depth); FATAL_CODE_END ``` -**Build modes:** -- **Debug/STRIP_ALL**: All checks enabled, full error messages with file:line info -- **FINAL_STRIP**: All checks compiled out (zero runtime cost, ~500-600 bytes saved) - -**Testing requirement:** +**DO NOT** use for expected errors (file not found, network errors). -Before committing code with fatal error checks, verify both modes compile: +**Build modes:** +- Debug/STRIP_ALL: All checks enabled +- FINAL_STRIP: All checks stripped (~500-600 bytes saved) +**Test both:** ```bash -# Normal build (checks enabled) -cmake -S . -B build -cmake --build build && cd build && ctest - -# FINAL_STRIP build (checks stripped) -cmake -S . -B build_final -DDEMO_FINAL_STRIP=ON -cmake --build build_final +cmake -S . -B build && cmake --build build -j4 && cd build && ctest +./scripts/build_final.sh ``` -Or use the convenience script: +### Script Maintenance +After hierarchy changes (moving files, renaming), verify: ```bash -./scripts/build_final.sh +./scripts/check_all.sh +./scripts/gen_coverage_report.sh ``` -**Important:** Never use `FATAL_CHECK` for conditions that can legitimately occur during normal operation. These are for programming errors only. - -### Script Maintenance After Hierarchy Changes - -After any major source hierarchy change (moving, renaming, or reorganizing files), you **must** review and update all scripts in the `scripts/` directory to ensure they remain functional. - -**When to perform this review:** -- Moving source files to new directories (e.g., `src/platform.cc` → `src/platform/platform.cc`) -- Renaming source files or directories -- Reorganizing the build system (CMake changes, new subdirectories) -- Changing asset locations or formats - -**Scripts that commonly require updates:** -- `scripts/check_all.sh` - Build verification and testing -- `scripts/gen_coverage_report.sh` - Code coverage analysis -- `scripts/build_win.sh` - Windows cross-compilation -- `scripts/gen_assets.sh` - Asset generation pipeline -- Any scripts with hardcoded paths or assumptions about file locations - -**Verification steps:** -1. Run `./scripts/check_all.sh` to verify builds and tests still work -2. Run `./scripts/gen_coverage_report.sh` to ensure coverage tracking handles new paths -3. Test platform-specific scripts if applicable (`build_win.sh`, `run_win.sh`) -4. Check for error messages about missing files or incorrect paths -5. Update documentation if script behavior has changed - -**Recent example:** -When `src/platform.cc` was moved to `src/platform/platform.cc`, the coverage script (`gen_coverage_report.sh`) initially failed with "unable to open /Users/skal/demo/src/platform.cc" due to stale coverage data. The fix required: -- Adding 'source' to `LCOV_OPTS` ignore list to handle missing source files -- Enabling automatic cleanup of the `build_coverage/` directory before each run -- Testing the script to verify it handles the new file structure - -**Automation:** -The `./scripts/check_all.sh` script is designed to catch most issues automatically. Always run it before committing hierarchy changes to ensure no regressions in build or test infrastructure. - +Update scripts with hardcoded paths. diff --git a/doc/FETCH_DEPS.md b/doc/FETCH_DEPS.md deleted file mode 100644 index ce62db2..0000000 --- a/doc/FETCH_DEPS.md +++ /dev/null @@ -1,68 +0,0 @@ -# Fetching Third-Party Dependencies - -This project intentionally does NOT vendor large third-party libraries. - -Currently required: - -## miniaudio - -Single-header audio library. - -Source: -https://github.com/mackron/miniaudio - -Required file: -- miniaudio.h - -Expected location: -third_party/miniaudio.h - -### Automatic fetch - -Use one of the provided scripts: -- scripts/project_init.sh -- scripts/project_init.bat - -### Manual fetch - -Download miniaudio.h from: -https://raw.githubusercontent.com/mackron/miniaudio/master/miniaudio.h - -and place it into: -third_party/miniaudio.h - -## wgpu-native - -WebGPU implementation via wgpu-native. - -### Installation - -**macOS:** -```bash -brew install wgpu-native -``` - -**Other platforms:** -Please install `wgpu-native` such that `libwgpu_native` (static or shared) is in your library path and headers are in your include path (under `webgpu/`). - -## glfw3webgpu - -Helper library for creating WebGPU surfaces from GLFW windows. - -### Automatic fetch - -Use one of the provided scripts: -- scripts/project_init.sh -- scripts/project_init.bat - -These scripts will download `glfw3webgpu.h` and `glfw3webgpu.c` into `third_party/glfw3webgpu`. - -## UPX - -Executable packer for binary compression (Linux/Windows only). -On macOS, the build script defaults to `strip` and `gzexe` due to UPX compatibility issues. - -### Installation - -**Linux/Windows:** -Download the appropriate release from https://github.com/upx/upx/releases and ensure the `upx` executable is in your PATH. diff --git a/doc/HOWTO.md b/doc/HOWTO.md index 55580ba..967b554 100644 --- a/doc/HOWTO.md +++ b/doc/HOWTO.md @@ -1,468 +1,218 @@ # How To -This document describes the common commands for building and testing the project. - -## Features - -* **Real-time Audio Synthesis**: The demo features a multi-voice synthesizer that generates audio in real-time from spectrograms. -* **Dynamic Sound Updates**: Spectrograms can be updated dynamically and safely during runtime for evolving soundscapes. -* **Unified Audio Engine**: The `AudioEngine` class manages audio subsystem initialization, ensuring correct setup order and eliminating initialization fragility. +Common commands for building and testing. ## Building ### Debug Build - -To run the demo in fullscreen mode, use the `--fullscreen` command-line option: - ```bash cmake -S . -B build -cmake --build build -./build/demo64k --fullscreen +cmake --build build -j4 +./build/demo64k ``` -To run in a specific resolution, use the `--resolution` option: -```bash -./build/demo64k --resolution 1024x768 -``` +Options: +- `--fullscreen`: Run in fullscreen +- `--resolution WxH`: Set window size (e.g., 1024x768) +- `--seek TIME`: Jump to timestamp (debug builds only) -Keyboard Controls: -* `Esc`: Exit the demo. -* `F`: Toggle fullscreen mode. +Keyboard: `Esc` (exit), `F` (toggle fullscreen) ### Size-Optimized Build - ```bash cmake -S . -B build -DDEMO_SIZE_OPT=ON -cmake --build build +cmake --build build -j4 ``` -### Final / Strip Build - -To produce the smallest possible binary (stripping all unnecessary code like command-line parsing and debug info), use the `DEMO_STRIP_ALL` option: - +### Strip Build (64k Target) ```bash cmake -S . -B build -DDEMO_STRIP_ALL=ON -cmake --build build +cmake --build build -j4 ``` -In this mode, the demo will always start in fullscreen. - -### Final-Final Build (Maximum Stripping) - -For the absolute smallest binary (removing ALL error checking), use the `DEMO_FINAL_STRIP` option: +Always starts in fullscreen. Full error checking enabled. +### Final Build (Maximum Stripping) ```bash -# Method 1: CMake target (from normal build directory) -cd build -make final - -# Method 2: Convenience script (creates build_final/ directory) ./scripts/build_final.sh - -# Method 3: Manual CMake configuration +# or cmake -S . -B build_final -DDEMO_FINAL_STRIP=ON -cmake --build build_final +cmake --build build_final -j4 ``` +⚠️ Removes ALL error checking. Use only for final release. -**⚠️ WARNING:** FINAL_STRIP removes ALL error checking (bounds checks, null checks, assertions). This saves ~500-600 bytes but means crashes will have no error messages. Use ONLY for final release builds, never for development or testing. - -**Build Mode Hierarchy:** -- **Debug**: Full error checking + debug features -- **STRIP_ALL**: Full error checking, no debug features (~64k target) -- **FINAL_STRIP**: No error checking, no debug features (absolute minimum size) - -### Developer Build (All Options) - -To enable all features at once (tests, tools, size optimizations, and stripping) for a comprehensive check: +**Build Hierarchy:** +- Debug: Full checks + debug features +- STRIP_ALL: Full checks, no debug (~64k target) +- FINAL_STRIP: No checks, no debug (absolute minimum) +### Developer Build ```bash cmake -S . -B build -DDEMO_ALL_OPTIONS=ON -cmake --build build +cmake --build build -j4 ``` +Enables tests, tools, size optimizations. -### Build System Notes +## Build System -**Incremental Builds**: The build system tracks all source files (.cc, .h) and asset files (.wgsl shaders, .spec audio, .obj meshes) as dependencies. Editing any file will trigger the necessary rebuilds automatically. +**Dependency Tracking**: CMake tracks 42 demo + 17 test assets. Editing shaders/audio auto-triggers rebuild. -**Asset Dependency Tracking**: CMake tracks 42 demo assets and 17 test assets individually. Changing a shader file (e.g., `assets/final/shaders/renderer_3d.wgsl`) automatically regenerates the asset bundle and recompiles dependent files. No manual workarounds needed. - -**Header Organization**: The `asset_manager` system is split into three headers for faster incremental builds: -- `asset_manager_dcl.h`: Forward declarations (use in headers) +**Header Organization**: +- `asset_manager_dcl.h`: Forward declarations - `asset_manager.h`: Core API (GetAsset/DropAsset) -- `asset_manager_utils.h`: Typed helpers (TextureAsset/MeshAsset) - -Include only what you need to minimize rebuild times. - -## git cloning - -if you have the public ssh key authorized on the VPS, you can use - -`git clone ssh://git@51.38.51.127/~/demo.git` +- `asset_manager_utils.h`: Typed helpers -to clone the repo and work on it. +## Git Clone +```bash +git clone ssh://git@51.38.51.127/~/demo.git +``` ## Audio System -### AudioEngine - -The audio subsystem uses `AudioEngine` to manage initialization and lifecycle. This ensures correct setup order and eliminates initialization fragility. - -**Usage in production code:** - +### AudioEngine API ```cpp #include "audio/audio_engine.h" -// Initialize audio backend (miniaudio) audio_init(); - -// Initialize audio engine (manages synth + tracker) static AudioEngine g_audio_engine; g_audio_engine.init(); -// In main loop: update with music time +// Main loop g_audio_engine.update(music_time); -// Cleanup g_audio_engine.shutdown(); audio_shutdown(); ``` -**What to use AudioEngine for:** -- Initialization: `engine.init()` replaces separate `synth_init()` + `tracker_init()` calls -- Updates: `engine.update(music_time)` replaces `tracker_update()` -- Cleanup: `engine.shutdown()` ensures proper teardown -- Seeking: `engine.seek(time)` for timeline navigation (debug builds only) +**Methods:** +- `init()`: Initialize synth + tracker +- `update(music_time)`: Update music state +- `shutdown()`: Cleanup +- `seek(time)`: Jump to timestamp (debug only) -**Direct synth API usage:** -For performance-critical or low-level operations, direct synth API calls are valid: -- `synth_register_spectrogram()` - Register audio samples -- `synth_trigger_voice()` - Trigger sound playback -- `synth_get_output_peak()` - Get audio peak for visualization -- `synth_render()` - Low-level audio rendering +**Direct Synth APIs** (performance-critical): +- `synth_register_spectrogram()`: Register samples +- `synth_trigger_voice()`: Trigger playback +- `synth_get_output_peak()`: Get audio level +- `synth_render()`: Low-level rendering **Testing:** -Tests should use `AudioEngine` for initialization: - ```cpp -#include "audio/audio_engine.h" - -void test_example() { - AudioEngine engine; - engine.init(); - - // Test code here - engine.update(1.0f); - - engine.shutdown(); -} +AudioEngine engine; +engine.init(); +engine.update(1.0f); +engine.shutdown(); ``` -For low-level synth-only tests, you can still call `synth_init()` directly. - ## Auxiliary Texture Masking -The project supports inter-effect texture sharing via the auxiliary texture registry. This allows effects to generate textures (masks, shadow maps, etc.) and make them available to other effects within the same frame. - -**Common use case:** Screen-space partitioning where different scenes render to complementary regions. - +Share textures between effects: ```cpp -class MaskGeneratorEffect : public Effect { - void init(MainSequence* demo) override { - demo_ = demo; - // Register a named auxiliary texture - demo->register_auxiliary_texture("my_mask", width_, height_); - } - - void compute(WGPUCommandEncoder encoder, ...) override { - // Generate mask to auxiliary texture - WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); - // ... render mask - } - - void render(WGPURenderPassEncoder pass, ...) override { - // Use mask in scene rendering - WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); - // ... render scene with mask - } -}; - -class MaskUserEffect : public Effect { - void render(WGPURenderPassEncoder pass, ...) override { - // Reuse mask from MaskGeneratorEffect - WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); - // ... render scene with inverted mask - } -}; -``` - -See `doc/MASKING_SYSTEM.md` for detailed architecture and examples. - -## Debugging - -### Seeking / Fast-Forward -In non-stripped builds, you can jump to any timestamp in the demo. This will simulate all audio logic and GPU physics (compute shaders) frame-by-frame from the start until the target time, then begin real-time playback. +// Generator effect +demo->register_auxiliary_texture("mask_name", width, height); +WGPUTextureView view = demo_->get_auxiliary_view("mask_name"); -```bash -./build/demo64k --seek 15.5 +// Consumer effect +WGPUTextureView view = demo_->get_auxiliary_view("mask_name"); ``` +See `doc/MASKING_SYSTEM.md` for details. -## Demo Choreography - -### Sequence Compiler -The demo timeline is managed via a textual description in `assets/demo.seq`. This file is transpiled into C++ code during the build process. +## Demo Timeline -**Format:** +Edit `assets/demo.seq`: ```text -# Starts a new sequence layer (global_start, priority) SEQUENCE 0.0 0 - # Adds an effect to the sequence (ClassName, local_start, local_end, priority, [constructor_args...]) EFFECT HeptagonEffect 0.0 60.0 0 ``` +Rebuild to update timeline. -To update the demo's timing or layering, simply edit `assets/demo.seq` and rebuild. - -## Tools - -If you are on macOS and want to test the Windows build: - -1. Build it: `./scripts/build_win.sh` -2. Run it: `./scripts/run_win.sh` - -Note: WebGPU support in Wine requires a Vulkan-capable backend (like MoltenVK on macOS). -Note2: make sure you run the script `./scripts/fetch_win_deps.sh` before, to install Win64 dependencies. - -### Testing - -**Commit Policy**: Always run tests before committing. Refer to `CONTRIBUTING.md` for details. - -To build and run the tests, you need to enable the `DEMO_BUILD_TESTS` option in CMake. Refer to the "Developer Build (All Options)" section for the easiest way to enable this. - -Available test suites: -* `HammingWindowTest`: Verifies the properties of the Hamming window function. -* `MathUtilsTest`: Verifies basic math utilities. -* `SynthEngineTest`: Verifies the core functionality of the audio synthesizer. -* `SpectoolEndToEndTest`: Performs an end-to-end test of the `spectool` by generating a WAV file, analyzing it, and verifying the output. -* `SequenceSystemTest`: Tests the logic of the sequence and effect system (activation, timing, priority), but **not** actual GPU rendering output, as this requires extensive mocking. +## Testing +**Run all tests:** ```bash cmake -S . -B build -DDEMO_BUILD_TESTS=ON -cmake --build build -cd build -ctest -cd .. +cmake --build build -j4 +cd build && ctest ``` -### Code Coverage (macOS only) - -To generate an HTML code coverage report for the project: - -1. **Install lcov:** - ```bash - brew install lcov - ``` - -2. **Run the coverage script:** - ```bash - ./scripts/gen_coverage_report.sh [optional_target_dir] - ``` - This script will configure the build with `-DDEMO_ENABLE_COVERAGE=ON`, run the tests, and generate a report in `build_coverage/coverage_report/`. It will then attempt to open the report in your browser. - - **Examples:** - * Full project report: - ```bash - ./scripts/gen_coverage_report.sh - ``` - * Report for `src/procedural` only: - ```bash - ./scripts/gen_coverage_report.sh src/procedural - ``` - -## Tools - -### Updating Submodules - -To ensure all `third_party/` submodules are updated to the latest "Tip of Tree" (ToT) from their respective remote repositories and to resolve any local diffs, follow these steps: - -1. **Enter the submodule directory** (e.g., `third_party/wgpu-native`): - ```bash - cd third_party/wgpu-native - ``` - -2. **Fetch the latest changes from the remote**: - ```bash - git fetch - ``` - -3. **Identify the default branch** (e.g., `main`, `master`, `trunk`). You can see remote branches with `git branch -r`. For `wgpu-native`, the main development branch is typically `trunk`. - -4. **Checkout the default branch and hard reset to its remote state** (this discards any local changes and ensures a clean ToT): - ```bash - git checkout trunk - git reset --hard origin/trunk - ``` - (Replace `trunk` with the correct branch name if different for other submodules.) - -5. **Return to the superproject's root directory**: - ```bash - cd ../.. - ``` - -6. **Update the superproject's record of the submodule**: This stages the change in the superproject's `.git` index, updating the commit hash for the submodule. - ```bash - git add third_party/wgpu-native - ``` - -7. **Commit the submodule update** in the superproject: - ```bash - git commit -m "chore: Update third_party/wgpu-native submodule" - ``` - (Adjust the commit message as appropriate for other submodules.) - -Repeat these steps for any other submodules in `third_party/` that need updating. - -### Spectrogram Tool (`spectool`) - -A command-line tool for analyzing WAV and MP3 files into spectrograms and playing them back. - -#### Building the Tool - -To build `spectool`, you need to enable the `DEMO_BUILD_TOOLS` option in CMake. +**Key tests:** +- `HammingWindowTest`: Window function properties +- `MathUtilsTest`: Math utilities +- `SynthEngineTest`: Audio synthesis +- `SequenceSystemTest`: Timeline logic +## Code Coverage (macOS) ```bash -cmake -S . -B build -DDEMO_BUILD_TOOLS=ON -cmake --build build +brew install lcov +./scripts/gen_coverage_report.sh [target_dir] ``` -The executable will be located at `build/spectool`. - -#### Usage -**Analyze an audio file:** -```bash -./build/spectool analyze path/to/input.wav path/to/output.spec -# or -./build/spectool analyze path/to/input.mp3 path/to/output.spec -``` +## Tools -**Play a spectrogram file:** +### Windows Cross-Compilation ```bash -./build/spectool play path/to/input.spec +./scripts/fetch_win_deps.sh +./scripts/build_win.sh +./scripts/run_win.sh ``` -### Spectrogram Viewer (`specview`) - -A command-line tool for visualizing spectrogram files in ASCII art. - -#### Building the Tool - -`specview` is built along with `spectool` when enabling `DEMO_BUILD_TOOLS`. - +### spectool (Audio Analysis) ```bash cmake -S . -B build -DDEMO_BUILD_TOOLS=ON -cmake --build build -``` -The executable will be located at `build/specview`. +cmake --build build -j4 -#### Usage +# Analyze +./build/spectool analyze input.wav output.spec -**View a spectrogram file:** -```bash -./build/specview path/to/input.spec +# Play +./build/spectool play input.spec ``` -### Audio Analysis & Playback Tool (`specplay`) - -A diagnostic tool for analyzing and playing .spec spectrogram files and .wav audio files. Useful for debugging audio issues, detecting clipping, and comparing spectrograms to source audio. - -#### Building the Tool - -`specplay` is built along with other tools when enabling `DEMO_BUILD_TOOLS`. - +### specview (Visualization) ```bash -cmake -S . -B build -DDEMO_BUILD_TOOLS=ON -cmake --build build +./build/specview input.spec ``` -The executable will be located at `build/specplay`. - -#### Usage -**Play and analyze a .spec file:** +### specplay (Diagnostic) ```bash -./build/specplay path/to/input.spec +./build/specplay input.spec +# or +./build/specplay input.wav ``` +Output: Peak, RMS, clipping detection. -**Play and analyze a .wav file:** +### Submodule Updates ```bash -./build/specplay path/to/input.wav +cd third_party/wgpu-native +git fetch +git checkout trunk +git reset --hard origin/trunk +cd ../.. +git add third_party/wgpu-native +git commit -m "chore: Update wgpu-native" ``` -#### Output Example -``` -Loading .spec: version=1, dct_size=512, frames=68 -PCM stats: Peak=0.403, RMS=0.058 -Playing 1.09 seconds... Press Ctrl+C to stop. -[CLIP at sample 1234: 1.523] # Only shown if clipping detected -Playback complete. -``` - -#### Use Cases -- **Detect clipping**: Peak > 1.0 indicates samples will clip -- **Compare loudness**: RMS shows average energy (loudness) -- **Verify spectrograms**: Ensure .spec matches source .wav -- **Debug distortion**: Quickly test individual samples - -See `tools/specplay_README.md` for detailed documentation and future enhancement ideas. - -### Asset Management System - -This system allows embedding binary assets directly into the demo executable. - -#### Defining Assets +## Asset Management -Assets are defined in `assets/final/demo_assets.txt` (for the demo) and `assets/final/test_assets_list.txt` (for tests). Each line specifies: -* `ASSET_NAME`: The identifier for the asset in C++ (e.g., `KICK_1`). -* `filename.ext`: The path to the asset file (relative to `assets/final/`). -* `NONE`: Compression type (currently only `NONE` is supported). -* `"Description"`: An optional description. - -Example `assets/final/demo_assets.txt` entry: +### Define Assets +Edit `assets/final/demo_assets.txt`: ``` -KICK_1, kick1.spec, NONE, "A drum kick sample" +KICK_1, kick1.spec, NONE, "Drum kick" ``` -#### Re-generating Assets - -To re-analyze source audio files (WAV/MP3) into spectrograms and update the embedded assets in the source tree, use the provided script: - +### Regenerate ```bash ./scripts/gen_assets.sh ``` +Converts WAV → .spec, packs into C++ arrays. -This script: -1. Ensures `spectool` and `asset_packer` are built. -2. Converts source audio files in `assets/wav/` to `.spec` files in `assets/final/`. -3. Runs `asset_packer` to update `src/assets.h` and `src/assets_data.cc`. - -#### Building with Assets - -The build system automatically runs `asset_packer` whenever the asset lists are modified. The generated files are located in the build directory (`build/src/`). - -To build the demo with the latest assets: - -```bash -cmake -S . -B build -cmake --build build -``` - -#### Accessing Assets in Code - -Include `assets.h` and use the `GetAsset` function: - +### Use Assets ```cpp #include "assets.h" -// ... -size_t asset_size; -const uint8_t* my_asset = GetAsset(AssetId::ASSET_SAMPLE_142, &asset_size); -// ... -// For lazy decompression (scaffolding only): -// DropAsset(AssetId::ASSET_SAMPLE_142, my_asset); +size_t size; +const uint8_t* data = GetAsset(AssetId::KICK_1, &size); +// Use data... +// DropAsset(AssetId::KICK_1, data); // For compressed assets only ``` + +Build system auto-runs `asset_packer` when asset lists change. diff --git a/doc/MASKING_SYSTEM.md b/doc/MASKING_SYSTEM.md index d468d48..86d6a74 100644 --- a/doc/MASKING_SYSTEM.md +++ b/doc/MASKING_SYSTEM.md @@ -1,240 +1,125 @@ # Auxiliary Texture Masking System -## Overview +## Purpose -The auxiliary texture masking system allows effects to share textures within a single frame render. Primary use case: **screen-space partitioning** where multiple effects render to complementary regions of the framebuffer. +Share textures between effects within a single frame for **screen-space partitioning** (split-screen, portals, picture-in-picture). -## Use Case +## Concept -**Problem:** Render two different 3D scenes to different regions of the screen (split-screen, portals, picture-in-picture). +- Effect1: Generate mask (1 = region A, 0 = region B) +- Effect1: Render scene A where mask = 1 +- Effect2: Reuse mask, render scene B where mask = 0 +- Both render to same framebuffer -**Solution:** -- Effect1 generates a mask (1 = Effect1's region, 0 = Effect2's region) -- Effect1 renders scene A where mask = 1 -- Effect2 reuses the mask and renders scene B where mask = 0 -- Both render to the same framebuffer in the same frame +## Design Choice: Mask Texture vs Stencil Buffer -## Architecture Choice: Mask Texture vs Stencil Buffer +**Chosen: Mask Texture** -### Option 1: Stencil Buffer (NOT CHOSEN) -**Pros:** Hardware-accelerated, fast early-rejection -**Cons:** 8-bit limitation, complex pipeline config, hard to debug - -### Option 2: Mask Texture (CHOSEN) -**Pros:** -- Flexible (soft edges, gradients, any format) -- Debuggable (visualize mask as texture) -- Reusable (multiple effects can read same mask) +Pros: +- Flexible (soft edges, gradients) +- Debuggable (visualize as texture) +- Reusable (multiple effects read same mask) - Simple pipeline setup -**Cons:** -- Requires auxiliary texture (~4 MB for 1280x720 RGBA8) +Cons: +- Memory cost (~4 MB for 1280x720 RGBA8) - Fragment shader discard (slightly slower than stencil) -**Verdict:** Mask texture flexibility and debuggability outweigh performance cost. +**Not Chosen: Stencil Buffer** -## Implementation +Pros: Hardware-accelerated, fast early-rejection +Cons: 8-bit limitation, complex pipeline, hard to debug -### MainSequence Auxiliary Texture Registry +## API Reference ```cpp class MainSequence { public: - // Register a named auxiliary texture (call once in effect init) + // Register once in effect init void register_auxiliary_texture(const char* name, int width, int height); - // Get texture view for reading/writing (call every frame) + // Get view every frame WGPUTextureView get_auxiliary_view(const char* name); - - private: - struct AuxiliaryTexture { - WGPUTexture texture; - WGPUTextureView view; - int width, height; - }; - std::map<std::string, AuxiliaryTexture> auxiliary_textures_; }; ``` -### Effect Lifecycle - -``` -Init Phase (once): - Effect1.init(): register_auxiliary_texture("mask_1", width, height) - Effect2.init(): (no registration - reuses Effect1's mask) - -Compute Phase (every frame): - Effect1.compute(): Generate mask to get_auxiliary_view("mask_1") - -Scene Pass (every frame, shared render pass): - Effect1.render(): Sample mask, discard if < 0.5, render scene A - Effect2.render(): Sample mask, discard if > 0.5, render scene B -``` - -### Render Flow Diagram - -``` -Frame N: -┌───────────────────────────────────────────────────────────┐ -│ Compute Phase: │ -│ Effect1.compute() │ -│ └─ Generate mask → auxiliary_textures_["mask_1"] │ -│ │ -├───────────────────────────────────────────────────────────┤ -│ Scene Pass (all effects share framebuffer A + depth): │ -│ Effect1.render() [priority 5] │ -│ ├─ Sample auxiliary_textures_["mask_1"] │ -│ ├─ Discard fragments where mask < 0.5 │ -│ └─ Render 3D scene A → framebuffer A │ -│ │ -│ Effect2.render() [priority 10] │ -│ ├─ Sample auxiliary_textures_["mask_1"] │ -│ ├─ Discard fragments where mask > 0.5 │ -│ └─ Render 3D scene B → framebuffer A │ -│ │ -│ Result: framebuffer A contains both scenes, partitioned │ -├───────────────────────────────────────────────────────────┤ -│ Post-Process Chain: │ -│ A ⟷ B ⟷ Screen │ -└───────────────────────────────────────────────────────────┘ -``` - -## Example: Circular Portal Effect - -### Effect1: Render Scene A (inside portal) +## Usage Pattern ```cpp -class PortalSceneEffect : public Effect { - public: - PortalSceneEffect(const GpuContext& ctx) : Effect(ctx) {} - - void init(MainSequence* demo) override { - demo_ = demo; - demo->register_auxiliary_texture("portal_mask", width_, height_); - // ... create pipelines - } - - void compute(WGPUCommandEncoder encoder, ...) override { - // Generate circular mask (portal region) - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... render fullscreen quad with circular mask shader - } - - void render(WGPURenderPassEncoder pass, ...) override { - // Render 3D scene, discard outside portal - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... bind mask, render scene with mask test - } -}; -``` - -### Effect2: Render Scene B (outside portal) +// Effect1 (mask generator) +void init(MainSequence* demo) { + demo->register_auxiliary_texture("portal_mask", width, height); +} -```cpp -class OutsideSceneEffect : public Effect { - public: - OutsideSceneEffect(const GpuContext& ctx) : Effect(ctx) {} +void compute(WGPUCommandEncoder encoder, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Generate mask to texture +} - void init(MainSequence* demo) override { - demo_ = demo; - // Don't register - reuse PortalSceneEffect's mask - } +void render(WGPURenderPassEncoder pass, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Sample mask, discard where mask < 0.5, render scene A +} - void render(WGPURenderPassEncoder pass, ...) override { - // Render 3D scene, discard inside portal - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... bind mask, render scene with inverted mask test - } -}; +// Effect2 (mask consumer) +void render(WGPURenderPassEncoder pass, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Sample mask, discard where mask > 0.5, render scene B +} ``` -### Mask Generation Shader +## Shader Example +**Mask Generation:** ```wgsl -// portal_mask.wgsl -@group(0) @binding(0) var<uniform> uniforms: MaskUniforms; - @fragment fn fs_main(@builtin(position) p: vec4<f32>) -> @location(0) vec4<f32> { let uv = p.xy / uniforms.resolution; - let center = vec2<f32>(0.5, 0.5); - let radius = 0.3; - - let dist = length(uv - center); - let mask = f32(dist < radius); // 1.0 inside circle, 0.0 outside - - return vec4<f32>(mask, mask, mask, 1.0); + let dist = length(uv - vec2(0.5)); + let mask = f32(dist < 0.3); // Circular portal + return vec4<f32>(mask); } ``` -### Scene Rendering with Mask - +**Masked Rendering:** ```wgsl -// scene_with_mask.wgsl -@group(0) @binding(2) var mask_sampler: sampler; @group(0) @binding(3) var mask_texture: texture_2d<f32>; @fragment fn fs_main(in: VertexOutput) -> @location(0) vec4<f32> { - // Sample mask - let screen_uv = in.position.xy / uniforms.resolution; - let mask_value = textureSample(mask_texture, mask_sampler, screen_uv).r; - - // Effect1: Discard outside portal (mask = 0) - if (mask_value < 0.5) { - discard; - } + let uv = in.position.xy / uniforms.resolution; + let mask = textureSample(mask_texture, mask_sampler, uv).r; - // Effect2: Invert test - discard inside portal (mask = 1) - // if (mask_value > 0.5) { discard; } + if (mask < 0.5) { discard; } // Effect1: discard outside + // if (mask > 0.5) { discard; } // Effect2: discard inside - // Render scene return compute_scene_color(in); } ``` ## Memory Impact -Each auxiliary texture: **width × height × 4 bytes** -- 1280×720 RGBA8: ~3.7 MB -- 1920×1080 RGBA8: ~8.3 MB +Per texture: **width × height × 4 bytes** +- 1280×720: ~3.7 MB +- 1920×1080: ~8.3 MB -For 2-3 masks: 10-25 MB total (acceptable overhead). +Typical usage: 2-3 masks = 10-25 MB (acceptable overhead) ## Use Cases -1. **Split-screen**: Vertical/horizontal partition -2. **Portals**: Circular/arbitrary shape windows to other scenes -3. **Picture-in-picture**: Small viewport in corner -4. **Masked transitions**: Wipe effects between scenes -5. **Shadow maps**: Pre-generated in compute, used in render -6. **Reflection probes**: Generated once, reused by multiple objects - -## Alternatives Considered - -### Effect-Owned Texture (No MainSequence changes) -```cpp -auto effect1 = std::make_shared<Effect1>(...); -auto effect2 = std::make_shared<Effect2>(...); -effect2->set_mask_source(effect1->get_mask_view()); -``` - -**Pros:** No MainSequence changes -**Cons:** Manual wiring, effects coupled, less flexible - -**Verdict:** Not chosen. Registry approach is cleaner and more maintainable. - -## Future Extensions - -- **Multi-channel masks**: RGBA mask for 4 independent regions -- **Mipmap support**: For hierarchical queries -- **Compression**: Quantize masks to R8 (1 byte per pixel) -- **Enum-based lookup**: Replace string keys for size optimization +1. Split-screen (vertical/horizontal) +2. Portals (arbitrary shape windows) +3. Picture-in-picture +4. Masked transitions (wipe effects) +5. Shadow maps (pre-generated in compute) +6. Reflection probes ## Size Impact -- MainSequence changes: ~100 lines (~500 bytes code) -- std::map usage: ~1 KB overhead (low priority for CRT removal) -- Runtime memory: 4-8 MB per mask (acceptable) +- Code: ~100 lines (~500 bytes) +- Runtime memory: 4-8 MB per mask ---- +## Future Extensions -*Document created: February 8, 2026* +- Multi-channel masks (RGBA = 4 regions) +- Mipmap support +- R8 compression (1 byte/pixel) +- Enum-based lookup (replace string keys) diff --git a/doc/PROCEDURAL.md b/doc/PROCEDURAL.md deleted file mode 100644 index c6bf688..0000000 --- a/doc/PROCEDURAL.md +++ /dev/null @@ -1,24 +0,0 @@ -# Procedural textures - -## the idea -We need a way to produce textures procedurally. -These texture can be generated on the CPU or the -GPU with a shader and the proper rendering target, -and then sent back to main memory. - -## What textures? - -The procedure can be use to pre-calc env maps -and lighting maps, or textures (fractals, fonts, etc.) - -## how - -This could be integrated in the asset system as -a special "compression" case (instead of "NONE", you -have "PROC(the_function_name_to_call)" as compression -type). - -## code - -let's have a proper 'src/procedural' sub-directory -with all the code related to procedural textures. diff --git a/doc/SCENE_FORMAT.md b/doc/SCENE_FORMAT.md index 679ab5e..fcf5e16 100644 --- a/doc/SCENE_FORMAT.md +++ b/doc/SCENE_FORMAT.md @@ -1,59 +1,49 @@ # Scene Binary Format (SCN1) -This document describes the binary format used for 3D scenes exported from Blender. +Binary format for 3D scenes exported from Blender. -## Overview +## File Layout - **Extension:** `.bin` or `.scene` - **Endianness:** Little Endian -- **Layout:** Header followed by sequential blocks. +- **Structure:** Header + Object Blocks -## Header (16 bytes) +## Format Specification -| Offset | Type | Description | -|--------|----------|-------------| -| 0 | char[4] | Magic bytes "SCN1" | -| 4 | uint32_t | Number of objects | -| 8 | uint32_t | Number of cameras (reserved) | -| 12 | uint32_t | Number of lights (reserved) | - -## Object Block - -Repeated `num_objects` times. +```cpp +// Header (16 bytes) +struct Header { + char magic[4]; // "SCN1" + uint32_t num_objects; + uint32_t num_cameras; // Reserved + uint32_t num_lights; // Reserved +}; -| Offset | Type | Description | -|--------|----------|-------------| -| 0 | char[64] | Object Name (UTF-8, null-padded) | -| 64 | uint32_t | Object Type (Enum) | -| 68 | vec3 | Position (x, y, z) - 12 bytes | -| 80 | quat | Rotation (x, y, z, w) - 16 bytes | -| 96 | vec3 | Scale (x, y, z) - 12 bytes | -| 108 | vec4 | Color (r, g, b, a) - 16 bytes | -| 124 | uint32_t | Mesh Name Length (N) | -| 128 | char[N] | Mesh Asset Name (if N > 0) | -| 128+N | float | Mass | -| 132+N | float | Restitution | -| 136+N | uint32_t | Is Static (0=Dynamic, 1=Static) | +// Object Block (repeated num_objects times) +struct ObjectBlock { + char name[64]; // UTF-8, null-padded + uint32_t type; // ObjectType enum + vec3 position; // 12 bytes + quat rotation; // 16 bytes (x, y, z, w) + vec3 scale; // 12 bytes + vec4 color; // 16 bytes (r, g, b, a) + uint32_t mesh_len; // Mesh name length (N) + char mesh_name[N]; // If N > 0 + float mass; + float restitution; + uint32_t is_static; // 0=Dynamic, 1=Static +}; +``` ### Object Types ```cpp enum class ObjectType { - CUBE = 0, - SPHERE = 1, - PLANE = 2, - TORUS = 3, - BOX = 4, - SKYBOX = 5, - MESH = 6 + CUBE = 0, SPHERE = 1, PLANE = 2, TORUS = 3, + BOX = 4, SKYBOX = 5, MESH = 6 }; ``` -## Coordinate System - -- **Position**: Blender coordinates (Z-up) should be converted to engine coordinates (Y-up) by the exporter or loader. Currently raw export. -- **Rotation**: Blender quaternions are (w, x, y, z). Exporter writes (x, y, z, w). Engine uses (x, y, z, w). - ## Asset Resolution -Mesh assets are referenced by name string (e.g., "MESH_CUBE"). The loader uses `GetAssetIdByName` to resolve this to a runtime `AssetId`. +Mesh names (e.g., "MESH_CUBE") resolve to runtime `AssetId` via `GetAssetIdByName`. diff --git a/doc/SEQUENCE.md b/doc/SEQUENCE.md index 7aa951d..954c816 100644 --- a/doc/SEQUENCE.md +++ b/doc/SEQUENCE.md @@ -4,45 +4,13 @@ This document describes the `.seq` file format used to define demo timelines. ## Overview -Sequence files (`.seq`) define the timeline and layering of visual effects in the demo. They are compiled by `seq_compiler` into C++ code at build time. - -## File Location +Sequence files (`.seq`) define the timeline and layering of visual effects. They are compiled by `seq_compiler` into C++ code at build time. +**Locations:** - Demo sequence: `assets/demo.seq` - Compiler: `tools/seq_compiler.cc` - Generated output: `src/generated/timeline.cc` -## Command Line Usage - -### Basic Compilation -```bash -./build/seq_compiler assets/demo.seq src/generated/timeline.cc -``` - -### Validation Only -```bash -./build/seq_compiler assets/demo.seq -``` -Validates syntax without generating code. - -### Gantt Chart Visualization - -**ASCII Gantt Chart:** -```bash -./build/seq_compiler assets/demo.seq --gantt=timeline.txt -``` - -**HTML/SVG Gantt Chart (Recommended):** -```bash -./build/seq_compiler assets/demo.seq --gantt-html=timeline.html -``` -Open `timeline.html` in browser for interactive visualization with hover tooltips. - -**Combined Modes:** -```bash -./build/seq_compiler assets/demo.seq timeline.cc --gantt=t.txt --gantt-html=t.html -``` - --- ## Syntax Reference @@ -51,41 +19,32 @@ Open `timeline.html` in browser for interactive visualization with hover tooltip ``` # BPM 120 ``` -Specifies beats per minute for the demo. Used to convert beat notation to seconds. +Specifies beats per minute. Used to convert beat notation to seconds. ### END_DEMO Directive ``` END_DEMO <time> ``` -Specifies when the demo should automatically exit (optional). -- If not specified, demo runs indefinitely until user closes window -- Supports beat notation (e.g., `64b`) or seconds (e.g., `32.0`) +Optional auto-exit time. Supports beat notation (`64b`) or seconds (`32.0`). ### SEQUENCE Declaration ``` SEQUENCE <global_start> <priority> ["optional_name"] [optional_end] - EFFECT <EffectClassName> <local_start> <local_end> <priority> [constructor_args...] + EFFECT <+|=|-> <EffectClassName> <local_start> <local_end> [constructor_args...] ``` **Parameters:** -- `global_start`: When this sequence starts (beats or seconds) -- `priority`: Render order between sequences (higher = rendered later/on top) - - Use 0-9 for scene effects - - Use 10+ for post-processing -- `"optional_name"`: Optional human-readable name in quotes (e.g., `"Opening Scene"`) - - If specified, the name will be displayed in Gantt charts for better readability - - Helps identify sequences when visualizing complex timelines -- `[optional_end]`: Optional end time in brackets (e.g., `[30.0]`) - - If specified, all effects in the sequence will be forcefully ended at this time - - Time is relative to the sequence start - - If omitted, effects run until their individual end times +- `global_start`: Sequence start time (beats or seconds) +- `priority`: Render order (0-9 for scenes, 10+ for post-processing) +- `"optional_name"`: Optional display name for Gantt charts +- `[optional_end]`: Optional sequence end time (forces effect termination) **Examples:** ``` -SEQUENCE 0 0 # Basic sequence (no name, no explicit end) +SEQUENCE 0 0 # Basic sequence SEQUENCE 0 0 "Intro" # Named sequence -SEQUENCE 0 0 [5.0] # Sequence with explicit end time -SEQUENCE 0 0 "Action Scene" [10.0] # Named sequence with explicit end time +SEQUENCE 0 0 [5.0] # Explicit end time +SEQUENCE 0 0 "Action Scene" [10.0] # Both name and end time ``` ### EFFECT Declaration @@ -93,29 +52,20 @@ SEQUENCE 0 0 "Action Scene" [10.0] # Named sequence with explicit end time EFFECT <+|=|-> <EffectClassName> <local_start> <local_end> [constructor_args...] ``` -**Parameters:** -- Priority modifier (one of): - - `+`: Increment priority (or start at 0 if first effect) - - `=`: Keep same priority as previous effect - - `-`: Decrement priority (or start at -1 if first effect, for background layers) -- `EffectClassName`: C++ class name (must be declared in `demo_effects.h`) -- `local_start`: Start time relative to sequence start (beats or seconds) -- `local_end`: End time relative to sequence start (beats or seconds) -- `constructor_args`: Optional additional parameters (rarely used) +**Priority Modifiers:** +- `+`: Increment priority (or start at 0 if first) +- `=`: Keep same priority as previous +- `-`: Decrement priority (or start at -1 if first, for backgrounds) -**Priority System:** -- Effects are assigned priorities based on their order in the file -- First effect: `+` starts at 0, `-` starts at -1, `=` starts at 0 -- Subsequent effects: `+` increments, `=` keeps same, `-` decrements -- Priorities determine render order (lower = drawn first/background) -- Effects should be listed in desired priority order within each sequence +**Parameters:** +- `EffectClassName`: C++ class from `demo_effects.h` +- `local_start`, `local_end`: Time relative to sequence start +- `constructor_args`: Optional (rarely used, most effects use standard params only) --- ## Time Notation -The sequence file supports flexible time notation: - | Notation | Example | Description | |----------|---------|-------------| | Integer beats | `0`, `64`, `128` | No decimal point = beats | @@ -123,118 +73,33 @@ The sequence file supports flexible time notation: | Decimal seconds | `0.0`, `32.0`, `64.0` | Decimal point = seconds | | Explicit seconds | `32.0s`, `64.0s` | Suffix 's' = seconds | -**Examples at 120 BPM:** -- Beat 0 = 0.0 seconds -- Beat 64 = 32.0 seconds -- Beat 120 = 60.0 seconds +**At 120 BPM:** Beat 64 = 32.0 seconds, Beat 120 = 60.0 seconds --- -## Effect System - -### Constructor Parameters - -**Standard Parameters (ALL effects receive these automatically):** -- `WGPUDevice device`: WebGPU device handle -- `WGPUQueue queue`: Command queue for GPU operations -- `WGPUTextureFormat format`: Surface texture format +## Runtime Parameters -Most effects only use these standard parameters. If an effect needs custom initialization, parameters are specified after the priority field. - -**Example (hypothetical custom effect):** -``` -EFFECT CustomEffect 0 10 0 1.5 "param" -``` -Translates to: -```cpp -std::make_shared<CustomEffect>(device, queue, format, 1.5, "param") -``` - -### Runtime Parameters - -All effects receive these dynamic parameters every frame in their `render()` method: +All effects receive these parameters every frame in `render()`: | Parameter | Type | Description | |-----------|------|-------------| -| `time` | float | Global time in seconds (from demo start) | -| `beat` | float | Current beat fraction (0.0 to 1.0 within each beat) | -| `intensity` | float | Audio peak intensity (0.0 to 1.0, for beat detection) | -| `aspect_ratio` | float | Screen width/height ratio | - ---- - -## Available Effects - -### Scene Effects -Render 3D geometry to the framebuffer with depth testing. - -#### HeptagonEffect -Animated geometric heptagon shape. -- **Uses:** `time` (animation), `beat` (rotation), `aspect_ratio` (projection) - -#### ParticlesEffect -GPU-simulated particle system. -- **Uses:** `time` (physics), `intensity` (emission rate) - -#### Hybrid3DEffect -3D objects with camera movement. -- **Uses:** `time` (camera paths), `beat` (object animation), `aspect_ratio` - -#### FlashCubeEffect -Large background cube with Perlin noise texture. -- **Uses:** `time` (rotation), `intensity` (flash trigger on beat hits) -- **Note:** Typically used with priority -1 for background layer - -### Post-Process Effects -Full-screen shaders applied in priority order after scene rendering. - -#### GaussianBlurEffect -Gaussian blur with beat pulsation. -- **Uses:** `intensity` (blur size pulsates 0.5x-2.5x based on audio peak) - -#### SolarizeEffect -Color inversion with alternating red/blue tints. -- **Uses:** `time` (alternates every 2 seconds between color schemes) - -#### ChromaAberrationEffect -RGB channel separation effect. -- **Uses:** `time` (animation), `intensity` (separation amount) - -#### ThemeModulationEffect -Brightness cycling (bright/dark alternation). -- **Uses:** `time` (cycles every 8 seconds: 1.0 bright to 0.35 dark) - -#### FadeEffect -Fade to/from black. -- **Uses:** `time` (fades in first 2s, fades out after 36s - hardcoded timing) - -#### FlashEffect -White flash on strong beat hits. -- **Uses:** `intensity` (triggers flash when > 0.7, exponential decay) +| `time` | float | Global time in seconds | +| `beat` | float | Current beat fraction (0.0-1.0) | +| `intensity` | float | Audio peak (0.0-1.0, for beat sync) | +| `aspect_ratio` | float | Screen width/height | --- -## Tips & Best Practices - -### Layering -- Scene effects render to framebuffer with depth testing -- Post-processing effects are full-screen shaders applied in sequence priority order -- Use negative priority for background elements (skybox, far objects) -- Higher sequence priority = later in post-process chain (rendered on top) +## Priority System -### Priority Guidelines -- **Scene Effects:** priority 0-9 - - Background: -1 (e.g., skybox, distant objects) - - Midground: 0-5 (main geometry) - - Foreground: 6-9 (overlays, particles) -- **Post-Process:** priority 10+ - - Apply in order: blur → distortion → color grading +**Scene Effects (0-9):** +- `-1`: Background (skybox, far objects) +- `0-5`: Midground (main geometry) +- `6-9`: Foreground (overlays, particles) -### Timing -- Use beat notation for music-synchronized effects -- Use seconds for absolute timing -- Keep effect durations reasonable (avoid very short < 0.1s unless intentional) -- Use sequence `[end_time]` to forcefully terminate long-running effects +**Post-Process (10+):** +- Applied in order: blur → distortion → color grading +- Higher priority = rendered later (on top) --- @@ -243,16 +108,16 @@ White flash on strong beat hits. ### Basic Sequence ``` SEQUENCE 0 0 - EFFECT + FlashEffect 0.0 0.5 # Priority 0 (first with +) - EFFECT + HeptagonEffect 0.2 10 # Priority 1 (increment) + EFFECT + FlashEffect 0.0 0.5 # Priority 0 + EFFECT + HeptagonEffect 0.2 10 # Priority 1 ``` -### Same Priority (Layered Effects) +### Same Priority Layering ``` SEQUENCE 0 0 EFFECT + Flash 0.0 0.5 # Priority 0 - EFFECT = Fade 0.1 0.3 # Priority 0 (same as Flash) - EFFECT + Other 0.2 3 # Priority 1 (increment) + EFFECT = Fade 0.1 0.3 # Priority 0 (same layer) + EFFECT + Other 0.2 3 # Priority 1 ``` ### Background Elements @@ -263,93 +128,70 @@ SEQUENCE 0 0 EFFECT + MainEffect 0 10 # Priority 0 (foreground) ``` -### Sequence with Explicit End Time +### Sequence with Explicit End ``` SEQUENCE 8b 0 [5.0] - EFFECT + Particles 0 120 # Would run 120s, but sequence ends at 5s + EFFECT + Particles 0 120 # Runs until 5s (sequence end) ``` ### Post-Processing Chain ``` SEQUENCE 0 10 - EFFECT + GaussianBlur 0 60 # Priority 0 (applied first) - EFFECT + ChromaAberration 0 60 # Priority 1 (applied second) - EFFECT + Solarize 0 60 # Priority 2 (applied last) + EFFECT + GaussianBlur 0 60 # Applied first + EFFECT + ChromaAberration 0 60 # Applied second + EFFECT + Solarize 0 60 # Applied last ``` -### Music-Synchronized Effects +### Music-Synchronized ``` # BPM 120 -SEQUENCE 0b 0 # Start at beat 0 (0.0s) - EFFECT + Flash 0b 1b # Flash from beat 0 to beat 1 (0-0.5s) - EFFECT + Heptagon 4b 8b # Main effect from beat 4 to 8 (2-4s) +SEQUENCE 0b 0 + EFFECT + Flash 0b 1b # Beat 0-1 (0-0.5s) + EFFECT + Heptagon 4b 8b # Beat 4-8 (2-4s) ``` --- -## Debugging & Visualization +## Visualization ### Gantt Charts -Gantt charts help visualize the timeline structure: - -**ASCII Chart (Terminal-Friendly):** +**ASCII Chart (Terminal):** +```bash +./build/seq_compiler assets/demo.seq --gantt=timeline.txt ``` -Time (s): 0 5 10 15 20 - |----|----|----|----|----| -SEQ@0s [pri=0] - ████████████████████████ (0-20s) - FlashEffect [pri=1] - ▓▓·················· (0-1s) - HeptagonEffect [pri=0] - ▓▓▓▓▓▓▓▓▓▓▓▓········ (0-10s) +**HTML/SVG Chart (Recommended):** +```bash +./build/seq_compiler assets/demo.seq --gantt-html=timeline.html ``` +Interactive visualization with hover tooltips, color coding, and zoom/pan. -**HTML/SVG Chart (Interactive):** -- Color-coded sequences and effects -- Hover tooltips with full details -- Invalid time ranges highlighted in red -- Professional dark theme -- Zoom/pan support -- Easy to share or present - -### Validation - -Run validation without code generation to catch syntax errors: +### Validation Only ```bash ./build/seq_compiler assets/demo.seq -# Output: "Validation successful: 14 sequences, explicit end time." ``` - -### Common Issues Caught by Validation -- Invalid time ranges (end < start) -- Missing effect classes -- Syntax errors -- Malformed time notation +Validates syntax without generating code. --- -## Integration with Build System +## Integration -The sequence compiler is automatically invoked during build via CMake: +CMake automatically invokes the compiler on `assets/demo.seq` changes: ```cmake add_custom_command( OUTPUT ${GENERATED_DIR}/timeline.cc COMMAND seq_compiler assets/demo.seq ${GENERATED_DIR}/timeline.cc DEPENDS assets/demo.seq - COMMENT "Compiling demo sequence..." ) ``` -Editing `assets/demo.seq` triggers automatic recompilation and rebuilds affected targets. - --- ## See Also -- `doc/HOWTO.md` - Building and running the demo -- `doc/CONTRIBUTING.md` - Coding guidelines - `assets/demo.seq` - Current demo timeline - `src/gpu/demo_effects.h` - Available effect classes +- `doc/EFFECTS_CATALOG.md` - Detailed effect reference (if needed) +- `doc/HOWTO.md` - Building and running diff --git a/doc/SPECTRAL_BRUSH_EDITOR.md b/doc/SPECTRAL_BRUSH_EDITOR.md index 7ea0270..a7d0e3a 100644 --- a/doc/SPECTRAL_BRUSH_EDITOR.md +++ b/doc/SPECTRAL_BRUSH_EDITOR.md @@ -2,43 +2,26 @@ ## Concept -The **Spectral Brush Editor** is a web-based tool for creating procedural audio by tracing spectrograms with parametric curves. It enables compact representation of audio samples for 64k demos. +Replace large `.spec` assets with procedural C++ code (50-100× compression). -### Goal - -Replace large `.spec` asset files with tiny procedural C++ code: -- **Before:** 5 KB binary `.spec` file -- **After:** ~100 bytes of C++ code calling `draw_bezier_curve()` - -### Workflow +**Before:** 5 KB binary `.spec` file +**After:** ~100 bytes C++ code calling `draw_bezier_curve()` +**Workflow:** ``` -.wav file → Load in editor → Trace with spectral brushes → Export params + C++ code - ↓ - (later) -procedural_params.txt → Load in editor → Adjust curves → Re-export +.wav → Load in editor → Trace with Bezier curves → Export procedural_params.txt + C++ code ``` --- ## Core Primitive: "Spectral Brush" -A spectral brush consists of two components: - ### 1. Central Curve (Bezier) -Traces a path through time-frequency space: -``` -{freq_bin, amplitude} = bezier(frame_number) -``` - -**Properties:** -- Control points: `[(frame, freq_hz, amplitude), ...]` -- Interpolation: Linear (between control points) -- Future: Cubic Bezier, Catmull-Rom splines +Traces time-frequency path: `{freq_bin, amplitude} = bezier(frame_number)` -**Example:** +**Example control points:** ```javascript -control_points: [ +[ {frame: 0, freq_hz: 200.0, amplitude: 0.9}, // Attack {frame: 20, freq_hz: 80.0, amplitude: 0.7}, // Sustain {frame: 100, freq_hz: 50.0, amplitude: 0.0} // Decay @@ -46,168 +29,59 @@ control_points: [ ``` ### 2. Vertical Profile -At each frame, applies a shape **vertically** in frequency bins around the central curve. +Shapes "brush stroke" around curve at each frame. **Profile Types:** - -#### Gaussian (Smooth harmonic) -``` -amplitude(dist) = exp(-(dist² / σ²)) -``` -- **σ (sigma)**: Width in frequency bins -- Use case: Musical tones, bass notes, melodic lines - -#### Decaying Sinusoid (Textured/resonant) -``` -amplitude(dist) = exp(-decay * dist) * cos(ω * dist) -``` -- **decay**: Falloff rate -- **ω (omega)**: Oscillation frequency -- Use case: Metallic sounds, bells, resonant tones - -#### Noise (Textured/gritty) -``` -amplitude(dist) = random(seed, dist) * noise_amplitude -``` -- **seed**: Deterministic RNG seed -- Use case: Hi-hats, cymbals, textured sounds - -#### Composite (Combinable) -```javascript -{ - type: "composite", - operation: "add" | "subtract" | "multiply", - profiles: [ - {type: "gaussian", sigma: 30.0}, - {type: "noise", amplitude: 0.1, seed: 42} - ] -} -``` - ---- - -## Visual Model - -``` -Frequency (bins) - ^ - | -512 | * ← Gaussian profile (frame 80) - | * * * -256 | **** * * * ← Gaussian profile (frame 50) - | ** ** * * -128 | ** Curve * * ← Central Bezier curve - | ** * - 64 | ** * - | * * - 0 +--*--*--*--*--*--*--*--*---→ Time (frames) - 0 10 20 30 40 50 60 70 80 - - At each frame: - 1. Evaluate curve → get freq_bin_0 and amplitude - 2. Draw profile vertically at that frame -``` +- **Gaussian**: `exp(-(dist² / σ²))` - Musical tones, bass +- **Decaying Sinusoid**: `exp(-decay * dist) * cos(ω * dist)` - Metallic sounds +- **Noise**: `random(seed, dist) * amplitude` - Hi-hats, cymbals +- **Composite**: Combine multiple profiles (add/subtract/multiply) --- ## File Formats -### A. `procedural_params.txt` (Human-readable, re-editable) - +### A. `procedural_params.txt` (Human-readable) ```text -# Kick drum spectral brush definition METADATA dct_size=512 num_frames=100 sample_rate=32000 CURVE bezier - CONTROL_POINT 0 200.0 0.9 # frame, freq_hz, amplitude + CONTROL_POINT 0 200.0 0.9 CONTROL_POINT 20 80.0 0.7 - CONTROL_POINT 50 60.0 0.3 CONTROL_POINT 100 50.0 0.0 PROFILE gaussian sigma=30.0 - PROFILE_ADD noise amplitude=0.1 seed=42 -END_CURVE - -CURVE bezier - CONTROL_POINT 0 500.0 0.5 - CONTROL_POINT 30 300.0 0.2 - PROFILE decaying_sinusoid decay=0.15 frequency=0.8 END_CURVE ``` -**Purpose:** -- Load back into editor for re-editing -- Human-readable, version-control friendly -- Can be hand-tweaked in text editor - ### B. C++ Code (Ready to compile) - ```cpp -// Generated from procedural_params.txt -// File: src/audio/gen_kick_procedural.cc - #include "audio/spectral_brush.h" void gen_kick_procedural(float* spec, int dct_size, int num_frames) { - // Curve 0: Low-frequency punch with noise texture - { - const float frames[] = {0.0f, 20.0f, 50.0f, 100.0f}; - const float freqs[] = {200.0f, 80.0f, 60.0f, 50.0f}; - const float amps[] = {0.9f, 0.7f, 0.3f, 0.0f}; - - draw_bezier_curve(spec, dct_size, num_frames, - frames, freqs, amps, 4, - PROFILE_GAUSSIAN, 30.0f); - - draw_bezier_curve_add(spec, dct_size, num_frames, - frames, freqs, amps, 4, - PROFILE_NOISE, 0.1f, 42.0f); - } - - // Curve 1: High-frequency attack - { - const float frames[] = {0.0f, 30.0f}; - const float freqs[] = {500.0f, 300.0f}; - const float amps[] = {0.5f, 0.2f}; + const float frames[] = {0.0f, 20.0f, 100.0f}; + const float freqs[] = {200.0f, 80.0f, 50.0f}; + const float amps[] = {0.9f, 0.7f, 0.0f}; - draw_bezier_curve(spec, dct_size, num_frames, - frames, freqs, amps, 2, - PROFILE_DECAYING_SINUSOID, 0.15f, 0.8f); - } + draw_bezier_curve(spec, dct_size, num_frames, + frames, freqs, amps, 3, + PROFILE_GAUSSIAN, 30.0f); } - -// Usage in demo_assets.txt: -// KICK_PROC, PROC(gen_kick_procedural), NONE, "Procedural kick drum" ``` -**Purpose:** -- Copy-paste into `src/audio/procedural_samples.cc` -- Compile directly into demo -- Zero runtime parsing overhead - --- ## C++ Runtime API -### New Files +### Files: `src/audio/spectral_brush.{h,cc}` -#### `src/audio/spectral_brush.h` +**Key functions:** ```cpp -#pragma once -#include <cstdint> - enum ProfileType { - PROFILE_GAUSSIAN = 0, - PROFILE_DECAYING_SINUSOID = 1, - PROFILE_NOISE = 2 + PROFILE_GAUSSIAN, + PROFILE_DECAYING_SINUSOID, + PROFILE_NOISE }; -// Evaluate linear Bezier interpolation at frame t -float evaluate_bezier_linear(const float* control_frames, - const float* control_values, - int n_points, - float frame); - -// Draw spectral brush: Bezier curve with vertical profile void draw_bezier_curve(float* spectrogram, int dct_size, int num_frames, const float* control_frames, const float* control_freqs_hz, @@ -217,281 +91,105 @@ void draw_bezier_curve(float* spectrogram, int dct_size, int num_frames, float profile_param1, float profile_param2 = 0.0f); -// Additive variant (for compositing profiles) -void draw_bezier_curve_add(float* spectrogram, int dct_size, int num_frames, - const float* control_frames, - const float* control_freqs_hz, - const float* control_amps, - int n_control_points, - ProfileType profile_type, - float profile_param1, - float profile_param2 = 0.0f); +float evaluate_bezier_linear(const float* control_frames, + const float* control_values, + int n_points, + float frame); -// Profile evaluation float evaluate_profile(ProfileType type, float distance, float param1, float param2); - -// Home-brew deterministic RNG (small, portable) -uint32_t spectral_brush_rand(uint32_t seed); ``` -**Future Extensions:** -- Cubic Bezier interpolation: `evaluate_bezier_cubic()` -- Generic loader: `gen_from_params(const PrimitiveData*)` - --- -## Editor Architecture +## Editor UI ### Technology Stack -- **HTML5 Canvas**: Spectrogram visualization -- **Web Audio API**: Playback (IDCT → audio) -- **Pure JavaScript**: No dependencies -- **Reuse from existing editor**: `dct.js` (IDCT implementation) - -### Editor UI Layout - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Spectral Brush Editor [Load .wav] │ -├─────────────────────────────────────────────────────────────┤ -│ │ -│ ┌───────────────────────────────────────────────────┐ │ -│ │ │ │ -│ │ CANVAS (Spectrogram Display) │ T │ -│ │ │ O │ -│ │ • Background: Reference .wav (gray, transparent) │ O │ -│ │ • Foreground: Procedural curves (colored) │ L │ -│ │ • Bezier control points (draggable circles) │ S │ -│ │ │ │ -│ │ │ [+]│ -│ │ │ [-]│ -│ │ │ [x]│ -│ └───────────────────────────────────────────────────┘ │ -│ │ -├─────────────────────────────────────────────────────────────┤ -│ Profile: [Gaussian ▾] Sigma: [████████░░] 30.0 │ -│ Curves: [Curve 0 ▾] Amplitude: [██████░░░░] 0.8 │ -├─────────────────────────────────────────────────────────────┤ -│ [1] Play Procedural [2] Play Original [Space] Pause │ -│ [Save Params] [Generate C++] [Undo] [Redo] │ -└─────────────────────────────────────────────────────────────┘ -``` +- HTML5 Canvas (visualization) +- Web Audio API (playback) +- Pure JavaScript (no dependencies) +- Reuse `dct.js` from existing editor -### Features (Phase 1: Minimal Working Version) - -#### Editing -- Click to place Bezier control points -- Drag to adjust control points (frame, frequency, amplitude) -- Delete control points (right-click or Delete key) -- Undo/Redo support (action history) - -#### Visualization -- Dual-layer canvas: - - **Background**: Reference spectrogram (semi-transparent gray) - - **Foreground**: Procedural spectrogram (colored) -- Log-scale frequency axis (musical perception) -- Control points: Draggable circles with labels - -#### Audio Playback -- **Key '1'**: Play procedural sound -- **Key '2'**: Play original .wav -- **Space**: Play/pause toggle - -#### File I/O -- Load .wav or .spec (reference sound) -- Save procedural_params.txt (re-editable) -- Generate C++ code (copy-paste ready) - -#### Undo/Redo -- Action history with snapshots -- Commands: Add control point, Move control point, Delete control point, Change profile -- **Ctrl+Z**: Undo -- **Ctrl+Shift+Z**: Redo +### Key Features +- Dual-layer canvas (reference + procedural spectrograms) +- Drag control points to adjust curves +- Real-time spectrogram rendering +- Audio playback (keys 1/2 for procedural/original) +- Undo/Redo (Ctrl+Z, Ctrl+Shift+Z) +- Load .wav/.spec, save params, generate C++ code ### Keyboard Shortcuts - | Key | Action | |-----|--------| -| **1** | Play procedural sound | -| **2** | Play original .wav | -| **Space** | Play/pause | -| **Delete** | Delete selected control point | -| **Esc** | Deselect all | -| **Ctrl+Z** | Undo | -| **Ctrl+Shift+Z** | Redo | -| **Ctrl+S** | Save procedural_params.txt | -| **Ctrl+Shift+S** | Generate C++ code | +| 1 | Play procedural | +| 2 | Play original | +| Space | Play/pause | +| Delete | Remove control point | +| Ctrl+Z | Undo | +| Ctrl+S | Save params | --- -## Implementation Plan +## Implementation Phases -### Phase 1: C++ Runtime (Foundation) -**Files:** `src/audio/spectral_brush.h`, `src/audio/spectral_brush.cc` +### Phase 1: C++ Runtime +**Files:** `src/audio/spectral_brush.{h,cc}`, `src/tests/test_spectral_brush.cc` **Tasks:** -- [ ] Define API (`ProfileType`, `draw_bezier_curve()`, etc.) -- [ ] Implement linear Bezier interpolation -- [ ] Implement Gaussian profile evaluation -- [ ] Implement home-brew RNG (for future noise support) -- [ ] Add unit tests (`src/tests/test_spectral_brush.cc`) - -**Deliverable:** Compiles, tests pass - ---- +- Define API (ProfileType, draw_bezier_curve, evaluate_profile) +- Implement linear Bezier interpolation +- Implement Gaussian profile +- Add unit tests +- **Deliverable:** Compiles, tests pass ### Phase 2: Editor Core -**Files:** `tools/spectral_editor/index.html`, `script.js`, `style.css`, `dct.js` (reuse) +**Files:** `tools/spectral_editor/{index.html, script.js, style.css, dct.js}` **Tasks:** -- [ ] HTML structure (canvas, controls, file input) -- [ ] Canvas rendering (dual-layer: reference + procedural) -- [ ] Bezier curve editor (place/drag/delete control points) -- [ ] Profile controls (Gaussian sigma slider) -- [ ] Real-time spectrogram rendering -- [ ] Audio playback (IDCT → Web Audio API) -- [ ] Undo/Redo system - -**Deliverable:** Interactive editor, can trace .wav files - ---- +- HTML structure (canvas, controls, file input) +- Bezier curve editor (place/drag/delete control points) +- Dual-layer canvas rendering +- Real-time spectrogram generation +- Audio playback (IDCT → Web Audio API) +- Undo/Redo system +- **Deliverable:** Interactive editor, can trace .wav files ### Phase 3: File I/O **Tasks:** -- [ ] Load .wav (decode, FFT/STFT → spectrogram) -- [ ] Load .spec (binary format parser) -- [ ] Save procedural_params.txt (text format writer) -- [ ] Generate C++ code (code generation template) -- [ ] Load procedural_params.txt (re-editing workflow) - -**Deliverable:** Full save/load cycle works - ---- - -### Phase 4: Polish & Documentation -**Tasks:** -- [ ] UI refinements (tooltips, visual feedback) -- [ ] Keyboard shortcut overlay (press '?' to show help) -- [ ] Error handling (invalid files, audio context failures) -- [ ] User guide (README.md in tools/spectral_editor/) -- [ ] Example files (kick.wav → kick_procedural.cc) - -**Deliverable:** Production-ready tool +- Load .wav (decode, STFT → spectrogram) +- Load .spec (binary parser) +- Save procedural_params.txt (text format) +- Generate C++ code (template) +- Load procedural_params.txt (re-editing) +- **Deliverable:** Full save/load cycle --- ## Design Decisions -### 1. Bezier Interpolation -- **Current:** Linear (simple, fast, small code) -- **Future:** Cubic Bezier, Catmull-Rom splines - -### 2. Parameter Limits -- **Editor UI:** Soft ranges (reasonable defaults, not enforced) -- **Examples:** - - Sigma: 1.0 - 100.0 (suggested range, user can type beyond) - - Amplitude: 0.0 - 1.0 (suggested, can exceed for overdrive effects) - -### 3. Random Number Generator -- **Implementation:** Home-brew deterministic RNG -- **Reason:** Independence, small code, repeatable results -- **Algorithm:** Simple LCG or xorshift - -### 4. Code Generation -- **Current:** Single function per sound (e.g., `gen_kick_procedural()`) -- **Future:** Generic `gen_from_params(const PrimitiveData*)` with data tables - -### 5. Initial UI Scope -- **Phase 1:** Bezier + Gaussian only -- **Rationale:** Validate workflow before adding complexity -- **Future:** Decaying sinusoid, noise, composite profiles +- **Bezier:** Linear interpolation (Phase 1), cubic later +- **Profiles:** Gaussian only (Phase 1), others later +- **Parameters:** Soft UI limits, no enforced bounds +- **RNG:** Home-brew deterministic (small, repeatable) +- **Code gen:** Single function per sound (generic loader later) --- -## Future Extensions - -### Bezier Enhancements -- [ ] Cubic Bezier interpolation (smoother curves) -- [ ] Catmull-Rom splines (automatic tangent control) -- [ ] Bezier curve tangent handles (manual control) - -### Additional Profiles -- [ ] Decaying sinusoid (metallic sounds) -- [ ] Noise (textured sounds) -- [ ] Composite profiles (add/subtract/multiply) -- [ ] User-defined profiles (custom formulas) +## Size Impact -### Multi-Dimensional Bezier -- [ ] `{freq, amplitude, oscillator_freq, decay} = bezier(frame)` -- [ ] Per-parameter control curves +**Example: Kick drum** -### Advanced Features -- [ ] Frequency snapping (snap to musical notes: C4, D4, E4, etc.) -- [ ] Amplitude envelope presets (ADSR) -- [ ] Profile library (save/load custom profiles) -- [ ] Batch processing (convert entire folder of .wav files) +**Before (Binary):** +- 512 bins × 100 frames × 4 bytes = 200 KB uncompressed +- ~5 KB compressed (zlib) -### Code Generation -- [ ] Generic `gen_from_params()` with data tables -- [ ] Optimization: Merge similar curves -- [ ] Size estimation (preview bytes saved vs. original .spec) +**After (Procedural):** +- 4 control points × 3 arrays × 4 floats = ~48 bytes data +- Function call overhead = ~20 bytes +- **Total: ~100 bytes** (50-100× reduction) ---- - -## Testing Strategy - -### C++ Runtime Tests -**File:** `src/tests/test_spectral_brush.cc` - -**Test cases:** -- Linear Bezier interpolation (verify values at control points) -- Gaussian profile evaluation (verify falloff curve) -- Full `draw_bezier_curve()` (verify spectrogram output) -- Edge cases (0 control points, 1 control point, out-of-range frames) - -### Editor Tests -**Manual testing workflow:** -1. Load example .wav (kick drum) -2. Place 3-4 control points to trace low-frequency punch -3. Adjust Gaussian sigma -4. Play procedural sound (should resemble original) -5. Save procedural_params.txt -6. Generate C++ code -7. Copy C++ code into demo, compile, verify runtime output matches editor +**Trade-off:** Runtime CPU cost, acceptable for 64k demo. --- -## Size Impact Estimate - -**Example: Kick drum sample** - -**Before (Binary .spec):** -- DCT size: 512 -- Num frames: 100 -- Size: 512 × 100 × 4 bytes = 200 KB (uncompressed) -- Compressed (zlib): ~5-10 KB - -**After (Procedural C++):** -```cpp -// 4 control points × 3 arrays × 4 values = ~48 bytes of data -const float frames[] = {0.0f, 20.0f, 50.0f, 100.0f}; -const float freqs[] = {200.0f, 80.0f, 60.0f, 50.0f}; -const float amps[] = {0.9f, 0.7f, 0.3f, 0.0f}; -draw_bezier_curve(...); // ~20 bytes function call - -// Total: ~100 bytes -``` - -**Compression ratio:** 50-100× reduction! - -**Trade-off:** Runtime CPU cost (generation vs. lookup), but acceptable for 64k demo. - ---- - -## References - -- **Bezier curves:** https://en.wikipedia.org/wiki/B%C3%A9zier_curve -- **DCT/IDCT:** Existing implementation in `src/audio/dct.cc` -- **Web Audio API:** https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API -- **Spectrogram visualization:** Existing editor in `tools/editor/` (to be replaced) +*See TODO.md for detailed implementation tasks.* diff --git a/doc/TRACKER.md b/doc/TRACKER.md index f3a34a3..6e71951 100644 --- a/doc/TRACKER.md +++ b/doc/TRACKER.md @@ -1,56 +1,17 @@ # Minimal Audio Tracker -in addition to being able to generate spectrograms (aka "Samples") on the -fly and play them at once, we need a way to assemble samples (assets or -generated) into modifiable patterns and loops. Like what the Trackers were -doing in the old time. - -## The idea - -A script can take a 'tracker music' text file that describes the sequence -of samples (a 'pattern') in a first part. In a second part, these sequence -a laid out with a timestamp (potentially overlapping) to generate the full -music score. - -The patterns' samples (spectrograms) are not yet generated, it's just the -'musical score' here. We still need to 'play' the score but with modifiers -applied. - -### Modifiers - -For diversity, these essential patterns can be modified on the fly before -being generated as spectrogram on the fly. -Modifiers could be: - * randomize (for drums, e.g.) - * add accents and stresses - * modulate volume - * add distortion or noise - * add 'grain' and static noise - * flanger, vocoding, robotic voice, etc. - * etc. - -These modifiers are applied to a predefined pattern just before it's -generated for playback (or assembling in the final track). - -### How would that work in practice? - -The musical score is a text file, that a tool will convert to run-time -code to be compiled in the final binary demo64k. -This generated code can be mixed with fixed code from the demo codebase -itself (explosion predefined at a given time ,etc.) -The baking is done at compile time, and the code will go in src/generated/ +System for assembling audio samples (assets or procedural) into patterns and loops. ## .track File Format -### Timing System +### Timing Convention -**Unit-less Timing Convention:** -- All time values are **unit-less** (not beats or seconds) -- Convention: **1 unit = 4 beats** -- Conversion to seconds: `seconds = units * (4 / BPM) * 60` +**Unit-less timing:** +- **1 unit = 4 beats** +- Conversion: `seconds = units * (4 / BPM) * 60` - At 120 BPM: 1 unit = 2 seconds -This makes patterns independent of BPM - changing BPM only affects playback speed, not pattern structure. +Patterns are BPM-independent. Changing BPM only affects playback speed. ### File Structure @@ -96,29 +57,20 @@ PATTERN melody LENGTH 1.0 0.50, NOTE_G4, 0.8, 0.0 ``` -### Conversion Reference - -At 120 BPM (1 unit = 4 beats = 2 seconds): - -| Units | Beats | Seconds | Description | -|-------|-------|---------|-------------| -| 0.00 | 0 | 0.0 | Start | -| 0.25 | 1 | 0.5 | Quarter | -| 0.50 | 2 | 1.0 | Half | -| 0.75 | 3 | 1.5 | Three-quarter | -| 1.00 | 4 | 2.0 | Full pattern | - ### Pattern Length -- `LENGTH` parameter is optional, defaults to 1.0 -- Can be any value (0.5 for half-length, 2.0 for double-length, etc.) -- Events must be within range `[0.0, LENGTH]` +- `LENGTH` parameter defaults to 1.0 if omitted +- Can be any value (0.5 for half-length, 2.0 for double-length) +- Events must be within `[0.0, LENGTH]` -Example of half-length pattern: +Example half-length pattern: ``` PATTERN short_fill LENGTH 0.5 # 2 beats = 1 second at 120 BPM 0.00, ASSET_HIHAT, 0.7, 0.0 0.50, ASSET_HIHAT, 0.6, 0.0 # 0.50 * 0.5 = 1 beat into the pattern ``` +### Future: Modifiers +Potential runtime modifiers (not yet implemented): +- Randomization, accents, volume modulation, distortion, noise, effects diff --git a/doc/test_demo_README.md b/doc/test_demo_README.md index bbce023..6ccada0 100644 --- a/doc/test_demo_README.md +++ b/doc/test_demo_README.md @@ -1,273 +1,122 @@ -# test_demo - Audio/Visual Synchronization Debug Tool +# test_demo - Audio/Visual Sync Debug Tool -## Overview - -A minimal standalone tool for debugging audio/visual synchronization in the demo without the complexity of the full demo64k timeline. +Minimal standalone tool for debugging audio/visual synchronization. ## Features - -- **Simple Drum Beat**: Kick-snare pattern with crash every 4th bar -- **NOTE_A4 Reference Tone**: 440 Hz tone plays at start of each bar for testing -- **Screen Flash Effect**: Visual feedback synchronized to audio peaks -- **16 Second Duration**: 8 bars at 120 BPM -- **Variable Tempo Mode**: Tests tempo scaling (1.0x ↔ 1.5x and 1.0x ↔ 0.66x) -- **Peak Logging**: Export audio peaks to file for gnuplot visualization +- Kick-snare drum pattern with crash every 4th bar +- NOTE_A4 reference tone (440 Hz) at bar starts +- Screen flash synchronized to audio peaks +- 16 seconds (8 bars at 120 BPM) +- Variable tempo mode (1.0x ↔ 1.5x / 0.66x) +- Peak logging for gnuplot ## Building - ```bash cmake --build build --target test_demo ``` ## Usage -### Basic Usage - +### Basic ```bash build/test_demo ``` ### Command-Line Options - ``` - --help Show help message and exit - --fullscreen Run in fullscreen mode - --resolution WxH Set window resolution (e.g., 1024x768) - --tempo Enable tempo variation test mode - --log-peaks FILE Log audio peaks at each beat (32 samples for 16s) - --log-peaks-fine Log at each frame for fine analysis (~960 samples) - (use with --log-peaks for millisecond resolution) +--help Show help +--fullscreen Fullscreen mode +--resolution WxH Window size (e.g., 1024x768) +--tempo Enable tempo variation +--log-peaks FILE Log audio peaks (beat-aligned) +--log-peaks-fine Log per-frame (~960 samples) ``` ### Examples - -#### Run in fullscreen ```bash +# Fullscreen build/test_demo --fullscreen -``` -#### Custom resolution with tempo testing -```bash -build/test_demo --resolution 1024x768 --tempo -``` +# Tempo test +build/test_demo --tempo -#### Log audio peaks for analysis (beat-aligned) -```bash +# Peak logging build/test_demo --log-peaks peaks.txt -``` +gnuplot -p -e "plot 'peaks.txt' using 2:3 with lines" -After running, visualize with gnuplot: -```bash -gnuplot -p -e "set xlabel 'Time (s)'; set ylabel 'Peak'; plot 'peaks.txt' using 2:3 with lines title 'Raw Peak'" -``` - -#### Log audio peaks with fine resolution (every frame) -```bash +# Fine-grained logging build/test_demo --log-peaks peaks_fine.txt --log-peaks-fine ``` -This logs at ~60 Hz (every frame) instead of every beat, providing millisecond-resolution data for detailed synchronization analysis. Produces ~960 samples for the 16-second demo. - -## Keyboard Controls - -- **ESC**: Exit the demo -- **F**: Toggle fullscreen +**Keyboard:** `Esc` (exit), `F` (toggle fullscreen) ## Audio Pattern +8 bars, 120 BPM: +- Bars 1-2, 4-6, 8: Kick-Snare-Kick-Snare + A4 tone +- Bars 3, 7: Kick+Crash-Snare-Kick-Snare + A4 tone (landmarks) -The demo plays a repeating drum pattern: - -``` -Bar 1-2: Kick-Snare-Kick-Snare (with A4 note) -Bar 3: Kick+Crash-Snare-Kick-Snare (with A4 note) ← Crash landmark -Bar 4-6: Kick-Snare-Kick-Snare (with A4 note) -Bar 7: Kick+Crash-Snare-Kick-Snare (with A4 note) ← Crash landmark -Bar 8: Kick-Snare-Kick-Snare (with A4 note) -``` - -**Timing:** -- 120 BPM (2 beats per second) -- 1 bar = 4 beats = 2 seconds -- Total duration: 8 bars = 16 seconds - -**Crash Landmarks:** -- First crash at T=4.0s (bar 3, beat 8) -- Second crash at T=12.0s (bar 7, beat 24) - -## Tempo Test Mode (`--tempo`) +**Crash landmarks:** T=4.0s (bar 3), T=12.0s (bar 7) -When enabled with `--tempo` flag, the demo alternates tempo scaling every bar: +## Tempo Mode +Alternates every bar: +- Even bars: 1.0x → 1.5x (accelerate) +- Odd bars: 1.0x → 0.66x (decelerate) -- **Even bars (0, 2, 4, 6)**: Accelerate from 1.0x → 1.5x -- **Odd bars (1, 3, 5, 7)**: Decelerate from 1.0x → 0.66x +Music time drifts from physical time. Patterns respect tempo scaling. -This tests the variable tempo system where music time advances independently of physical time: -- `music_time += dt * tempo_scale` -- Pattern triggering respects tempo scaling -- Audio samples play at normal pitch (no pitch shifting) +## Peak Logging -**Console Output with --tempo:** +**Beat-aligned** (32 samples): ``` -[T=0.50, MusicT=0.56, Beat=1, Frac=0.12, Peak=0.72, Tempo=1.25x] -``` - -**Expected Behavior:** -- Physical time always advances at 1:1 (16 seconds = 16 seconds) -- Music time advances faster during acceleration, slower during deceleration -- By end of demo: Music time > Physical time (due to net acceleration) - -## Peak Logging Format - -The `--log-peaks` option writes a text file with three columns. - -### Beat-Aligned Mode (default) - -Logs once per beat (32 samples for 16 seconds): - -``` -# Audio peak log from test_demo -# Mode: beat-aligned -# To plot with gnuplot: -# gnuplot -p -e "set xlabel 'Time (s)'; set ylabel 'Peak'; plot 'peaks.txt' using 2:3 with lines title 'Raw Peak'" # Columns: beat_number clock_time raw_peak -# 0 0.000000 0.850000 1 0.500000 0.720000 -2 1.000000 0.800000 -... ``` -**Columns:** -1. **beat_number**: Beat index (0, 1, 2, ...) -2. **clock_time**: Physical time in seconds -3. **raw_peak**: Audio peak value (0.0-1.0+) - -### Fine-Grained Mode (`--log-peaks-fine`) - -Logs at every frame (approximately 960 samples for 16 seconds at 60 Hz): - +**Fine-grained** (~960 samples): ``` -# Audio peak log from test_demo -# Mode: fine (per-frame) -# To plot with gnuplot: -# gnuplot -p -e "set xlabel 'Time (s)'; set ylabel 'Peak'; plot 'peaks_fine.txt' using 2:3 with lines title 'Raw Peak'" # Columns: frame_number clock_time raw_peak beat_number -# 0 0.000000 0.850000 0 1 0.016667 0.845231 0 -2 0.033333 0.823445 0 -3 0.050000 0.802891 0 -... -30 0.500000 0.720000 1 -31 0.516667 0.715234 1 -... ``` -**Columns:** -1. **frame_number**: Frame index (0, 1, 2, ...) -2. **clock_time**: Physical time in seconds (millisecond precision) -3. **raw_peak**: Audio peak value (0.0-1.0+) -4. **beat_number**: Corresponding beat index (for correlation with beat-aligned mode) - -**Use Cases:** - -*Beat-Aligned Mode:* -- Verify audio/visual synchronization at beat boundaries -- Detect clipping at specific beats (peak > 1.0) -- Analyze tempo scaling effects on pattern triggering -- Compare expected vs actual beat times - -*Fine-Grained Mode:* -- Millisecond-resolution synchronization analysis -- Detect frame-level timing jitter or drift -- Analyze audio envelope shape and attack/decay characteristics -- Debug precise flash-to-audio alignment issues -- Identify sub-beat audio artifacts or glitches +**Use cases:** +- Beat-aligned: Verify sync at beat boundaries, detect clipping +- Fine-grained: Millisecond-resolution analysis, detect jitter ## Files +- `src/test_demo.cc`: Main executable (~220 lines) +- `assets/test_demo.track`: Drum pattern +- `assets/test_demo.seq`: Visual timeline +- `src/generated/test_demo_*.cc`: Generated code -- **`src/test_demo.cc`**: Main executable (~220 lines) -- **`assets/test_demo.track`**: Drum pattern and NOTE_A4 definition -- **`assets/test_demo.seq`**: Visual timeline (FlashEffect) -- **`src/generated/test_demo_timeline.cc`**: Generated timeline (auto) -- **`src/generated/test_demo_music.cc`**: Generated music data (auto) +## Verification -## Verification Checklist +**Normal mode:** +- Flashes every ~0.5s, synchronized with kicks +- Crash landmarks at T=4.0s and T=12.0s +- No audio glitches -### Normal Mode -- [ ] Visual flashes occur every ~0.5 seconds -- [ ] Audio plays (kick-snare drum pattern audible) -- [ ] A4 tone (440 Hz) audible at start of each bar -- [ ] Synchronization: Flash happens simultaneously with kick hit -- [ ] Crash landmark: Larger flash + cymbal crash at T=4.0s and T=12.0s -- [ ] Auto-exit: Demo stops cleanly at 16 seconds -- [ ] Console timing: Peak values spike when kicks hit (Peak > 0.7) -- [ ] No audio glitches: Smooth playback, no stuttering +**Tempo mode:** +- Bar 0 accelerates, bar 1 decelerates +- Music time drifts from physical time +- Smooth transitions -### Tempo Test Mode -- [ ] Bar 0 accelerates (1.0x → 1.5x) -- [ ] Bar 1 decelerates (1.0x → 0.66x) -- [ ] Music time drifts from physical time -- [ ] Audio still syncs with flashes -- [ ] Smooth transitions at bar boundaries -- [ ] Physical time = 16s at end -- [ ] Music time > 16s at end -- [ ] Console shows tempo value - -### Peak Logging -- [ ] File created when `--log-peaks` specified -- [ ] Contains beat_number, clock_time, raw_peak columns -- [ ] Gnuplot command in header comment -- [ ] One row per beat (32 rows for 16 seconds) -- [ ] Peak values match console output -- [ ] Gnuplot visualization works +**Peak logging:** +- File created with correct format +- Gnuplot visualization works ## Troubleshooting -**Flash appears before audio:** -- Cause: Audio latency too high -- Fix: Reduce ring buffer size in `ring_buffer.h` - -**Flash appears after audio:** -- Cause: Ring buffer under-filled -- Fix: Increase pre-fill duration in `audio_render_ahead()` - -**No flash at all:** -- Cause: Peak threshold not reached -- Check: Console shows Peak > 0.7 -- Fix: Increase `visual_peak` multiplier in code (currently 8.0×) - -**A4 note not audible:** -- Cause: NOTE_A4 volume too low or procedural generation issue -- Check: Console shows correct sample count (4 samples) -- Fix: Increase volume in test_demo.track (currently 0.5) - -**Peak log file empty:** -- Cause: Demo exited before first beat -- Check: File has header comments -- Fix: Ensure demo runs for at least 0.5 seconds +**Flash before audio:** Reduce ring buffer size (`ring_buffer.h`) +**Flash after audio:** Increase pre-fill (`audio_render_ahead()`) +**No flash:** Increase `visual_peak` multiplier (currently 8.0×) +**A4 not audible:** Increase volume in `test_demo.track` +**Empty log:** Ensure demo runs ≥0.5s ## Design Rationale -**Why separate from demo64k?** -- Isolated testing environment -- No timeline complexity -- Faster iteration cycles -- Independent verification - -**Why use main demo assets?** -- Avoid asset system conflicts (AssetId enum collision) -- Reuse existing samples -- No size overhead -- Simpler CMake integration - -**Why 16 seconds?** -- Long enough for verification (8 bars) -- Short enough for quick tests -- Crash landmarks at 25% and 75% for easy reference - -**Why NOTE_A4?** -- Standard reference tone (440 Hz) -- Easily identifiable pitch -- Tests procedural note generation -- Minimal code impact +**Why separate?** Isolated testing, no timeline complexity +**Why 16 seconds?** Long enough to verify, short enough to iterate +**Why NOTE_A4?** Standard 440 Hz reference tone +**Why crash landmarks?** Easy visual/audio synchronization verification |