1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
# 64k Demo Project
Goal:
- Produce a <=64k native demo binary
- Same C++ codebase for Windows, macOS, Linux
Graphics:
- WebGPU via wgpu-native
- WGSL shaders
- Hybrid rendering: Rasterized proxy geometry + SDF raymarching
Audio:
- 32 kHz, 16-bit stereo
- Procedurally generated samples
- Real-time additive synthesis from spectrograms (IDCT)
- Variable tempo system with music time abstraction
- Event-based pattern triggering for dynamic tempo scaling
- Modifiable Loops and Patterns, w/ script to generate them (like a Tracker)
- Unified AudioEngine for lifecycle management (eliminates initialization fragility)
Constraints:
- Size-sensitive
- Minimal dependencies
- Explicit control over all allocations
Style:
- Demoscene
- No engine abstractions
---
## Project Roadmap
**Note:** For detailed history of recently completed milestones, see `COMPLETED.md`.
### Current Status
- Audio system: Sample-accurate synchronization achieved. Uses hardware playback time as master clock. Variable tempo support integrated. **Pipeline optimized (Task #72)**: Zero heap allocations per frame, direct ring buffer writes, explicit clipping. Comprehensive test coverage maintained.
- Build system: Optimized with proper asset dependency tracking
- Shader system: Modular with comprehensive compilation tests. **WGSL composability improved**: Common utilities extracted (`math/common_utils.wgsl`) with 12 call sites deduplicated across renderer shaders.
- 3D rendering: Hybrid SDF/rasterization with BVH acceleration and binary scene loader. **Object data loading and parsing pipeline enhanced for primitives (e.g., plane_distance).**
- Asset pipeline: Blender export script and binary scene ingestion supported
- Error handling: **Dual macro system**: `FATAL_XXX` for programming errors (abort), `CHECK_RETURN` for recoverable errors (graceful return). Messages stripped in STRIP_ALL builds.
---
## Next Up
- **Task #5: Spectral Brush Editor** [IN PROGRESS - February 6, 2026]
- Create web-based tool for procedurally tracing audio spectrograms
- Replace large .spec assets with tiny C++ code (50-100× compression)
- Phase 1: C++ runtime (`spectral_brush.h/cc` - Bezier curves + Gaussian profiles)
- Phase 2: Editor UI (HTML/JS canvas, dual-layer visualization, keyboard shortcuts)
- Phase 3: File I/O (load .wav/.spec, export procedural_params.txt + C++ code)
- See `doc/SPECTRAL_BRUSH_EDITOR.md` for complete design
- **Task #72: Audio Pipeline Streamlining** [COMPLETED - February 8, 2026]
- ✅ Optimize data flow: Zero heap allocations per frame achieved
- ✅ Direct additive mixing: Ring buffer two-phase write API
- ✅ Precision: float32 internal pipeline with explicit clipping
- **Visuals & Content**
- [ ] **Task #52: Procedural SDF Font**: Minimal bezier/spline set for [A-Z, 0-9] and SDF rendering.
- [ ] **Task #53: Particles Shader Polish**: Improve visual quality of particles.
- [ ] **Task #55: SDF Random Planes Intersection**: Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection.
- **Tooling & Optimization**
- [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling.
- [x] **Task #39: Visual Debugging System**: Implemented wireframe primitives (Sphere, Cone, Cross, Trajectory) for debugging.
- [ ] **Task #53: Particles Shader Polish**: Improve visual quality of particles.
- [ ] **Task #55: SDF Random Planes Intersection**: Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection.
- **Tooling & Optimization**
- [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling.
---
## Future Goals
- **Task #36: Blender Exporter**: Create script to export scenes to internal binary format. (Deprioritized)
- **Task #5: Implement Spectrogram Editor**
- [ ] Develop a web-based tool (`tools/editor`) for creating and editing `.spec` files visually.
- **Task #21: Shader Optimization**
- [ ] Use macros or code generation to factorize common WGSL code (normals, bump, lighting).
- [ ] Implement Tri-planar mapping for better procedural textures.
- [ ] **Task #18-B: GPU BVH & Shadows**: Optimize scene queries with a GPU-based BVH.
- **Phase 2: Advanced Size Optimization**
- [ ] **Task #22: Windows Native Platform**: Replace GLFW with minimal native Windows API.
- [ ] **Task #28: Spectrogram Quantization**: Quantize spectrograms to logarithmic frequency and uint16_t.
- [ ] **Task #35: CRT Replacement**: Investigation and implementation of CRT-free entry point.
---
*For a detailed list of all completed tasks, see the git history.*
## Recently Completed (February 2026)
- **WGSL Shader Composability** - Extracted common utilities to `math/common_utils.wgsl`:
- `transform_normal()` - 2 call sites (renderer_3d, mesh_render)
- `spherical_uv()` / `spherical_uv_from_dir()` - 8 call sites (renderer_3d, skybox)
- `grid_pattern()` - 2 call sites (renderer_3d)
- Size savings: ~200 bytes net
- **Test Suite Optimization** - JitteredAudioBackendTest: 3.5s → 0.07s (50x speedup)
- Reduced test duration and sleep times
- Full CI suite now <1 second
- **CHECK_RETURN Macro System** - Error handling for recoverable errors:
- `CHECK_RETURN_IF()` - Simple validation with return
- `CHECK_RETURN_BEGIN/END` - Complex validation with cleanup
- `WARN_IF()` - Non-fatal warnings
- Applied to 5 call sites (asset_manager, test_demo)
- Size impact: ~500 bytes saved in STRIP_ALL builds
## Architectural Overview
### Hybrid 3D Renderer
- **Core Idea**: Uses standard rasterization to draw proxy hulls (boxes), then raymarches inside the fragment shader to find the exact SDF surface.
- **Transforms**: Uses `inv_model` matrices to perform all raymarching in local object space, handling rotation and non-uniform scaling correctly.
- **Shadows**: Instance-based shadow casting with self-shadowing prevention (`skip_idx`).
### Sequence & Effect System
- **Effect**: Abstract base for visual elements. Supports `compute` and `render` phases.
- **Sequence**: Timeline of effects with start/end times.
- **MainSequence**: Top-level coordinator and framebuffer manager.
- **seq_compiler**: Transpiles `assets/demo.seq` into C++ `timeline.cc`.
### Asset & Build System
- **asset_packer**: Embeds binary assets (like `.spec` files) into C++ arrays.
- **Runtime Manager**: O(1) retrieval with lazy procedural generation support.
- **Automation**: `gen_assets.sh`, `build_win.sh`, and `check_all.sh` for multi-platform validation.
### Audio Engine
- **Synthesis**: Real-time additive synthesis from spectrograms via FFT-based IDCT (O(N log N)). Stereo output (32kHz, 16-bit, interleaved L/R). Uses orthonormal DCT-II/DCT-III transforms with Numerical Recipes reordering method.
- **Variable Tempo**: Music time abstraction with configurable tempo_scale. Tempo changes don't affect pitch.
- **Event-Based Tracker**: Individual TrackerEvents trigger as separate voices with dynamic beat calculation. Notes within patterns respect tempo scaling.
- **Backend Abstraction**: `AudioBackend` interface with `MiniaudioBackend` (production), `MockAudioBackend` (testing), and `WavDumpBackend` (offline rendering).
- **Dynamic Updates**: Double-buffered spectrograms for live thread-safe updates.
- **Procedural Library**: Melodies and spectral filters (noise, comb) generated at runtime.
- **Pattern System**: TrackerPatterns contain lists of TrackerEvents (beat, sample_id, volume, pan). Events trigger individually based on elapsed music time.
|
