1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
|
# 64k Demo Project
Goal:
- Produce a <=64k native demo binary
- Same C++ codebase for Windows, macOS, Linux
Graphics:
- WebGPU via wgpu-native
- WGSL shaders
- Hybrid rendering: Rasterized proxy geometry + SDF raymarching
Audio:
- 32 kHz, 16-bit stereo
- Procedurally generated samples
- Real-time additive synthesis from spectrograms (IDCT)
- Variable tempo system with music time abstraction
- Event-based pattern triggering for dynamic tempo scaling
- Modifiable Loops and Patterns, w/ script to generate them (like a Tracker)
- Unified AudioEngine for lifecycle management (eliminates initialization fragility)
Constraints:
- Size-sensitive
- Minimal dependencies
- Explicit control over all allocations
Style:
- Demoscene
- No engine abstractions
---
## Project Roadmap
**Note:** For detailed history of recently completed milestones, see `COMPLETED.md`.
### Current Status
- Audio system: Sample-accurate synchronization achieved. Uses hardware playback time as master clock. Variable tempo support integrated. **Pipeline optimized (Task #72)**: Zero heap allocations per frame, direct ring buffer writes, explicit clipping. Comprehensive test coverage maintained.
- Build system: Optimized with proper asset dependency tracking
- Shader system: **Parameterization complete**: UniformHelper template, per-frame dynamic params, .seq syntax support. Modular with comprehensive compilation tests. **WGSL composability improved**: Common utilities extracted (`math/common_utils.wgsl`) with 12 call sites deduplicated across renderer shaders.
- 3D rendering: Hybrid SDF/rasterization with BVH acceleration and binary scene loader. **Object data loading and parsing pipeline enhanced for primitives (e.g., plane_distance).**
- Asset pipeline: Blender export script and binary scene ingestion supported
- Error handling: **Dual macro system**: `FATAL_XXX` for programming errors (abort), `CHECK_RETURN` for recoverable errors (graceful return). Messages stripped in STRIP_ALL builds.
- Testing: **32/33 tests passing (97%)** - Uniform buffer alignment fixed (Task #74). DemoEffectsTest fails due to wgpu_native library bug (not project code).
---
## Next Up
- **Task #5: Spectral Brush Editor** [IN PROGRESS]
- Create web-based tool for procedurally tracing audio spectrograms
- Replace large .spec assets with tiny C++ code (50-100× compression)
- Phase 1: C++ runtime (`spectral_brush.h/cc` - Bezier curves + Gaussian profiles)
- Phase 2: Editor UI (HTML/JS canvas, dual-layer visualization, keyboard shortcuts)
- Phase 3: File I/O (load .wav/.spec, export procedural_params.txt + C++ code)
- See `doc/SPECTRAL_BRUSH_EDITOR.md` for complete design
- **Visuals & Content**
- [ ] **Task #52: Procedural SDF Font**: Minimal bezier/spline set for [A-Z, 0-9] and SDF rendering
- [ ] **Task #53: Particles Shader Polish**: Improve visual quality of particles
- [ ] **Task #55: SDF Random Planes Intersection**: Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection
- **Tooling & Optimization**
- [ ] **Task #54: Tracy Integration**: Integrate Tracy debugger for performance profiling
---
## Design Docs Quick Reference
For detailed documentation, use Read tool to load specific docs:
- **doc/TRACKER.md**: Audio pattern system with unit-less timing (1 unit = 4 beats). Text-based music score compiled to C++ runtime.
- **doc/3D.md**: Hybrid SDF raymarching with BVH acceleration and Position Based Dynamics physics.
- **doc/ASSET_SYSTEM.md**: Build-time asset packer with 16-byte alignment, enum-based O(1) retrieval, procedural generation support.
- **doc/BUILD.md**: Multi-platform builds (Debug/STRIP_ALL/FINAL_STRIP), cross-compilation, size reporting.
- **doc/SPECTRAL_BRUSH_EDITOR.md**: Web tool for tracing spectrograms with Bezier curves (50-100× compression).
- **doc/SEQUENCE.md**: .seq timeline format with BPM notation, priority modifiers, Gantt visualization.
- **doc/MASKING_SYSTEM.md**: Auxiliary texture registry for inter-effect screen-space partitioning.
- **doc/SCENE_FORMAT.md**: Binary scene format (SCN1) with object transforms, physics, mesh references.
- **doc/test_demo_README.md**: 16s audio/visual sync test tool with tempo variation and peak logging.
- **doc/CONTEXT_MAINTENANCE.md**: Context hygiene protocol (archive to COMPLETED.md monthly, keep Tier 1 files lean).
---
## Future Goals
- **Task #36: Blender Exporter**: Create script to export scenes to internal binary format. (Deprioritized)
- **Task #21: Shader Optimization**
- [ ] Use macros or code generation to factorize common WGSL code (normals, bump, lighting).
- [ ] Implement Tri-planar mapping for better procedural textures.
- [ ] **Task #18-B: GPU BVH & Shadows**: Optimize scene queries with a GPU-based BVH.
- **Phase 2: Advanced Size Optimization**
- [ ] **Task #22: Windows Native Platform**: Replace GLFW with minimal native Windows API.
- [ ] **Task #28: Spectrogram Quantization**: Quantize spectrograms to logarithmic frequency and uint16_t.
- [ ] **Task #35: CRT Replacement**: Investigation and implementation of CRT-free entry point.
---
*For a detailed list of all completed tasks, see the git history.*
## Recently Completed (February 2026)
- **WGSL Uniform Buffer Validation (Task #75)** (February 9) - Standardized uniform buffer layout across all effects. Created validation tool (`validate_uniforms.py`) integrated into build. All effects now use `CommonPostProcessUniforms` (binding 2) + effect-specific params (binding 3). Added `UNIFORM_BUFFER_GUIDELINES.md`.
- **Uniform Buffer Alignment (Task #74)** (February 9) - Fixed WGSL `vec3<f32>` alignment issues. Changed padding from `vec3` to three `f32` fields in multiple shaders. Demo runs with 0 validation errors. Test suite: 32/33 passing (97%).
- **Shader Parametrization (Task #73)** (February 8) - Full uniform parameter system with .seq syntax. FlashEffect, ChromaAberrationEffect, GaussianBlurEffect now support dynamic parameters. Size: ~400-500 bytes. See `doc/COMPLETED.md` for details.
## Architectural Overview
### Hybrid 3D Renderer
- **Core Idea**: Uses standard rasterization to draw proxy hulls (boxes), then raymarches inside the fragment shader to find the exact SDF surface.
- **Transforms**: Uses `inv_model` matrices to perform all raymarching in local object space, handling rotation and non-uniform scaling correctly.
- **Shadows**: Instance-based shadow casting with self-shadowing prevention (`skip_idx`).
### Sequence & Effect System
- **Effect**: Abstract base for visual elements. Supports `compute` and `render` phases.
- **Sequence**: Timeline of effects with start/end times.
- **MainSequence**: Top-level coordinator and framebuffer manager.
- **seq_compiler**: Transpiles `assets/demo.seq` into C++ `timeline.cc`.
### Asset & Build System
- **asset_packer**: Embeds binary assets (like `.spec` files) into C++ arrays.
- **Runtime Manager**: O(1) retrieval with lazy procedural generation support.
- **Automation**: `gen_assets.sh`, `build_win.sh`, and `check_all.sh` for multi-platform validation.
### Audio Engine
- **Synthesis**: Real-time additive synthesis from spectrograms via FFT-based IDCT (O(N log N)). Stereo output (32kHz, 16-bit, interleaved L/R). Uses orthonormal DCT-II/DCT-III transforms with Numerical Recipes reordering method.
- **Variable Tempo**: Music time abstraction with configurable tempo_scale. Tempo changes don't affect pitch.
- **Event-Based Tracker**: Individual TrackerEvents trigger as separate voices with dynamic beat calculation. Notes within patterns respect tempo scaling.
- **Backend Abstraction**: `AudioBackend` interface with `MiniaudioBackend` (production), `MockAudioBackend` (testing), and `WavDumpBackend` (offline rendering).
- **Dynamic Updates**: Double-buffered spectrograms for live thread-safe updates.
- **Procedural Library**: Melodies and spectral filters (noise, comb) generated at runtime.
- **Pattern System**: TrackerPatterns contain lists of TrackerEvents (beat, sample_id, volume, pan). Events trigger individually based on elapsed music time.
|
