1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
|
# 64k Demo Project
Goal:
- Produce a <=64k native demo binary
- Same C++ codebase for Windows, macOS, Linux
Graphics:
- WebGPU via wgpu-native
- WGSL shaders
- Single fullscreen pass initially
Audio:
- 32 kHz, 16-bit mono
- Procedurally generated samples
- No decoding, no assets
Constraints:
- Size-sensitive
- Minimal dependencies
- Explicit control over all allocations
Style:
- Demoscene
- No engine abstractions
Incoming tasks in no particular order:
- [x] 1. add a fullscreen mode (as command line option)
- [x] 2. parse the keyboard key. Exit the demo when 'esc' is pressed. Toggle full-screen when 'f' is pressed.
- [x] 3. add binary crunchers for all platforms
- [ ] 4. add cross-compilation for PC+linux (x86_64) and PC+Windows (.exe binary)
- [x] PC+Windows (.exe binary) via MinGW
- [ ] PC+linux (x86_64)
- 5. implement a spectrogram editor for representing .spec with elementary shapes (bezier curves, lines, random noise, rectangles...) as a mean of compression / spec generation
- 6. add a scripting tool to edit the demo (compiled into the binary at the end)
- [x] 7. compile wgpu-native in optimized mode (not unoptimized)
- [x] 8. add a #define STRIP_ALL to remove all unnecessary code for the final build (for instance, command-line args parsing, or unnecessary options, constant parameters to function calls, etc.)
- [x] 9. work on the compact in-line and off-line asset system (@ASSET_SYSTEM.md)
- 10. optimize spectool to remove trailing zeroes from a spec file before saving it
- [x] 11. implement a general [time / timer / beat] system in the demo, for effects timing
- 12. Implement 'Sequence' and 'Effect' system:
- `Effect` interface: `start()`, `draw(time, beat, intensity)`, `end()`.
- `Sequence` manager: Handles a list of potentially overlapping effects.
- `time` is relative to sequence start. `beat` is a [0..1] pulse. `intensity` is signal strength (e.g. music level).
## Session Decisions and Current State
### Asset Management System:
- **Architecture**: Implemented a C++ tool (`asset_packer`) that bundles external files into hex-encoded C arrays.
- **Lookup**: Uses a highly efficient array-based lookup (AssetRecord) for O(1) retrieval of raw byte data at runtime.
- **Descriptors**: Assets are defined in `assets/final/demo_assets.txt` (for the demo) and `assets/final/test_assets_list.txt` (for tests).
- **Organization**: Common retrieval logic is decoupled and located in `src/util/asset_manager.cc`, ensuring generated data files only contain raw binary blobs.
- **Lazy Decompression**: Scaffolding implemented for future on-demand decompression support.
### Build System:
- **Production Pipeline**: Automated the entire assembly process via a `final` CMake target (`make final`).
- **Automation**: This target builds the tools, runs the `gen_assets.sh` script to re-analyze audio and regenerate sources, and then performs final binary stripping and crunching (using `strip` and `gzexe` on macOS).
- **Windows Cross-Compilation**: Implemented a full pipeline from macOS to Windows x86_64 using MinGW.
- `scripts/fetch_win_deps.sh`: Downloads pre-compiled GLFW and `wgpu-native` binaries.
- `scripts/build_win.sh`: Cross-compiles the demo, bundles MinGW DLLs, and crunches the binary.
- `scripts/run_win.sh`: Executes the resulting `.exe` using Wine.
- `scripts/crunch_win.sh`: Strips and packs the Windows binary using UPX (LZMA).
- `scripts/analyze_win_bloat.sh`: Reports section sizes and top symbols for size optimization.
### Audio Engine (Synth):
- **Architecture**: Real-time additive synthesis from spectrograms using Inverse Discrete Cosine Transform (IDCT).
- **Dynamic Updates**: Implemented a double-buffering (flip-flop) mechanism for thread-safe, real-time updates of spectrogram data.
- **Peak Detection**: Real-time output peak detection with exponential decay for smooth visual synchronization.
- **Procedural Melody**: Implemented a shared library (`src/audio/gen.cc`) for generating note spectrograms at runtime. Supports melody pasting with overlapping frames and spectral post-processing.
- **Spectral Filters**: Implemented runtime spectral effects including noise (grit), lowpass filtering, and comb filtering (phaser/flanger effects).
- **Timing System**: Implemented a beat-based timing system (BPM) used for both audio generation and visual synchronization.
### WebGPU Integration:
- **Resource Management**: Introduced `GpuBuffer`, `RenderPass`, and `ComputePass` abstractions to group pipeline and bind group resources.
- **Compute Shaders**: Implemented a high-performance particle system (10,000 particles) using compute shaders for physics and audio-reactive updates.
- **Visuals**: Pulsating heptagon and a compute-driven particle field synchronized with audio peaks and global time.
### WebGPU Integration Fixes:
Several critical issues were resolved to ensure stable WebGPU operation across platforms:
- **Surface Creation (macOS)**: Fixed a `g_surface` assertion failure by adding platform-specific compile definitions (`-DGLFW_EXPOSE_NATIVE_COCOA`) to `CMakeLists.txt`. This allows `glfw3webgpu` to access the native window handles required for surface creation.
- **Texture Usage**: Resolved a validation error (`RENDER_ATTACHMENT` usage missing) by explicitly setting `g_config.usage = WGPUTextureUsage_RenderAttachment` in the surface configuration.
- **Render Pass Validation**: Fixed a "Depth slice provided but view is not 3D" error by ensuring `WGPURenderPassColorAttachment` is correctly initialized, specifically setting `resolveTarget = nullptr` and `depthSlice = WGPU_DEPTH_SLICE_UNDEFINED`.
### Optimizations:
- **Audio Decoding**: Disabled FLAC, WAV, MP3, and all encoding features in `miniaudio` for the runtime demo build (via `MA_NO_FLAC`, `MA_NO_WAV`, etc.). This reduced the packed Windows binary size by ~100KB (461KB -> 356KB). `spectool` retains full decoding capabilities.
- **Build Stripping**: Implemented `DEMO_STRIP_ALL` CMake option to remove command-line parsing, debug info, and non-essential error handling strings.
### Future Optimizations (Phase 2):
- **Windows Platform Layer**: Replace the static GLFW library with a minimal, native Windows API implementation (`CreateWindow`, `PeekMessage`) to significantly reduce binary size.
- **Asset Compression**: Implement logarithmic frequency storage and quantization for `.spec` files.
- **CRT Replacement**: investigate minimal C runtime alternatives.
### WebGPU Portability Layer:
To maintain a single codebase while supporting different `wgpu-native` versions (Native macOS/Linux headers vs. Windows/MinGW v0.19.4.1), a portability layer was implemented in `src/gpu/gpu.cc`:
- **Header Mapping**: Conditional inclusion of `<webgpu/webgpu.h>` for Windows vs. `<webgpu.h>` for native builds.
- **Type Shims**: Implementation of `WGPUStringView` as a simple `const char*` for older APIs.
- **Callback Signatures**: Handles the difference between 4-argument (Windows/Old) and 5-argument (macOS/New) callback signatures for `wgpuInstanceRequestAdapter` and `wgpuAdapterRequestDevice`, including the use of callback info structs on newer APIs.
- **API Lifecycle**:
- **Wait Mechanism**: Abstraction of `wgpuInstanceWaitAny` (new) vs. `wgpuInstanceProcessEvents` (old).
- **Struct Differences**:
- **Color Attachments**: Conditional removal of the `depthSlice` member in `WGPURenderPassColorAttachment`, which is not present in v0.19.
- **Error Handling**: Abstracted `wgpuDeviceSetUncapturedErrorCallback` usage.
- **Surface Creation**: Custom logic in `glfw3webgpu.c` to handle `WGPUSurfaceSourceWindowsHWND` (new) vs. `WGPUSurfaceDescriptorFromWindowsHWND` (old).
### Coding Style:
- **Standard**: Strictly enforced project-specific rules in `CONTRIBUTING.md`.
- **Cleanup**: Automate removal of trailing whitespaces and addition of missing newlines at EOF across all source files.
- **Constraints**: No `auto`, no C++ style casts (`static_cast`, etc.), mandatory 3-line headers.
### Design Decision: Spectrogram Data Representation
* **Current State:** Spectrogram frequency bins are stored linearly in `.spec` files and processed as such by the core audio engine (using standard DCT/IDCT). The JavaScript spectrogram editor maps this linear data to a logarithmic scale for visualization and interaction.
* **Future Optimization (TODO):** The `.spec` file format will be revisited to:
* Store frequencies logarithmically.
* Use `uint16_t` instead of `float` for spectral values.
* **Impact:** This aims to achieve better compression while retaining fine frequency resolution relevant to human perception. It will primarily affect the code responsible for saving to and reading from `.spec` files, requiring conversions between the new format and the linear float format used internally by the audio engine.
### Development Workflow:
- **Testing**: Comprehensive test suite including `AssetManagerTest`, `SynthEngineTest`, `HammingWindowTest`, and `SpectoolEndToEndTest`. All tests are verified before committing.
|
