1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
|
# 64k Demo Project
Goal:
- Produce a <=64k native demo binary
- Same C++ codebase for Windows, macOS, Linux
Graphics:
- WebGPU via wgpu-native
- WGSL shaders
- Single fullscreen pass initially
Audio:
- 32 kHz, 16-bit mono
- Procedurally generated samples
- No decoding, no assets
Constraints:
- Size-sensitive
- Minimal dependencies
- Explicit control over all allocations
Style:
- Demoscene
- No engine abstractions
Incoming tasks in no particular order:
- [x] 1. add a fullscreen mode (as command line option)
- [x] 2. parse the keyboard key. Exit the demo when 'esc' is pressed. Toggle full-screen when 'f' is pressed.
- [x] 3. add binary crunchers for all platforms
- 4. add cross-compilation for PC+linux (x86_64) and PC+Windows (.exe binary)
- 5. implement a spectrogram editor for representing .spec with elementary shapes (bezier curves, lines, random noise, rectangles...) as a mean of compression / spec generation
- 6. add a scripting tool to edit the demo (compiled into the binary at the end)
- [x] 7. compile wgpu-native in optimized mode (not unoptimized)
- [x] 8. add a #define STRIP_ALL to remove all unnecessary code for the final build (for instance, command-line args parsing, or unnecessary options, constant parameters to function calls, etc.)
- [x] 9. work on the compact in-line and off-line asset system (@ASSET_SYSTEM.md)
- 10. optimize spectool to remove trailing zeroes from a spec file before saving it
- 11. implement a general [time / timer / beat] system in the demo, for effects timing
## Session Decisions and Current State
### Asset Management System:
- **Architecture**: Implemented a C++ tool (`asset_packer`) that bundles external files into hex-encoded C arrays.
- **Lookup**: Uses a highly efficient array-based lookup (AssetRecord) for O(1) retrieval of raw byte data at runtime.
- **Descriptors**: Assets are defined in `assets/final/demo_assets.txt` (for the demo) and `assets/final/test_assets_list.txt` (for tests).
- **Organization**: Common retrieval logic is decoupled and located in `src/util/asset_manager.cc`, ensuring generated data files only contain raw binary blobs.
- **Lazy Decompression**: Scaffolding implemented for future on-demand decompression support.
### Build System:
- **Production Pipeline**: Automated the entire assembly process via a `final` CMake target (`make final`).
- **Automation**: This target builds the tools, runs the `gen_assets.sh` script to re-analyze audio and regenerate sources, and then performs final binary stripping and crunching (using `strip` and `gzexe` on macOS).
- **Scripts**:
- `scripts/gen_assets.sh`: Re-analyzes source audio files into `.spec` format and triggers the `asset_packer`.
- `scripts/crunch_demo.sh`: Performs the final size optimization and compression.
### Audio Engine (Synth):
- **Architecture**: Real-time additive synthesis from spectrograms using Inverse Discrete Cosine Transform (IDCT).
- **Dynamic Updates**: Implemented a double-buffering (flip-flop) mechanism for thread-safe, real-time updates of spectrogram data. The main thread writes to a back buffer, which is then atomically swapped to become the active front buffer for the audio thread, avoiding real-time allocations and locks.
- **Peak Detection**: Real-time output peak detection with exponential decay for smooth visual synchronization.
- **Sample Generation**: Samples are generated in the frequency domain (spectrograms) and converted to time-domain audio using IDCT with a Hamming window.
- **Audio Input Support (`spectool`)**: Supports WAV and MP3 input for analysis (leveraging `miniaudio`'s built-in decoders). AAC is explicitly *not* supported due to complexity and dependency constraints.
### Tools Developed:
- `spectool`: A command-line tool for analyzing WAV/MP3 files into `.spec` spectrogram format and for playing back `.spec` files through the synth engine.
- `specview`: A command-line tool for visualizing `.spec` spectrogram files in ASCII art.
- `asset_packer`: A build-time tool for embedding assets into the binary.
### WebGPU Integration:
- **Strategy**: Uses system-installed `wgpu-native` (e.g., via Homebrew) for the implementation.
- **Surface Creation**: Uses `glfw3webgpu` helper library to abstract platform-specific surface creation from GLFW windows, keeping the codebase clean and cross-platform.
- **Headers**: Uses standard `<webgpu.h>` provided by the system install.
- **Visuals**: Pulsating heptagon synchronized with audio peaks, with automatic aspect ratio correction.
### Coding Style:
- **Standard**: Adopted a consistent coding style enforced by `.clang-format`.
- **Rules**: 2-space indentation, no tabs, 80-column line limit.
- **Headers**: Mandatory 3-line descriptive header at the top of every `.h` and `.cc` file.
- **File Extension**: All C++ source files renamed from `.cpp` to `.cc`.
### Platform & Input:
- **Windowing**: Uses GLFW for cross-platform window management.
- **Input**: Supports 'Esc' and 'Q' for quitting, and 'F' for toggling fullscreen.
### Design Decision: Spectrogram Data Representation
* **Current State:** Spectrogram frequency bins are stored linearly in `.spec` files and processed as such by the core audio engine (using standard DCT/IDCT). The JavaScript spectrogram editor maps this linear data to a logarithmic scale for visualization and interaction.
* **Future Optimization (TODO):** The `.spec` file format will be revisited to:
* Store frequencies logarithmically.
* Use `uint16_t` instead of `float` for spectral values.
* **Impact:** This aims to achieve better compression while retaining fine frequency resolution relevant to human perception. It will primarily affect the code responsible for saving to and reading from `.spec` files, requiring conversions between the new format and the linear float format used internally by the audio engine.
### Development Workflow:
- **Commit Policy**: Explicit rule to always run the full test suite before preparing any commit, documented in `CONTRIBUTING.md`.
- **Testing**: Comprehensive test suite including `AssetManagerTest`, `SynthEngineTest`, `HammingWindowTest`, and `SpectoolEndToEndTest`.
### Spectrogram Generation Module
* **Goal:** Introduce programmatic functions to synthesize `.spec` data for creating custom sounds and textures.
* **Core Functionality:** Implemented `NoteParams` struct and `generate_note_spectrogram_data` function.
* **Key Features:**
* Synthesizes notes with configurable `base_freq`, `duration`, `amplitude`, and `delay`.
* Supports basic ADSR-like envelope (Attack).
* Includes vibrato (rate, depth).
* Allows control over harmonic content (`num_harmonics`, `harmonic_amplitude_decay`).
* Adds subtle variations via `pitch_randomness_stddev` and `amplitude_randomness_stddev`.
* **Data Flow:**
1. `NoteParams` -> `generate_note_time_domain()` (produces time-domain samples using project constants for `SAMPLE_RATE` and `DCT_SIZE`)
2. Time-domain samples -> `generate_spectral_data_from_time_domain()` (applies window, DCT, produces spectral frames)
3. Spectral frames -> `save_spectrogram_to_spec_file()` (writes to `.spec` file).
* **Future Work:** Extend envelopes (Sustain, Decay, Release), add more waveform types, implement noise generation, advanced harmonic control, and explore `.spec` file format optimizations.
|
