1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
|
# Spectral Brush Editor (Task #5)
## Concept
Replace large `.spec` assets with procedural C++ code (50-100× compression).
**Before:** 5 KB binary `.spec` file
**After:** ~100 bytes C++ code calling `draw_bezier_curve()`
**Workflow:**
```
.wav → Load in editor → Trace with Bezier curves → Export procedural_params.txt + C++ code
```
---
## Core Primitive: "Spectral Brush"
### 1. Central Curve (Bezier)
Traces time-frequency path: `{freq_bin, amplitude} = bezier(frame_number)`
**Example control points:**
```javascript
[
{frame: 0, freq_hz: 200.0, amplitude: 0.9}, // Attack
{frame: 20, freq_hz: 80.0, amplitude: 0.7}, // Sustain
{frame: 100, freq_hz: 50.0, amplitude: 0.0} // Decay
]
```
### 2. Vertical Profile
Shapes "brush stroke" around curve at each frame.
**Profile Types:**
- **Gaussian**: `exp(-(dist² / σ²))` - Musical tones, bass
- **Decaying Sinusoid**: `exp(-decay * dist) * cos(ω * dist)` - Metallic sounds
- **Noise**: `random(seed, dist) * amplitude` - Hi-hats, cymbals
- **Composite**: Combine multiple profiles (add/subtract/multiply)
---
## File Formats
### A. `procedural_params.txt` (Human-readable)
```text
METADATA dct_size=512 num_frames=100 sample_rate=32000
CURVE bezier
CONTROL_POINT 0 200.0 0.9
CONTROL_POINT 20 80.0 0.7
CONTROL_POINT 100 50.0 0.0
PROFILE gaussian sigma=30.0
END_CURVE
```
### B. C++ Code (Ready to compile)
```cpp
#include "audio/spectral_brush.h"
void gen_kick_procedural(float* spec, int dct_size, int num_frames) {
const float frames[] = {0.0f, 20.0f, 100.0f};
const float freqs[] = {200.0f, 80.0f, 50.0f};
const float amps[] = {0.9f, 0.7f, 0.0f};
draw_bezier_curve(spec, dct_size, num_frames,
frames, freqs, amps, 3,
PROFILE_GAUSSIAN, 30.0f);
}
```
---
## C++ Runtime API
### Files: `src/audio/spectral_brush.{h,cc}`
**Key functions:**
```cpp
enum ProfileType {
PROFILE_GAUSSIAN,
PROFILE_DECAYING_SINUSOID,
PROFILE_NOISE
};
void draw_bezier_curve(float* spectrogram, int dct_size, int num_frames,
const float* control_frames,
const float* control_freqs_hz,
const float* control_amps,
int n_control_points,
ProfileType profile_type,
float profile_param1,
float profile_param2 = 0.0f);
float evaluate_bezier_linear(const float* control_frames,
const float* control_values,
int n_points,
float frame);
float evaluate_profile(ProfileType type, float distance,
float param1, float param2);
```
---
## Editor UI
### Technology Stack
- HTML5 Canvas (visualization)
- Web Audio API (playback)
- Pure JavaScript (no dependencies)
- Reuse `dct.js` from existing editor
### Key Features
- Dual-layer canvas (reference + procedural spectrograms)
- Drag control points to adjust curves
- Real-time spectrogram rendering
- Audio playback (keys 1/2 for procedural/original)
- Undo/Redo (Ctrl+Z, Ctrl+Shift+Z)
- Load .wav/.spec, save params, generate C++ code
### Keyboard Shortcuts
| Key | Action |
|-----|--------|
| 1 | Play procedural |
| 2 | Play original |
| Space | Play/pause |
| Delete | Remove control point |
| Ctrl+Z | Undo |
| Ctrl+S | Save params |
---
## Implementation Phases
### Phase 1: C++ Runtime
**Files:** `src/audio/spectral_brush.{h,cc}`, `src/tests/test_spectral_brush.cc`
**Tasks:**
- Define API (ProfileType, draw_bezier_curve, evaluate_profile)
- Implement linear Bezier interpolation
- Implement Gaussian profile
- Add unit tests
- **Deliverable:** Compiles, tests pass
### Phase 2: Editor Core
**Files:** `tools/spectral_editor/{index.html, script.js, style.css, dct.js}`
**Tasks:**
- HTML structure (canvas, controls, file input)
- Bezier curve editor (place/drag/delete control points)
- Dual-layer canvas rendering
- Real-time spectrogram generation
- Audio playback (IDCT → Web Audio API)
- Undo/Redo system
- **Deliverable:** Interactive editor, can trace .wav files
### Phase 3: File I/O
**Tasks:**
- Load .wav (decode, STFT → spectrogram)
- Load .spec (binary parser)
- Save procedural_params.txt (text format)
- Generate C++ code (template)
- Load procedural_params.txt (re-editing)
- **Deliverable:** Full save/load cycle
---
## Design Decisions
- **Bezier:** Linear interpolation (Phase 1), cubic later
- **Profiles:** Gaussian only (Phase 1), others later
- **Parameters:** Soft UI limits, no enforced bounds
- **RNG:** Home-brew deterministic (small, repeatable)
- **Code gen:** Single function per sound (generic loader later)
---
## Size Impact
**Example: Kick drum**
**Before (Binary):**
- 512 bins × 100 frames × 4 bytes = 200 KB uncompressed
- ~5 KB compressed (zlib)
**After (Procedural):**
- 4 control points × 3 arrays × 4 floats = ~48 bytes data
- Function call overhead = ~20 bytes
- **Total: ~100 bytes** (50-100× reduction)
**Trade-off:** Runtime CPU cost, acceptable for 64k demo.
---
*See TODO.md for detailed implementation tasks.*
|
