1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
|
# MQ Spectral Editor
McAulay-Quatieri sinusoidal analysis and synthesis tool.
## Usage
```bash
open tools/mq_editor/index.html
```
1. Click **Open WAV** (or **⚗ Test WAV** for a built-in 440+660 Hz test signal)
2. Click **Extract Partials**
3. Press **1** to play synthesized, **2** to play original
## UI
- **Top bar:** title + loaded filename
- **Toolbar:** file, extraction, playback controls and parameters
- **Main view:** log-scale time-frequency spectrogram with partial trajectories
- **Right panel:** synthesis checkboxes (integrate phase, disable jitter, disable spread)
- **Mini-spectrum** (bottom-right overlay): FFT slice at mouse/playhead time
- Blue/orange gradient: original spectrum; green/yellow: synthesized
- Green bars: detected spectral peaks
- Red lines (2px): partial f0 positions at current time
## Parameters
- **Hop Size:** 64–1024 samples (default 256)
- **Threshold:** dB floor for peak detection (default −60 dB)
- **f·Power:** checkbox — weight spectrum by frequency (`f·FFT_Power(f)`) before peak detection, accentuating high-frequency peaks
- **Keep %:** slider to limit how many partials are shown/synthesized
## Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `1` | Play synthesized audio |
| `2` | Play original audio |
| `E` | Extract Partials |
| `P` | Toggle raw peak overlay |
| `A` | Toggle mini-spectrum: original ↔ synthesized |
| `Esc` | Deselect partial |
| Shift+scroll | Zoom time axis |
| Scroll | Pan time axis |
## Architecture
- `index.html` — UI, playback, extraction orchestration, keyboard shortcuts
- `editor.js` — Property panel and amplitude bezier editor for selected partials
- `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache
- `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting
- `mq_synth.js` — Oscillator bank synthesis from extracted partials
- `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom
### viewer.js coordinate API
| Method | Description |
|--------|-------------|
| `timeToX(t)` | time → canvas X (current view) |
| `canvasToTime(x)` | canvas X → time |
| `freqToY(f)` | freq → canvas Y (log scale) |
| `canvasToFreq(y)` | canvas Y → freq |
| `freqLogNorm(f)` | freq → normalized [0..1] log position |
| `normToFreq(n)` | normalized [0..1] → freq |
| `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range |
| `partialColor(p)` | partial index → display color |
## Algorithm
1. **STFT:** Overlapping Hann windows, radix-2 FFT
2. **Peak Detection:** Local maxima above threshold + parabolic interpolation; optional `f·Power(f)` frequency weighting to accentuate high-frequency peaks
3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance, candidate persistence
4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames
5. **Bezier Fitting:** Cubic curves with control points at t/3 and 2t/3
## Implementation Status
- [x] Phase 1: MQ extraction + visualization
- [x] Log-scale spectrogram with power-law colormap
- [x] Zoom (shift+scroll) and pan (scroll)
- [x] Axis ticks, mouse tooltip
- [x] Partial tracking with candidate system
- [x] Mini-spectrum overlay with log-scale frequency axis
- [x] Overlay cursor (separate canvas, no main redraw on mousemove)
- [x] Backward pass for onset recovery
- [x] Phase 2: Synthesis preview
- [x] Replica oscillator bank (spread, jitter, phase integration)
- [x] Synthesis debug checkboxes (disable jitter/spread)
- [x] Synthesized STFT cache for FFT comparison
- [x] Phase 3: Editing UI
- [x] Partial selection with property panel (freq/amp/synth tabs)
- [x] Amplitude bezier drag editor
- [x] Synth params with jog sliders (decay, jitter, spread)
- [x] Auto-spread detection per partial and global
- [x] Mute / delete partials
- [ ] Phase 4: Export (.spec + C++ code generation)
## See Also
- Design doc: `doc/SPECTRAL_BRUSH_2.md`
|
