1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
|
# MQ Spectral Editor
McAulay-Quatieri sinusoidal analysis and synthesis tool.
## Usage
```bash
open tools/mq_editor/index.html
```
1. Click **Open WAV** (or **⚗ Test WAV** for a built-in 440+660 Hz test signal)
2. Click **Extract Partials**
3. Press **1** to play synthesized, **2** to play original
## UI
- **Top bar:** title + loaded filename
- **Toolbar:** file, extraction, playback controls and parameters
- **Main view:** log-scale time-frequency spectrogram with partial trajectories
- **Right panel:** per-partial mode toggle (Sinusoid / Resonator), synth params, global checkboxes
- **Mini-spectrum** (bottom-right overlay): FFT slice at mouse/playhead time
- Blue/orange gradient: original spectrum; green/yellow: synthesized
- Green bars: detected spectral peaks
- Red lines (2px): partial f0 positions at current time
## Parameters
- **Hop Size:** 64–1024 samples (default 256)
- **Threshold:** dB floor for peak detection (default −60 dB)
- **f·Power:** checkbox — weight spectrum by frequency (`f·FFT_Power(f)`) before peak detection, accentuating high-frequency peaks
- **Keep %:** slider to limit how many partials are shown/synthesized
## Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `1` | Play synthesized audio |
| `2` | Play original audio |
| `E` | Extract Partials |
| `P` | Toggle raw peak overlay |
| `A` | Toggle mini-spectrum: original ↔ synthesized |
| `Esc` | Deselect partial |
| Shift+scroll | Zoom time axis |
| Scroll | Pan time axis |
## Architecture
- `index.html` — UI, playback, extraction orchestration, keyboard shortcuts
- `editor.js` — Property panel and amplitude bezier editor for selected partials
- `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache
- `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting
- `mq_synth.js` — Oscillator bank + two-pole resonator synthesis from extracted partials
- `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom
### viewer.js coordinate API
| Method | Description |
|--------|-------------|
| `timeToX(t)` | time → canvas X (current view) |
| `canvasToTime(x)` | canvas X → time |
| `freqToY(f)` | freq → canvas Y (log scale) |
| `canvasToFreq(y)` | canvas Y → freq |
| `freqLogNorm(f)` | freq → normalized [0..1] log position |
| `normToFreq(n)` | normalized [0..1] → freq |
| `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range |
| `partialColor(p)` | partial index → display color |
## Resonator Synthesis Mode
Each partial has a **per-partial synthesis mode** selectable in the **Synth** tab:
### Sinusoid (default)
Replica oscillator bank — direct additive sinusoids with optional spread/jitter/decay shaping.
### Resonator
Two-pole IIR bandpass resonator driven by band-limited noise:
```
y[n] = 2r·cos(ω₀)·y[n-1] − r²·y[n-2] + A(t)·√(1−r²)·noise[n]
```
- **ω₀** = `2π·f₀(t)/SR` — recomputed each sample from the freq Bezier curve (handles glides/vibrato)
- **A(t)** — amp Bezier curve scales excitation continuously
- **√(1−r²)** — power normalization, keeps output level ≈ sinusoidal mode at `gainComp = 1`
- **noise[n]** — deterministic per-partial LCG (reproducible renders)
**Parameters:**
| Param | Default | Range | Meaning |
|-------|---------|-------|---------|
| `r (pole)` | 0.995 | [0, 0.9999] | Pole radius. r→1 = narrow BW / long ring. r→0 = wide / fast decay. |
| `gain` | 1.0 | [0, ∞) | Output multiplier on top of power normalization. |
**Coefficient translation from spread:**
`r = exp(−π · BW / SR)` where `BW = f₀ · (spread_above + spread_below) / 2`.
For a partial at 440 Hz with `spread = 0.02`: `BW ≈ 8.8 Hz`, `r ≈ exp(−π·8.8/32000) ≈ 0.9991`.
**When to use:**
- Metallic / percussive partials with natural exponential decay
- Wide spectral peaks (large spread) where a bandpass filter is more physically accurate
- Comparing resonator vs. sinusoidal timbre on the same partial
**Note:** `RES` badge appears in the panel header when a partial is in resonator mode.
---
## Algorithm
1. **STFT:** Overlapping Hann windows, radix-2 FFT
2. **Peak Detection:** Local maxima above threshold + parabolic interpolation; optional `f·Power(f)` frequency weighting to accentuate high-frequency peaks
3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance, candidate persistence
4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames
5. **Bezier Fitting:** Cubic curves with control points at t/3 and 2t/3
## Implementation Status
- [x] Phase 1: MQ extraction + visualization
- [x] Log-scale spectrogram with power-law colormap
- [x] Zoom (shift+scroll) and pan (scroll)
- [x] Axis ticks, mouse tooltip
- [x] Partial tracking with candidate system
- [x] Mini-spectrum overlay with log-scale frequency axis
- [x] Overlay cursor (separate canvas, no main redraw on mousemove)
- [x] Backward pass for onset recovery
- [x] Phase 2: Synthesis preview
- [x] Replica oscillator bank (spread, jitter, phase integration)
- [x] Synthesis debug checkboxes (disable jitter/spread)
- [x] Synthesized STFT cache for FFT comparison
- [x] Phase 3: Editing UI
- [x] Partial selection with property panel (freq/amp/synth tabs)
- [x] Amplitude bezier drag editor
- [x] Synth params with jog sliders (decay, jitter, spread)
- [x] Auto-spread detection per partial and global
- [x] Mute / delete partials
- [x] Per-partial resonator synthesis mode (Synth tab toggle)
- [ ] Phase 4: Export (.spec + C++ code generation)
## See Also
- Design doc: `doc/SPECTRAL_BRUSH_2.md`
|
