1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
|
# MQ Spectral Editor
McAulay-Quatieri sinusoidal analysis and synthesis tool.
## Usage
```bash
open tools/mq_editor/index.html
```
1. Click **Open WAV** (or **⚗ Test WAV** for a built-in 440+660 Hz test signal)
2. Click **Extract Partials**
3. Press **1** to play synthesized, **2** to play original
## UI
- **Top bar:** title + loaded filename
- **Toolbar:** file, extraction, playback controls and parameters
- **Main view:** log-scale time-frequency spectrogram with partial trajectories
- **Right panel:** per-partial mode toggle (Sinusoid / Resonator), synth params, global checkboxes
- **Mini-spectrum** (bottom-right overlay): FFT slice at mouse/playhead time
- Blue/orange gradient: original spectrum; green/yellow: synthesized
- Green bars: detected spectral peaks
- Red lines (2px): partial f0 positions at current time
## Parameters
Parameters are grouped into four sections in the toolbar.
### STFT
| Param | Default | Description |
|-------|---------|-------------|
| **Hop** | 256 | STFT hop size in samples. Smaller = finer time resolution, more frames, slower. |
### Peak Detection
| Param | Default | Description |
|-------|---------|-------------|
| **Threshold (dB)** | −20 | Minimum spectral peak amplitude. Peaks below this are ignored. |
| **Prominence (dB)** | 1.0 | How much a peak must rise above its surrounding valley. Suppresses weak shoulders. |
| **f·Power** | off | Weight spectrum by `f × Power(f)` before detection. Boosts high-frequency peaks relative to low-frequency ones. |
### Tracking
| Param | Default | Description |
|-------|---------|-------------|
| **Birth** | 3 | Frames a candidate must persist before becoming a partial. Higher = fewer spurious bursts. |
| **Death** | 5 | Frames a partial can go unmatched before termination. Higher = bridges short gaps. |
| **Phase Wt** | 2.0 | Weight of phase prediction error in the peak-matching cost function. Higher = stricter phase coherence. |
| **Min Len** | 10 | Minimum frame count for a partial to survive after tracking. Discards very short partials. |
### Filter
| Param | Default | Description |
|-------|---------|-------------|
| **Keep %** | 100% | Retain only the strongest N% of extracted partials by peak amplitude. |
## Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `1` | Play synthesized audio |
| `2` | Play original audio |
| `E` | Extract Partials |
| `P` | Toggle raw peak overlay |
| `A` | Toggle mini-spectrum: original ↔ synthesized |
| `Esc` | Deselect partial |
| Shift+scroll | Zoom time axis |
| Scroll | Pan time axis |
## Architecture
- `index.html` — UI, playback, extraction orchestration, keyboard shortcuts
- `editor.js` — Property panel and amplitude bezier editor for selected partials
- `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache
- `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting
- `mq_synth.js` — Oscillator bank + two-pole resonator synthesis from extracted partials
- `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom
### viewer.js coordinate API
| Method | Description |
|--------|-------------|
| `timeToX(t)` | time → canvas X (current view) |
| `canvasToTime(x)` | canvas X → time |
| `freqToY(f)` | freq → canvas Y (log scale) |
| `canvasToFreq(y)` | canvas Y → freq |
| `freqLogNorm(f)` | freq → normalized [0..1] log position |
| `normToFreq(n)` | normalized [0..1] → freq |
| `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range |
| `partialColor(p)` | partial index → display color |
## Post-Synthesis Filters
Global LP/HP filters applied after the oscillator bank, before normalization.
Sliders in the **Synthesis** panel; display shows -3 dB cutoff frequency.
| Control | Filter | Formula | Bypass |
|---------|--------|---------|--------|
| **LP k1** | `y[n] = k1·x[n] + (1−k1)·y[n−1]` | `-3dB: cos(ω) = (2−2k−k²)/(2(1−k))` | k1 = 1.0 |
| **HP k2** | `y[n] = k2·(y[n−1] + x[n] − x[n−1])` | `-3dB from peak: cos(ω) = 2k/(1+k²)` | k2 = 1.0 |
Both default to 1.0 (bypass). Frequency display uses `audioBuffer.sampleRate` when loaded, falls back to 44100 Hz.
---
## Resonator Synthesis Mode
Each partial has a **per-partial synthesis mode** selectable in the **Synth** tab:
### Sinusoid (default)
Replica oscillator bank — direct additive sinusoids with optional spread/jitter/decay shaping.
### Resonator
Two-pole IIR bandpass resonator driven by band-limited noise:
```
y[n] = 2r·cos(ω₀)·y[n-1] − r²·y[n-2] + A(t)·√(1−r²)·noise[n]
```
- **ω₀** = `2π·f₀(t)/SR` — recomputed each sample from the freq Bezier curve (handles glides/vibrato)
- **A(t)** — amp Bezier curve scales excitation continuously
- **√(1−r²)** — power normalization, keeps output level ≈ sinusoidal mode at `gainComp = 1`
- **noise[n]** — deterministic per-partial LCG (reproducible renders)
**Parameters:**
| Param | Default | Range | Meaning |
|-------|---------|-------|---------|
| `r (pole)` | 0.995 | [0, 0.9999] | Pole radius. r→1 = narrow BW / long ring. r→0 = wide / fast decay. |
| `gain` | 1.0 | [0, ∞) | Output multiplier on top of power normalization. |
**Coefficient translation from spread:**
`r = exp(−π · BW / SR)` where `BW = f₀ · (spread_above + spread_below) / 2`.
For a partial at 440 Hz with `spread = 0.02`: `BW ≈ 8.8 Hz`, `r ≈ exp(−π·8.8/32000) ≈ 0.9991`.
**When to use:**
- Metallic / percussive partials with natural exponential decay
- Wide spectral peaks (large spread) where a bandpass filter is more physically accurate
- Comparing resonator vs. sinusoidal timbre on the same partial
**Note:** `RES` badge appears in the panel header when a partial is in resonator mode.
---
## Algorithm
1. **STFT:** Overlapping Hann windows, radix-2 FFT
2. **Peak Detection:** Local maxima above threshold + parabolic interpolation. Includes **Prominence Filtering** (rejects peaks not significantly higher than surroundings). Optional `f·Power(f)` weighting.
3. **Forward Tracking:** Phase-coherent birth/death/continuation. Configurable `Birth`, `Death`, `Phase Wt`, and `Min Len`. Uses velocity prediction and phase advance to resolve ambiguous peak matches.
4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames
5. **Bezier Fitting:** Cubic curves optimized via **Least-Squares** (minimizes error across all points).
## Implementation Status
- [x] Phase 1: MQ extraction + visualization
- [x] Log-scale spectrogram with power-law colormap
- [x] Zoom (shift+scroll) and pan (scroll)
- [x] Axis ticks, mouse tooltip
- [x] Partial tracking with candidate system
- [x] Mini-spectrum overlay with log-scale frequency axis
- [x] Overlay cursor (separate canvas, no main redraw on mousemove)
- [x] Backward pass for onset recovery
- [x] Phase 2: Synthesis preview
- [x] Replica oscillator bank (spread, jitter, phase integration)
- [x] Synthesis debug checkboxes (disable jitter/spread)
- [x] Synthesized STFT cache for FFT comparison
- [x] Phase 3: Editing UI
- [x] Partial selection with property panel (freq/amp/synth tabs)
- [x] Amplitude bezier drag editor
- [x] Synth params with jog sliders (decay, jitter, spread)
- [x] Auto-spread detection per partial and global
- [x] Mute / delete partials
- [x] Per-partial resonator synthesis mode (Synth tab toggle)
- [x] Global LP/HP post-synthesis filter sliders with Hz display
- [ ] Phase 4: Export (.spec + C++ code generation)
## See Also
- Design doc: `doc/SPECTRAL_BRUSH_2.md`
|
