diff options
| -rw-r--r-- | tools/mq_editor/README.md | 102 |
1 files changed, 64 insertions, 38 deletions
diff --git a/tools/mq_editor/README.md b/tools/mq_editor/README.md index 71ad90f..4250d33 100644 --- a/tools/mq_editor/README.md +++ b/tools/mq_editor/README.md @@ -8,57 +8,83 @@ McAulay-Quatieri sinusoidal analysis and synthesis tool. open tools/mq_editor/index.html ``` -1. Load WAV file -2. Adjust MQ parameters (FFT size, hop, threshold) -3. Click "Extract Partials" -4. View extracted sinusoidal trajectories with bezier curve fits +1. Click **Open WAV** (or **⚗ Test WAV** for a built-in 440+660 Hz test signal) +2. Click **Extract Partials** +3. Press **1** to play synthesized, **2** to play original -## Features (Phase 1) +## UI -- **MQ Extraction:** FFT → peak detection → trajectory tracking → bezier fitting -- **Visualization:** Time-frequency plot with raw trajectories and bezier overlays -- **Real-time:** Adjustable extraction parameters +- **Top bar:** title + loaded filename +- **Toolbar:** file, extraction, playback controls and parameters +- **Main view:** log-scale time-frequency spectrogram with partial trajectories +- **Right panel:** synthesis checkboxes (integrate phase, disable jitter, disable spread) +- **Mini-spectrum** (bottom-right overlay): FFT slice at mouse/playhead time + - Blue/orange gradient: original spectrum; green/yellow: synthesized + - Green bars: detected spectral peaks + - Red lines (2px): partial f0 positions at current time ## Parameters -- **FFT Size:** 1024-4096 (default 2048) -- **Hop Size:** 64-2048 samples (default 512, 75% overlap) -- **Threshold:** -80 to -20 dB (default -60 dB) +- **Hop Size:** 64–1024 samples (default 256) +- **Threshold:** dB floor for peak detection (default −60 dB) +- **Keep %:** slider to limit how many partials are shown/synthesized + +## Keyboard Shortcuts + +| Key | Action | +|-----|--------| +| `1` | Play synthesized audio | +| `2` | Play original audio | +| `P` | Toggle raw peak overlay | +| `A` | Toggle mini-spectrum: original ↔ synthesized | +| Shift+scroll | Zoom time axis | +| Scroll | Pan time axis | ## Architecture -- `index.html` - UI and integration -- `fft.js` - Fast Fourier Transform (Cooley-Tukey radix-2) -- `mq_extract.js` - MQ algorithm (peak detection, tracking, bezier fitting) -- `viewer.js` - Visualization (spectrogram, partials, zoom, mouse interaction) -- `test_fft.html` - Isolated FFT correctness tests (open in browser, no server needed) +- `index.html` — UI, playback, extraction orchestration +- `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache +- `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting +- `mq_synth.js` — Oscillator bank synthesis from extracted partials +- `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom -## Implementation Status +### viewer.js coordinate API -- [x] Phase 1: MQ extraction + visualization - - [x] Spectrogram rendering with power-law colormap (gamma=0.3) - - [x] Horizontal zoom (shift+mousewheel) and scroll (mousewheel) - - [x] Axis ticks and labels - - [x] Mouse tooltip (time/frequency/dB intensity) - - [x] Improved partial tracking (frequency-dependent threshold, candidate system) - - [x] Original WAV playback with animated playhead - - [x] Keyboard shortcuts ('1' synthesized, '2' original) - - [x] Mini-spectrum viewer (bottom-right overlay) - - [x] STFT cache for optimized FFT access -- [x] Phase 2: JS synthesizer (preview playback) - - [x] Bezier curve evaluation (cubic) - - [x] Replica oscillator bank with frequency spread and jitter - - [x] PCM synthesis from extracted partials - - [x] Playback via Web Audio API -- [ ] Phase 3: Editing UI (drag control points, replicas) -- [ ] Phase 4: Export (.txt + C++ code generation) +| Method | Description | +|--------|-------------| +| `timeToX(t)` | time → canvas X (current view) | +| `canvasToTime(x)` | canvas X → time | +| `freqToY(f)` | freq → canvas Y (log scale) | +| `canvasToFreq(y)` | canvas Y → freq | +| `freqLogNorm(f)` | freq → normalized [0..1] log position | +| `normToFreq(n)` | normalized [0..1] → freq | +| `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range | +| `partialColor(p)` | partial index → display color | ## Algorithm -1. **STFT:** Overlapping Hann windows, radix-2 Cooley-Tukey FFT (from spectral_editor/dct.js) -2. **Peak Detection:** Local maxima above threshold, parabolic interpolation -3. **Tracking:** Birth/death/continuation (50 Hz tolerance, 2-frame persistence) -4. **Bezier Fitting:** Cubic curves (4 control points), fixed endpoints +1. **STFT:** Overlapping Hann windows, radix-2 FFT +2. **Peak Detection:** Local maxima above threshold + parabolic interpolation +3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance, candidate persistence +4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames +5. **Bezier Fitting:** Cubic curves with control points at t/3 and 2t/3 + +## Implementation Status + +- [x] Phase 1: MQ extraction + visualization + - [x] Log-scale spectrogram with power-law colormap + - [x] Zoom (shift+scroll) and pan (scroll) + - [x] Axis ticks, mouse tooltip + - [x] Partial tracking with candidate system + - [x] Mini-spectrum overlay with log-scale frequency axis + - [x] Overlay cursor (separate canvas, no main redraw on mousemove) + - [x] Backward pass for onset recovery +- [x] Phase 2: Synthesis preview + - [x] Replica oscillator bank (spread, jitter, phase integration) + - [x] Synthesis debug checkboxes (disable jitter/spread) + - [x] Synthesized STFT cache for FFT comparison +- [ ] Phase 3: Editing UI (drag control points, replicas) +- [ ] Phase 4: Export (.spec + C++ code generation) ## See Also |