diff options
Diffstat (limited to 'tools')
| -rw-r--r-- | tools/mq_editor/PHASE_TRACKING_PLAN.md | 73 | ||||
| -rw-r--r-- | tools/mq_editor/README.md | 101 | ||||
| -rw-r--r-- | tools/mq_editor/app.js | 565 | ||||
| -rw-r--r-- | tools/mq_editor/editor.js | 165 | ||||
| -rw-r--r-- | tools/mq_editor/fft.js | 6 | ||||
| -rw-r--r-- | tools/mq_editor/index.html | 832 | ||||
| -rw-r--r-- | tools/mq_editor/mq_extract.js | 360 | ||||
| -rw-r--r-- | tools/mq_editor/mq_synth.js | 137 | ||||
| -rw-r--r-- | tools/mq_editor/style.css | 104 | ||||
| -rw-r--r-- | tools/mq_editor/test_fft.html | 229 | ||||
| -rw-r--r-- | tools/mq_editor/utils.js | 83 | ||||
| -rw-r--r-- | tools/mq_editor/viewer.js | 366 |
12 files changed, 1774 insertions, 1247 deletions
diff --git a/tools/mq_editor/PHASE_TRACKING_PLAN.md b/tools/mq_editor/PHASE_TRACKING_PLAN.md new file mode 100644 index 0000000..5111692 --- /dev/null +++ b/tools/mq_editor/PHASE_TRACKING_PLAN.md @@ -0,0 +1,73 @@ +# Implementation Plan: Phase-Coherent Partial Tracking + +This document outlines the plan to integrate phase prediction into the existing MQ tracking algorithm. The core idea is to use phase coherence as a primary factor for linking peaks across frames, making the tracking more robust, especially for crossing or closely-spaced partials. + +--- + +### **Stage 1: Cache Per-Bin Phase in `fft.js`** + +**Objective:** Augment the `STFTCache` to compute and store the phase angle for every frequency bin in every frame, making it available for the tracking algorithm. + +1. **Locate the FFT Processing Loop:** + * In `tools/mq_editor/fft.js`, within the `STFTCache` class (likely in the constructor or an initialization method), find the loop that iterates through each frame to compute the FFT. + * This is where `squaredAmplitude` is currently being calculated from the `real` and `imag` components. + +2. **Compute and Store Phase:** + * In the same loop, immediately after calculating the squared amplitude, calculate the phase for each bin. + * Create a new `Float32Array` for phase, let's call it `ph`. + * Inside the bin loop, compute: `ph[k] = Math.atan2(imag[k], real[k]);` + * Store this new phase array in the frame object, parallel to the existing `squaredAmplitude` array. + + **Resulting Change in `fft.js`:** + + ```javascript + // Inside the STFTCache frame processing loop... + + // Existing code: + const sq = new Float32Array(this.fftSize / 2); + for (let k = 0; k < this.fftSize / 2; k++) { + sq[k] = real[k] * real[k] + imag[k] * imag[k]; + } + + // New code to add: + const ph = new Float32Array(this.fftSize / 2); + for (let k = 0; k < this.fftSize / 2; k++) { + ph[k] = Math.atan2(imag[k], real[k]); + } + + // Update the stored frame object + this.frames.push({ + time: t, + squaredAmplitude: sq, + phase: ph // The newly cached phase data + }); + ``` + +--- + +### **Stage 2: Utilize Phase for Tracking in `mq_extract.js`** + +**Objective:** Modify the main forward/backward tracking algorithm to use phase coherence for identifying and linking peaks. + +1. **Extract Interpolated Peak Phase:** + * In `tools/mq_editor/mq_extract.js`, find the function responsible for peak detection within a single frame (e.g., `findPeaks`). + * This function currently takes a `squaredAmplitude` array. It must now also access the corresponding `phase` array from the cached frame data. + * When a peak is found at bin `k`, use the same parabolic interpolation logic that calculates the true frequency and amplitude to also calculate the **true phase**. This involves interpolating the phase values from bins `k-1`, `k`, and `k+1`. + * **Crucially, this interpolation must handle phase wrapping.** A helper function will be needed to correctly find the shortest angular distance between phase values. + +2. **Update Tracking Data Structures:** + * The data structures holding candidate and live partials must be updated to store the phase of each point in the trajectory, not just frequency and amplitude. + +3. **Implement Phase Prediction Logic:** + * In the main tracking loop that steps from frame `n` to `n+1`: + * For each active partial, calculate its `predictedPhase` for frame `n+1`. + * `phase_delta = (2 * Math.PI * last_freq * params.hopSize) / params.sampleRate;` + * `predictedPhase = last_phase + phase_delta;` + +4. **Refine the Candidate Matching Score:** + * Modify the logic that links a partial to peaks in the next frame. + * Instead of matching based on frequency proximity alone, calculate a `cost` based on both frequency and phase deviation: + * `freqError = Math.abs(peak.freq - partial.last_freq);` + * `phaseError = Math.abs(normalize_angle(peak.phase - predictedPhase));` // Difference on a circle + * `cost = (freq_weight * freqError) + (phase_weight * phaseError);` + * The peak with the lowest `cost` below a certain threshold is the correct continuation. The `phase_weight` should be high, as a low phase error is a strong indicator of a correct match. diff --git a/tools/mq_editor/README.md b/tools/mq_editor/README.md index c1f2732..7d7d06b 100644 --- a/tools/mq_editor/README.md +++ b/tools/mq_editor/README.md @@ -8,9 +8,9 @@ McAulay-Quatieri sinusoidal analysis and synthesis tool. open tools/mq_editor/index.html ``` -1. Click **Open WAV** (or **âš— Test WAV** for a built-in 440+660 Hz test signal) -2. Click **Extract Partials** -3. Press **1** to play synthesized, **2** to play original +1. Click **Open WAV** +2. Click **Extract Partials** (optional — Explore/Contour modes work immediately after load) +3. Press **1** to play original, **2** to play synthesized, **3** to play current partial ## UI @@ -25,32 +25,92 @@ open tools/mq_editor/index.html ## Parameters -- **Hop Size:** 64–1024 samples (default 256) -- **Threshold:** dB floor for peak detection (default −60 dB) -- **Prominence:** Min dB height of a peak above its surrounding "valley floor" (default 1.0 dB). Filters out insignificant local maxima. -- **f·Power:** checkbox — weight spectrum by frequency (`f·FFT_Power(f)`) before peak detection, accentuating high-frequency peaks -- **Keep %:** slider to limit how many partials are shown/synthesized +Parameters are grouped into four sections in the toolbar. + +### STFT +| Param | Default | Description | +|-------|---------|-------------| +| **Hop** | 256 | STFT hop size in samples. Smaller = finer time resolution, more frames, slower. | + +### Peak Detection +| Param | Default | Description | +|-------|---------|-------------| +| **Threshold (dB)** | −20 | Minimum spectral peak amplitude. Peaks below this are ignored. | +| **Prominence (dB)** | 1.0 | How much a peak must rise above its surrounding valley. Suppresses weak shoulders. | +| **f·Power** | off | Weight spectrum by `f × Power(f)` before detection. Boosts high-frequency peaks relative to low-frequency ones. | + +### Tracking +| Param | Default | Description | +|-------|---------|-------------| +| **Birth** | 3 | Frames a candidate must persist before becoming a partial. Higher = fewer spurious bursts. | +| **Death** | 5 | Frames a partial can go unmatched before termination. Higher = bridges short gaps. | +| **Phase Wt** | 2.0 | Weight of phase prediction error in the peak-matching cost function. Higher = stricter phase coherence. | +| **Min Len** | 10 | Minimum frame count for a partial to survive after tracking. Discards very short partials. | + +### Filter +| Param | Default | Description | +|-------|---------|-------------| +| **Keep %** | 100% | Retain only the strongest N% of extracted partials by peak amplitude. | + +## Interactive Partial Creation + +Three ways to add partials beyond automatic extraction: + +### ⊕ Explore — Peak Tracking (key: `X`) + +Hover the mouse over any spectral peak in the spectrogram. The tracker +snaps to the nearest detected peak and runs MQ tracking forward and +backward from the cursor position, showing an **orange dashed preview**. +Click to commit the partial. + +- Works immediately after WAV load (no extraction required) +- Uses **Birth / Death / Phase Wt** params from the toolbar +- Good for: strong peaks missed by global extraction, fine-grained control + +### ≋ Contour — Iso-Energy Tracking (key: `C`) + +Hover over any region of the spectrogram. The tracker follows the +iso-energy contour at the energy level under the cursor (`Eâ‚€ = mag(t, f)`), +frame by frame, without requiring a spectral peak. Preview shown in **cyan**. +Click to commit. + +- Designed for **broad, diffuse bass regions** where no peaks are detected +- Search window: ±15% in frequency; declares a miss if nearest bin deviates >15 dB +- Spread is auto-detected on commit (naturally comes out large for broad regions) +- Good for: bass smear, noisy resonances, sub-threshold energy + +Both modes are mutually exclusive. `Escape` exits either mode. +Committed partials are prepended to the partial list with full undo support. ## Keyboard Shortcuts | Key | Action | |-----|--------| -| `1` | Play synthesized audio | -| `2` | Play original audio | +| `1` | Play original audio | +| `2` | Play synthesized audio | +| `3` | Play selected partial alone | | `E` | Extract Partials | +| `N` | New partial (flat 440 Hz, full duration) | +| `X` | Toggle ⊕ Explore mode (peak tracking) | +| `C` | Toggle ≋ Contour mode (iso-energy tracking) | | `P` | Toggle raw peak overlay | | `A` | Toggle mini-spectrum: original ↔ synthesized | -| `Esc` | Deselect partial | +| `Delete` | Delete selected partial | +| `Esc` | Exit explore/contour mode · deselect partial | +| `Ctrl+Z` | Undo | +| `Ctrl+Y` / `Ctrl+Shift+Z` | Redo | | Shift+scroll | Zoom time axis | | Scroll | Pan time axis | ## Architecture - `index.html` — UI, playback, extraction orchestration, keyboard shortcuts +- `style.css` — All UI styles (extracted from inline styles) - `editor.js` — Property panel and amplitude bezier editor for selected partials - `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache - `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting - `mq_synth.js` — Oscillator bank + two-pole resonator synthesis from extracted partials +- `utils.js` — Shared helpers: `evalBezier`, `evalBezierAmp`, `clamp`, canvas coordinate utilities - `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom ### viewer.js coordinate API @@ -66,6 +126,19 @@ open tools/mq_editor/index.html | `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range | | `partialColor(p)` | partial index → display color | +## Amplitude Bezier Editor + +The bottom panel shows the amplitude envelope for the selected partial as a **Lagrange interpolating curve** through four control points P0–P3. + +- **P0, P3** (endpoints): drag vertically to adjust amplitude at the start/end of the partial. Time positions are fixed to `t_start`/`t_end`. +- **P1, P2** (inner points): drag in **both axes** — vertically for amplitude, horizontally for time position. The ordering constraint `t0 < t1 < t2 < t3` is enforced automatically. + +Cursor hints: `ns-resize` for P0/P3, `move` for P1/P2. + +All four time and amplitude values are also editable as numbers in the property panel above. + +--- + ## Post-Synthesis Filters Global LP/HP filters applied after the oscillator bank, before normalization. @@ -123,7 +196,7 @@ For a partial at 440 Hz with `spread = 0.02`: `BW ≈ 8.8 Hz`, `r ≈ exp(−π 1. **STFT:** Overlapping Hann windows, radix-2 FFT 2. **Peak Detection:** Local maxima above threshold + parabolic interpolation. Includes **Prominence Filtering** (rejects peaks not significantly higher than surroundings). Optional `f·Power(f)` weighting. -3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance. Includes **Predictive Kinematic Tracking** (uses velocity to track rapidly moving partials). +3. **Forward Tracking:** Phase-coherent birth/death/continuation. Configurable `Birth`, `Death`, `Phase Wt`, and `Min Len`. Uses velocity prediction and phase advance to resolve ambiguous peak matches. 4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames 5. **Bezier Fitting:** Cubic curves optimized via **Least-Squares** (minimizes error across all points). @@ -143,12 +216,14 @@ For a partial at 440 Hz with `spread = 0.02`: `BW ≈ 8.8 Hz`, `r ≈ exp(−π - [x] Synthesized STFT cache for FFT comparison - [x] Phase 3: Editing UI - [x] Partial selection with property panel (freq/amp/synth tabs) - - [x] Amplitude bezier drag editor + - [x] Amplitude bezier drag editor (P1/P2 horizontally movable, ordering constrained) - [x] Synth params with jog sliders (decay, jitter, spread) - [x] Auto-spread detection per partial and global - [x] Mute / delete partials - [x] Per-partial resonator synthesis mode (Synth tab toggle) - [x] Global LP/HP post-synthesis filter sliders with Hz display + - [x] Explore mode: interactive peak tracking from mouse position (`X`) + - [x] Contour mode: iso-energy tracking for bass/diffuse regions (`C`) - [ ] Phase 4: Export (.spec + C++ code generation) ## See Also diff --git a/tools/mq_editor/app.js b/tools/mq_editor/app.js new file mode 100644 index 0000000..9df00fb --- /dev/null +++ b/tools/mq_editor/app.js @@ -0,0 +1,565 @@ +// MQ Editor — application glue (extracted from index.html) + +// LP: y[n] = k*x[n] + (1-k)*y[n-1] => -3dB at cos(w) = (2-2k-k²)/(2(1-k)) +function k1ToHz(k, sr) { + if (k >= 1.0) return sr / 2; + const cosW = (2 - 2*k - k*k) / (2*(1 - k)); + return Math.acos(clamp(cosW, -1, 1)) * sr / (2 * Math.PI); +} +// HP: y[n] = k*(y[n-1]+x[n]-x[n-1]) => -3dB from peak at cos(w) = 2k/(1+k²) +function k2ToHz(k, sr) { + if (k >= 1.0) return 0; + const cosW = 2*k / (1 + k*k); + return Math.acos(clamp(cosW, -1, 1)) * sr / (2 * Math.PI); +} +function fmtHz(f) { + return f >= 1000 ? (f/1000).toFixed(1) + 'k' : Math.round(f) + 'Hz'; +} +function getSR() { return (typeof audioBuffer !== 'undefined' && audioBuffer) ? audioBuffer.sampleRate : 44100; } + +// Params dropdown toggle +(function() { + const btn = document.getElementById('paramsBtn'); + const panel = document.getElementById('paramsPanel'); + btn.addEventListener('click', (e) => { + e.stopPropagation(); + const open = panel.classList.toggle('open'); + btn.classList.toggle('params-open', open); + }); + document.addEventListener('click', (e) => { + if (!panel.contains(e.target) && e.target !== btn) { + panel.classList.remove('open'); + btn.classList.remove('params-open'); + } + }); +})(); + +// LP/HP slider live display +document.getElementById('lpK1').addEventListener('input', function() { + const k = parseFloat(this.value); + const f = k1ToHz(k, getSR()); + document.getElementById('lpK1Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); +}); +document.getElementById('hpK2').addEventListener('input', function() { + const k = parseFloat(this.value); + const f = k2ToHz(k, getSR()); + document.getElementById('hpK2Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); +}); + +// Show/hide global resonator params when forceResonator toggled +document.getElementById('forceResonator').addEventListener('change', function() { + document.getElementById('globalResParams').style.display = this.checked ? '' : 'none'; +}); +document.getElementById('globalR').addEventListener('input', function() { + document.getElementById('globalRVal').textContent = parseFloat(this.value).toFixed(4); +}); +document.getElementById('globalGain').addEventListener('input', function() { + document.getElementById('globalGainVal').textContent = parseFloat(this.value).toFixed(2); +}); +let audioBuffer = null; +let viewer = null; +let audioContext = null; +let currentSource = null; +let extractedPartials = null; +let stftCache = null; +let exploreMode = false; // false | 'peak' | 'contour' + +function setExploreMode(mode) { // false | 'peak' | 'contour' + exploreMode = mode; + document.getElementById('exploreBtn').classList.toggle('explore-active', mode === 'peak'); + document.getElementById('contourBtn').classList.toggle('contour-active', mode === 'contour'); + if (viewer) viewer.setExploreMode(mode); +} + +// Undo/redo +const undoStack = []; +const redoStack = []; + +function _updateUndoRedoBtns() { + document.getElementById('undoBtn').disabled = undoStack.length === 0; + document.getElementById('redoBtn').disabled = redoStack.length === 0; +} + +function pushUndo() { + undoStack.push(JSON.parse(JSON.stringify(extractedPartials || []))); + redoStack.length = 0; + if (undoStack.length > 50) undoStack.shift(); + _updateUndoRedoBtns(); +} + +function refreshPartialsView(selectIdx = -1) { + editor.setPartials(extractedPartials); + if (viewer) { + viewer.setPartials(extractedPartials); + viewer.setKeepCount(extractedPartials && extractedPartials.length > 0 ? getKeepCount() : 0); + viewer.selectPartial(selectIdx); + } +} + +function _applySnapshot(snap) { + extractedPartials = snap; + refreshPartialsView(); + _updateUndoRedoBtns(); +} + +function undo() { + if (!undoStack.length) return; + redoStack.push(JSON.parse(JSON.stringify(extractedPartials || []))); + _applySnapshot(undoStack.pop()); +} + +function redo() { + if (!redoStack.length) return; + undoStack.push(JSON.parse(JSON.stringify(extractedPartials || []))); + _applySnapshot(redoStack.pop()); +} + +const wavFile = document.getElementById('wavFile'); +const chooseFileBtn = document.getElementById('chooseFileBtn'); +const extractBtn = document.getElementById('extractBtn'); +const autoSpreadAllBtn = document.getElementById('autoSpreadAllBtn'); +const playBtn = document.getElementById('playBtn'); +const stopBtn = document.getElementById('stopBtn'); +const canvas = document.getElementById('canvas'); +const status = document.getElementById('status'); +const fileLabel = document.getElementById('fileLabel'); + +const hopSize = document.getElementById('hopSize'); +const threshold = document.getElementById('threshold'); +const prominence = document.getElementById('prominence'); +const freqWeightCb = document.getElementById('freqWeight'); +const birthPersistenceEl = document.getElementById('birthPersistence'); +const deathAgeEl = document.getElementById('deathAge'); +const phaseErrorWeightEl = document.getElementById('phaseErrorWeight'); +const minLengthEl = document.getElementById('minLength'); +const keepPct = document.getElementById('keepPct'); +const keepPctLabel = document.getElementById('keepPctLabel'); +const fftSize = 1024; // Fixed + +function getKeepCount() { + return Math.max(1, Math.ceil(extractedPartials.length * parseInt(keepPct.value) / 100)); +} + +keepPct.addEventListener('input', () => { + keepPctLabel.textContent = keepPct.value + '%'; + if (viewer && extractedPartials) viewer.setKeepCount(getKeepCount()); +}); + +// --- Editor --- +const editor = new PartialEditor(); +editor.onPartialDeleted = () => { + if (viewer && extractedPartials) + viewer.setKeepCount(extractedPartials.length > 0 ? getKeepCount() : 0); +}; +editor.onBeforeChange = pushUndo; + +// Initialize audio context +function initAudioContext() { + if (!audioContext) { + audioContext = new (window.AudioContext || window.webkitAudioContext)(); + } +} + +// Shared: initialize editor from an AudioBuffer +function loadAudioBuffer(buffer, label) { + audioBuffer = buffer; + initAudioContext(); + extractBtn.disabled = false; + playBtn.disabled = false; + setStatus('Computing STFT cache...', 'info'); + + // Reset partials from previous file + extractedPartials = null; + editor.setPartials(null); + + setTimeout(() => { + const signal = audioBuffer.getChannelData(0); + stftCache = new STFTCache(signal, audioBuffer.sampleRate, fftSize, Math.max(64, parseInt(hopSize.value) || 64)); + setStatus(`${label} — ${audioBuffer.duration.toFixed(2)}s, ${audioBuffer.sampleRate}Hz, ${audioBuffer.numberOfChannels}ch (${stftCache.getNumFrames()} frames cached)`, 'info'); + + // Pre-compute peak frames so explore mode works immediately (before Extract) + const peakFrames = []; + for (let i = 0; i < stftCache.getNumFrames(); ++i) { + const f = stftCache.getFrameAtIndex(i); + peakFrames.push({ + time: f.time, + peaks: detectPeaks(f.squaredAmplitude, f.phase, fftSize, audioBuffer.sampleRate, + parseFloat(threshold.value), freqWeightCb.checked, + parseFloat(prominence.value)), + }); + } + + if (viewer) viewer.destroy(); + viewer = new SpectrogramViewer(canvas, audioBuffer, stftCache); + viewer.setFrames(peakFrames); + document.getElementById('exploreBtn').disabled = false; + document.getElementById('contourBtn').disabled = false; + editor.setViewer(viewer); + viewer.onPartialSelect = (i) => editor.onPartialSelect(i); + viewer.onRender = () => editor.onRender(); + viewer.onBeforeChange = pushUndo; + viewer.onExploreMove = (time, freq) => { + let partial = null; + if (exploreMode === 'peak') { + if (!viewer.frames || viewer.frames.length === 0) return; + partial = trackFromSeed(viewer.frames, time, freq, { + hopSize: Math.max(64, parseInt(hopSize.value) || 64), + sampleRate: audioBuffer.sampleRate, + deathAge: parseInt(deathAgeEl.value), + phaseErrorWeight: parseFloat(phaseErrorWeightEl.value), + }); + } else if (exploreMode === 'contour') { + partial = trackIsoContour(stftCache, time, freq, { + sampleRate: audioBuffer.sampleRate, + deathAge: parseInt(deathAgeEl.value), + }); + } + viewer.setPreviewPartial(partial); + }; + viewer.onExploreCommit = (partial) => { + if (!extractedPartials) extractedPartials = []; + pushUndo(); + const {spread_above, spread_below} = autodetectSpread(partial, stftCache, fftSize, audioBuffer.sampleRate); + if (!partial.harmonics) partial.harmonics = { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; + partial.harmonics.spread_above = spread_above; + partial.harmonics.spread_below = spread_below; + extractedPartials.unshift(partial); + refreshPartialsView(0); + setStatus(`${exploreMode}: added partial (${extractedPartials.length} total)`, 'info'); + }; + }, 10); +} + +// File chooser button +chooseFileBtn.addEventListener('click', () => wavFile.click()); + +// Load WAV file +wavFile.addEventListener('change', async (e) => { + const file = e.target.files[0]; + if (!file) return; + + fileLabel.textContent = file.name; + setStatus('Loading WAV...', 'info'); + try { + const arrayBuffer = await file.arrayBuffer(); + const ctx = new AudioContext(); + const buf = await ctx.decodeAudioData(arrayBuffer); + loadAudioBuffer(buf, `Loaded: ${file.name}`); + } catch (err) { + setStatus('Error loading WAV: ' + err.message, 'error'); + console.error(err); + } +}); + +// Update cache when hop size changes +hopSize.addEventListener('change', () => { + const val = Math.max(64, parseInt(hopSize.value) || 64); + hopSize.value = val; + if (stftCache) { + setStatus('Updating STFT cache...', 'info'); + setTimeout(() => { + stftCache.setHopSize(val); + setStatus(`Cache updated (${stftCache.getNumFrames()} frames)`, 'info'); + if (viewer) viewer.render(); + }, 10); + } +}); + +function runExtraction() { + if (!stftCache) return; + + setStatus('Extracting partials...', 'info'); + extractBtn.disabled = true; + + setTimeout(() => { + try { + const params = { + fftSize: fftSize, + hopSize: parseInt(hopSize.value), + threshold: parseFloat(threshold.value), + prominence: parseFloat(prominence.value), + freqWeight: freqWeightCb.checked, + birthPersistence: parseInt(birthPersistenceEl.value), + deathAge: parseInt(deathAgeEl.value), + phaseErrorWeight: parseFloat(phaseErrorWeightEl.value), + minLength: parseInt(minLengthEl.value), + sampleRate: audioBuffer.sampleRate + }; + + const result = extractPartials(params, stftCache); + + // Sort by decreasing peak amplitude + result.partials.sort((a, b) => { + const peakA = a.amps.reduce((m, v) => Math.max(m, v), 0); + const peakB = b.amps.reduce((m, v) => Math.max(m, v), 0); + return peakB - peakA; + }); + + extractedPartials = result.partials; + autoSpreadAll(); + viewer.setFrames(result.frames); + setStatus(`Extracted ${result.partials.length} partials`, 'info'); + refreshPartialsView(); + + } catch (err) { + setStatus('Extraction error: ' + err.message, 'error'); + console.error(err); + } + extractBtn.disabled = false; + autoSpreadAllBtn.disabled = false; + document.getElementById('newPartialBtn').disabled = false; + document.getElementById('clearAllBtn').disabled = false; + undoStack.length = 0; redoStack.length = 0; _updateUndoRedoBtns(); + }, 50); +} + +extractBtn.addEventListener('click', () => { + if (!audioBuffer) return; + runExtraction(); +}); + +function createNewPartial() { + if (!audioBuffer || !extractedPartials) return; + pushUndo(); + const dur = audioBuffer.duration; + const newPartial = { + times: [0, dur], + freqs: [440, 440], + amps: [1.0, 1.0], + phases: [0, 0], + muted: false, + freqCurve: { + t0: 0, t1: dur / 3, t2: dur * 2 / 3, t3: dur, + v0: 440, v1: 440, v2: 440, v3: 440, + a0: 1.0, a1: 1.0, a2: 1.0, a3: 1.0, + }, + harmonics: { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }, + }; + extractedPartials.unshift(newPartial); + refreshPartialsView(0); +} + +function clearAllPartials() { + if (!extractedPartials || extractedPartials.length === 0) return; + pushUndo(); + extractedPartials = []; + refreshPartialsView(); +} + +document.getElementById('newPartialBtn').addEventListener('click', createNewPartial); +document.getElementById('clearAllBtn').addEventListener('click', clearAllPartials); +document.getElementById('exploreBtn').addEventListener('click', () => setExploreMode(exploreMode === 'peak' ? false : 'peak')); +document.getElementById('contourBtn').addEventListener('click', () => setExploreMode(exploreMode === 'contour' ? false : 'contour')); +document.getElementById('undoBtn').addEventListener('click', undo); +document.getElementById('redoBtn').addEventListener('click', redo); + +function autoSpreadAll() { + if (!extractedPartials || !stftCache) return; + const fs = stftCache.fftSize; + const sr = audioBuffer.sampleRate; + const defaults = { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; + for (const p of extractedPartials) { + const {spread_above, spread_below} = autodetectSpread(p, stftCache, fs, sr); + if (!p.harmonics) p.harmonics = { ...defaults }; + p.harmonics.spread_above = spread_above; + p.harmonics.spread_below = spread_below; + } + if (viewer) viewer.render(); + const sel = viewer ? viewer.selectedPartial : -1; + if (sel >= 0) editor.onPartialSelect(sel); + setStatus(`Auto-spread applied to ${extractedPartials.length} partials`, 'info'); +} + +autoSpreadAllBtn.addEventListener('click', autoSpreadAll); + +threshold.addEventListener('change', () => { + if (stftCache) runExtraction(); +}); + +freqWeightCb.addEventListener('change', () => { + if (stftCache) runExtraction(); +}); + +for (const el of [birthPersistenceEl, deathAgeEl, phaseErrorWeightEl, minLengthEl]) { + el.addEventListener('change', () => { if (stftCache) runExtraction(); }); +} + +function playAudioBuffer(buffer, statusMsg) { + const startTime = audioContext.currentTime; + currentSource = audioContext.createBufferSource(); + currentSource.buffer = buffer; + currentSource.connect(audioContext.destination); + currentSource.start(); + currentSource.onended = () => { + currentSource = null; + playBtn.disabled = false; + stopBtn.disabled = true; + viewer.setPlayheadTime(-1); + setStatus('Stopped', 'info'); + }; + playBtn.disabled = true; + stopBtn.disabled = false; + setStatus(statusMsg, 'info'); + function tick() { + if (!currentSource) return; + viewer.setPlayheadTime(audioContext.currentTime - startTime); + requestAnimationFrame(tick); + } + tick(); +} + +function stopAudio() { + if (currentSource) { + try { currentSource.stop(); } catch (e) {} + currentSource = null; + } + if (viewer) viewer.setPlayheadTime(-1); + playBtn.disabled = false; + stopBtn.disabled = true; + setStatus('Stopped', 'info'); +} + +function playOriginal() { + if (!audioBuffer || !audioContext) return; + stopAudio(); + playAudioBuffer(audioBuffer, 'Playing...'); +} + +// Play audio +playBtn.addEventListener('click', playOriginal); + +// Stop audio +stopBtn.addEventListener('click', () => { + stopAudio(); +}); + +function setStatus(msg, type = '') { + status.innerHTML = msg; + status.className = type; +} + +function getSynthParams() { + const forceResonator = document.getElementById('forceResonator').checked; + const lpK1Raw = parseFloat(document.getElementById('lpK1').value); + const hpK2Raw = parseFloat(document.getElementById('hpK2').value); + return { + integratePhase: document.getElementById('integratePhase').checked, + opts: { + disableJitter: document.getElementById('disableJitter').checked, + disableSpread: document.getElementById('disableSpread').checked, + forceResonator, + forceRGain: forceResonator && document.getElementById('forceRGain').checked, + globalR: parseFloat(document.getElementById('globalR').value), + globalGain: parseFloat(document.getElementById('globalGain').value), + k1: lpK1Raw < 1.0 ? lpK1Raw : null, + k2: hpK2Raw < 1.0 ? hpK2Raw : null, + }, + }; +} + +// Synthesize partials and return an AudioBuffer, trimmed to [t_start-margin, t_end+margin] +function getAudioBuffer(partials, margin = 0) { + const sr = audioBuffer.sampleRate; + const {integratePhase, opts} = getSynthParams(); + const pcm = synthesizeMQ(partials, sr, audioBuffer.duration, integratePhase, opts); + let startSample = 0, endSample = pcm.length; + if (margin >= 0 && partials.length > 0) { + const times = partials.flatMap(p => p.times); + const tStart = Math.min(...times); + const tEnd = Math.max(...times); + startSample = Math.max(0, Math.floor((tStart - margin) * sr)); + endSample = Math.min(pcm.length, Math.ceil((tEnd + margin) * sr)); + } + const trimmed = pcm.subarray(startSample, endSample); + const buf = audioContext.createBuffer(1, trimmed.length, sr); + buf.getChannelData(0).set(trimmed); + return buf; +} + +// Play synthesized audio +function playSynthesized() { + if (!extractedPartials || extractedPartials.length === 0) { + setStatus('No partials extracted yet', 'warn'); + return; + } + if (!audioBuffer || !audioContext) return; + + stopAudio(); + + setStatus('Synthesizing...', 'info'); + + const keepCount = getKeepCount(); + const partialsToUse = extractedPartials.slice(0, keepCount).filter(p => !p.muted); + setStatus(`Synthesizing ${partialsToUse.length}/${extractedPartials.length} partials (${keepPct.value}%)...`, 'info'); + + if (viewer) { + const {integratePhase, opts} = getSynthParams(); + const pcm = synthesizeMQ(partialsToUse, audioBuffer.sampleRate, audioBuffer.duration, + integratePhase, opts); + viewer.setSynthStftCache(new STFTCache(pcm, audioBuffer.sampleRate, fftSize, parseInt(hopSize.value))); + } + + const synthBuffer = getAudioBuffer(partialsToUse); + playAudioBuffer(synthBuffer, `Playing synthesized (${partialsToUse.length}/${extractedPartials.length} partials, ${keepPct.value}%)...`); +} + +// Keyboard shortcuts +document.addEventListener('keydown', (e) => { + if (e.target.tagName === 'INPUT' || e.target.tagName === 'TEXTAREA') return; + if (e.ctrlKey && e.code === 'KeyZ' && !e.shiftKey) { + e.preventDefault(); undo(); return; + } else if (e.ctrlKey && (e.code === 'KeyY' || (e.code === 'KeyZ' && e.shiftKey))) { + e.preventDefault(); redo(); return; + } else if (e.code === 'KeyN' && !e.ctrlKey && !e.metaKey) { + e.preventDefault(); createNewPartial(); return; + } + if (e.code === 'Digit1') { + e.preventDefault(); + if (!playBtn.disabled) playOriginal(); + } else if (e.code === 'Digit2') { + e.preventDefault(); + playSynthesized(); + } else if (e.code === 'Digit3') { + e.preventDefault(); + const sel = viewer ? viewer.selectedPartial : -1; + if (sel < 0 || !extractedPartials || !audioBuffer || !audioContext) return; + const partial = extractedPartials[sel]; + if (!partial) return; + stopAudio(); + playAudioBuffer(getAudioBuffer([partial], 0.05), `Playing partial #${sel}...`); + } else if (e.code === 'KeyP') { + e.preventDefault(); + if (viewer) viewer.togglePeaks(); + } else if (e.code === 'KeyA') { + e.preventDefault(); + if (viewer) { + viewer.showSynthFFT = !viewer.showSynthFFT; + viewer.renderSpectrum(); + } + } else if (e.code === 'KeyE') { + e.preventDefault(); + if (!extractBtn.disabled) extractBtn.click(); + } else if (e.code === 'KeyX' && !e.ctrlKey && !e.metaKey) { + e.preventDefault(); + if (!document.getElementById('exploreBtn').disabled) setExploreMode(exploreMode === 'peak' ? false : 'peak'); + } else if (e.code === 'KeyC' && !e.ctrlKey && !e.metaKey) { + e.preventDefault(); + if (!document.getElementById('contourBtn').disabled) setExploreMode(exploreMode === 'contour' ? false : 'contour'); + } else if (e.code === 'Delete' || e.code === 'Backspace') { + e.preventDefault(); + document.getElementById('deletePartialBtn').click(); + } else if (e.code === 'Escape') { + if (exploreMode) { setExploreMode(false); return; } + if (viewer) viewer.selectPartial(-1); + } +}); + +// Curve tab switching +document.querySelectorAll('.tab-btn').forEach(btn => { + btn.addEventListener('click', () => { + document.querySelectorAll('.tab-btn').forEach(b => b.classList.remove('active')); + btn.classList.add('active'); + document.querySelectorAll('.tab-pane').forEach(p => p.style.display = 'none'); + document.getElementById('tab' + btn.dataset.tab).style.display = ''; + }); +}); diff --git a/tools/mq_editor/editor.js b/tools/mq_editor/editor.js index 97d8a7a..b07664e 100644 --- a/tools/mq_editor/editor.js +++ b/tools/mq_editor/editor.js @@ -20,6 +20,8 @@ class PartialEditor { // Callback: called after a partial is deleted so the host can update keepCount this.onPartialDeleted = null; + // Callback: called before any mutation (for undo/redo) + this.onBeforeChange = null; // Private state this._selectedIndex = -1; @@ -85,11 +87,11 @@ class PartialEditor { muteBtn.style.color = partial.muted ? '#fa4' : ''; this._buildCurveGrid(this._freqGrid, partial, 'freqCurve', 'f', index); - this._buildCurveGrid(this._ampGrid, partial, 'ampCurve', 'a', index); + this._buildCurveGrid(this._ampGrid, partial, 'freqCurve', 'a', index, 'a'); this._buildSynthGrid(partial, index); } - _buildCurveGrid(grid, partial, curveKey, valueLabel, partialIndex) { + _buildCurveGrid(grid, partial, curveKey, valueLabel, partialIndex, valueKey = 'v') { grid.innerHTML = ''; const curve = partial[curveKey]; if (!curve) { grid.style.color = '#444'; grid.textContent = 'none'; return; } @@ -107,10 +109,10 @@ class PartialEditor { const vInput = document.createElement('input'); vInput.type = 'number'; - vInput.value = curveKey === 'freqCurve' ? curve['v' + i].toFixed(2) : curve['v' + i].toFixed(4); - vInput.step = curveKey === 'freqCurve' ? '1' : '0.0001'; + vInput.value = valueKey === 'v' ? curve['v' + i].toFixed(2) : curve[valueKey + i].toFixed(4); + vInput.step = valueKey === 'v' ? '1' : '0.0001'; vInput.title = valueLabel + i; - vInput.addEventListener('change', this._makeCurveUpdater(partialIndex, curveKey, 'v', i)); + vInput.addEventListener('change', this._makeCurveUpdater(partialIndex, curveKey, valueKey, i)); grid.appendChild(lbl); grid.appendChild(tInput); @@ -122,8 +124,8 @@ class PartialEditor { const grid = this._synthGrid; grid.innerHTML = ''; - const repDefaults = { decay_alpha: 0.1, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; - const resDefaults = { r: 0.995, gainComp: 1.0 }; + const harmDefaults = { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; + const resDefaults = { r: 0.995, gainComp: 1.0 }; const isResonator = !!(partial.resonator && partial.resonator.enabled); @@ -154,16 +156,17 @@ class PartialEditor { sinSection.style.cssText = 'display:contents;'; sinSection.dataset.section = 'sinusoid'; - const rep = partial.replicas || {}; + const harm = partial.harmonics || {}; const sinParams = [ - { key: 'decay_alpha', label: 'decay', step: '0.001' }, + { key: 'decay', label: 'h.decay', step: '0.01', max: '0.90' }, + { key: 'freq_mult', label: 'h.freq', step: '0.01' }, { key: 'jitter', label: 'jitter', step: '0.001' }, { key: 'spread_above', label: 'spread ↑', step: '0.001' }, { key: 'spread_below', label: 'spread ↓', step: '0.001' }, ]; const sinInputs = {}; for (const p of sinParams) { - const val = rep[p.key] != null ? rep[p.key] : repDefaults[p.key]; + const val = harm[p.key] != null ? harm[p.key] : harmDefaults[p.key]; const lbl = document.createElement('span'); lbl.textContent = p.label; const inp = document.createElement('input'); @@ -171,17 +174,28 @@ class PartialEditor { inp.value = val.toFixed(3); inp.step = p.step; inp.min = '0'; + if (p.max) inp.max = p.max; inp.addEventListener('change', (e) => { if (!this.partials) return; - const v = parseFloat(e.target.value); + let v = parseFloat(e.target.value); if (isNaN(v)) return; - if (!this.partials[index].replicas) this.partials[index].replicas = { ...repDefaults }; - this.partials[index].replicas[p.key] = v; + if (p.max) v = Math.min(v, parseFloat(p.max)); + if (!this.partials[index].harmonics) this.partials[index].harmonics = { ...harmDefaults }; + this.partials[index].harmonics[p.key] = v; if (this.viewer) this.viewer.render(); }); sinInputs[p.key] = inp; - const jog = this._makeJogSlider(inp, partial, index, p, repDefaults); + const jog = this._makeJogSlider(inp, { + step: parseFloat(p.step), + max: p.max ? parseFloat(p.max) : undefined, + onUpdate: (newVal) => { + if (!this.partials || !this.partials[index]) return; + if (!this.partials[index].harmonics) this.partials[index].harmonics = { ...harmDefaults }; + this.partials[index].harmonics[p.key] = newVal; + if (this.viewer) this.viewer.render(); + } + }); const wrap = document.createElement('div'); wrap.className = 'synth-field-wrap'; wrap.appendChild(inp); @@ -203,9 +217,9 @@ class PartialEditor { const sr = this.viewer ? this.viewer.audioBuffer.sampleRate : 44100; const fs = sc ? sc.fftSize : 2048; const {spread_above, spread_below} = autodetectSpread(p, sc, fs, sr); - if (!p.replicas) p.replicas = { ...repDefaults }; - p.replicas.spread_above = spread_above; - p.replicas.spread_below = spread_below; + if (!p.harmonics) p.harmonics = { ...harmDefaults }; + p.harmonics.spread_above = spread_above; + p.harmonics.spread_below = spread_below; sinInputs['spread_above'].value = spread_above.toFixed(4); sinInputs['spread_below'].value = spread_below.toFixed(4); }); @@ -245,45 +259,15 @@ class PartialEditor { if (this.viewer) this.viewer.render(); }); - // Inline jog slider for resonator params - const step = parseFloat(p.step); - const sensitivity = step * 5; - const jog = document.createElement('div'); - jog.className = 'jog-slider'; - const thumb = document.createElement('div'); - thumb.className = 'jog-thumb'; - jog.appendChild(thumb); - let dragging = false, startX = 0, startVal = 0; - const onMove = (ev) => { - if (!dragging) return; - const dx = ev.clientX - startX; - const half = jog.offsetWidth / 2; - const clamped = Math.max(-half, Math.min(half, dx)); - thumb.style.transition = 'none'; - thumb.style.left = `calc(50% - 3px + ${clamped}px)`; - const newVal = Math.max(parseFloat(inp.min) || 0, - Math.min(parseFloat(inp.max) || 1e9, startVal + dx * sensitivity)); - inp.value = newVal.toFixed(4); - if (!this.partials || !this.partials[index]) return; - if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; - this.partials[index].resonator[p.key] = newVal; - if (this.viewer) this.viewer.render(); - }; - const onUp = () => { - if (!dragging) return; - dragging = false; - thumb.style.transition = ''; - thumb.style.left = 'calc(50% - 3px)'; - document.removeEventListener('mousemove', onMove); - document.removeEventListener('mouseup', onUp); - }; - jog.addEventListener('mousedown', (ev) => { - dragging = true; - startX = ev.clientX; - startVal = parseFloat(inp.value) || 0; - document.addEventListener('mousemove', onMove); - document.addEventListener('mouseup', onUp); - ev.preventDefault(); + const jog = this._makeJogSlider(inp, { + step: parseFloat(p.step), + decimals: 4, + onUpdate: (newVal) => { + if (!this.partials || !this.partials[index]) return; + if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; + this.partials[index].resonator[p.key] = newVal; + if (this.viewer) this.viewer.render(); + } }); const wrap = document.createElement('div'); @@ -322,29 +306,32 @@ class PartialEditor { }); } - _makeJogSlider(inp, partial, index, p, defaults) { + _makeJogSlider(inp, options) { + const {step, onUpdate, decimals = 3} = options; + const min = options.min != null ? options.min : + (inp.min !== '' && !isNaN(parseFloat(inp.min)) ? parseFloat(inp.min) : 0); + const max = options.max != null ? options.max : + (inp.max !== '' && !isNaN(parseFloat(inp.max)) ? parseFloat(inp.max) : Infinity); + const sensitivity = step * 5; + const slider = document.createElement('div'); slider.className = 'jog-slider'; const thumb = document.createElement('div'); thumb.className = 'jog-thumb'; slider.appendChild(thumb); - const sensitivity = parseFloat(p.step) * 5; let startX = 0, startVal = 0, dragging = false; const onMove = (e) => { if (!dragging) return; const dx = e.clientX - startX; const half = slider.offsetWidth / 2; - const clamped = Math.max(-half, Math.min(half, dx)); + const clamped = clamp(dx, -half, half); thumb.style.transition = 'none'; thumb.style.left = `calc(50% - 3px + ${clamped}px)`; - const newVal = Math.max(0, startVal + dx * sensitivity); - inp.value = newVal.toFixed(3); - if (!this.partials || !this.partials[index]) return; - if (!this.partials[index].replicas) this.partials[index].replicas = { ...defaults }; - this.partials[index].replicas[p.key] = newVal; - if (this.viewer) this.viewer.render(); + const newVal = clamp(startVal + dx * sensitivity, min, max); + inp.value = newVal.toFixed(decimals); + onUpdate(newVal); }; const onUp = () => { @@ -359,7 +346,7 @@ class PartialEditor { slider.addEventListener('mousedown', (e) => { dragging = true; startX = e.clientX; - startVal = Math.max(0, parseFloat(inp.value) || 0); + startVal = Math.max(min, parseFloat(inp.value) || 0); document.addEventListener('mousemove', onMove); document.addEventListener('mouseup', onUp); e.preventDefault(); @@ -373,6 +360,7 @@ class PartialEditor { if (!this.partials) return; const val = parseFloat(e.target.value); if (isNaN(val)) return; + if (this.onBeforeChange) this.onBeforeChange(); this.partials[partialIndex][curveKey][field + pointIndex] = val; if (this.viewer) this.viewer.render(); }; @@ -381,6 +369,7 @@ class PartialEditor { _setupButtons() { document.getElementById('mutePartialBtn').addEventListener('click', () => { if (this._selectedIndex < 0 || !this.partials) return; + if (this.onBeforeChange) this.onBeforeChange(); const p = this.partials[this._selectedIndex]; p.muted = !p.muted; if (this.viewer) this.viewer.render(); @@ -389,6 +378,7 @@ class PartialEditor { document.getElementById('deletePartialBtn').addEventListener('click', () => { if (this._selectedIndex < 0 || !this.partials || !this.viewer) return; + if (this.onBeforeChange) this.onBeforeChange(); this.partials.splice(this._selectedIndex, 1); this.viewer.selectPartial(-1); if (this.onPartialDeleted) this.onPartialDeleted(); @@ -462,7 +452,7 @@ class PartialEditor { } // Bezier curve - const curve = partial.ampCurve; + const curve = partial.freqCurve; if (!curve) return; const color = this.viewer ? this.viewer.partialColor(this._selectedIndex) : '#f44'; @@ -474,7 +464,7 @@ class PartialEditor { const t = curve.t0 + (curve.t3 - curve.t0) * i / 120; const x = this._tToX(t); if (x < -1 || x > W + 1) { started = false; continue; } - const y = this._ampToY(evalBezier(curve, t)); + const y = this._ampToY(evalBezierAmp(curve, t)); if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); } ctx.stroke(); @@ -482,7 +472,7 @@ class PartialEditor { // Control points for (let i = 0; i < 4; ++i) { const x = this._tToX(curve['t' + i]); - const y = this._ampToY(curve['v' + i]); + const y = this._ampToY(curve['a' + i]); ctx.fillStyle = color; ctx.beginPath(); ctx.arc(x, y, 6, 0, 2 * Math.PI); @@ -503,14 +493,14 @@ class PartialEditor { canvas.addEventListener('mousedown', (e) => { if (this._selectedIndex < 0 || !this.partials) return; const partial = this.partials[this._selectedIndex]; - if (!partial || !partial.ampCurve) return; - const rect = canvas.getBoundingClientRect(); - const x = e.clientX - rect.left, y = e.clientY - rect.top; - const curve = partial.ampCurve; + if (!partial || !partial.freqCurve) return; + const {x, y} = getCanvasCoords(e, canvas); + const curve = partial.freqCurve; for (let i = 0; i < 4; ++i) { - if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['v' + i]) - y) <= 8) { + if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['a' + i]) - y) <= 8) { + if (this.onBeforeChange) this.onBeforeChange(); this._dragPointIndex = i; - canvas.style.cursor = 'grabbing'; + canvas.style.cursor = (i === 1 || i === 2) ? 'move' : 'ns-resize'; e.preventDefault(); return; } @@ -518,14 +508,18 @@ class PartialEditor { }); canvas.addEventListener('mousemove', (e) => { - const rect = canvas.getBoundingClientRect(); - const x = e.clientX - rect.left, y = e.clientY - rect.top; + const {x, y} = getCanvasCoords(e, canvas); if (this._dragPointIndex >= 0) { - const curve = this.partials[this._selectedIndex].ampCurve; + const curve = this.partials[this._selectedIndex].freqCurve; const i = this._dragPointIndex; - curve['t' + i] = Math.max(0, Math.min(this.viewer ? this.viewer.t_max : 1e6, this._xToT(x))); - curve['v' + i] = Math.max(0, this._yToAmp(y)); + curve['a' + i] = Math.max(0, this._yToAmp(y)); + // Inner control points are also horizontally draggable + if (i === 1) { + curve.t1 = clamp(this._xToT(x), curve.t0 + 1e-4, curve.t2 - 1e-4); + } else if (i === 2) { + curve.t2 = clamp(this._xToT(x), curve.t1 + 1e-4, curve.t3 - 1e-4); + } this._renderAmpEditor(); if (this.viewer) this.viewer.render(); e.preventDefault(); @@ -534,15 +528,16 @@ class PartialEditor { // Cursor hint if (this._selectedIndex >= 0 && this.partials) { - const curve = this.partials[this._selectedIndex]?.ampCurve; + const curve = this.partials[this._selectedIndex]?.freqCurve; if (curve) { - let near = false; + let cursor = 'crosshair'; for (let i = 0; i < 4; ++i) { - if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['v' + i]) - y) <= 8) { - near = true; break; + if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['a' + i]) - y) <= 8) { + cursor = (i === 1 || i === 2) ? 'move' : 'ns-resize'; + break; } } - canvas.style.cursor = near ? 'grab' : 'crosshair'; + canvas.style.cursor = cursor; } } }); diff --git a/tools/mq_editor/fft.js b/tools/mq_editor/fft.js index 10a5b45..0d54eae 100644 --- a/tools/mq_editor/fft.js +++ b/tools/mq_editor/fft.js @@ -132,18 +132,20 @@ class STFTCache { windowed[i] = frame[i] * w; } - // Compute FFT, store only squared amplitudes (re*re + im*im, no sqrt) + // Compute FFT, store squared amplitudes and phase const fftOut = realFFT(windowed); const squaredAmplitude = new Float32Array(this.fftSize / 2); + const phase = new Float32Array(this.fftSize / 2); for (let i = 0; i < this.fftSize / 2; ++i) { const re = fftOut[i * 2]; const im = fftOut[i * 2 + 1]; squaredAmplitude[i] = re * re + im * im; + phase[i] = Math.atan2(im, re); // Cache phase for tracking const db = 10 * Math.log10(Math.max(squaredAmplitude[i], 1e-20)); if (db > this.maxDB) this.maxDB = db; } - this.frames.push({time, offset, squaredAmplitude}); + this.frames.push({time, offset, squaredAmplitude, phase}); } if (!isFinite(this.maxDB)) this.maxDB = 0; diff --git a/tools/mq_editor/index.html b/tools/mq_editor/index.html index a2daff5..efbd73d 100644 --- a/tools/mq_editor/index.html +++ b/tools/mq_editor/index.html @@ -3,252 +3,7 @@ <head> <meta charset="utf-8"> <title>MQ Spectral Editor</title> - <style> - body { - font-family: monospace; - font-size: 14px; - margin: 20px; - background: #1a1a1a; - color: #ddd; - } - .page-title { - display: flex; - align-items: baseline; - gap: 16px; - margin-bottom: 10px; - } - .page-title h2 { margin: 0; } - #fileLabel { - font-size: 13px; - color: #8af; - opacity: 0.8; - } - .toolbar { - margin-bottom: 10px; - padding: 10px; - background: #2a2a2a; - border-radius: 4px; - } - button { - background: #3a3a3a; - color: #ddd; - border: 1px solid #555; - padding: 8px 16px; - margin-right: 8px; - cursor: pointer; - border-radius: 4px; - } - button:hover { background: #4a4a4a; } - button:disabled { opacity: 0.5; cursor: not-allowed; } - #extractBtn { background: #666; color: #fff; font-weight: bold; border-color: #888; } - input[type="file"] { display: none; } - .params { - display: inline-block; - margin-left: 20px; - } - label { - margin-right: 8px; - } - input[type="number"], select { - width: 80px; - background: #3a3a3a; - color: #ddd; - border: 1px solid #555; - padding: 4px; - border-radius: 3px; - } - .main-area { - display: flex; - align-items: flex-start; - gap: 10px; - margin-top: 10px; - } - #canvas { - border: 1px solid #555; - background: #000; - cursor: crosshair; - display: block; - flex-shrink: 0; - } - .right-panel { - background: #2a2a2a; - border: 1px solid #555; - border-radius: 4px; - padding: 12px; - min-width: 260px; - max-width: 260px; - max-height: 700px; - overflow-y: auto; - display: flex; - flex-direction: column; - gap: 6px; - box-sizing: border-box; - } - .panel-title { - font-size: 12px; - color: #888; - text-transform: uppercase; - letter-spacing: 1px; - border-bottom: 1px solid #444; - padding-bottom: 6px; - margin-bottom: 2px; - display: flex; - align-items: center; - gap: 6px; - } - .right-panel label { - display: flex; - align-items: center; - gap: 6px; - margin: 0; - cursor: pointer; - font-size: 14px; - } - /* Partial properties */ - .prop-row { - display: flex; - justify-content: space-between; - align-items: baseline; - font-size: 13px; - padding: 2px 0; - } - .prop-label { color: #777; font-size: 12px; } - .curve-tabs { - display: flex; - gap: 2px; - margin-top: 8px; - margin-bottom: 4px; - } - .tab-btn { - flex: 1; - padding: 4px 0; - font-size: 12px; - margin: 0; - background: #222; - border-color: #444; - color: #888; - } - .tab-btn.active { - background: #3a3a3a; - border-color: #666; - color: #ddd; - } - .curve-grid { - display: grid; - grid-template-columns: 18px 1fr 1fr; - gap: 3px 4px; - align-items: center; - } - .curve-grid span { color: #666; font-size: 11px; } - .curve-grid input[type="number"] { - width: 100%; - background: #333; - color: #ccc; - border: 1px solid #444; - padding: 2px 4px; - border-radius: 2px; - font-size: 11px; - font-family: monospace; - box-sizing: border-box; - } - .curve-grid input[type="number"]:focus { - border-color: #666; - outline: none; - } - .partial-actions { - display: flex; - gap: 4px; - margin-top: 8px; - } - .partial-actions button { - flex: 1; - padding: 4px 6px; - font-size: 12px; - margin: 0; - } - .synth-grid { - display: grid; - grid-template-columns: auto 1fr; - gap: 4px 8px; - align-items: center; - } - .synth-grid span { color: #888; font-size: 12px; } - .synth-field-wrap { - display: flex; - align-items: center; - gap: 4px; - } - .synth-field-wrap input[type="number"] { - flex: 1; - min-width: 0; - background: #333; - color: #ccc; - border: 1px solid #444; - padding: 2px 4px; - border-radius: 2px; - font-size: 11px; - font-family: monospace; - box-sizing: border-box; - } - .synth-field-wrap input[type="number"]:focus { border-color: #666; outline: none; } - .jog-slider { - width: 44px; - height: 16px; - background: #1e1e1e; - border: 1px solid #444; - border-radius: 3px; - cursor: ew-resize; - position: relative; - flex-shrink: 0; - user-select: none; - overflow: hidden; - } - .jog-slider::before { - content: ''; - position: absolute; - left: 50%; top: 3px; bottom: 3px; - width: 1px; - background: #484848; - transform: translateX(-50%); - } - .jog-thumb { - position: absolute; - top: 3px; bottom: 3px; - width: 6px; - background: #888; - border-radius: 2px; - left: calc(50% - 3px); - transition: left 0.12s ease; - } - .jog-slider:hover .jog-thumb { background: #aaa; } - .synth-grid input[type="number"]:focus { border-color: #666; outline: none; } - .synth-section { - border-top: 1px solid #444; - padding-top: 8px; - margin-top: auto; - } - /* resonator mode badge shown in partial header color swatch area */ - .res-badge { - font-size: 9px; - color: #8cf; - border: 1px solid #8cf; - border-radius: 2px; - padding: 0 3px; - vertical-align: middle; - margin-left: 4px; - opacity: 0.8; - } - #status { - margin-top: 10px; - padding: 8px; - background: #2a2a2a; - border-radius: 4px; - min-height: 20px; - } - .info { color: #4af; } - .warn { color: #fa4; } - .error { color: #f44; } - </style> + <link rel="stylesheet" href="style.css"> </head> <body> <div class="page-title"> @@ -258,53 +13,99 @@ <div class="toolbar"> <input type="file" id="wavFile" accept=".wav"> - <button id="chooseFileBtn">📂 Open WAV</button> - <button id="testWavBtn">âš— Test WAV</button> - <button id="extractBtn" disabled>Extract Partials</button> - <button id="autoSpreadAllBtn" disabled>Auto Spread All</button> - <button id="playBtn" disabled>â–¶ Play</button> - <button id="stopBtn" disabled>â– Stop</button> - - <div class="params"> - <label>Hop:</label> - <input type="number" id="hopSize" value="256" min="64" max="1024" step="64"> - - <label>Threshold (dB):</label> - <input type="number" id="threshold" value="-20" step="any"> - - <label>Prominence (dB):</label> - <input type="number" id="prominence" value="1.0" step="0.1" min="0"> - - <label style="margin-left:16px;" title="Weight spectrum by frequency before peak detection (f * FFT_Power(f)), accentuates high-frequency peaks"> - <input type="checkbox" id="freqWeight"> f·Power - </label> - - <label style="margin-left:16px;">Keep:</label> - <input type="range" id="keepPct" min="1" max="100" value="100" style="width:100px; vertical-align:middle;"> - <span id="keepPctLabel" style="margin-left:4px;">100%</span> - </div> + <span class="toolbar-group"> + <button id="chooseFileBtn">📂 Open WAV</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-group"> + <button id="extractBtn" disabled>Extract Partials</button> + <button id="autoSpreadAllBtn" disabled>Auto Spread All</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-group"> + <button id="playBtn" disabled>â–¶ Play</button> + <button id="stopBtn" disabled>â– Stop</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-group"> + <button id="newPartialBtn" disabled>+ Partial</button> + <button id="clearAllBtn" disabled>✕ Clear All</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-group"> + <button id="exploreBtn" disabled>⊕ Explore</button> + <button id="contourBtn" disabled>≋ Contour</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-group"> + <button id="undoBtn" disabled>↩ Undo</button> + <button id="redoBtn" disabled>↪ Redo</button> + </span> + <span class="toolbar-sep"></span> + <span class="toolbar-wrap"> + <button id="paramsBtn">âš™ Params</button> + <div id="paramsPanel"> + <div class="param-group"> + <span class="group-label">STFT</span> + <label title="STFT hop size in samples. Smaller = finer time resolution, more frames, slower.">Hop</label> + <input type="number" id="hopSize" value="256" min="64" max="1024" step="64" style="width:60px;"> + </div> + <div class="param-group"> + <span class="group-label">Peak Detect</span> + <label title="Minimum spectral peak amplitude in dB. Peaks below this are ignored.">Threshold (dB)</label> + <input type="number" id="threshold" value="-20" step="any"> + <label title="Minimum prominence in dB: how much a peak must stand above its surrounding valley. Suppresses weak shoulders.">Prominence (dB)</label> + <input type="number" id="prominence" value="1.0" step="0.1" min="0"> + <label title="Weight spectrum by frequency before peak detection: f × Power(f). Boosts high-frequency peaks relative to low-frequency ones."> + <input type="checkbox" id="freqWeight"> f·Power + </label> + </div> + <div class="param-group"> + <span class="group-label">Tracking</span> + <label title="Frames a candidate must persist consecutively before being promoted to a tracked partial. Higher = fewer spurious short bursts.">Birth</label> + <input type="number" id="birthPersistence" value="3" min="1" max="10" step="1" style="width:50px;"> + <label title="Frames a partial can go unmatched before it is terminated. Higher = bridges short gaps; lower = cuts off quickly.">Death</label> + <input type="number" id="deathAge" value="5" min="1" max="20" step="1" style="width:50px;"> + <label title="Weight of phase prediction error in the peak-matching cost function. Higher = stricter phase coherence required to continue a partial.">Phase Wt</label> + <input type="number" id="phaseErrorWeight" value="2.0" min="0" max="10" step="0.5" style="width:55px;"> + <label title="Minimum number of frames a tracked partial must span. Shorter partials are discarded after tracking.">Min Len</label> + <input type="number" id="minLength" value="10" min="1" max="50" step="1" style="width:50px;"> + </div> + </div> + </span> </div> <div class="main-area"> - <div style="display:flex; flex-direction:column; gap:6px; flex-shrink:0;"> - <div style="position: relative;"> + <div class="canvas-col"> + <div class="canvas-wrap"> <canvas id="canvas" width="1400" height="600"></canvas> - <canvas id="cursorCanvas" width="1400" height="600" style="position:absolute;top:0;left:0;pointer-events:none;"></canvas> + <canvas id="cursorCanvas" width="1400" height="600"></canvas> + <canvas id="playheadCanvas" width="1400" height="600"></canvas> + <!-- Partial spectrum viewer (bottom-right overlay, left of main spectrum) --> + <div id="partialSpectrumViewer"> + <canvas id="partialSpectrumCanvas" width="200" height="100"></canvas> + </div> <!-- Mini spectrum viewer (bottom-right overlay) --> - <div id="spectrumViewer" style="position: absolute; bottom: 10px; right: 10px; width: 400px; height: 100px; background: rgba(30, 30, 30, 0.9); border: 1px solid #555; border-radius: 3px; pointer-events: none;"> + <div id="spectrumViewer"> <canvas id="spectrumCanvas" width="400" height="100"></canvas> </div> + <!-- Keep slider (bottom-left overlay) --> + <div id="keepOverlay"> + <span title="Keep only the strongest N% of extracted partials, ranked by peak amplitude.">Keep</span> + <input type="range" id="keepPct" min="1" max="100" value="100"> + <span id="keepPctLabel">100%</span> + </div> </div> <!-- Amplitude bezier editor (shown when partial selected) --> - <div id="ampEditPanel" style="display:none;"> - <div style="font-size:10px; color:#555; padding:2px 0 3px 1px; display:flex; align-items:center; gap:10px; text-transform:uppercase; letter-spacing:0.5px;"> + <div id="ampEditPanel"> + <div class="amp-edit-header"> <span>Amplitude</span> - <span id="ampEditTitle" style="color:#777; text-transform:none; letter-spacing:0;"></span> - <span style="color:#333; text-transform:none; letter-spacing:0;">drag control points · Esc to deselect</span> + <span id="ampEditTitle"></span> + <span class="amp-edit-hint">drag control points · Esc to deselect</span> </div> - <canvas id="ampEditCanvas" width="1400" height="120" style="border:1px solid #2a2a2a; background:#0e0e0e; cursor:crosshair; display:block;"></canvas> + <canvas id="ampEditCanvas" width="1400" height="120"></canvas> </div> </div> @@ -313,7 +114,7 @@ <div id="partialProps" style="display:none;"> <div class="panel-title"> <span id="propTitle">Partial #—</span> - <span id="propSwatch" style="display:inline-block;width:10px;height:10px;border-radius:2px;flex-shrink:0;"></span> + <span id="propSwatch"></span> </div> <div class="prop-row"> <span class="prop-label">Peak</span> @@ -339,11 +140,11 @@ </div> <div class="partial-actions"> <button id="mutePartialBtn">Mute</button> - <button id="deletePartialBtn">Delete</button> + <button id="deletePartialBtn">Delete <kbd>Del</kbd></button> </div> </div> - <div id="noSelMsg" style="color:#555;font-size:13px;padding:2px 0;">Click a partial to select</div> + <div id="noSelMsg">Click a partial to select</div> <!-- Synthesis options (always at bottom) --> <div class="synth-section"> @@ -352,478 +153,45 @@ <label><input type="checkbox" id="disableJitter"> Disable jitter</label> <label><input type="checkbox" id="disableSpread"> Disable spread</label> <label title="Test mode: force resonator synthesis for all partials (ignores per-partial mode setting)"><input type="checkbox" id="forceResonator"> Resonator (all)</label> - <div id="globalResParams" style="display:none;margin-top:4px;padding:4px 0 2px 12px;border-left:2px solid #555;"> - <label style="display:flex;align-items:center;gap:6px;" title="Global pole radius r in (0,1). Applied to all partials in resonator mode."> + <div id="globalResParams" style="display:none;"> + <label title="Global pole radius r in (0,1). Applied to all partials in resonator mode."> r (pole) - <input type="range" id="globalR" min="0.75" max="0.9999" step="0.0001" value="0.995" style="flex:1;min-width:0;"> - <span id="globalRVal" style="width:44px;text-align:right;">0.9950</span> + <input type="range" id="globalR" min="0.75" max="0.9999" step="0.0001" value="0.995"> + <span id="globalRVal" class="slider-val">0.9950</span> </label> - <label style="display:flex;align-items:center;gap:6px;" title="Global gain compensation applied to all partials in resonator mode."> + <label title="Global gain compensation applied to all partials in resonator mode."> gain - <input type="range" id="globalGain" min="0.0" max="4.0" step="0.01" value="1.0" style="flex:1;min-width:0;"> - <span id="globalGainVal" style="width:44px;text-align:right;">1.00</span> + <input type="range" id="globalGain" min="0.0" max="4.0" step="0.01" value="1.0"> + <span id="globalGainVal" class="slider-val">1.00</span> </label> <label title="Override per-partial r/gain with global values during playback"><input type="checkbox" id="forceRGain"> force r/gain</label> </div> <div style="margin-top:6px;"> - <label style="display:flex;align-items:center;gap:6px;" title="LP filter coefficient k1 in (0,1]. 1.0 = bypass."> + <label title="LP filter coefficient k1 in (0,1]. 1.0 = bypass."> LP k1 - <input type="range" id="lpK1" min="0.001" max="1.0" step="0.001" value="1.0" style="flex:1;min-width:0;"> - <span id="lpK1Val" style="width:44px;text-align:right;">bypass</span> + <input type="range" id="lpK1" min="0.001" max="1.0" step="0.001" value="1.0"> + <span id="lpK1Val" class="slider-val">bypass</span> </label> - <label style="display:flex;align-items:center;gap:6px;" title="HP filter coefficient k2 in (0,1]. 1.0 = bypass."> + <label title="HP filter coefficient k2 in (0,1]. 1.0 = bypass."> HP k2 - <input type="range" id="hpK2" min="0.001" max="1.0" step="0.001" value="1.0" style="flex:1;min-width:0;"> - <span id="hpK2Val" style="width:44px;text-align:right;">bypass</span> + <input type="range" id="hpK2" min="0.001" max="1.0" step="0.001" value="1.0"> + <span id="hpK2Val" class="slider-val">bypass</span> </label> </div> </div> </div> </div> - <div id="tooltip" style="position: fixed; display: none; background: #2a2a2a; padding: 4px 8px; border: 1px solid #555; border-radius: 3px; pointer-events: none; font-size: 11px; z-index: 1000;"></div> + <div id="tooltip"></div> <div id="status">Load a WAV file to begin...</div> + <script src="utils.js"></script> <script src="fft.js"></script> <script src="mq_extract.js"></script> <script src="mq_synth.js"></script> <script src="viewer.js"></script> <script src="editor.js"></script> - <script> - // LP: y[n] = k*x[n] + (1-k)*y[n-1] => -3dB at cos(w) = (2-2k-k²)/(2(1-k)) - function k1ToHz(k, sr) { - if (k >= 1.0) return sr / 2; - const cosW = (2 - 2*k - k*k) / (2*(1 - k)); - return Math.acos(Math.max(-1, Math.min(1, cosW))) * sr / (2 * Math.PI); - } - // HP: y[n] = k*(y[n-1]+x[n]-x[n-1]) => -3dB from peak at cos(w) = 2k/(1+k²) - function k2ToHz(k, sr) { - if (k >= 1.0) return 0; - const cosW = 2*k / (1 + k*k); - return Math.acos(Math.max(-1, Math.min(1, cosW))) * sr / (2 * Math.PI); - } - function fmtHz(f) { - return f >= 1000 ? (f/1000).toFixed(1) + 'k' : Math.round(f) + 'Hz'; - } - function getSR() { return (typeof audioBuffer !== 'undefined' && audioBuffer) ? audioBuffer.sampleRate : 44100; } - - // LP/HP slider live display - document.getElementById('lpK1').addEventListener('input', function() { - const k = parseFloat(this.value); - const f = k1ToHz(k, getSR()); - document.getElementById('lpK1Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); - }); - document.getElementById('hpK2').addEventListener('input', function() { - const k = parseFloat(this.value); - const f = k2ToHz(k, getSR()); - document.getElementById('hpK2Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); - }); - - // Show/hide global resonator params when forceResonator toggled - document.getElementById('forceResonator').addEventListener('change', function() { - document.getElementById('globalResParams').style.display = this.checked ? '' : 'none'; - }); - document.getElementById('globalR').addEventListener('input', function() { - document.getElementById('globalRVal').textContent = parseFloat(this.value).toFixed(4); - }); - document.getElementById('globalGain').addEventListener('input', function() { - document.getElementById('globalGainVal').textContent = parseFloat(this.value).toFixed(2); - }); - let audioBuffer = null; - let viewer = null; - let audioContext = null; - let currentSource = null; - let extractedPartials = null; - let stftCache = null; - - const wavFile = document.getElementById('wavFile'); - const chooseFileBtn = document.getElementById('chooseFileBtn'); - const extractBtn = document.getElementById('extractBtn'); - const autoSpreadAllBtn = document.getElementById('autoSpreadAllBtn'); - const playBtn = document.getElementById('playBtn'); - const stopBtn = document.getElementById('stopBtn'); - const canvas = document.getElementById('canvas'); - const status = document.getElementById('status'); - const fileLabel = document.getElementById('fileLabel'); - - const hopSize = document.getElementById('hopSize'); - const threshold = document.getElementById('threshold'); - const prominence = document.getElementById('prominence'); - const freqWeightCb = document.getElementById('freqWeight'); - const keepPct = document.getElementById('keepPct'); - const keepPctLabel = document.getElementById('keepPctLabel'); - const fftSize = 1024; // Fixed - - function getKeepCount() { - return Math.max(1, Math.ceil(extractedPartials.length * parseInt(keepPct.value) / 100)); - } - - keepPct.addEventListener('input', () => { - keepPctLabel.textContent = keepPct.value + '%'; - if (viewer && extractedPartials) viewer.setKeepCount(getKeepCount()); - }); - - // --- Editor --- - const editor = new PartialEditor(); - editor.onPartialDeleted = () => { - if (viewer && extractedPartials) - viewer.setKeepCount(extractedPartials.length > 0 ? getKeepCount() : 0); - }; - - // Initialize audio context - function initAudioContext() { - if (!audioContext) { - audioContext = new (window.AudioContext || window.webkitAudioContext)(); - } - } - - // Shared: initialize editor from an AudioBuffer - function loadAudioBuffer(buffer, label) { - audioBuffer = buffer; - initAudioContext(); - extractBtn.disabled = false; - playBtn.disabled = false; - setStatus('Computing STFT cache...', 'info'); - - // Reset partials from previous file - extractedPartials = null; - editor.setPartials(null); - - setTimeout(() => { - const signal = audioBuffer.getChannelData(0); - stftCache = new STFTCache(signal, audioBuffer.sampleRate, fftSize, Math.max(64, parseInt(hopSize.value) || 64)); - setStatus(`${label} — ${audioBuffer.duration.toFixed(2)}s, ${audioBuffer.sampleRate}Hz, ${audioBuffer.numberOfChannels}ch (${stftCache.getNumFrames()} frames cached)`, 'info'); - viewer = new SpectrogramViewer(canvas, audioBuffer, stftCache); - editor.setViewer(viewer); - viewer.onPartialSelect = (i) => editor.onPartialSelect(i); - viewer.onRender = () => editor.onRender(); - if (label.startsWith('Test WAV')) validateTestWAVPeaks(stftCache); - }, 10); - } - - // File chooser button - chooseFileBtn.addEventListener('click', () => wavFile.click()); - - // Load WAV file - wavFile.addEventListener('change', async (e) => { - const file = e.target.files[0]; - if (!file) return; - - fileLabel.textContent = file.name; - setStatus('Loading WAV...', 'info'); - try { - const arrayBuffer = await file.arrayBuffer(); - const ctx = new AudioContext(); - const buf = await ctx.decodeAudioData(arrayBuffer); - loadAudioBuffer(buf, `Loaded: ${file.name}`); - } catch (err) { - setStatus('Error loading WAV: ' + err.message, 'error'); - console.error(err); - } - }); - - // Test WAV: generate synthetic signal (two sine waves) in-memory - document.getElementById('testWavBtn').addEventListener('click', () => { - initAudioContext(); - const SR = 32000; - const duration = 2.0; - const numSamples = SR * duration; - - // Two sine waves: 440 Hz (A4) + 660 Hz (E5, perfect fifth), equal amplitude - const buf = audioContext.createBuffer(1, numSamples, SR); - const data = buf.getChannelData(0); - for (let i = 0; i < numSamples; ++i) { - data[i] = 0.5 * Math.sin(2 * Math.PI * 440 * i / SR) - + 0.5 * Math.sin(2 * Math.PI * 660 * i / SR); - } - - fileLabel.textContent = 'test-440+660hz.wav'; - loadAudioBuffer(buf, 'Test WAV: 440Hz + 660Hz (2s, 32kHz)'); - }); - - // Update cache when hop size changes - hopSize.addEventListener('change', () => { - const val = Math.max(64, parseInt(hopSize.value) || 64); - hopSize.value = val; - if (stftCache) { - setStatus('Updating STFT cache...', 'info'); - setTimeout(() => { - stftCache.setHopSize(val); - setStatus(`Cache updated (${stftCache.getNumFrames()} frames)`, 'info'); - if (viewer) viewer.render(); - }, 10); - } - }); - - function runExtraction() { - if (!stftCache) return; - - setStatus('Extracting partials...', 'info'); - extractBtn.disabled = true; - - setTimeout(() => { - try { - const params = { - fftSize: fftSize, - hopSize: parseInt(hopSize.value), - threshold: parseFloat(threshold.value), - prominence: parseFloat(prominence.value), - freqWeight: freqWeightCb.checked, - sampleRate: audioBuffer.sampleRate - }; - - const result = extractPartials(params, stftCache); - - // Sort by decreasing peak amplitude - result.partials.sort((a, b) => { - const peakA = a.amps.reduce((m, v) => Math.max(m, v), 0); - const peakB = b.amps.reduce((m, v) => Math.max(m, v), 0); - return peakB - peakA; - }); - - extractedPartials = result.partials; - editor.setPartials(result.partials); - viewer.setFrames(result.frames); - setStatus(`Extracted ${result.partials.length} partials`, 'info'); - viewer.setPartials(result.partials); - viewer.setKeepCount(getKeepCount()); - viewer.selectPartial(-1); - - } catch (err) { - setStatus('Extraction error: ' + err.message, 'error'); - console.error(err); - } - extractBtn.disabled = false; - autoSpreadAllBtn.disabled = false; - }, 50); - } - - extractBtn.addEventListener('click', () => { - if (!audioBuffer) return; - runExtraction(); - }); - - autoSpreadAllBtn.addEventListener('click', () => { - if (!extractedPartials || !stftCache) return; - const fs = stftCache.fftSize; - const sr = audioBuffer.sampleRate; - const defaults = { decay_alpha: 0.1, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; - for (const p of extractedPartials) { - const {spread_above, spread_below} = autodetectSpread(p, stftCache, fs, sr); - if (!p.replicas) p.replicas = { ...defaults }; - p.replicas.spread_above = spread_above; - p.replicas.spread_below = spread_below; - } - if (viewer) viewer.render(); - const sel = viewer ? viewer.selectedPartial : -1; - if (sel >= 0) editor.onPartialSelect(sel); - setStatus(`Auto-spread applied to ${extractedPartials.length} partials`, 'info'); - }); - - threshold.addEventListener('change', () => { - if (stftCache) runExtraction(); - }); - - freqWeightCb.addEventListener('change', () => { - if (stftCache) runExtraction(); - }); - - function playAudioBuffer(buffer, statusMsg) { - const startTime = audioContext.currentTime; - currentSource = audioContext.createBufferSource(); - currentSource.buffer = buffer; - currentSource.connect(audioContext.destination); - currentSource.start(); - currentSource.onended = () => { - currentSource = null; - playBtn.disabled = false; - stopBtn.disabled = true; - viewer.setPlayheadTime(-1); - setStatus('Stopped', 'info'); - }; - playBtn.disabled = true; - stopBtn.disabled = false; - setStatus(statusMsg, 'info'); - function tick() { - if (!currentSource) return; - viewer.setPlayheadTime(audioContext.currentTime - startTime); - requestAnimationFrame(tick); - } - tick(); - } - - function stopAudio() { - if (currentSource) { - try { currentSource.stop(); } catch (e) {} - currentSource = null; - } - if (viewer) viewer.setPlayheadTime(-1); - playBtn.disabled = false; - stopBtn.disabled = true; - setStatus('Stopped', 'info'); - } - - // Play audio - playBtn.addEventListener('click', () => { - if (!audioBuffer || !audioContext) return; - stopAudio(); - playAudioBuffer(audioBuffer, 'Playing...'); - }); - - // Stop audio - stopBtn.addEventListener('click', () => { - stopAudio(); - }); - - function setStatus(msg, type = '') { - status.innerHTML = msg; - status.className = type; - } - - // Play synthesized audio - function playSynthesized() { - if (!extractedPartials || extractedPartials.length === 0) { - setStatus('No partials extracted yet', 'warn'); - return; - } - if (!audioBuffer || !audioContext) return; - - stopAudio(); - - setStatus('Synthesizing...', 'info'); - - const keepCount = getKeepCount(); - const partialsToUse = extractedPartials.slice(0, keepCount).filter(p => !p.muted); - setStatus(`Synthesizing ${partialsToUse.length}/${extractedPartials.length} partials (${keepPct.value}%)...`, 'info'); - - const integratePhase = document.getElementById('integratePhase').checked; - const disableJitter = document.getElementById('disableJitter').checked; - const disableSpread = document.getElementById('disableSpread').checked; - const forceResonator = document.getElementById('forceResonator').checked; - const lpK1Raw = parseFloat(document.getElementById('lpK1').value); - const hpK2Raw = parseFloat(document.getElementById('hpK2').value); - const k1 = lpK1Raw < 1.0 ? lpK1Raw : null; - const k2 = hpK2Raw < 1.0 ? hpK2Raw : null; - const forceRGain = forceResonator && document.getElementById('forceRGain').checked; - const globalR = parseFloat(document.getElementById('globalR').value); - const globalGain = parseFloat(document.getElementById('globalGain').value); - const pcm = synthesizeMQ(partialsToUse, audioBuffer.sampleRate, audioBuffer.duration, - integratePhase, {disableJitter, disableSpread, forceResonator, - forceRGain, globalR, globalGain, k1, k2}); - - if (viewer) { - viewer.setSynthStftCache(new STFTCache(pcm, audioBuffer.sampleRate, fftSize, parseInt(hopSize.value))); - } - - const synthBuffer = audioContext.createBuffer(1, pcm.length, audioBuffer.sampleRate); - synthBuffer.getChannelData(0).set(pcm); - playAudioBuffer(synthBuffer, `Playing synthesized (${partialsToUse.length}/${extractedPartials.length} partials, ${keepPct.value}%)...`); - } - - // Keyboard shortcuts - document.addEventListener('keydown', (e) => { - if (e.target.tagName === 'INPUT' || e.target.tagName === 'TEXTAREA') return; - if (e.code === 'Digit1') { - e.preventDefault(); - playSynthesized(); - } else if (e.code === 'Digit2') { - e.preventDefault(); - if (!playBtn.disabled) { - playBtn.click(); - } - } else if (e.code === 'KeyP') { - e.preventDefault(); - if (viewer) viewer.togglePeaks(); - } else if (e.code === 'KeyA') { - e.preventDefault(); - if (viewer) { - viewer.showSynthFFT = !viewer.showSynthFFT; - viewer.renderSpectrum(); - } - } else if (e.code === 'KeyE') { - e.preventDefault(); - if (!extractBtn.disabled) extractBtn.click(); - } else if (e.code === 'Escape') { - if (viewer) viewer.selectPartial(-1); - } - }); - - // Curve tab switching - document.querySelectorAll('.tab-btn').forEach(btn => { - btn.addEventListener('click', () => { - document.querySelectorAll('.tab-btn').forEach(b => b.classList.remove('active')); - btn.classList.add('active'); - document.querySelectorAll('.tab-pane').forEach(p => p.style.display = 'none'); - document.getElementById('tab' + btn.dataset.tab).style.display = ''; - }); - }); - - // --- Test WAV peak validation --- - function validateTestWAVPeaks(cache) { - const SR = cache.sampleRate; - const N = cache.fftSize; - const binWidth = SR / N; // Hz per bin - const numBins = N / 2; - const numBars = 100; // mini-spectrum bar count - - // Use a mid-signal frame (avoid edge effects) - const midFrame = cache.frames[Math.floor(cache.frames.length / 2)]; - if (!midFrame) { console.error('[TestWAV] No frames computed'); return; } - const sq = midFrame.squaredAmplitude; - const t = midFrame.time; - - console.group('[TestWAV] Peak validation @ t=' + t.toFixed(3) + 's'); - - // Top 5 bins by magnitude - const ranked = Array.from(sq) - .map((v, i) => ({ bin: i, freq: i * binWidth, db: 10 * Math.log10(Math.max(v, 1e-20)) })) - .sort((a, b) => b.db - a.db); - console.log('Top 5 FFT bins:'); - ranked.slice(0, 5).forEach(x => - console.log(` bin ${x.bin.toString().padStart(3)}: ${x.freq.toFixed(1).padStart(7)}Hz ${x.db.toFixed(1)}dB`)); - - // Expected bins for 440/660 Hz - const bin440 = Math.round(440 / binWidth); - const bin660 = Math.round(660 / binWidth); - const db440 = 10 * Math.log10(Math.max(sq[bin440], 1e-20)); - const db660 = 10 * Math.log10(Math.max(sq[bin660], 1e-20)); - console.log(`440Hz → bin ${bin440} (${(bin440 * binWidth).toFixed(1)}Hz): ${db440.toFixed(1)}dB`); - console.log(`660Hz → bin ${bin660} (${(bin660 * binWidth).toFixed(1)}Hz): ${db660.toFixed(1)}dB`); - - // Validate: 440/660 Hz must be in top-10 - const top10Freqs = ranked.slice(0, 10).map(x => x.freq); - const pass440 = top10Freqs.some(f => Math.abs(f - 440) < binWidth * 2); - const pass660 = top10Freqs.some(f => Math.abs(f - 660) < binWidth * 2); - console.log('Peak check: 440Hz ' + (pass440 ? 'PASS ✓' : 'FAIL ✗') + - ', 660Hz ' + (pass660 ? 'PASS ✓' : 'FAIL ✗')); - - // Mini-spectrum: which bar do these peaks land in? - const bar440 = Math.floor(bin440 * numBars / numBins); - const bar660 = Math.floor(bin660 * numBars / numBins); - const sampledBin440 = Math.floor(bar440 * numBins / numBars); - const sampledBin660 = Math.floor(bar660 * numBars / numBars); - console.log('Mini-spectrum (linear scale, 100 bars):'); - console.log(` 440Hz (bin ${bin440}) → bar ${bar440}/100 [bar samples bin ${sampledBin440} = ${(sampledBin440 * binWidth).toFixed(1)}Hz]`); - console.log(` 660Hz (bin ${bin660}) → bar ${bar660}/100 [bar samples bin ${Math.floor(bar660 * numBins / numBars)} = ${(Math.floor(bar660 * numBins / numBars) * binWidth).toFixed(1)}Hz]`); - if (bar440 < 5 || bar660 < 5) { - console.warn(' âš BUG: peaks fall in bars ' + bar440 + ' and ' + bar660 + - ' (leftmost ~' + Math.max(bar440, bar660) * 2 + 'px of 200px canvas)' + - ' — linear scale hides low-frequency peaks. Need log-scale bar mapping.'); - } - - // Main spectrogram: confirm bins are in draw range - const mainFreqStart = 20, mainFreqEnd = 16000; - const inRange440 = 440 >= mainFreqStart && 440 <= mainFreqEnd; - const inRange660 = 660 >= mainFreqStart && 660 <= mainFreqEnd; - const norm440 = (Math.log2(440) - Math.log2(mainFreqStart)) / (Math.log2(mainFreqEnd) - Math.log2(mainFreqStart)); - const norm660 = (Math.log2(660) - Math.log2(mainFreqStart)) / (Math.log2(mainFreqEnd) - Math.log2(mainFreqStart)); - console.log('Main spectrogram (log Y-axis, 600px):'); - console.log(` 440Hz: in range=${inRange440}, y=${Math.round(600 * (1 - norm440))}px, db=${db440.toFixed(1)}dB → intensity=${Math.min(1, Math.pow(Math.max(0, (db440 + 80) / 80), 2)).toFixed(2)}`); - console.log(` 660Hz: in range=${inRange660}, y=${Math.round(600 * (1 - norm660))}px, db=${db660.toFixed(1)}dB → intensity=${Math.min(1, Math.pow(Math.max(0, (db660 + 80) / 80), 2)).toFixed(2)}`); - - console.groupEnd(); - } - </script> + <script src="app.js"></script> </body> </html> diff --git a/tools/mq_editor/mq_extract.js b/tools/mq_editor/mq_extract.js index 97fbb00..47c21b9 100644 --- a/tools/mq_editor/mq_extract.js +++ b/tools/mq_editor/mq_extract.js @@ -9,28 +9,48 @@ function extractPartials(params, stftCache) { const frames = []; for (let i = 0; i < numFrames; ++i) { const cachedFrame = stftCache.getFrameAtIndex(i); - const squaredAmp = stftCache.getSquaredAmplitude(cachedFrame.time); - const peaks = detectPeaks(squaredAmp, fftSize, sampleRate, threshold, freqWeight, prominence); + const squaredAmp = cachedFrame.squaredAmplitude; + const phase = cachedFrame.phase; + const peaks = detectPeaks(squaredAmp, phase, fftSize, sampleRate, threshold, freqWeight, prominence); frames.push({time: cachedFrame.time, peaks}); } - const partials = trackPartials(frames); + const partials = trackPartials(frames, params); // Second pass: extend partials leftward to recover onset frames expandPartialsLeft(partials, frames); for (const partial of partials) { partial.freqCurve = fitBezier(partial.times, partial.freqs); - partial.ampCurve = fitBezier(partial.times, partial.amps); + const ac = fitBezier(partial.times, partial.amps); + partial.freqCurve.a0 = ac.v0; partial.freqCurve.a1 = ac.v1; + partial.freqCurve.a2 = ac.v2; partial.freqCurve.a3 = ac.v3; } return {partials, frames}; } +// Helper to interpolate phase via quadratic formula on unwrapped neighbors. +// This provides a more accurate phase estimate at the sub-bin peak location. +function phaseInterp(p_minus, p_center, p_plus, p_frac) { + // unwrap neighbors relative to center + let dp_minus = p_minus - p_center; + while (dp_minus > Math.PI) dp_minus -= 2 * Math.PI; + while (dp_minus < -Math.PI) dp_minus += 2 * Math.PI; + + let dp_plus = p_plus - p_center; + while (dp_plus > Math.PI) dp_plus -= 2 * Math.PI; + while (dp_plus < -Math.PI) dp_plus += 2 * Math.PI; + + const p_interp = p_center + (dp_plus - dp_minus) * p_frac * 0.5 + (dp_plus + dp_minus) * 0.5 * p_frac * p_frac; + return p_interp; +} + // Detect spectral peaks via local maxima + parabolic interpolation // squaredAmp: pre-computed re*re+im*im per bin +// phase: pre-computed atan2(im,re) per bin // freqWeight: if true, weight by f before peak detection (f * Power(f)) -function detectPeaks(squaredAmp, fftSize, sampleRate, thresholdDB, freqWeight, prominenceDB = 0) { +function detectPeaks(squaredAmp, phase, fftSize, sampleRate, thresholdDB, freqWeight, prominenceDB = 0) { const mag = new Float32Array(fftSize / 2); const binHz = sampleRate / fftSize; for (let i = 0; i < fftSize / 2; ++i) { @@ -62,56 +82,84 @@ function detectPeaks(squaredAmp, fftSize, sampleRate, thresholdDB, freqWeight, p if (mag[i] - valley < prominenceDB) continue; } - // Parabolic interpolation for sub-bin accuracy + // Parabolic interpolation for sub-bin accuracy on frequency, amplitude, and phase const alpha = mag[i-1]; const beta = mag[i]; const gamma = mag[i+1]; const p = 0.5 * (alpha - gamma) / (alpha - 2*beta + gamma); + + const p_phase = phaseInterp(phase[i-1], phase[i], phase[i+1], p); const freq = (i + p) * sampleRate / fftSize; const ampDB = beta - 0.25 * (alpha - gamma) * p; - peaks.push({freq, amp: Math.pow(10, ampDB / 20)}); + peaks.push({freq, amp: Math.pow(10, ampDB / 20), phase: p_phase}); } } return peaks; } -// Track partials across frames (birth/death/continuation) -function trackPartials(frames) { +// Helper to compute shortest angle difference (e.g., between -pi and pi) +function normalizeAngle(angle) { + return angle - 2 * Math.PI * Math.floor((angle + Math.PI) / (2 * Math.PI)); +} + +// Find best matching peak for a predicted freq/phase. Returns {bestIdx, bestCost}. +function findBestPeak(peaks, matched, predictedFreq, predictedPhase, tol, phaseErrorWeight) { + let bestIdx = -1, bestCost = Infinity; + for (let i = 0; i < peaks.length; ++i) { + if (matched.has(i)) continue; + const pk = peaks[i]; + const freqError = Math.abs(pk.freq - predictedFreq); + if (freqError > tol) continue; + const phaseError = Math.abs(normalizeAngle(pk.phase - predictedPhase)); + const cost = freqError + phaseErrorWeight * phaseError * predictedFreq; + if (cost < bestCost) { bestCost = cost; bestIdx = i; } + } + return { bestIdx, bestCost }; +} + +// Track partials across frames using phase coherence for robust matching. +function trackPartials(frames, params) { + const { + sampleRate, hopSize, + birthPersistence = 3, + deathAge = 5, + minLength = 10, + phaseErrorWeight = 2.0 + } = params; const partials = []; const activePartials = []; const candidates = []; // pre-birth const trackingRatio = 0.05; // 5% frequency tolerance const minTrackingHz = 20; - const birthPersistence = 3; // frames before partial is born - const deathAge = 5; // frames without match before death - const minLength = 10; // frames required to keep partial for (const frame of frames) { const matched = new Set(); - // Continue active partials + // --- Continue active partials --- for (const partial of activePartials) { const lastFreq = partial.freqs[partial.freqs.length - 1]; + const lastPhase = partial.phases[partial.phases.length - 1]; const velocity = partial.velocity || 0; - const predicted = lastFreq + velocity; + const predictedFreq = lastFreq + velocity; - const tol = Math.max(lastFreq * trackingRatio, minTrackingHz); - let bestIdx = -1, bestDist = Infinity; - - for (let i = 0; i < frame.peaks.length; ++i) { - if (matched.has(i)) continue; - const dist = Math.abs(frame.peaks[i].freq - predicted); - if (dist < tol && dist < bestDist) { bestDist = dist; bestIdx = i; } - } + // Predict phase for the current frame based on the last frame's frequency. + // Multiply by (age+1) to account for frames missed during a gap. + const phaseAdvance = 2 * Math.PI * lastFreq * (partial.age + 1) * hopSize / sampleRate; + const predictedPhase = lastPhase + phaseAdvance; + + const tol = Math.max(predictedFreq * trackingRatio, minTrackingHz); + // Find the peak in the new frame with the lowest cost (freq + phase error). + const { bestIdx } = findBestPeak(frame.peaks, matched, predictedFreq, predictedPhase, tol, phaseErrorWeight); if (bestIdx >= 0) { const pk = frame.peaks[bestIdx]; partial.times.push(frame.time); partial.freqs.push(pk.freq); partial.amps.push(pk.amp); + partial.phases.push(pk.phase); partial.age = 0; partial.velocity = pk.freq - lastFreq; matched.add(bestIdx); @@ -120,52 +168,54 @@ function trackPartials(frames) { } } - // Advance candidates + // --- Advance candidates --- for (let i = candidates.length - 1; i >= 0; --i) { const cand = candidates[i]; const lastFreq = cand.freqs[cand.freqs.length - 1]; + const lastPhase = cand.phases[cand.phases.length - 1]; const velocity = cand.velocity || 0; - const predicted = lastFreq + velocity; + const predictedFreq = lastFreq + velocity; - const tol = Math.max(lastFreq * trackingRatio, minTrackingHz); - let bestIdx = -1, bestDist = Infinity; + // Candidates die on first miss so age is always 0 here, but kept consistent. + const phaseAdvance = 2 * Math.PI * lastFreq * hopSize / sampleRate; + const predictedPhase = lastPhase + phaseAdvance; - for (let j = 0; j < frame.peaks.length; ++j) { - if (matched.has(j)) continue; - const dist = Math.abs(frame.peaks[j].freq - predicted); - if (dist < tol && dist < bestDist) { bestDist = dist; bestIdx = j; } - } + const tol = Math.max(predictedFreq * trackingRatio, minTrackingHz); + const { bestIdx } = findBestPeak(frame.peaks, matched, predictedFreq, predictedPhase, tol, phaseErrorWeight); if (bestIdx >= 0) { const pk = frame.peaks[bestIdx]; cand.times.push(frame.time); cand.freqs.push(pk.freq); cand.amps.push(pk.amp); + cand.phases.push(pk.phase); cand.velocity = pk.freq - lastFreq; matched.add(bestIdx); + // "graduate" a candidate to a full partial if (cand.times.length >= birthPersistence) { activePartials.push(cand); candidates.splice(i, 1); } } else { - candidates.splice(i, 1); + candidates.splice(i, 1); // kill candidate } } - // Spawn candidates from unmatched peaks + // --- Spawn new candidates from unmatched peaks --- for (let i = 0; i < frame.peaks.length; ++i) { if (matched.has(i)) continue; const pk = frame.peaks[i]; candidates.push({ times: [frame.time], freqs: [pk.freq], - amps: [pk.amp], + amps: [pk.amp], + phases: [pk.phase], age: 0, velocity: 0 }); } - // Kill aged-out partials + // --- Kill aged-out partials --- for (let i = activePartials.length - 1; i >= 0; --i) { if (activePartials[i].age > deathAge) { if (activePartials[i].times.length >= minLength) partials.push(activePartials[i]); @@ -174,7 +224,7 @@ function trackPartials(frames) { } } - // Collect remaining active partials + // --- Collect remaining active partials --- for (const partial of activePartials) { if (partial.times.length >= minLength) partials.push(partial); } @@ -193,6 +243,8 @@ function expandPartialsLeft(partials, frames) { for (let i = 0; i < frames.length; ++i) timeToIdx.set(frames[i].time, i); for (const partial of partials) { + if (!partial.phases) partial.phases = []; // Ensure old partials have phase array + let startIdx = timeToIdx.get(partial.times[0]); if (startIdx == null || startIdx === 0) continue; @@ -213,6 +265,7 @@ function expandPartialsLeft(partials, frames) { partial.times.unshift(frame.time); partial.freqs.unshift(pk.freq); partial.amps.unshift(pk.amp); + partial.phases.unshift(pk.phase); } } } @@ -283,7 +336,200 @@ function autodetectSpread(partial, stftCache, fftSize, sampleRate) { return {spread_above, spread_below}; } -// Fit cubic bezier to trajectory using least-squares for inner control points +// Track a single partial starting from a (time, freq) seed position. +// Snaps to nearest spectral peak, then tracks forward and backward. +// Returns a partial object (with freqCurve), or null if no peak found near seed. +function trackFromSeed(frames, seedTime, seedFreq, params) { + if (!frames || frames.length === 0) return null; + + // Find nearest frame to seedTime + let seedFrameIdx = 0; + let bestDt = Infinity; + for (let i = 0; i < frames.length; ++i) { + const dt = Math.abs(frames[i].time - seedTime); + if (dt < bestDt) { bestDt = dt; seedFrameIdx = i; } + } + + // Snap to nearest spectral peak within 10% freq tolerance + const seedFrame = frames[seedFrameIdx]; + const snapTol = Math.max(seedFreq * 0.10, 50); + let seedPeak = null, bestDist = snapTol; + for (const pk of seedFrame.peaks) { + const d = Math.abs(pk.freq - seedFreq); + if (d < bestDist) { bestDist = d; seedPeak = pk; } + } + if (!seedPeak) return null; + + const { hopSize, sampleRate, deathAge = 5, phaseErrorWeight = 2.0 } = params; + const trackingRatio = 0.05; + const minTrackingHz = 20; + + // Forward pass from seed frame + const times = [seedFrame.time]; + const freqs = [seedPeak.freq]; + const amps = [seedPeak.amp]; + const phases = [seedPeak.phase]; + + let fwdFreq = seedPeak.freq, fwdPhase = seedPeak.phase, fwdVel = 0, fwdAge = 0; + for (let i = seedFrameIdx + 1; i < frames.length; ++i) { + const predicted = fwdFreq + fwdVel; + const predPhase = fwdPhase + 2 * Math.PI * fwdFreq * (fwdAge + 1) * hopSize / sampleRate; + const tol = Math.max(predicted * trackingRatio, minTrackingHz); + const { bestIdx } = findBestPeak(frames[i].peaks, new Set(), predicted, predPhase, tol, phaseErrorWeight); + if (bestIdx >= 0) { + const pk = frames[i].peaks[bestIdx]; + times.push(frames[i].time); + freqs.push(pk.freq); + amps.push(pk.amp); + phases.push(pk.phase); + fwdVel = pk.freq - fwdFreq; + fwdFreq = pk.freq; fwdPhase = pk.phase; fwdAge = 0; + } else { + fwdAge++; + if (fwdAge > deathAge) break; + } + } + + // Backward pass from seed frame + const bwdTimes = [], bwdFreqs = [], bwdAmps = [], bwdPhases = []; + let bwdFreq = seedPeak.freq, bwdAge = 0; + for (let i = seedFrameIdx - 1; i >= 0; --i) { + const tol = Math.max(bwdFreq * trackingRatio, minTrackingHz); + let bestIdx = -1, bDist = tol; + for (let j = 0; j < frames[i].peaks.length; ++j) { + const d = Math.abs(frames[i].peaks[j].freq - bwdFreq); + if (d < bDist) { bDist = d; bestIdx = j; } + } + if (bestIdx >= 0) { + const pk = frames[i].peaks[bestIdx]; + bwdTimes.unshift(frames[i].time); + bwdFreqs.unshift(pk.freq); + bwdAmps.unshift(pk.amp); + bwdPhases.unshift(pk.phase); + bwdFreq = pk.freq; bwdAge = 0; + } else { + bwdAge++; + if (bwdAge > deathAge) break; + } + } + + const allTimes = [...bwdTimes, ...times]; + const allFreqs = [...bwdFreqs, ...freqs]; + const allAmps = [...bwdAmps, ...amps]; + const allPhases = [...bwdPhases, ...phases]; + + if (allTimes.length < 2) return null; + + const freqCurve = fitBezier(allTimes, allFreqs); + const ac = fitBezier(allTimes, allAmps); + freqCurve.a0 = ac.v0; freqCurve.a1 = ac.v1; + freqCurve.a2 = ac.v2; freqCurve.a3 = ac.v3; + + return { + times: allTimes, freqs: allFreqs, amps: allAmps, phases: allPhases, + muted: false, freqCurve, + harmonics: { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }, + }; +} + +// Track an iso-energy contour starting from (seedTime, seedFreq). +// Instead of following spectral peaks, follows where energy ≈ seedEnergy. +// Useful for broad/diffuse bass regions with no detectable peaks. +// Returns a partial with large default spread, or null if seed energy is zero. +function trackIsoContour(stftCache, seedTime, seedFreq, params) { + const { sampleRate, deathAge = 8 } = params; + const numFrames = stftCache.getNumFrames(); + const fftSize = stftCache.fftSize; + const binHz = sampleRate / fftSize; + const halfBins = fftSize / 2; + + // Find seed frame + let seedFrameIdx = 0, bestDt = Infinity; + for (let i = 0; i < numFrames; ++i) { + const dt = Math.abs(stftCache.getFrameAtIndex(i).time - seedTime); + if (dt < bestDt) { bestDt = dt; seedFrameIdx = i; } + } + + const seedFrame = stftCache.getFrameAtIndex(seedFrameIdx); + const seedBin = clamp(Math.round(seedFreq / binHz), 1, halfBins - 2); + const targetSq = seedFrame.squaredAmplitude[seedBin]; + if (targetSq <= 0) return null; + const targetDB = 10 * Math.log10(targetSq); + + const trackingRatio = 0.15; // larger search window than peak tracker + const minTrackHz = 30; + const maxDbDev = 15; // dB: declare miss if nothing within this range + + // Find bin minimizing |dB(b) - targetDB| near refBin, with mild position bias. + function findContourBin(sq, refBin) { + const tol = Math.max(refBin * binHz * trackingRatio, minTrackHz); + const tolBins = Math.ceil(tol / binHz); + const lo = Math.max(1, refBin - tolBins); + const hi = Math.min(halfBins - 2, refBin + tolBins); + let bestBin = -1, bestCost = Infinity; + for (let b = lo; b <= hi; ++b) { + const dE = Math.abs(10 * Math.log10(Math.max(sq[b], 1e-20)) - targetDB); + if (dE > maxDbDev) continue; + const dPos = Math.abs(b - refBin) / Math.max(1, tolBins); + const cost = dE + 3 * dPos; // energy match dominates, position breaks ties + if (cost < bestCost) { bestCost = cost; bestBin = b; } + } + return bestBin; + } + + const times = [seedFrame.time]; + const freqs = [seedBin * binHz]; + const amps = [Math.sqrt(Math.max(0, targetSq))]; + + // Forward pass + let fwdBin = seedBin, fwdAge = 0; + for (let i = seedFrameIdx + 1; i < numFrames; ++i) { + const frame = stftCache.getFrameAtIndex(i); + const b = findContourBin(frame.squaredAmplitude, fwdBin); + if (b >= 0) { + times.push(frame.time); + freqs.push(b * binHz); + amps.push(Math.sqrt(Math.max(0, frame.squaredAmplitude[b]))); + fwdBin = b; fwdAge = 0; + } else { if (++fwdAge > deathAge) break; } + } + + // Backward pass + const bwdTimes = [], bwdFreqs = [], bwdAmps = []; + let bwdBin = seedBin, bwdAge = 0; + for (let i = seedFrameIdx - 1; i >= 0; --i) { + const frame = stftCache.getFrameAtIndex(i); + const b = findContourBin(frame.squaredAmplitude, bwdBin); + if (b >= 0) { + bwdTimes.unshift(frame.time); + bwdFreqs.unshift(b * binHz); + bwdAmps.unshift(Math.sqrt(Math.max(0, frame.squaredAmplitude[b]))); + bwdBin = b; bwdAge = 0; + } else { if (++bwdAge > deathAge) break; } + } + + const allTimes = [...bwdTimes, ...times]; + const allFreqs = [...bwdFreqs, ...freqs]; + const allAmps = [...bwdAmps, ...amps]; + if (allTimes.length < 2) return null; + + const freqCurve = fitBezier(allTimes, allFreqs); + const ac = fitBezier(allTimes, allAmps); + freqCurve.a0 = ac.v0; freqCurve.a1 = ac.v1; + freqCurve.a2 = ac.v2; freqCurve.a3 = ac.v3; + + return { + times: allTimes, freqs: allFreqs, amps: allAmps, + phases: new Array(allTimes.length).fill(0), + muted: false, freqCurve, + harmonics: { decay: 0.0, freq_mult: 2.0, jitter: 0.05, spread_above: 0.15, spread_below: 0.15 }, + }; +} + +// Fit interpolating curve to trajectory via least-squares for inner control point values. +// Inner knots fixed at u=1/3 and u=2/3 (t = t0+dt/3, t0+2*dt/3). +// The curve passes through all 4 control points (Lagrange interpolation). +// TODO: support arbitrary number of inner control points function fitBezier(times, values) { const n = times.length - 1; const t0 = times[0], v0 = values[0]; @@ -291,43 +537,35 @@ function fitBezier(times, values) { const dt = t3 - t0; if (dt <= 1e-9 || n < 2) { - // Linear fallback for too few points or zero duration return {t0, v0, t1: t0 + dt / 3, v1: v0 + (v3 - v0) / 3, t2: t0 + 2 * dt / 3, v2: v0 + 2 * (v3 - v0) / 3, t3, v3}; } - // Least squares solve for v1, v2 - // Bezier: B(u) = (1-u)^3*v0 + 3(1-u)^2*u*v1 + 3(1-u)*u^2*v2 + u^3*v3 - // Target_i = val_i - (1-u)^3*v0 - u^3*v3 - // Model_i = A_i*v1 + B_i*v2 - // A_i = 3(1-u)^2*u - // B_i = 3(1-u)*u^2 + // Lagrange basis with inner knots at u1=1/3, u2=2/3 + // l1(u) = u*(u-2/3)*(u-1) / ((1/3)*(1/3-2/3)*(1/3-1)) = 13.5*u*(u-2/3)*(u-1) + // l2(u) = u*(u-1/3)*(u-1) / ((2/3)*(2/3-1/3)*(2/3-1)) = -13.5*u*(u-1/3)*(u-1) + // l0(u) = (u-1/3)*(u-2/3)*(u-1) / ((-1/3)*(-2/3)*(-1)) = -4.5*(u-1/3)*(u-2/3)*(u-1) + // l3(u) = u*(u-1/3)*(u-2/3) / ((2/3)*(1/3)) = 4.5*u*(u-1/3)*(u-2/3) + // Least-squares: minimize Σ(l1*v1 + l2*v2 - target_i)^2 + // target_i = values[i] - l0*v0 - l3*v3 let sA2 = 0, sB2 = 0, sAB = 0, sAT = 0, sBT = 0; for (let i = 0; i <= n; ++i) { - const u = (times[i] - t0) / dt; - const u2 = u * u; - const u3 = u2 * u; - const invU = 1.0 - u; - const invU2 = invU * invU; - const invU3 = invU2 * invU; - - const A = 3 * invU2 * u; - const B = 3 * invU * u2; - const target = values[i] - (invU3 * v0 + u3 * v3); - - sA2 += A * A; - sB2 += B * B; - sAB += A * B; - sAT += A * target; - sBT += B * target; + const u = (times[i] - t0) / dt; + const l0 = -4.5 * (u - 1/3) * (u - 2/3) * (u - 1); + const l1 = 13.5 * u * (u - 2/3) * (u - 1); + const l2 = -13.5 * u * (u - 1/3) * (u - 1); + const l3 = 4.5 * u * (u - 1/3) * (u - 2/3); + const A = l1, B = l2; + const target = values[i] - l0 * v0 - l3 * v3; + sA2 += A * A; sB2 += B * B; sAB += A * B; + sAT += A * target; sBT += B * target; } const det = sA2 * sB2 - sAB * sAB; let v1, v2; if (Math.abs(det) < 1e-9) { - // Fallback to simple 1/3, 2/3 heuristic if matrix is singular const idx1 = Math.round(n / 3); const idx2 = Math.round(2 * n / 3); v1 = values[idx1]; diff --git a/tools/mq_editor/mq_synth.js b/tools/mq_editor/mq_synth.js index 2d4cf1b..e5f7e1a 100644 --- a/tools/mq_editor/mq_synth.js +++ b/tools/mq_editor/mq_synth.js @@ -1,18 +1,5 @@ // MQ Synthesizer -// Replica oscillator bank for sinusoidal synthesis, plus two-pole resonator mode - -// Evaluate cubic bezier curve at time t -function evalBezier(curve, t) { - const dt = curve.t3 - curve.t0; - if (dt <= 0) return curve.v0; - let u = (t - curve.t0) / dt; - u = Math.max(0, Math.min(1, u)); - const u1 = 1.0 - u; - return u1*u1*u1 * curve.v0 + - 3*u1*u1*u * curve.v1 + - 3*u1*u*u * curve.v2 + - u*u*u * curve.v3; -} +// Harmonic oscillator bank for sinusoidal synthesis, plus two-pole resonator mode // Deterministic LCG PRNG function randFloat(seed, min, max) { @@ -20,9 +7,26 @@ function randFloat(seed, min, max) { return min + (seed / 0x100000000) * (max - min); } +// Build harmonic list from harmonics config. +// Fundamental (ratio=1.0, ampMult=1.0) is always first. +// Then harmonics at n*freq_mult for n=1,2,... with ampMult=decay^n (added on top). +function buildHarmonics(harmonics) { + const decay = Math.min(harmonics.decay ?? 0.0, 0.90); + const freqMult = harmonics.freq_mult ?? 2.0; + const result = [{ ratio: 1.0, ampMult: 1.0 }]; // fundamental always + if (decay > 0) { + for (let n = 1; ; ++n) { + const ampMult = Math.pow(decay, n); + if (ampMult < 0.001) break; + result.push({ ratio: n * freqMult, ampMult }); + } + } + return result; +} + // Synthesize audio from MQ partials -// partials: array of {freqCurve, ampCurve, replicas?, resonator?} -// replicas: {offsets, decay_alpha, jitter, spread_above, spread_below} +// partials: array of {freqCurve (with a0-a3 for amp), harmonics?, resonator?} +// harmonics: {decay, freq_mult, jitter, spread_above, spread_below} // resonator: {enabled, r, gainComp} — two-pole resonator mode per partial // integratePhase: true = accumulate 2Ï€*f/SR per sample (correct for varying freq) // false = 2Ï€*f*t (simpler, only correct for constant freq) @@ -32,15 +36,12 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt const numSamples = Math.floor(sampleRate * duration); const pcm = new Float32Array(numSamples); - const jitterMult = options.disableJitter ? 0 : 1; - const spreadMult = options.disableSpread ? 0 : 1; - - const defaultReplicas = { - offsets: [1.0], - decay_alpha: 0.1, - jitter: 0.05, - spread_above: 0.02, - spread_below: 0.02 + const defaultHarmonics = { + decay: 0.0, + freq_mult: 1.0, + jitter: 0.05, + spread_above: 0.02, + spread_below: 0.02 }; // Pre-build per-partial configs with fixed spread/jitter and phase accumulators @@ -48,7 +49,6 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt for (let p = 0; p < partials.length; ++p) { const partial = partials[p]; const fc = partial.freqCurve; - const ac = partial.ampCurve; if ((partial.resonator && partial.resonator.enabled) || options.forceResonator) { // --- Two-pole resonator mode --- @@ -56,34 +56,41 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt // r controls pole radius (bandwidth): r→1 = narrow, r→0 = wide. // gainNorm = sqrt(1 - r²) normalises steady-state output power to ~A. const res = partial.resonator || {}; - const r = options.forceRGain ? Math.min(0.9999, Math.max(0, options.globalR)) - : (res.r != null ? Math.min(0.9999, Math.max(0, res.r)) : 0.995); + const r = options.forceRGain ? clamp(options.globalR, 0, 0.9999) + : (res.r != null ? clamp(res.r, 0, 0.9999) : 0.995); const gainComp = options.forceRGain ? options.globalGain : (res.gainComp != null ? res.gainComp : 1.0); const gainNorm = Math.sqrt(Math.max(0, 1.0 - r * r)); + + // Build harmonic list (jitter/spread not applied to resonator) + const harm = partial.harmonics || defaultHarmonics; + const harmonicList = buildHarmonics(harm); + configs.push({ mode: 'resonator', - fc, ac, + fc, r, gainComp, gainNorm, - y1: 0.0, y2: 0.0, + harmonicList, + y1: new Float64Array(harmonicList.length), + y2: new Float64Array(harmonicList.length), noiseSeed: ((p * 1664525 + 1013904223) & 0xFFFFFFFF) >>> 0 }); } else { - // --- Sinusoidal (replica) mode --- - const rep = partial.replicas != null ? partial.replicas : defaultReplicas; - const offsets = rep.offsets != null ? rep.offsets : [1.0]; - const decay_alpha = rep.decay_alpha != null ? rep.decay_alpha : 0.0; - const jitter = rep.jitter != null ? rep.jitter : 0.0; - const spread_above = rep.spread_above != null ? rep.spread_above : 0.0; - const spread_below = rep.spread_below != null ? rep.spread_below : 0.0; + // --- Sinusoidal (harmonic) mode --- + const harm = partial.harmonics || defaultHarmonics; + const spread_above = harm.spread_above ?? 0.0; + const spread_below = harm.spread_below ?? 0.0; + const jitter = harm.jitter ?? 0.0; + const harmonicList = buildHarmonics(harm); const replicaData = []; - for (let r = 0; r < offsets.length; ++r) { - const spread = spreadMult * randFloat(p * 67890 + r * 999, -spread_below, spread_above); - const initPhase = randFloat(p * 67890 + r, 0.0, 1.0) * (jitter * jitterMult) * 2.0 * Math.PI; - replicaData.push({ratio: offsets[r], spread, phase: initPhase}); + for (let h = 0; h < harmonicList.length; ++h) { + const hc = harmonicList[h]; + const spread = randFloat(p * 67890 + h * 999, -spread_below, spread_above); + const initPhase = randFloat(p * 67890 + h, 0.0, 1.0) * jitter * 2.0 * Math.PI; + replicaData.push({ ratio: hc.ratio, ampMult: hc.ampMult, spread, phase: initPhase }); } - configs.push({ mode: 'sinusoid', fc, ac, decay_alpha, replicaData }); + configs.push({ mode: 'sinusoid', fc, replicaData }); } } @@ -93,37 +100,43 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt for (let p = 0; p < configs.length; ++p) { const cfg = configs[p]; - const {fc, ac} = cfg; + const {fc} = cfg; if (cfg.mode === 'resonator') { - if (t < fc.t0 || t > fc.t3) { cfg.y1 = 0.0; cfg.y2 = 0.0; continue; } + if (t < fc.t0 || t > fc.t3) { + cfg.y1.fill(0.0); cfg.y2.fill(0.0); continue; + } - const f0 = evalBezier(fc, t); - const A = evalBezier(ac, t); - const omega = 2.0 * Math.PI * f0 / sampleRate; - const b1 = 2.0 * cfg.r * Math.cos(omega); + const f0 = evalBezier(fc, t); + const A = evalBezierAmp(fc, t); - // LCG noise excitation (deterministic per-partial) + // LCG noise excitation (deterministic per-partial, shared across harmonics) cfg.noiseSeed = (Math.imul(1664525, cfg.noiseSeed) + 1013904223) >>> 0; const noise = cfg.noiseSeed / 0x100000000 * 2.0 - 1.0; - const x = A * cfg.gainNorm * noise; - const y = b1 * cfg.y1 - cfg.r * cfg.r * cfg.y2 + x; - cfg.y2 = cfg.y1; - cfg.y1 = y; - sample += y * cfg.gainComp; + for (let h = 0; h < cfg.harmonicList.length; ++h) { + const hc = cfg.harmonicList[h]; + const fh = f0 * hc.ratio; + const omega = 2.0 * Math.PI * fh / sampleRate; + const b1 = 2.0 * cfg.r * Math.cos(omega); + + const x = A * cfg.gainNorm * noise * hc.ampMult; + const y = b1 * cfg.y1[h] - cfg.r * cfg.r * cfg.y2[h] + x; + cfg.y2[h] = cfg.y1[h]; + cfg.y1[h] = y; + sample += y * cfg.gainComp; + } } else { if (t < fc.t0 || t > fc.t3) continue; const f0 = evalBezier(fc, t); - const A0 = evalBezier(ac, t); - const {decay_alpha, replicaData} = cfg; + const A0 = evalBezierAmp(fc, t); - for (let r = 0; r < replicaData.length; ++r) { - const rep = replicaData[r]; - const f = f0 * rep.ratio * (1.0 + rep.spread); - const A = A0 * Math.exp(-decay_alpha * Math.abs(f - f0)); + for (let h = 0; h < cfg.replicaData.length; ++h) { + const rep = cfg.replicaData[h]; + const f = f0 * rep.ratio * (1.0 + rep.spread); + const A = A0 * rep.ampMult; let phase; if (integratePhase) { @@ -145,7 +158,7 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt // LP: y[n] = k1*x[n] + (1-k1)*y[n-1] — options.k1 in (0,1], omit to bypass // HP: y[n] = k2*(y[n-1] + x[n] - x[n-1]) — options.k2 in (0,1], omit to bypass if (options.k1 != null) { - const k1 = Math.max(0, Math.min(1, options.k1)); + const k1 = clamp(options.k1, 0, 1); let y = 0.0; for (let i = 0; i < numSamples; ++i) { y = k1 * pcm[i] + (1.0 - k1) * y; @@ -153,7 +166,7 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt } } if (options.k2 != null) { - const k2 = Math.max(0, Math.min(1, options.k2)); + const k2 = clamp(options.k2, 0, 1); let y = 0.0, xp = 0.0; for (let i = 0; i < numSamples; ++i) { const x = pcm[i]; diff --git a/tools/mq_editor/style.css b/tools/mq_editor/style.css new file mode 100644 index 0000000..07a404a --- /dev/null +++ b/tools/mq_editor/style.css @@ -0,0 +1,104 @@ +/* MQ Spectral Editor */ + +/* === Base === */ +body { font-family: monospace; font-size: 14px; margin: 20px; background: #1a1a1a; color: #ddd; } +label { margin-right: 4px; } +input[type="number"], select { width: 80px; background: #3a3a3a; color: #ddd; border: 1px solid #555; padding: 4px; border-radius: 3px; } +input[type="file"] { display: none; } + +/* === Layout === */ +.page-title { display: flex; align-items: baseline; gap: 16px; margin-bottom: 10px; } +.page-title h2 { margin: 0; } +.main-area { display: flex; align-items: flex-start; gap: 10px; margin-top: 10px; } +.canvas-col { display: flex; flex-direction: column; gap: 6px; flex-shrink: 0; } +.canvas-wrap { position: relative; } + +/* === Toolbar === */ +.toolbar { margin-bottom: 10px; padding: 10px; background: #2a2a2a; border-radius: 4px; display: flex; align-items: center; flex-wrap: wrap; gap: 4px; } +.toolbar-group { display: inline-flex; align-items: center; gap: 2px; } +.toolbar-group button { margin-right: 0; } +.toolbar-sep { align-self: stretch; width: 1px; background: #555; margin: 3px 6px; } +.toolbar-wrap { position: relative; display: inline-block; } + +/* === Buttons === */ +button { background: #3a3a3a; color: #ddd; border: 1px solid #555; padding: 8px 16px; margin-right: 8px; cursor: pointer; border-radius: 4px; } +button:hover { background: #4a4a4a; } +button:disabled { opacity: 0.5; cursor: not-allowed; } +#extractBtn { background: #666; color: #fff; font-weight: bold; border-color: #888; } +button.explore-active { background: #554; border-color: #aa8; color: #ffd; } +button.contour-active { background: #145; border-color: #0cc; color: #aff; } + +/* === Params dropdown === */ +#paramsPanel { display: none; position: absolute; z-index: 200; top: 100%; left: 0; margin-top: 4px; background: #222; border: 1px solid #555; border-radius: 4px; padding: 6px 0; white-space: nowrap; box-shadow: 0 4px 12px rgba(0,0,0,.5); } +#paramsPanel.open { display: block; } +#paramsBtn.params-open { background: #4a4a4a; border-color: #888; } +.param-group { display: grid; grid-template-columns: auto 1fr; gap: 4px 10px; padding: 8px 14px; border-bottom: 1px solid #333; align-items: center; } +.param-group:last-child { border-bottom: none; } +.group-label { grid-column: 1 / -1; font-size: 9px; color: #666; text-transform: uppercase; letter-spacing: 1px; } + +/* === Canvas & overlays === */ +#canvas { border: 1px solid #555; background: #000; cursor: crosshair; display: block; flex-shrink: 0; } +#cursorCanvas, #playheadCanvas { position: absolute; top: 0; left: 0; pointer-events: none; } +#partialSpectrumViewer { position: absolute; bottom: 10px; right: 420px; width: 200px; height: 100px; background: rgba(30,30,30,.9); border: 1px solid #555; border-radius: 3px; pointer-events: none; } +#spectrumViewer { position: absolute; bottom: 10px; right: 10px; width: 400px; height: 100px; background: rgba(30,30,30,.9); border: 1px solid #555; border-radius: 3px; pointer-events: none; } +#keepOverlay { position: absolute; bottom: 10px; left: 10px; background: rgba(30,30,30,.88); border: 1px solid #555; border-radius: 3px; padding: 4px 8px; display: flex; align-items: center; gap: 6px; font-size: 12px; color: #aaa; user-select: none; } +#keepOverlay input[type="range"] { width: 90px; } + +/* === Amp edit panel === */ +#ampEditPanel { display: none; } +.amp-edit-header { font-size: 10px; color: #555; padding: 2px 0 3px 1px; display: flex; align-items: center; gap: 10px; text-transform: uppercase; letter-spacing: .5px; } +#ampEditTitle { color: #777; text-transform: none; letter-spacing: 0; } +.amp-edit-hint { color: #333; text-transform: none; letter-spacing: 0; } +#ampEditCanvas { border: 1px solid #2a2a2a; background: #0e0e0e; cursor: crosshair; display: block; } + +/* === Right panel === */ +.right-panel { background: #2a2a2a; border: 1px solid #555; border-radius: 4px; padding: 12px; min-width: 260px; max-width: 260px; max-height: 700px; overflow-y: auto; display: flex; flex-direction: column; gap: 6px; box-sizing: border-box; } +.panel-title { font-size: 12px; color: #888; text-transform: uppercase; letter-spacing: 1px; border-bottom: 1px solid #444; padding-bottom: 6px; margin-bottom: 2px; display: flex; align-items: center; gap: 6px; } +.right-panel label { display: flex; align-items: center; gap: 6px; margin: 0; cursor: pointer; font-size: 14px; } +.right-panel label input[type="range"] { flex: 1; min-width: 0; } +#noSelMsg { color: #555; font-size: 13px; padding: 2px 0; } + +/* === Partial properties === */ +.prop-row { display: flex; justify-content: space-between; align-items: baseline; font-size: 13px; padding: 2px 0; } +.prop-label { color: #777; font-size: 12px; } +#propSwatch { display: inline-block; width: 10px; height: 10px; border-radius: 2px; flex-shrink: 0; } +.partial-actions { display: flex; gap: 4px; margin-top: 8px; } +.partial-actions button { flex: 1; padding: 4px 6px; font-size: 12px; margin: 0; } +kbd { font-size: 10px; opacity: 0.55; } + +/* === Curve tabs & grid === */ +.curve-tabs { display: flex; gap: 2px; margin-top: 8px; margin-bottom: 4px; } +.tab-btn { flex: 1; padding: 4px 0; font-size: 12px; margin: 0; background: #222; border-color: #444; color: #888; } +.tab-btn.active { background: #3a3a3a; border-color: #666; color: #ddd; } +.curve-grid { display: grid; grid-template-columns: 18px 1fr 1fr; gap: 3px 4px; align-items: center; } +.curve-grid span { color: #666; font-size: 11px; } +.curve-grid input[type="number"], +.synth-field-wrap input[type="number"] { width: 100%; background: #333; color: #ccc; border: 1px solid #444; padding: 2px 4px; border-radius: 2px; font-size: 11px; font-family: monospace; box-sizing: border-box; } +.curve-grid input[type="number"]:focus, +.synth-field-wrap input[type="number"]:focus, +.synth-grid input[type="number"]:focus { border-color: #666; outline: none; } + +/* === Synth section === */ +.synth-section { border-top: 1px solid #444; padding-top: 8px; margin-top: auto; } +.synth-grid { display: grid; grid-template-columns: auto 1fr; gap: 4px 8px; align-items: center; } +.synth-grid span { color: #888; font-size: 12px; } +.synth-field-wrap { display: flex; align-items: center; gap: 4px; } +#globalResParams { margin-top: 4px; padding: 4px 0 2px 12px; border-left: 2px solid #555; } +.slider-val { width: 44px; text-align: right; } + +/* === Jog slider === */ +.jog-slider { width: 44px; height: 16px; background: #1e1e1e; border: 1px solid #444; border-radius: 3px; cursor: ew-resize; position: relative; flex-shrink: 0; user-select: none; overflow: hidden; } +.jog-slider::before { content: ''; position: absolute; left: 50%; top: 3px; bottom: 3px; width: 1px; background: #484848; transform: translateX(-50%); } +.jog-thumb { position: absolute; top: 3px; bottom: 3px; width: 6px; background: #888; border-radius: 2px; left: calc(50% - 3px); transition: left .12s ease; } +.jog-slider:hover .jog-thumb { background: #aaa; } + +/* === Resonator badge === */ +.res-badge { font-size: 9px; color: #8cf; border: 1px solid #8cf; border-radius: 2px; padding: 0 3px; vertical-align: middle; margin-left: 4px; opacity: .8; } + +/* === Status & tooltip === */ +#fileLabel { font-size: 13px; color: #8af; opacity: .8; } +#status { margin-top: 10px; padding: 8px; background: #2a2a2a; border-radius: 4px; min-height: 20px; } +.info { color: #4af; } +.warn { color: #fa4; } +.error { color: #f44; } +#tooltip { position: fixed; display: none; background: #2a2a2a; padding: 4px 8px; border: 1px solid #555; border-radius: 3px; pointer-events: none; font-size: 11px; z-index: 1000; } diff --git a/tools/mq_editor/test_fft.html b/tools/mq_editor/test_fft.html deleted file mode 100644 index b4e7f48..0000000 --- a/tools/mq_editor/test_fft.html +++ /dev/null @@ -1,229 +0,0 @@ -<!DOCTYPE html> -<!-- - test_fft.html — Isolated FFT correctness tests for fft.js - Open directly in a browser (no server needed). - - Tests fftForward(), realFFT(), and STFTCache from fft.js. - - Test summary: - 1 DC impulse — all-ones input → bin 0 must equal N - 2 Single tone — 440 Hz pure sine → peak bin matches expected frequency - 3 STFT magnitude — STFTCache.getMagnitudeDB() returns loud value at 440 Hz - 4 Pair equal — 220 + 880 Hz, both peaks found - 5 Pair equal — 440 + 1320 Hz, both peaks found - 6 Pair wide — 300 + 3000 Hz (decade separation), both peaks found - 7 Pair extreme — 100 + 8000 Hz (80× ratio), both peaks found - 8 Pair unequal — 440 Hz + 2000 Hz at 10:1 amplitude, weak peak still found - 9 Triplet chord — C4 (261.63) + E4 (329.63) + G4 (392) major chord - 10 Triplet octaves — 110 + 220 + 440 Hz octave stack - 11 Triplet harmonic — 500 + 1500 + 4500 Hz (1:3:9 ratio) - 12 Triplet unequal — 440 + 880 + 1760 Hz at 1.0 / 0.5 / 0.25 (decaying harmonics) - - Pass criteria: - - Each expected frequency bin is found within ±guard bins (guard ≈ 2 × bin_width). - - Bin width = SR / N = 32000 / 4096 ≈ 7.8 Hz. - - All spectra are drawn as linear-magnitude plots (0..Nyquist on x-axis). - Colored vertical markers show expected frequency positions. ---> -<html> -<head> -<meta charset="utf-8"> -<title>FFT Test</title> -<style> - body { font-family: monospace; background: #111; color: #ccc; padding: 20px; } - h2 { color: #fff; } - canvas { border: 1px solid #444; display: block; margin: 10px 0; } - .pass { color: #4f4; } - .fail { color: #f44; } - pre { background: #222; padding: 10px; } -</style> -</head> -<body> -<h2>FFT Isolation Test</h2> -<pre id="log"></pre> -<canvas id="spectrum" width="800" height="200"></canvas> -<script src="fft.js"></script> -<script> -const log = document.getElementById('log'); -function print(msg, cls) { - const span = document.createElement('span'); - if (cls) span.className = cls; - span.textContent = msg + '\n'; - log.appendChild(span); -} - -// --- Test 1: Single bin impulse (DC) --- -{ - const N = 8; - const real = new Float32Array([1,1,1,1,1,1,1,1]); - const imag = new Float32Array(N); - fftForward(real, imag, N); - const ok = Math.abs(real[0] - 8) < 1e-4 && Math.abs(imag[0]) < 1e-4; - print(`Test 1 DC impulse: real[0]=${real[0].toFixed(4)} ${ok?'PASS':'FAIL'}`, ok?'pass':'fail'); -} - -// --- Test 2: 440 Hz peak detection --- -{ - const SR = 32000; - const N = 2048; - const signal = new Float32Array(N); - for (let i = 0; i < N; ++i) - signal[i] = Math.sin(2 * Math.PI * 440 * i / SR); - - const spectrum = realFFT(signal); - - let peakBin = 0, peakVal = 0; - for (let i = 0; i < N / 2; ++i) { - const re = spectrum[i * 2], im = spectrum[i * 2 + 1]; - const mag = re * re + im * im; - if (mag > peakVal) { peakVal = mag; peakBin = i; } - } - const peakFreq = peakBin * SR / N; - const ok = Math.abs(peakFreq - 440) < SR / N; - print(`Test 2 440Hz peak: bin=${peakBin} freq=${peakFreq.toFixed(1)}Hz ${ok?'PASS':'FAIL'}`, ok?'pass':'fail'); - - // Draw spectrum - const canvas = document.getElementById('spectrum'); - const ctx = canvas.getContext('2d'); - ctx.fillStyle = '#111'; - ctx.fillRect(0, 0, canvas.width, canvas.height); - ctx.strokeStyle = '#0af'; - ctx.beginPath(); - const halfN = N / 2; - for (let i = 0; i < halfN; ++i) { - const re = spectrum[i * 2], im = spectrum[i * 2 + 1]; - const mag = Math.sqrt(re * re + im * im) / (N / 2); - const x = i / halfN * canvas.width; - const y = canvas.height - mag * canvas.height; - i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y); - } - ctx.stroke(); - - // Mark 440Hz - ctx.strokeStyle = '#f80'; - ctx.beginPath(); - const x440 = peakBin / halfN * canvas.width; - ctx.moveTo(x440, 0); ctx.lineTo(x440, canvas.height); - ctx.stroke(); - ctx.fillStyle = '#f80'; - ctx.fillText(`440Hz (bin ${peakBin})`, x440 + 4, 16); -} - -// --- Test 3: STFT getMagnitudeDB at t=0 --- -{ - const SR = 32000; - const N = 44032; // ~1.375s - const signal = new Float32Array(N); - for (let i = 0; i < N; ++i) - signal[i] = Math.sin(2 * Math.PI * 440 * i / SR); - - const stft = new STFTCache(signal, SR, 2048, 512); - const db = stft.getMagnitudeDB(0.0, 440); - const ok = db > -10; - print(`Test 3 STFT getMagnitudeDB(0, 440Hz): ${db.toFixed(1)} dB ${ok?'PASS':'FAIL'}`, ok?'pass':'fail'); -} - -// Helper: find top-K peaks in spectrum (bin indices), ignoring neighbors within 'guard' bins -function findPeaks(spectrum, halfN, k, guard) { - const mags = new Float32Array(halfN); - for (let i = 0; i < halfN; ++i) { - const re = spectrum[i * 2], im = spectrum[i * 2 + 1]; - mags[i] = re * re + im * im; - } - const peaks = []; - const used = new Uint8Array(halfN); - for (let p = 0; p < k; ++p) { - let best = -1, bestVal = 0; - for (let i = 1; i < halfN; ++i) { - if (!used[i] && mags[i] > bestVal) { bestVal = mags[i]; best = i; } - } - if (best < 0) break; - peaks.push(best); - for (let g = Math.max(0, best - guard); g <= Math.min(halfN - 1, best + guard); ++g) - used[g] = 1; - } - return peaks; -} - -// Helper: draw labeled spectrum on a new canvas -function drawSpectrum(label, spectrum, N, SR, markerFreqs) { - const halfN = N / 2; - const colors = ['#f80', '#0f8', '#f0f', '#08f']; - const canvas = document.createElement('canvas'); - canvas.width = 800; canvas.height = 160; - const div = document.createElement('div'); - div.style.cssText = 'color:#888;font-family:monospace;font-size:12px;margin-top:8px'; - div.textContent = label; - document.body.appendChild(div); - document.body.appendChild(canvas); - - const ctx = canvas.getContext('2d'); - ctx.fillStyle = '#111'; - ctx.fillRect(0, 0, canvas.width, canvas.height); - ctx.strokeStyle = '#0af'; - ctx.beginPath(); - for (let i = 0; i < halfN; ++i) { - const re = spectrum[i * 2], im = spectrum[i * 2 + 1]; - const mag = Math.sqrt(re * re + im * im) / (N / 2); - const x = i / halfN * canvas.width; - const y = canvas.height - mag * canvas.height; - i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y); - } - ctx.stroke(); - - markerFreqs.forEach((f, idx) => { - const bin = Math.round(f * N / SR); - const x = bin / halfN * canvas.width; - ctx.strokeStyle = colors[idx % colors.length]; - ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, canvas.height); ctx.stroke(); - ctx.fillStyle = colors[idx % colors.length]; - ctx.fillText(`${f}Hz`, x + 3, 14 + idx * 14); - }); -} - -// Helper: test multi-frequency signal, return pass/fail -function testMultiFreq(label, freqs, amplitudes, SR, N, testNum) { - const signal = new Float32Array(N); - for (let i = 0; i < N; ++i) - for (let f = 0; f < freqs.length; ++f) - signal[i] += (amplitudes ? amplitudes[f] : 1.0) * Math.sin(2 * Math.PI * freqs[f] * i / SR); - - const spectrum = realFFT(signal); - const halfN = N / 2; - const guard = Math.ceil(SR / N * 2); // ~2 bins tolerance - const peaks = findPeaks(spectrum, halfN, freqs.length, guard); - - const detectedFreqs = peaks.map(b => (b * SR / N).toFixed(1)); - const allFound = freqs.every(f => { - const expectedBin = Math.round(f * N / SR); - return peaks.some(b => Math.abs(b - expectedBin) <= guard); - }); - - const freqStr = freqs.map((f, i) => amplitudes ? `${f}Hz@${amplitudes[i].toFixed(1)}` : `${f}Hz`).join(' + '); - print(`Test ${testNum} ${label} [${freqStr}]: peaks=[${detectedFreqs.join(', ')}] ${allFound?'PASS':'FAIL'}`, - allFound ? 'pass' : 'fail'); - - drawSpectrum(`Test ${testNum}: ${label} — ${freqStr}`, spectrum, N, SR, freqs); - return allFound; -} - -const SR = 32000, N = 4096; - -// --- Pairs --- -print('\n-- Pairs --'); -testMultiFreq('pair', [220, 880], null, SR, N, 4); -testMultiFreq('pair', [440, 1320], null, SR, N, 5); -testMultiFreq('pair', [300, 3000], null, SR, N, 6); -testMultiFreq('pair', [100, 8000], null, SR, N, 7); -testMultiFreq('pair unequal amp', [440, 2000], [1.0, 0.1], SR, N, 8); - -// --- Triplets --- -print('\n-- Triplets --'); -testMultiFreq('triplet', [261.63, 329.63, 392.00], null, SR, N, 9); // C E G chord -testMultiFreq('triplet', [110, 220, 440], null, SR, N, 10); // octave stack -testMultiFreq('triplet', [500, 1500, 4500], null, SR, N, 11); // harmonic series -testMultiFreq('triplet unequal', [440, 880, 1760], [1.0, 0.5, 0.25],SR, N, 12); // decaying harmonics -</script> -</body> -</html> diff --git a/tools/mq_editor/utils.js b/tools/mq_editor/utils.js new file mode 100644 index 0000000..7ab274e --- /dev/null +++ b/tools/mq_editor/utils.js @@ -0,0 +1,83 @@ +// Shared utilities for mq_editor + +// Evaluate interpolating curve at time t via Lagrange interpolation. +// The curve passes through all 4 control points at their stored time positions. +// Knot positions: u_k = (t_k - t0) / (t3 - t0), k=0..3. +// TODO: support arbitrary number of inner control points +function evalBezier(curve, t) { + const dt = curve.t3 - curve.t0; + if (dt <= 0) return curve.v0; + let u = (t - curve.t0) / dt; + u = clamp(u, 0, 1); + const u1 = (curve.t1 - curve.t0) / dt; + const u2 = (curve.t2 - curve.t0) / dt; + const d0 = (-u1) * (-u2) * (-1); + const d1 = u1 * (u1 - u2) * (u1 - 1); + const d2 = u2 * (u2 - u1) * (u2 - 1); + const d3 = (1 - u1) * (1 - u2); + if (Math.abs(d0) < 1e-9 || Math.abs(d1) < 1e-9 || Math.abs(d2) < 1e-9 || Math.abs(d3) < 1e-9) + return curve.v0 + (curve.v3 - curve.v0) * u; + const l0 = (u - u1) * (u - u2) * (u - 1) / d0; + const l1 = u * (u - u2) * (u - 1) / d1; + const l2 = u * (u - u1) * (u - 1) / d2; + const l3 = u * (u - u1) * (u - u2) / d3; + return l0 * curve.v0 + l1 * curve.v1 + l2 * curve.v2 + l3 * curve.v3; +} + +// Evaluate amplitude component of interpolating curve at time t +function evalBezierAmp(curve, t) { + const dt = curve.t3 - curve.t0; + if (dt <= 0) return curve.a0; + let u = (t - curve.t0) / dt; + u = clamp(u, 0, 1); + const u1 = (curve.t1 - curve.t0) / dt; + const u2 = (curve.t2 - curve.t0) / dt; + const d0 = (-u1) * (-u2) * (-1); + const d1 = u1 * (u1 - u2) * (u1 - 1); + const d2 = u2 * (u2 - u1) * (u2 - 1); + const d3 = (1 - u1) * (1 - u2); + if (Math.abs(d0) < 1e-9 || Math.abs(d1) < 1e-9 || Math.abs(d2) < 1e-9 || Math.abs(d3) < 1e-9) + return curve.a0 + (curve.a3 - curve.a0) * u; + const l0 = (u - u1) * (u - u2) * (u - 1) / d0; + const l1 = u * (u - u2) * (u - 1) / d1; + const l2 = u * (u - u1) * (u - 1) / d2; + const l3 = u * (u - u1) * (u - u2) / d3; + return l0 * curve.a0 + l1 * curve.a1 + l2 * curve.a2 + l3 * curve.a3; +} + +function clamp(v, lo, hi) { return v < lo ? lo : v > hi ? hi : v; } + +// Get canvas-relative {x, y} from a mouse event +function getCanvasCoords(e, canvas) { + const rect = canvas.getBoundingClientRect(); + return { x: e.clientX - rect.left, y: e.clientY - rect.top }; +} + +// Build upper/lower band point arrays for a frequency curve. +// factorAbove/factorBelow are fractional offsets (e.g. 0.02 = ±2%). +// freqMult: optional frequency scaling for harmonics (default 1.0). +// Returns { upper: [[x,y],...], lower: [[x,y],...] } +function buildBandPoints(viewer, curve, factorAbove, factorBelow, freqMult = 1.0) { + const STEPS = 60; + const upper = [], lower = []; + for (let i = 0; i <= STEPS; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; + if (t < viewer.t_view_min - 0.01 || t > viewer.t_view_max + 0.01) continue; + const f = evalBezier(curve, t) * freqMult; + upper.push([viewer.timeToX(t), viewer.freqToY(f * (1 + factorAbove))]); + lower.push([viewer.timeToX(t), viewer.freqToY(f * (1 - factorBelow))]); + } + return { upper, lower }; +} + +// Build center line points at freq * freqMult along the curve. +function buildCenterPoints(viewer, curve, freqMult = 1.0) { + const STEPS = 60; + const pts = []; + for (let i = 0; i <= STEPS; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; + if (t < viewer.t_view_min - 0.01 || t > viewer.t_view_max + 0.01) continue; + pts.push([viewer.timeToX(t), viewer.freqToY(evalBezier(curve, t) * freqMult)]); + } + return pts; +} diff --git a/tools/mq_editor/viewer.js b/tools/mq_editor/viewer.js index 76c57e2..82e9c24 100644 --- a/tools/mq_editor/viewer.js +++ b/tools/mq_editor/viewer.js @@ -38,7 +38,9 @@ class SpectrogramViewer { this.cursorCtx = this.cursorCanvas ? this.cursorCanvas.getContext('2d') : null; this.mouseX = -1; - // Playhead + // Playhead overlay + this.playheadCanvas = document.getElementById('playheadCanvas'); + this.playheadCtx = this.playheadCanvas ? this.playheadCanvas.getContext('2d') : null; this.playheadTime = -1; // -1 = not playing // Spectrum viewer @@ -48,11 +50,23 @@ class SpectrogramViewer { this.showSynthFFT = false; // Toggle: false=original, true=synth this.synthStftCache = null; + // Partial spectrum viewer + this.partialSpectrumCanvas = document.getElementById('partialSpectrumCanvas'); + this.partialSpectrumCtx = this.partialSpectrumCanvas ? this.partialSpectrumCanvas.getContext('2d') : null; + this._partialSpecCache = null; // {partialIndex, time, specData?} — see renderPartialSpectrum + // Selection and editing this.selectedPartial = -1; this.dragState = null; // {pointIndex: 0-3} this.onPartialSelect = null; // callback(index) this.onRender = null; // callback() called after each render (for synced panels) + this.onBeforeChange = null; // callback() called before any mutation (for undo/redo) + + // Explore mode + this.exploreMode = false; + this.previewPartial = null; + this.onExploreMove = null; // callback(time, freq) + this.onExploreCommit = null; // callback(partial) // Setup event handlers this.setupMouseHandlers(); @@ -100,7 +114,7 @@ class SpectrogramViewer { // DB value -> normalized intensity [0..1], relative to cache maxDB over 80dB range normalizeDB(magDB, maxDB) { - return Math.max(0, Math.min(1, (magDB - (maxDB - 80)) / 80)); + return clamp((magDB - (maxDB - 80)) / 80, 0, 1); } // Partial index -> display color @@ -118,10 +132,12 @@ class SpectrogramViewer { this.playheadTime = time; if (time >= 0) { this.spectrumTime = time; + this.renderSpectrum(); + this.renderPartialSpectrum(time); } else if (this.mouseX >= 0) { this.spectrumTime = this.canvasToTime(this.mouseX); } - this.render(); + this.drawPlayhead(); } setPartials(partials) { @@ -160,6 +176,7 @@ class SpectrogramViewer { } selectPartial(index) { + this._partialSpecCache = null; this.selectedPartial = index; this.render(); if (this.onPartialSelect) this.onPartialSelect(index); @@ -169,7 +186,7 @@ class SpectrogramViewer { hitTestPartial(x, y) { const THRESH = 10; let bestIdx = -1, bestDist = THRESH; - for (let p = 0; p < this.partials.length; ++p) { + for (let p = 0; p < this.partials.length && p < this.keepCount; ++p) { const curve = this.partials[p].freqCurve; if (!curve) continue; for (let i = 0; i <= 50; ++i) { @@ -209,6 +226,7 @@ class SpectrogramViewer { this.drawAxes(); this.drawPlayhead(); this.renderSpectrum(); + this.renderPartialSpectrum(this.spectrumTime, true); if (this.onRender) this.onRender(); } @@ -283,20 +301,14 @@ class SpectrogramViewer { _renderSpreadBand(partial, color) { const {ctx} = this; - const curve = partial.freqCurve; - const rep = partial.replicas || {}; - const sa = rep.spread_above != null ? rep.spread_above : 0.02; - const sb = rep.spread_below != null ? rep.spread_below : 0.02; + const curve = partial.freqCurve; + const harm = partial.harmonics || {}; + const sa = harm.spread_above != null ? harm.spread_above : 0.02; + const sb = harm.spread_below != null ? harm.spread_below : 0.02; + const decay = harm.decay != null ? harm.decay : 0.0; + const freqMult = harm.freq_mult != null ? harm.freq_mult : 2.0; - const STEPS = 60; - const upper = [], lower = []; - for (let i = 0; i <= STEPS; ++i) { - const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; - if (t < this.t_view_min - 0.01 || t > this.t_view_max + 0.01) continue; - const f = evalBezier(curve, t); - upper.push([this.timeToX(t), this.freqToY(f * (1 + sa))]); - lower.push([this.timeToX(t), this.freqToY(f * (1 - sb))]); - } + const {upper, lower} = buildBandPoints(this, curve, sa, sb); if (upper.length < 2) return; const savedAlpha = ctx.globalAlpha; @@ -327,14 +339,7 @@ class SpectrogramViewer { ctx.setLineDash([]); // 50% drop-off reference lines (dotted, dimmer) - const p5upper = [], p5lower = []; - for (let i = 0; i <= STEPS; ++i) { - const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; - if (t < this.t_view_min - 0.01 || t > this.t_view_max + 0.01) continue; - const f = evalBezier(curve, t); - p5upper.push([this.timeToX(t), this.freqToY(f * 1.50)]); - p5lower.push([this.timeToX(t), this.freqToY(f * 0.50)]); - } + const {upper: p5upper, lower: p5lower} = buildBandPoints(this, curve, 0.50, 0.50); if (p5upper.length >= 2) { ctx.globalAlpha = 0.55; ctx.strokeStyle = color; @@ -351,6 +356,56 @@ class SpectrogramViewer { ctx.setLineDash([]); } + // Harmonic bands (faint, fading with decay^n) + if (decay > 0) { + for (let n = 1; ; ++n) { + const ampMult = Math.pow(decay, n); + if (ampMult < 0.001) break; + const hRatio = n * freqMult; + + // Center line + const cpts = buildCenterPoints(this, curve, hRatio); + if (cpts.length >= 2) { + ctx.globalAlpha = ampMult * 0.85; + ctx.strokeStyle = color; + ctx.lineWidth = 1.5; + ctx.setLineDash([3, 4]); + ctx.beginPath(); + ctx.moveTo(cpts[0][0], cpts[0][1]); + for (let i = 1; i < cpts.length; ++i) ctx.lineTo(cpts[i][0], cpts[i][1]); + ctx.stroke(); + ctx.setLineDash([]); + } + + // Spread band fill + boundary dashes + const {upper: hu, lower: hl} = buildBandPoints(this, curve, sa, sb, hRatio); + if (hu.length >= 2) { + ctx.beginPath(); + ctx.moveTo(hu[0][0], hu[0][1]); + for (let i = 1; i < hu.length; ++i) ctx.lineTo(hu[i][0], hu[i][1]); + for (let i = hl.length - 1; i >= 0; --i) ctx.lineTo(hl[i][0], hl[i][1]); + ctx.closePath(); + ctx.fillStyle = color; + ctx.globalAlpha = ampMult * 0.12; + ctx.fill(); + + ctx.globalAlpha = ampMult * 0.55; + ctx.strokeStyle = color; + ctx.lineWidth = 1; + ctx.setLineDash([3, 5]); + ctx.beginPath(); + ctx.moveTo(hu[0][0], hu[0][1]); + for (let i = 1; i < hu.length; ++i) ctx.lineTo(hu[i][0], hu[i][1]); + ctx.stroke(); + ctx.beginPath(); + ctx.moveTo(hl[0][0], hl[0][1]); + for (let i = 1; i < hl.length; ++i) ctx.lineTo(hl[i][0], hl[i][1]); + ctx.stroke(); + ctx.setLineDash([]); + } + } + } + ctx.globalAlpha = savedAlpha; } @@ -389,24 +444,69 @@ class SpectrogramViewer { const h = this.cursorCanvas.height; ctx.clearRect(0, 0, this.cursorCanvas.width, h); if (x < 0) return; - ctx.strokeStyle = 'rgba(255, 60, 60, 0.7)'; + ctx.strokeStyle = this.exploreMode === 'contour' ? 'rgba(0,220,220,0.8)' + : this.exploreMode ? 'rgba(255,160,0,0.8)' + : 'rgba(255,60,60,0.7)'; ctx.lineWidth = 1; ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, h); ctx.stroke(); + if (this.exploreMode && this.previewPartial) { + this._drawPreviewPartial(ctx, this.previewPartial); + } + } + + setExploreMode(enabled) { + this.exploreMode = enabled; + if (!enabled) this.previewPartial = null; + this.drawMouseCursor(this.mouseX); + this.canvas.style.cursor = enabled ? 'cell' : 'crosshair'; + } + + setPreviewPartial(partial) { + this.previewPartial = partial; + this.drawMouseCursor(this.mouseX); + } + + _drawPreviewPartial(ctx, partial) { + const curve = partial.freqCurve; + if (!curve) return; + const col = this.exploreMode === 'contour' ? '0,220,220' : '255,160,0'; + ctx.save(); + ctx.strokeStyle = `rgba(${col},0.9)`; + ctx.lineWidth = 2; + ctx.setLineDash([6, 3]); + ctx.shadowColor = `rgba(${col},0.5)`; + ctx.shadowBlur = 6; + ctx.beginPath(); + let started = false; + for (let i = 0; i <= 80; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / 80; + const freq = evalBezier(curve, t); + if (t < this.t_view_min || t > this.t_view_max) continue; + if (freq < this.freqStart || freq > this.freqEnd) continue; + const px = this.timeToX(t); + const py = this.freqToY(freq); + if (!started) { ctx.moveTo(px, py); started = true; } else ctx.lineTo(px, py); + } + if (started) ctx.stroke(); + ctx.restore(); } drawPlayhead() { + if (!this.playheadCtx) return; + const ctx = this.playheadCtx; + const h = this.playheadCanvas.height; + ctx.clearRect(0, 0, this.playheadCanvas.width, h); if (this.playheadTime < 0) return; if (this.playheadTime < this.t_view_min || this.playheadTime > this.t_view_max) return; - const {ctx, canvas} = this; const x = this.timeToX(this.playheadTime); - ctx.strokeStyle = '#f00'; + ctx.strokeStyle = 'rgba(255, 80, 80, 0.9)'; ctx.lineWidth = 2; ctx.beginPath(); ctx.moveTo(x, 0); - ctx.lineTo(x, canvas.height); + ctx.lineTo(x, h); ctx.stroke(); } @@ -546,6 +646,129 @@ class SpectrogramViewer { ctx.fillText(useSynth ? 'SYNTH [a]' : 'ORIG [a]', 4, 10); } + // Draw synthesized power spectrum of the selected partial at `time` into partialSpectrumCanvas. + // X-axis: log frequency (same scale as main view). Y-axis: dB (normalised to peak). + // force=true bypasses cache — used by render() when params change. + // Otherwise cached on {partialIndex, time} for mouse-move performance. + renderPartialSpectrum(time, force = false) { + const ctx = this.partialSpectrumCtx; + if (!ctx) return; + + const canvas = this.partialSpectrumCanvas; + const width = canvas.width; + const height = canvas.height; + const p = this.selectedPartial; + + // Cache check — skip if same partial+time unless forced by param change + if (!force && this._partialSpecCache && + this._partialSpecCache.partialIndex === p && + this._partialSpecCache.time === time) return; + + ctx.fillStyle = '#1e1e1e'; + ctx.fillRect(0, 0, width, height); + ctx.font = '9px monospace'; + + if (p < 0 || !this.partials || p >= this.partials.length) { + ctx.fillStyle = '#333'; + ctx.fillText('no partial', 4, height / 2 + 4); + this._partialSpecCache = {partialIndex: p, time}; + return; + } + + const partial = this.partials[p]; + const curve = partial.freqCurve; + if (!curve || time < curve.t0 || time > curve.t3) { + ctx.fillStyle = '#333'; + ctx.fillText('out of range', 4, height / 2 + 4); + this._partialSpecCache = {partialIndex: p, time}; + return; + } + + // Synthesize window → FFT → power spectrum + const specData = this._computePartialSpectrum(partial, time); + this._partialSpecCache = {partialIndex: p, time, specData}; + + const {squaredAmp, maxDB, sampleRate, fftSize} = specData; + const numBins = fftSize / 2; + const binWidth = sampleRate / fftSize; + const color = this.partialColor(p); + const cr = parseInt(color[1] + color[1], 16); + const cg = parseInt(color[2] + color[2], 16); + const cb = parseInt(color[3] + color[3], 16); + + for (let px = 0; px < width; ++px) { + const fStart = this.normToFreq(px / width); + const fEnd = this.normToFreq((px + 1) / width); + const bStart = Math.max(0, Math.floor(fStart / binWidth)); + const bEnd = Math.min(numBins - 1, Math.ceil(fEnd / binWidth)); + if (bStart > bEnd) continue; + + let maxSq = 0; + for (let b = bStart; b <= bEnd; ++b) if (squaredAmp[b] > maxSq) maxSq = squaredAmp[b]; + + const magDB = 10 * Math.log10(Math.max(maxSq, 1e-20)); + const barH = Math.round(this.normalizeDB(magDB, maxDB) * (height - 12)); + if (barH <= 0) continue; + + const grad = ctx.createLinearGradient(0, height - barH, 0, height); + grad.addColorStop(0, color); + grad.addColorStop(1, `rgba(${cr},${cg},${cb},0.53)`); + ctx.fillStyle = grad; + ctx.fillRect(px, height - barH, 1, barH); + } + + ctx.fillStyle = color; + ctx.fillText('P#' + p + ' @' + time.toFixed(3) + 's', 4, 10); + } + + // Synthesise a 2048-sample Hann-windowed frame of `partial` centred on `time`, + // run FFT, and return {squaredAmp, maxDB, sampleRate, fftSize}. + // freqCurve times are shifted so synthesizeMQ's t=0 aligns with tStart = time - window/2. + _computePartialSpectrum(partial, time) { + const sampleRate = this.audioBuffer.sampleRate; + const FFT_SIZE = 2048; + const windowDuration = FFT_SIZE / sampleRate; + const tStart = time - windowDuration / 2; + + // Shift curve times so synthesis window [0, windowDuration] maps to [tStart, tStart+windowDuration] + const fc = partial.freqCurve; + const shiftedPartial = { + ...partial, + freqCurve: { + t0: fc.t0 - tStart, t1: fc.t1 - tStart, + t2: fc.t2 - tStart, t3: fc.t3 - tStart, + v0: fc.v0, v1: fc.v1, v2: fc.v2, v3: fc.v3, + a0: fc.a0, a1: fc.a1, a2: fc.a2, a3: fc.a3, + }, + }; + + const pcm = synthesizeMQ([shiftedPartial], sampleRate, windowDuration, true, {}); + + // Hann window + for (let i = 0; i < FFT_SIZE; ++i) { + pcm[i] *= 0.5 * (1 - Math.cos(2 * Math.PI * i / (FFT_SIZE - 1))); + } + + // FFT + const real = new Float32Array(FFT_SIZE); + const imag = new Float32Array(FFT_SIZE); + for (let i = 0; i < FFT_SIZE; ++i) real[i] = pcm[i]; + fftRadix2(real, imag, FFT_SIZE, 1); + + // Power spectrum + const squaredAmp = new Float32Array(FFT_SIZE / 2); + for (let i = 0; i < FFT_SIZE / 2; ++i) { + squaredAmp[i] = (real[i] * real[i] + imag[i] * imag[i]) / (FFT_SIZE * FFT_SIZE); + } + + // maxDB for normalizing the display + let maxSq = 1e-20; + for (let i = 0; i < squaredAmp.length; ++i) if (squaredAmp[i] > maxSq) maxSq = squaredAmp[i]; + const maxDB = 10 * Math.log10(maxSq); + + return {squaredAmp, maxDB, sampleRate, fftSize: FFT_SIZE}; + } + // --- View management --- updateViewBounds() { @@ -568,18 +791,34 @@ class SpectrogramViewer { this.t_center = (this.t_view_min + this.t_view_max) / 2; } + destroy() { + const {canvas} = this; + canvas.removeEventListener('mousedown', this._onMousedown); + canvas.removeEventListener('mousemove', this._onMousemove); + canvas.removeEventListener('mouseleave', this._onMouseleave); + canvas.removeEventListener('mouseup', this._onMouseup); + canvas.removeEventListener('wheel', this._onWheel); + } + setupMouseHandlers() { const {canvas, tooltip} = this; - canvas.addEventListener('mousedown', (e) => { - const rect = canvas.getBoundingClientRect(); - const x = e.clientX - rect.left; - const y = e.clientY - rect.top; + this._onMousedown = (e) => { + const {x, y} = getCanvasCoords(e, canvas); + + // Explore mode: commit preview on click + if (this.exploreMode) { + if (this.previewPartial && this.onExploreCommit) { + this.onExploreCommit(this.previewPartial); + } + return; + } // Check control point drag on selected partial if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { const ptIdx = this.hitTestControlPoint(x, y, this.partials[this.selectedPartial]); if (ptIdx >= 0) { + if (this.onBeforeChange) this.onBeforeChange(); this.dragState = { pointIndex: ptIdx }; canvas.style.cursor = 'grabbing'; e.preventDefault(); @@ -590,16 +829,15 @@ class SpectrogramViewer { // Otherwise: select partial by click const idx = this.hitTestPartial(x, y); this.selectPartial(idx); - }); + }; + canvas.addEventListener('mousedown', this._onMousedown); - canvas.addEventListener('mousemove', (e) => { - const rect = canvas.getBoundingClientRect(); - const x = e.clientX - rect.left; - const y = e.clientY - rect.top; + this._onMousemove = (e) => { + const {x, y} = getCanvasCoords(e, canvas); if (this.dragState) { - const t = Math.max(0, Math.min(this.t_max, this.canvasToTime(x))); - const v = Math.max(this.freqStart, Math.min(this.freqEnd, this.canvasToFreq(y))); + const t = clamp(this.canvasToTime(x), 0, this.t_max); + const v = clamp(this.canvasToFreq(y), this.freqStart, this.freqEnd); const partial = this.partials[this.selectedPartial]; const i = this.dragState.pointIndex; partial.freqCurve['t' + i] = t; @@ -614,42 +852,53 @@ class SpectrogramViewer { const time = this.canvasToTime(x); const freq = this.canvasToFreq(y); + + if (this.exploreMode && this.onExploreMove) { + this.onExploreMove(time, freq); // may call setPreviewPartial → redraws cursor canvas + } + const intensity = this.getIntensityAt(time, freq); if (this.playheadTime < 0) { this.spectrumTime = time; this.renderSpectrum(); + this.renderPartialSpectrum(time); } - // Cursor hint for control points - if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { - const ptIdx = this.hitTestControlPoint(x, y, this.partials[this.selectedPartial]); - canvas.style.cursor = ptIdx >= 0 ? 'grab' : 'crosshair'; - } else { - canvas.style.cursor = 'crosshair'; + // Cursor hint for control points (skip in explore mode) + if (!this.exploreMode) { + if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { + const ptIdx = this.hitTestControlPoint(x, y, this.partials[this.selectedPartial]); + canvas.style.cursor = ptIdx >= 0 ? 'grab' : 'crosshair'; + } else { + canvas.style.cursor = 'crosshair'; + } } tooltip.style.left = (e.clientX + 10) + 'px'; tooltip.style.top = (e.clientY + 10) + 'px'; tooltip.style.display = 'block'; tooltip.textContent = `${time.toFixed(3)}s, ${freq.toFixed(1)}Hz, ${intensity.toFixed(1)}dB`; - }); + }; + canvas.addEventListener('mousemove', this._onMousemove); - canvas.addEventListener('mouseleave', () => { + this._onMouseleave = () => { this.mouseX = -1; this.drawMouseCursor(-1); tooltip.style.display = 'none'; - }); + }; + canvas.addEventListener('mouseleave', this._onMouseleave); - canvas.addEventListener('mouseup', () => { + this._onMouseup = () => { if (this.dragState) { this.dragState = null; canvas.style.cursor = 'crosshair'; if (this.onPartialSelect) this.onPartialSelect(this.selectedPartial); } - }); + }; + canvas.addEventListener('mouseup', this._onMouseup); - canvas.addEventListener('wheel', (e) => { + this._onWheel = (e) => { e.preventDefault(); const delta = e.deltaY !== 0 ? e.deltaY : e.deltaX; @@ -674,7 +923,8 @@ class SpectrogramViewer { this.updateViewBounds(); this.render(); - }); + }; + canvas.addEventListener('wheel', this._onWheel); } // --- Utilities --- @@ -717,13 +967,3 @@ class SpectrogramViewer { } } -// Bezier evaluation (shared utility) -function evalBezier(curve, t) { - let u = (t - curve.t0) / (curve.t3 - curve.t0); - u = Math.max(0, Math.min(1, u)); - const u1 = 1 - u; - return u1*u1*u1 * curve.v0 + - 3*u1*u1*u * curve.v1 + - 3*u1*u*u * curve.v2 + - u*u*u * curve.v3; -} |
