diff options
Diffstat (limited to 'tools/mq_editor')
| -rw-r--r-- | tools/mq_editor/PHASE_TRACKING_PLAN.md | 73 | ||||
| -rw-r--r-- | tools/mq_editor/README.md | 79 | ||||
| -rw-r--r-- | tools/mq_editor/editor.js | 578 | ||||
| -rw-r--r-- | tools/mq_editor/fft.js | 6 | ||||
| -rw-r--r-- | tools/mq_editor/index.html | 380 | ||||
| -rw-r--r-- | tools/mq_editor/mq_extract.js | 258 | ||||
| -rw-r--r-- | tools/mq_editor/mq_synth.js | 142 | ||||
| -rw-r--r-- | tools/mq_editor/viewer.js | 418 |
8 files changed, 1718 insertions, 216 deletions
diff --git a/tools/mq_editor/PHASE_TRACKING_PLAN.md b/tools/mq_editor/PHASE_TRACKING_PLAN.md new file mode 100644 index 0000000..5111692 --- /dev/null +++ b/tools/mq_editor/PHASE_TRACKING_PLAN.md @@ -0,0 +1,73 @@ +# Implementation Plan: Phase-Coherent Partial Tracking + +This document outlines the plan to integrate phase prediction into the existing MQ tracking algorithm. The core idea is to use phase coherence as a primary factor for linking peaks across frames, making the tracking more robust, especially for crossing or closely-spaced partials. + +--- + +### **Stage 1: Cache Per-Bin Phase in `fft.js`** + +**Objective:** Augment the `STFTCache` to compute and store the phase angle for every frequency bin in every frame, making it available for the tracking algorithm. + +1. **Locate the FFT Processing Loop:** + * In `tools/mq_editor/fft.js`, within the `STFTCache` class (likely in the constructor or an initialization method), find the loop that iterates through each frame to compute the FFT. + * This is where `squaredAmplitude` is currently being calculated from the `real` and `imag` components. + +2. **Compute and Store Phase:** + * In the same loop, immediately after calculating the squared amplitude, calculate the phase for each bin. + * Create a new `Float32Array` for phase, let's call it `ph`. + * Inside the bin loop, compute: `ph[k] = Math.atan2(imag[k], real[k]);` + * Store this new phase array in the frame object, parallel to the existing `squaredAmplitude` array. + + **Resulting Change in `fft.js`:** + + ```javascript + // Inside the STFTCache frame processing loop... + + // Existing code: + const sq = new Float32Array(this.fftSize / 2); + for (let k = 0; k < this.fftSize / 2; k++) { + sq[k] = real[k] * real[k] + imag[k] * imag[k]; + } + + // New code to add: + const ph = new Float32Array(this.fftSize / 2); + for (let k = 0; k < this.fftSize / 2; k++) { + ph[k] = Math.atan2(imag[k], real[k]); + } + + // Update the stored frame object + this.frames.push({ + time: t, + squaredAmplitude: sq, + phase: ph // The newly cached phase data + }); + ``` + +--- + +### **Stage 2: Utilize Phase for Tracking in `mq_extract.js`** + +**Objective:** Modify the main forward/backward tracking algorithm to use phase coherence for identifying and linking peaks. + +1. **Extract Interpolated Peak Phase:** + * In `tools/mq_editor/mq_extract.js`, find the function responsible for peak detection within a single frame (e.g., `findPeaks`). + * This function currently takes a `squaredAmplitude` array. It must now also access the corresponding `phase` array from the cached frame data. + * When a peak is found at bin `k`, use the same parabolic interpolation logic that calculates the true frequency and amplitude to also calculate the **true phase**. This involves interpolating the phase values from bins `k-1`, `k`, and `k+1`. + * **Crucially, this interpolation must handle phase wrapping.** A helper function will be needed to correctly find the shortest angular distance between phase values. + +2. **Update Tracking Data Structures:** + * The data structures holding candidate and live partials must be updated to store the phase of each point in the trajectory, not just frequency and amplitude. + +3. **Implement Phase Prediction Logic:** + * In the main tracking loop that steps from frame `n` to `n+1`: + * For each active partial, calculate its `predictedPhase` for frame `n+1`. + * `phase_delta = (2 * Math.PI * last_freq * params.hopSize) / params.sampleRate;` + * `predictedPhase = last_phase + phase_delta;` + +4. **Refine the Candidate Matching Score:** + * Modify the logic that links a partial to peaks in the next frame. + * Instead of matching based on frequency proximity alone, calculate a `cost` based on both frequency and phase deviation: + * `freqError = Math.abs(peak.freq - partial.last_freq);` + * `phaseError = Math.abs(normalize_angle(peak.phase - predictedPhase));` // Difference on a circle + * `cost = (freq_weight * freqError) + (phase_weight * phaseError);` + * The peak with the lowest `cost` below a certain threshold is the correct continuation. The `phase_weight` should be high, as a low phase error is a strong indicator of a correct match. diff --git a/tools/mq_editor/README.md b/tools/mq_editor/README.md index 4250d33..c1f2732 100644 --- a/tools/mq_editor/README.md +++ b/tools/mq_editor/README.md @@ -17,7 +17,7 @@ open tools/mq_editor/index.html - **Top bar:** title + loaded filename - **Toolbar:** file, extraction, playback controls and parameters - **Main view:** log-scale time-frequency spectrogram with partial trajectories -- **Right panel:** synthesis checkboxes (integrate phase, disable jitter, disable spread) +- **Right panel:** per-partial mode toggle (Sinusoid / Resonator), synth params, global checkboxes - **Mini-spectrum** (bottom-right overlay): FFT slice at mouse/playhead time - Blue/orange gradient: original spectrum; green/yellow: synthesized - Green bars: detected spectral peaks @@ -27,6 +27,8 @@ open tools/mq_editor/index.html - **Hop Size:** 64–1024 samples (default 256) - **Threshold:** dB floor for peak detection (default −60 dB) +- **Prominence:** Min dB height of a peak above its surrounding "valley floor" (default 1.0 dB). Filters out insignificant local maxima. +- **f·Power:** checkbox — weight spectrum by frequency (`f·FFT_Power(f)`) before peak detection, accentuating high-frequency peaks - **Keep %:** slider to limit how many partials are shown/synthesized ## Keyboard Shortcuts @@ -35,17 +37,20 @@ open tools/mq_editor/index.html |-----|--------| | `1` | Play synthesized audio | | `2` | Play original audio | +| `E` | Extract Partials | | `P` | Toggle raw peak overlay | | `A` | Toggle mini-spectrum: original ↔ synthesized | +| `Esc` | Deselect partial | | Shift+scroll | Zoom time axis | | Scroll | Pan time axis | ## Architecture -- `index.html` — UI, playback, extraction orchestration +- `index.html` — UI, playback, extraction orchestration, keyboard shortcuts +- `editor.js` — Property panel and amplitude bezier editor for selected partials - `fft.js` — Cooley-Tukey radix-2 FFT + STFT cache - `mq_extract.js` — MQ algorithm: peak detection, forward tracking, backward expansion, bezier fitting -- `mq_synth.js` — Oscillator bank synthesis from extracted partials +- `mq_synth.js` — Oscillator bank + two-pole resonator synthesis from extracted partials - `viewer.js` — Visualization: coordinate API, spectrogram, partials, mini-spectrum, mouse/zoom ### viewer.js coordinate API @@ -61,13 +66,66 @@ open tools/mq_editor/index.html | `normalizeDB(db, maxDB)` | dB → intensity [0..1] over 80 dB range | | `partialColor(p)` | partial index → display color | +## Post-Synthesis Filters + +Global LP/HP filters applied after the oscillator bank, before normalization. +Sliders in the **Synthesis** panel; display shows -3 dB cutoff frequency. + +| Control | Filter | Formula | Bypass | +|---------|--------|---------|--------| +| **LP k1** | `y[n] = k1·x[n] + (1−k1)·y[n−1]` | `-3dB: cos(ω) = (2−2k−k²)/(2(1−k))` | k1 = 1.0 | +| **HP k2** | `y[n] = k2·(y[n−1] + x[n] − x[n−1])` | `-3dB from peak: cos(ω) = 2k/(1+k²)` | k2 = 1.0 | + +Both default to 1.0 (bypass). Frequency display uses `audioBuffer.sampleRate` when loaded, falls back to 44100 Hz. + +--- + +## Resonator Synthesis Mode + +Each partial has a **per-partial synthesis mode** selectable in the **Synth** tab: + +### Sinusoid (default) +Replica oscillator bank — direct additive sinusoids with optional spread/jitter/decay shaping. + +### Resonator +Two-pole IIR bandpass resonator driven by band-limited noise: + +``` +y[n] = 2r·cos(ω₀)·y[n-1] − r²·y[n-2] + A(t)·√(1−r²)·noise[n] +``` + +- **ω₀** = `2π·f₀(t)/SR` — recomputed each sample from the freq Bezier curve (handles glides/vibrato) +- **A(t)** — amp Bezier curve scales excitation continuously +- **√(1−r²)** — power normalization, keeps output level ≈ sinusoidal mode at `gainComp = 1` +- **noise[n]** — deterministic per-partial LCG (reproducible renders) + +**Parameters:** + +| Param | Default | Range | Meaning | +|-------|---------|-------|---------| +| `r (pole)` | 0.995 | [0, 0.9999] | Pole radius. r→1 = narrow BW / long ring. r→0 = wide / fast decay. | +| `gain` | 1.0 | [0, ∞) | Output multiplier on top of power normalization. | + +**Coefficient translation from spread:** +`r = exp(−π · BW / SR)` where `BW = f₀ · (spread_above + spread_below) / 2`. +For a partial at 440 Hz with `spread = 0.02`: `BW ≈ 8.8 Hz`, `r ≈ exp(−π·8.8/32000) ≈ 0.9991`. + +**When to use:** +- Metallic / percussive partials with natural exponential decay +- Wide spectral peaks (large spread) where a bandpass filter is more physically accurate +- Comparing resonator vs. sinusoidal timbre on the same partial + +**Note:** `RES` badge appears in the panel header when a partial is in resonator mode. + +--- + ## Algorithm 1. **STFT:** Overlapping Hann windows, radix-2 FFT -2. **Peak Detection:** Local maxima above threshold + parabolic interpolation -3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance, candidate persistence +2. **Peak Detection:** Local maxima above threshold + parabolic interpolation. Includes **Prominence Filtering** (rejects peaks not significantly higher than surroundings). Optional `f·Power(f)` weighting. +3. **Forward Tracking:** Birth/death/continuation with frequency-dependent tolerance. Includes **Predictive Kinematic Tracking** (uses velocity to track rapidly moving partials). 4. **Backward Expansion:** Second pass extends each partial leftward to recover onset frames -5. **Bezier Fitting:** Cubic curves with control points at t/3 and 2t/3 +5. **Bezier Fitting:** Cubic curves optimized via **Least-Squares** (minimizes error across all points). ## Implementation Status @@ -83,7 +141,14 @@ open tools/mq_editor/index.html - [x] Replica oscillator bank (spread, jitter, phase integration) - [x] Synthesis debug checkboxes (disable jitter/spread) - [x] Synthesized STFT cache for FFT comparison -- [ ] Phase 3: Editing UI (drag control points, replicas) +- [x] Phase 3: Editing UI + - [x] Partial selection with property panel (freq/amp/synth tabs) + - [x] Amplitude bezier drag editor + - [x] Synth params with jog sliders (decay, jitter, spread) + - [x] Auto-spread detection per partial and global + - [x] Mute / delete partials + - [x] Per-partial resonator synthesis mode (Synth tab toggle) + - [x] Global LP/HP post-synthesis filter sliders with Hz display - [ ] Phase 4: Export (.spec + C++ code generation) ## See Also diff --git a/tools/mq_editor/editor.js b/tools/mq_editor/editor.js new file mode 100644 index 0000000..97d8a7a --- /dev/null +++ b/tools/mq_editor/editor.js @@ -0,0 +1,578 @@ +// Partial Editor +// Property panel (right) and amplitude bezier editor (bottom) for selected partials + +class PartialEditor { + constructor() { + // DOM refs + this._propPanel = document.getElementById('partialProps'); + this._noSelMsg = document.getElementById('noSelMsg'); + this._ampPanel = document.getElementById('ampEditPanel'); + this._ampCanvas = document.getElementById('ampEditCanvas'); + this._ampTitle = document.getElementById('ampEditTitle'); + this._freqGrid = document.getElementById('freqCurveGrid'); + this._ampGrid = document.getElementById('ampCurveGrid'); + this._synthGrid = document.getElementById('synthGrid'); + this._ampCtx = this._ampCanvas ? this._ampCanvas.getContext('2d') : null; + + // References set by host + this.viewer = null; + this.partials = null; + + // Callback: called after a partial is deleted so the host can update keepCount + this.onPartialDeleted = null; + + // Private state + this._selectedIndex = -1; + this._dragPointIndex = -1; + this._amp = { tMin: 0, tMax: 1, ampTop: 1 }; + + this._setupButtons(); + this._setupAmpDrag(); + } + + // --- Public API --- + + setViewer(v) { this.viewer = v; } + setPartials(p) { this.partials = p; } + + // Wire to viewer.onPartialSelect + onPartialSelect(index) { + this._selectedIndex = index; + this._updatePropPanel(index); + this._showAmpEditor(index); + } + + // Wire to viewer.onRender — keeps amp editor in sync with zoom/scroll + onRender() { + this._renderAmpEditor(); + } + + // --- Property panel --- + + _updatePropPanel(index) { + if (index < 0 || !this.partials || index >= this.partials.length) { + this._propPanel.style.display = 'none'; + this._noSelMsg.style.display = 'block'; + return; + } + + this._propPanel.style.display = 'block'; + this._noSelMsg.style.display = 'none'; + + const partial = this.partials[index]; + const color = this.viewer ? this.viewer.partialColor(index) : '#888'; + + const titleEl = document.getElementById('propTitle'); + const badge = partial.resonator && partial.resonator.enabled + ? ' <span class="res-badge">RES</span>' : ''; + titleEl.innerHTML = 'Partial #' + index + badge; + document.getElementById('propSwatch').style.background = color; + + let peakAmp = 0, peakIdx = 0; + for (let i = 0; i < partial.amps.length; ++i) { + if (partial.amps[i] > peakAmp) { peakAmp = partial.amps[i]; peakIdx = i; } + } + document.getElementById('propPeak').textContent = + partial.freqs[peakIdx].toFixed(1) + ' Hz ' + peakAmp.toFixed(3); + + const t0 = partial.freqCurve ? partial.freqCurve.t0 : partial.times[0]; + const t3 = partial.freqCurve ? partial.freqCurve.t3 : partial.times[partial.times.length - 1]; + document.getElementById('propTime').textContent = + t0.toFixed(3) + 's\u2013' + t3.toFixed(3) + 's'; + + const muteBtn = document.getElementById('mutePartialBtn'); + muteBtn.textContent = partial.muted ? 'Unmute' : 'Mute'; + muteBtn.style.color = partial.muted ? '#fa4' : ''; + + this._buildCurveGrid(this._freqGrid, partial, 'freqCurve', 'f', index); + this._buildCurveGrid(this._ampGrid, partial, 'ampCurve', 'a', index); + this._buildSynthGrid(partial, index); + } + + _buildCurveGrid(grid, partial, curveKey, valueLabel, partialIndex) { + grid.innerHTML = ''; + const curve = partial[curveKey]; + if (!curve) { grid.style.color = '#444'; grid.textContent = 'none'; return; } + + for (let i = 0; i < 4; ++i) { + const lbl = document.createElement('span'); + lbl.textContent = 'P' + i; + + const tInput = document.createElement('input'); + tInput.type = 'number'; + tInput.value = curve['t' + i].toFixed(4); + tInput.step = '0.001'; + tInput.title = 't' + i; + tInput.addEventListener('change', this._makeCurveUpdater(partialIndex, curveKey, 't', i)); + + const vInput = document.createElement('input'); + vInput.type = 'number'; + vInput.value = curveKey === 'freqCurve' ? curve['v' + i].toFixed(2) : curve['v' + i].toFixed(4); + vInput.step = curveKey === 'freqCurve' ? '1' : '0.0001'; + vInput.title = valueLabel + i; + vInput.addEventListener('change', this._makeCurveUpdater(partialIndex, curveKey, 'v', i)); + + grid.appendChild(lbl); + grid.appendChild(tInput); + grid.appendChild(vInput); + } + } + + _buildSynthGrid(partial, index) { + const grid = this._synthGrid; + grid.innerHTML = ''; + + const repDefaults = { decay_alpha: 0.1, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; + const resDefaults = { r: 0.995, gainComp: 1.0 }; + + const isResonator = !!(partial.resonator && partial.resonator.enabled); + + // --- Mode toggle --- + const modeLbl = document.createElement('span'); + modeLbl.textContent = 'mode'; + + const modeWrap = document.createElement('div'); + modeWrap.style.cssText = 'display:flex;gap:3px;'; + + const btnSin = document.createElement('button'); + btnSin.textContent = 'Sinusoid'; + btnSin.className = 'tab-btn' + (isResonator ? '' : ' active'); + btnSin.style.cssText = 'flex:1;padding:2px 4px;font-size:11px;margin:0;'; + + const btnRes = document.createElement('button'); + btnRes.textContent = 'Resonator'; + btnRes.className = 'tab-btn' + (isResonator ? ' active' : ''); + btnRes.style.cssText = 'flex:1;padding:2px 4px;font-size:11px;margin:0;'; + + modeWrap.appendChild(btnSin); + modeWrap.appendChild(btnRes); + grid.appendChild(modeLbl); + grid.appendChild(modeWrap); + + // --- Sinusoid section --- + const sinSection = document.createElement('div'); + sinSection.style.cssText = 'display:contents;'; + sinSection.dataset.section = 'sinusoid'; + + const rep = partial.replicas || {}; + const sinParams = [ + { key: 'decay_alpha', label: 'decay', step: '0.001' }, + { key: 'jitter', label: 'jitter', step: '0.001' }, + { key: 'spread_above', label: 'spread ↑', step: '0.001' }, + { key: 'spread_below', label: 'spread ↓', step: '0.001' }, + ]; + const sinInputs = {}; + for (const p of sinParams) { + const val = rep[p.key] != null ? rep[p.key] : repDefaults[p.key]; + const lbl = document.createElement('span'); + lbl.textContent = p.label; + const inp = document.createElement('input'); + inp.type = 'number'; + inp.value = val.toFixed(3); + inp.step = p.step; + inp.min = '0'; + inp.addEventListener('change', (e) => { + if (!this.partials) return; + const v = parseFloat(e.target.value); + if (isNaN(v)) return; + if (!this.partials[index].replicas) this.partials[index].replicas = { ...repDefaults }; + this.partials[index].replicas[p.key] = v; + if (this.viewer) this.viewer.render(); + }); + sinInputs[p.key] = inp; + + const jog = this._makeJogSlider(inp, partial, index, p, repDefaults); + const wrap = document.createElement('div'); + wrap.className = 'synth-field-wrap'; + wrap.appendChild(inp); + wrap.appendChild(jog); + sinSection.appendChild(lbl); + sinSection.appendChild(wrap); + } + + // Auto-detect spread button + const autoLbl = document.createElement('span'); + autoLbl.textContent = 'spread'; + const autoBtn = document.createElement('button'); + autoBtn.textContent = 'Auto'; + autoBtn.title = 'Infer spread_above/below from frequency variance around the bezier curve'; + autoBtn.addEventListener('click', () => { + if (!this.partials) return; + const p = this.partials[index]; + const sc = this.viewer ? this.viewer.stftCache : null; + const sr = this.viewer ? this.viewer.audioBuffer.sampleRate : 44100; + const fs = sc ? sc.fftSize : 2048; + const {spread_above, spread_below} = autodetectSpread(p, sc, fs, sr); + if (!p.replicas) p.replicas = { ...repDefaults }; + p.replicas.spread_above = spread_above; + p.replicas.spread_below = spread_below; + sinInputs['spread_above'].value = spread_above.toFixed(4); + sinInputs['spread_below'].value = spread_below.toFixed(4); + }); + sinSection.appendChild(autoLbl); + sinSection.appendChild(autoBtn); + + // --- Resonator section --- + const resSection = document.createElement('div'); + resSection.style.cssText = 'display:contents;'; + resSection.dataset.section = 'resonator'; + + const resObj = partial.resonator || {}; + const resParams = [ + { key: 'r', label: 'r (pole)', step: '0.001', min: '0.75', max: '0.9999', + title: 'Pole radius. r→1 = narrow bandwidth / long ring. r→0 = wide / fast decay.' }, + { key: 'gainComp', label: 'gain', step: '0.01', min: '0', max: '100', + title: 'Output gain multiplier (gainNorm=√(1-r²) normalises power; use this to trim level).' }, + ]; + for (const p of resParams) { + const val = resObj[p.key] != null ? resObj[p.key] : resDefaults[p.key]; + const lbl = document.createElement('span'); + lbl.textContent = p.label; + lbl.title = p.title || ''; + const inp = document.createElement('input'); + inp.type = 'number'; + inp.value = val.toFixed(4); + inp.step = p.step; + inp.min = p.min || '0'; + inp.max = p.max || ''; + inp.title = p.title || ''; + inp.addEventListener('change', (e) => { + if (!this.partials) return; + const v = parseFloat(e.target.value); + if (isNaN(v)) return; + if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; + this.partials[index].resonator[p.key] = v; + if (this.viewer) this.viewer.render(); + }); + + // Inline jog slider for resonator params + const step = parseFloat(p.step); + const sensitivity = step * 5; + const jog = document.createElement('div'); + jog.className = 'jog-slider'; + const thumb = document.createElement('div'); + thumb.className = 'jog-thumb'; + jog.appendChild(thumb); + let dragging = false, startX = 0, startVal = 0; + const onMove = (ev) => { + if (!dragging) return; + const dx = ev.clientX - startX; + const half = jog.offsetWidth / 2; + const clamped = Math.max(-half, Math.min(half, dx)); + thumb.style.transition = 'none'; + thumb.style.left = `calc(50% - 3px + ${clamped}px)`; + const newVal = Math.max(parseFloat(inp.min) || 0, + Math.min(parseFloat(inp.max) || 1e9, startVal + dx * sensitivity)); + inp.value = newVal.toFixed(4); + if (!this.partials || !this.partials[index]) return; + if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; + this.partials[index].resonator[p.key] = newVal; + if (this.viewer) this.viewer.render(); + }; + const onUp = () => { + if (!dragging) return; + dragging = false; + thumb.style.transition = ''; + thumb.style.left = 'calc(50% - 3px)'; + document.removeEventListener('mousemove', onMove); + document.removeEventListener('mouseup', onUp); + }; + jog.addEventListener('mousedown', (ev) => { + dragging = true; + startX = ev.clientX; + startVal = parseFloat(inp.value) || 0; + document.addEventListener('mousemove', onMove); + document.addEventListener('mouseup', onUp); + ev.preventDefault(); + }); + + const wrap = document.createElement('div'); + wrap.className = 'synth-field-wrap'; + wrap.appendChild(inp); + wrap.appendChild(jog); + resSection.appendChild(lbl); + resSection.appendChild(wrap); + } + + // Show/hide helper + const applyMode = (resonator) => { + for (const el of sinSection.children) el.style.display = resonator ? 'none' : ''; + for (const el of resSection.children) el.style.display = resonator ? '' : 'none'; + btnSin.classList.toggle('active', !resonator); + btnRes.classList.toggle('active', resonator); + }; + + // Initial state + grid.appendChild(sinSection); + grid.appendChild(resSection); + applyMode(isResonator); + + // Toggle handlers + btnSin.addEventListener('click', () => { + if (!this.partials) return; + if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; + this.partials[index].resonator.enabled = false; + applyMode(false); + }); + btnRes.addEventListener('click', () => { + if (!this.partials) return; + if (!this.partials[index].resonator) this.partials[index].resonator = { ...resDefaults }; + this.partials[index].resonator.enabled = true; + applyMode(true); + }); + } + + _makeJogSlider(inp, partial, index, p, defaults) { + const slider = document.createElement('div'); + slider.className = 'jog-slider'; + const thumb = document.createElement('div'); + thumb.className = 'jog-thumb'; + slider.appendChild(thumb); + + const sensitivity = parseFloat(p.step) * 5; + let startX = 0, startVal = 0, dragging = false; + + const onMove = (e) => { + if (!dragging) return; + const dx = e.clientX - startX; + const half = slider.offsetWidth / 2; + const clamped = Math.max(-half, Math.min(half, dx)); + thumb.style.transition = 'none'; + thumb.style.left = `calc(50% - 3px + ${clamped}px)`; + const newVal = Math.max(0, startVal + dx * sensitivity); + inp.value = newVal.toFixed(3); + if (!this.partials || !this.partials[index]) return; + if (!this.partials[index].replicas) this.partials[index].replicas = { ...defaults }; + this.partials[index].replicas[p.key] = newVal; + if (this.viewer) this.viewer.render(); + }; + + const onUp = () => { + if (!dragging) return; + dragging = false; + thumb.style.transition = ''; + thumb.style.left = 'calc(50% - 3px)'; + document.removeEventListener('mousemove', onMove); + document.removeEventListener('mouseup', onUp); + }; + + slider.addEventListener('mousedown', (e) => { + dragging = true; + startX = e.clientX; + startVal = Math.max(0, parseFloat(inp.value) || 0); + document.addEventListener('mousemove', onMove); + document.addEventListener('mouseup', onUp); + e.preventDefault(); + }); + + return slider; + } + + _makeCurveUpdater(partialIndex, curveKey, field, pointIndex) { + return (e) => { + if (!this.partials) return; + const val = parseFloat(e.target.value); + if (isNaN(val)) return; + this.partials[partialIndex][curveKey][field + pointIndex] = val; + if (this.viewer) this.viewer.render(); + }; + } + + _setupButtons() { + document.getElementById('mutePartialBtn').addEventListener('click', () => { + if (this._selectedIndex < 0 || !this.partials) return; + const p = this.partials[this._selectedIndex]; + p.muted = !p.muted; + if (this.viewer) this.viewer.render(); + this._updatePropPanel(this._selectedIndex); + }); + + document.getElementById('deletePartialBtn').addEventListener('click', () => { + if (this._selectedIndex < 0 || !this.partials || !this.viewer) return; + this.partials.splice(this._selectedIndex, 1); + this.viewer.selectPartial(-1); + if (this.onPartialDeleted) this.onPartialDeleted(); + }); + } + + // --- Amplitude bezier editor --- + + _showAmpEditor(index) { + if (index < 0 || !this.partials || index >= this.partials.length) { + this._ampPanel.style.display = 'none'; + return; + } + this._ampPanel.style.display = 'block'; + const color = this.viewer ? this.viewer.partialColor(index) : '#888'; + this._ampTitle.textContent = 'Partial #' + index; + this._ampTitle.style.color = color; + this._renderAmpEditor(); + } + + _renderAmpEditor() { + if (this._selectedIndex < 0 || !this.partials || !this._ampCtx) return; + const partial = this.partials[this._selectedIndex]; + if (!partial) return; + + const canvas = this._ampCanvas; + const ctx = this._ampCtx; + const W = canvas.width, H = canvas.height; + + // Sync time range with viewer + const amp = this._amp; + amp.tMin = this.viewer ? this.viewer.t_view_min : 0; + amp.tMax = this.viewer ? this.viewer.t_view_max : 1; + amp.ampTop = Math.max(partial.amps.reduce((m, v) => Math.max(m, v), 0), 0.001) * 1.3; + + ctx.fillStyle = '#0e0e0e'; + ctx.fillRect(0, 0, W, H); + + // Horizontal grid (0, 25, 50, 75, 100% of ampTop) + ctx.lineWidth = 1; + ctx.font = '9px monospace'; + for (let k = 0; k <= 4; ++k) { + const a = amp.ampTop * k / 4; + const y = this._ampToY(a); + ctx.strokeStyle = k === 0 ? '#2a2a2a' : '#1a1a1a'; + ctx.beginPath(); ctx.moveTo(0, y); ctx.lineTo(W, y); ctx.stroke(); + ctx.fillStyle = '#383838'; + ctx.fillText(a.toFixed(3), W - 40, y - 2); + } + + // Vertical time grid (matching main view step) + if (this.viewer) { + const step = this.viewer.getAxisStep(amp.tMax - amp.tMin); + let t = Math.ceil(amp.tMin / step) * step; + ctx.strokeStyle = '#1a1a1a'; + ctx.fillStyle = '#383838'; + while (t <= amp.tMax) { + const x = this._tToX(t); + ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, H); ctx.stroke(); + ctx.fillText(t.toFixed(2) + 's', x + 2, H - 2); + t += step; + } + } + + // Raw amp data (faint dots) + ctx.fillStyle = '#2e2e2e'; + for (let i = 0; i < partial.times.length; ++i) { + const x = this._tToX(partial.times[i]); + if (x < -2 || x > W + 2) continue; + ctx.fillRect(x - 1, this._ampToY(partial.amps[i]) - 1, 2, 2); + } + + // Bezier curve + const curve = partial.ampCurve; + if (!curve) return; + + const color = this.viewer ? this.viewer.partialColor(this._selectedIndex) : '#f44'; + ctx.strokeStyle = color; + ctx.lineWidth = 2; + ctx.beginPath(); + let started = false; + for (let i = 0; i <= 120; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / 120; + const x = this._tToX(t); + if (x < -1 || x > W + 1) { started = false; continue; } + const y = this._ampToY(evalBezier(curve, t)); + if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); + } + ctx.stroke(); + + // Control points + for (let i = 0; i < 4; ++i) { + const x = this._tToX(curve['t' + i]); + const y = this._ampToY(curve['v' + i]); + ctx.fillStyle = color; + ctx.beginPath(); + ctx.arc(x, y, 6, 0, 2 * Math.PI); + ctx.fill(); + ctx.strokeStyle = '#fff'; + ctx.lineWidth = 1.5; + ctx.stroke(); + ctx.fillStyle = '#888'; + ctx.font = '9px monospace'; + ctx.fillText('P' + i, x + 8, y - 4); + } + } + + _setupAmpDrag() { + const canvas = this._ampCanvas; + if (!canvas) return; + + canvas.addEventListener('mousedown', (e) => { + if (this._selectedIndex < 0 || !this.partials) return; + const partial = this.partials[this._selectedIndex]; + if (!partial || !partial.ampCurve) return; + const rect = canvas.getBoundingClientRect(); + const x = e.clientX - rect.left, y = e.clientY - rect.top; + const curve = partial.ampCurve; + for (let i = 0; i < 4; ++i) { + if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['v' + i]) - y) <= 8) { + this._dragPointIndex = i; + canvas.style.cursor = 'grabbing'; + e.preventDefault(); + return; + } + } + }); + + canvas.addEventListener('mousemove', (e) => { + const rect = canvas.getBoundingClientRect(); + const x = e.clientX - rect.left, y = e.clientY - rect.top; + + if (this._dragPointIndex >= 0) { + const curve = this.partials[this._selectedIndex].ampCurve; + const i = this._dragPointIndex; + curve['t' + i] = Math.max(0, Math.min(this.viewer ? this.viewer.t_max : 1e6, this._xToT(x))); + curve['v' + i] = Math.max(0, this._yToAmp(y)); + this._renderAmpEditor(); + if (this.viewer) this.viewer.render(); + e.preventDefault(); + return; + } + + // Cursor hint + if (this._selectedIndex >= 0 && this.partials) { + const curve = this.partials[this._selectedIndex]?.ampCurve; + if (curve) { + let near = false; + for (let i = 0; i < 4; ++i) { + if (Math.hypot(this._tToX(curve['t' + i]) - x, this._ampToY(curve['v' + i]) - y) <= 8) { + near = true; break; + } + } + canvas.style.cursor = near ? 'grab' : 'crosshair'; + } + } + }); + + canvas.addEventListener('mouseup', () => { + if (this._dragPointIndex >= 0) { + this._dragPointIndex = -1; + canvas.style.cursor = 'crosshair'; + this._updatePropPanel(this._selectedIndex); // sync text inputs + } + }); + } + + // --- Coordinate helpers (amp canvas) --- + + _tToX(t) { + return (t - this._amp.tMin) / (this._amp.tMax - this._amp.tMin) * this._ampCanvas.width; + } + + _xToT(x) { + return this._amp.tMin + (x / this._ampCanvas.width) * (this._amp.tMax - this._amp.tMin); + } + + _ampToY(a) { + const PADY = 10, H = this._ampCanvas.height; + return PADY + (1 - a / this._amp.ampTop) * (H - 2 * PADY); + } + + _yToAmp(y) { + const PADY = 10, H = this._ampCanvas.height; + return (1 - (y - PADY) / (H - 2 * PADY)) * this._amp.ampTop; + } +} diff --git a/tools/mq_editor/fft.js b/tools/mq_editor/fft.js index 10a5b45..0d54eae 100644 --- a/tools/mq_editor/fft.js +++ b/tools/mq_editor/fft.js @@ -132,18 +132,20 @@ class STFTCache { windowed[i] = frame[i] * w; } - // Compute FFT, store only squared amplitudes (re*re + im*im, no sqrt) + // Compute FFT, store squared amplitudes and phase const fftOut = realFFT(windowed); const squaredAmplitude = new Float32Array(this.fftSize / 2); + const phase = new Float32Array(this.fftSize / 2); for (let i = 0; i < this.fftSize / 2; ++i) { const re = fftOut[i * 2]; const im = fftOut[i * 2 + 1]; squaredAmplitude[i] = re * re + im * im; + phase[i] = Math.atan2(im, re); // Cache phase for tracking const db = 10 * Math.log10(Math.max(squaredAmplitude[i], 1e-20)); if (db > this.maxDB) this.maxDB = db; } - this.frames.push({time, offset, squaredAmplitude}); + this.frames.push({time, offset, squaredAmplitude, phase}); } if (!isFinite(this.maxDB)) this.maxDB = 0; diff --git a/tools/mq_editor/index.html b/tools/mq_editor/index.html index 60076b3..a2daff5 100644 --- a/tools/mq_editor/index.html +++ b/tools/mq_editor/index.html @@ -6,6 +6,7 @@ <style> body { font-family: monospace; + font-size: 14px; margin: 20px; background: #1a1a1a; color: #ddd; @@ -39,6 +40,7 @@ } button:hover { background: #4a4a4a; } button:disabled { opacity: 0.5; cursor: not-allowed; } + #extractBtn { background: #666; color: #fff; font-weight: bold; border-color: #888; } input[type="file"] { display: none; } .params { display: inline-block; @@ -73,19 +75,26 @@ border: 1px solid #555; border-radius: 4px; padding: 12px; - min-width: 160px; + min-width: 260px; + max-width: 260px; + max-height: 700px; + overflow-y: auto; display: flex; flex-direction: column; - gap: 10px; + gap: 6px; + box-sizing: border-box; } - .right-panel .panel-title { - font-size: 11px; + .panel-title { + font-size: 12px; color: #888; text-transform: uppercase; letter-spacing: 1px; border-bottom: 1px solid #444; padding-bottom: 6px; margin-bottom: 2px; + display: flex; + align-items: center; + gap: 6px; } .right-panel label { display: flex; @@ -93,7 +102,141 @@ gap: 6px; margin: 0; cursor: pointer; + font-size: 14px; + } + /* Partial properties */ + .prop-row { + display: flex; + justify-content: space-between; + align-items: baseline; font-size: 13px; + padding: 2px 0; + } + .prop-label { color: #777; font-size: 12px; } + .curve-tabs { + display: flex; + gap: 2px; + margin-top: 8px; + margin-bottom: 4px; + } + .tab-btn { + flex: 1; + padding: 4px 0; + font-size: 12px; + margin: 0; + background: #222; + border-color: #444; + color: #888; + } + .tab-btn.active { + background: #3a3a3a; + border-color: #666; + color: #ddd; + } + .curve-grid { + display: grid; + grid-template-columns: 18px 1fr 1fr; + gap: 3px 4px; + align-items: center; + } + .curve-grid span { color: #666; font-size: 11px; } + .curve-grid input[type="number"] { + width: 100%; + background: #333; + color: #ccc; + border: 1px solid #444; + padding: 2px 4px; + border-radius: 2px; + font-size: 11px; + font-family: monospace; + box-sizing: border-box; + } + .curve-grid input[type="number"]:focus { + border-color: #666; + outline: none; + } + .partial-actions { + display: flex; + gap: 4px; + margin-top: 8px; + } + .partial-actions button { + flex: 1; + padding: 4px 6px; + font-size: 12px; + margin: 0; + } + .synth-grid { + display: grid; + grid-template-columns: auto 1fr; + gap: 4px 8px; + align-items: center; + } + .synth-grid span { color: #888; font-size: 12px; } + .synth-field-wrap { + display: flex; + align-items: center; + gap: 4px; + } + .synth-field-wrap input[type="number"] { + flex: 1; + min-width: 0; + background: #333; + color: #ccc; + border: 1px solid #444; + padding: 2px 4px; + border-radius: 2px; + font-size: 11px; + font-family: monospace; + box-sizing: border-box; + } + .synth-field-wrap input[type="number"]:focus { border-color: #666; outline: none; } + .jog-slider { + width: 44px; + height: 16px; + background: #1e1e1e; + border: 1px solid #444; + border-radius: 3px; + cursor: ew-resize; + position: relative; + flex-shrink: 0; + user-select: none; + overflow: hidden; + } + .jog-slider::before { + content: ''; + position: absolute; + left: 50%; top: 3px; bottom: 3px; + width: 1px; + background: #484848; + transform: translateX(-50%); + } + .jog-thumb { + position: absolute; + top: 3px; bottom: 3px; + width: 6px; + background: #888; + border-radius: 2px; + left: calc(50% - 3px); + transition: left 0.12s ease; + } + .jog-slider:hover .jog-thumb { background: #aaa; } + .synth-grid input[type="number"]:focus { border-color: #666; outline: none; } + .synth-section { + border-top: 1px solid #444; + padding-top: 8px; + margin-top: auto; + } + /* resonator mode badge shown in partial header color swatch area */ + .res-badge { + font-size: 9px; + color: #8cf; + border: 1px solid #8cf; + border-radius: 2px; + padding: 0 3px; + vertical-align: middle; + margin-left: 4px; + opacity: 0.8; } #status { margin-top: 10px; @@ -118,6 +261,7 @@ <button id="chooseFileBtn">📂 Open WAV</button> <button id="testWavBtn">⚗ Test WAV</button> <button id="extractBtn" disabled>Extract Partials</button> + <button id="autoSpreadAllBtn" disabled>Auto Spread All</button> <button id="playBtn" disabled>▶ Play</button> <button id="stopBtn" disabled>■ Stop</button> @@ -126,7 +270,14 @@ <input type="number" id="hopSize" value="256" min="64" max="1024" step="64"> <label>Threshold (dB):</label> - <input type="number" id="threshold" value="-60" step="any"> + <input type="number" id="threshold" value="-20" step="any"> + + <label>Prominence (dB):</label> + <input type="number" id="prominence" value="1.0" step="0.1" min="0"> + + <label style="margin-left:16px;" title="Weight spectrum by frequency before peak detection (f * FFT_Power(f)), accentuates high-frequency peaks"> + <input type="checkbox" id="freqWeight"> f·Power + </label> <label style="margin-left:16px;">Keep:</label> <input type="range" id="keepPct" min="1" max="100" value="100" style="width:100px; vertical-align:middle;"> @@ -135,21 +286,98 @@ </div> <div class="main-area"> - <div style="position: relative; flex-shrink: 0;"> - <canvas id="canvas" width="1400" height="600"></canvas> - <canvas id="cursorCanvas" width="1400" height="600" style="position:absolute;top:0;left:0;pointer-events:none;"></canvas> + <div style="display:flex; flex-direction:column; gap:6px; flex-shrink:0;"> + <div style="position: relative;"> + <canvas id="canvas" width="1400" height="600"></canvas> + <canvas id="cursorCanvas" width="1400" height="600" style="position:absolute;top:0;left:0;pointer-events:none;"></canvas> - <!-- Mini spectrum viewer (bottom-right overlay) --> - <div id="spectrumViewer" style="position: absolute; bottom: 10px; right: 10px; width: 400px; height: 100px; background: rgba(30, 30, 30, 0.9); border: 1px solid #555; border-radius: 3px; pointer-events: none;"> - <canvas id="spectrumCanvas" width="400" height="100"></canvas> + <!-- Mini spectrum viewer (bottom-right overlay) --> + <div id="spectrumViewer" style="position: absolute; bottom: 10px; right: 10px; width: 400px; height: 100px; background: rgba(30, 30, 30, 0.9); border: 1px solid #555; border-radius: 3px; pointer-events: none;"> + <canvas id="spectrumCanvas" width="400" height="100"></canvas> + </div> + </div> + + <!-- Amplitude bezier editor (shown when partial selected) --> + <div id="ampEditPanel" style="display:none;"> + <div style="font-size:10px; color:#555; padding:2px 0 3px 1px; display:flex; align-items:center; gap:10px; text-transform:uppercase; letter-spacing:0.5px;"> + <span>Amplitude</span> + <span id="ampEditTitle" style="color:#777; text-transform:none; letter-spacing:0;"></span> + <span style="color:#333; text-transform:none; letter-spacing:0;">drag control points · Esc to deselect</span> + </div> + <canvas id="ampEditCanvas" width="1400" height="120" style="border:1px solid #2a2a2a; background:#0e0e0e; cursor:crosshair; display:block;"></canvas> </div> </div> <div class="right-panel"> - <div class="panel-title">Synthesis</div> - <label><input type="checkbox" id="integratePhase" checked> Integrate phase</label> - <label><input type="checkbox" id="disableJitter"> Disable jitter</label> - <label><input type="checkbox" id="disableSpread"> Disable spread</label> + <!-- Partial properties (visible when a partial is selected) --> + <div id="partialProps" style="display:none;"> + <div class="panel-title"> + <span id="propTitle">Partial #—</span> + <span id="propSwatch" style="display:inline-block;width:10px;height:10px;border-radius:2px;flex-shrink:0;"></span> + </div> + <div class="prop-row"> + <span class="prop-label">Peak</span> + <span id="propPeak">—</span> + </div> + <div class="prop-row"> + <span class="prop-label">Time</span> + <span id="propTime">—</span> + </div> + <div class="curve-tabs"> + <button class="tab-btn active" data-tab="Freq">Freq</button> + <button class="tab-btn" data-tab="Amp">Amp</button> + <button class="tab-btn" data-tab="Synth">Synth</button> + </div> + <div class="tab-pane" id="tabFreq"> + <div class="curve-grid" id="freqCurveGrid"></div> + </div> + <div class="tab-pane" id="tabAmp" style="display:none;"> + <div class="curve-grid" id="ampCurveGrid"></div> + </div> + <div class="tab-pane" id="tabSynth" style="display:none;"> + <div class="synth-grid" id="synthGrid"></div> + </div> + <div class="partial-actions"> + <button id="mutePartialBtn">Mute</button> + <button id="deletePartialBtn">Delete</button> + </div> + </div> + + <div id="noSelMsg" style="color:#555;font-size:13px;padding:2px 0;">Click a partial to select</div> + + <!-- Synthesis options (always at bottom) --> + <div class="synth-section"> + <div class="panel-title">Synthesis</div> + <label><input type="checkbox" id="integratePhase" checked> Integrate phase</label> + <label><input type="checkbox" id="disableJitter"> Disable jitter</label> + <label><input type="checkbox" id="disableSpread"> Disable spread</label> + <label title="Test mode: force resonator synthesis for all partials (ignores per-partial mode setting)"><input type="checkbox" id="forceResonator"> Resonator (all)</label> + <div id="globalResParams" style="display:none;margin-top:4px;padding:4px 0 2px 12px;border-left:2px solid #555;"> + <label style="display:flex;align-items:center;gap:6px;" title="Global pole radius r in (0,1). Applied to all partials in resonator mode."> + r (pole) + <input type="range" id="globalR" min="0.75" max="0.9999" step="0.0001" value="0.995" style="flex:1;min-width:0;"> + <span id="globalRVal" style="width:44px;text-align:right;">0.9950</span> + </label> + <label style="display:flex;align-items:center;gap:6px;" title="Global gain compensation applied to all partials in resonator mode."> + gain + <input type="range" id="globalGain" min="0.0" max="4.0" step="0.01" value="1.0" style="flex:1;min-width:0;"> + <span id="globalGainVal" style="width:44px;text-align:right;">1.00</span> + </label> + <label title="Override per-partial r/gain with global values during playback"><input type="checkbox" id="forceRGain"> force r/gain</label> + </div> + <div style="margin-top:6px;"> + <label style="display:flex;align-items:center;gap:6px;" title="LP filter coefficient k1 in (0,1]. 1.0 = bypass."> + LP k1 + <input type="range" id="lpK1" min="0.001" max="1.0" step="0.001" value="1.0" style="flex:1;min-width:0;"> + <span id="lpK1Val" style="width:44px;text-align:right;">bypass</span> + </label> + <label style="display:flex;align-items:center;gap:6px;" title="HP filter coefficient k2 in (0,1]. 1.0 = bypass."> + HP k2 + <input type="range" id="hpK2" min="0.001" max="1.0" step="0.001" value="1.0" style="flex:1;min-width:0;"> + <span id="hpK2Val" style="width:44px;text-align:right;">bypass</span> + </label> + </div> + </div> </div> </div> @@ -161,7 +389,47 @@ <script src="mq_extract.js"></script> <script src="mq_synth.js"></script> <script src="viewer.js"></script> + <script src="editor.js"></script> <script> + // LP: y[n] = k*x[n] + (1-k)*y[n-1] => -3dB at cos(w) = (2-2k-k²)/(2(1-k)) + function k1ToHz(k, sr) { + if (k >= 1.0) return sr / 2; + const cosW = (2 - 2*k - k*k) / (2*(1 - k)); + return Math.acos(Math.max(-1, Math.min(1, cosW))) * sr / (2 * Math.PI); + } + // HP: y[n] = k*(y[n-1]+x[n]-x[n-1]) => -3dB from peak at cos(w) = 2k/(1+k²) + function k2ToHz(k, sr) { + if (k >= 1.0) return 0; + const cosW = 2*k / (1 + k*k); + return Math.acos(Math.max(-1, Math.min(1, cosW))) * sr / (2 * Math.PI); + } + function fmtHz(f) { + return f >= 1000 ? (f/1000).toFixed(1) + 'k' : Math.round(f) + 'Hz'; + } + function getSR() { return (typeof audioBuffer !== 'undefined' && audioBuffer) ? audioBuffer.sampleRate : 44100; } + + // LP/HP slider live display + document.getElementById('lpK1').addEventListener('input', function() { + const k = parseFloat(this.value); + const f = k1ToHz(k, getSR()); + document.getElementById('lpK1Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); + }); + document.getElementById('hpK2').addEventListener('input', function() { + const k = parseFloat(this.value); + const f = k2ToHz(k, getSR()); + document.getElementById('hpK2Val').textContent = k >= 1.0 ? 'bypass' : fmtHz(f); + }); + + // Show/hide global resonator params when forceResonator toggled + document.getElementById('forceResonator').addEventListener('change', function() { + document.getElementById('globalResParams').style.display = this.checked ? '' : 'none'; + }); + document.getElementById('globalR').addEventListener('input', function() { + document.getElementById('globalRVal').textContent = parseFloat(this.value).toFixed(4); + }); + document.getElementById('globalGain').addEventListener('input', function() { + document.getElementById('globalGainVal').textContent = parseFloat(this.value).toFixed(2); + }); let audioBuffer = null; let viewer = null; let audioContext = null; @@ -172,6 +440,7 @@ const wavFile = document.getElementById('wavFile'); const chooseFileBtn = document.getElementById('chooseFileBtn'); const extractBtn = document.getElementById('extractBtn'); + const autoSpreadAllBtn = document.getElementById('autoSpreadAllBtn'); const playBtn = document.getElementById('playBtn'); const stopBtn = document.getElementById('stopBtn'); const canvas = document.getElementById('canvas'); @@ -180,6 +449,8 @@ const hopSize = document.getElementById('hopSize'); const threshold = document.getElementById('threshold'); + const prominence = document.getElementById('prominence'); + const freqWeightCb = document.getElementById('freqWeight'); const keepPct = document.getElementById('keepPct'); const keepPctLabel = document.getElementById('keepPctLabel'); const fftSize = 1024; // Fixed @@ -193,6 +464,13 @@ if (viewer && extractedPartials) viewer.setKeepCount(getKeepCount()); }); + // --- Editor --- + const editor = new PartialEditor(); + editor.onPartialDeleted = () => { + if (viewer && extractedPartials) + viewer.setKeepCount(extractedPartials.length > 0 ? getKeepCount() : 0); + }; + // Initialize audio context function initAudioContext() { if (!audioContext) { @@ -208,11 +486,18 @@ playBtn.disabled = false; setStatus('Computing STFT cache...', 'info'); + // Reset partials from previous file + extractedPartials = null; + editor.setPartials(null); + setTimeout(() => { const signal = audioBuffer.getChannelData(0); stftCache = new STFTCache(signal, audioBuffer.sampleRate, fftSize, Math.max(64, parseInt(hopSize.value) || 64)); setStatus(`${label} — ${audioBuffer.duration.toFixed(2)}s, ${audioBuffer.sampleRate}Hz, ${audioBuffer.numberOfChannels}ch (${stftCache.getNumFrames()} frames cached)`, 'info'); viewer = new SpectrogramViewer(canvas, audioBuffer, stftCache); + editor.setViewer(viewer); + viewer.onPartialSelect = (i) => editor.onPartialSelect(i); + viewer.onRender = () => editor.onRender(); if (label.startsWith('Test WAV')) validateTestWAVPeaks(stftCache); }, 10); } @@ -283,6 +568,8 @@ fftSize: fftSize, hopSize: parseInt(hopSize.value), threshold: parseFloat(threshold.value), + prominence: parseFloat(prominence.value), + freqWeight: freqWeightCb.checked, sampleRate: audioBuffer.sampleRate }; @@ -296,29 +583,52 @@ }); extractedPartials = result.partials; + editor.setPartials(result.partials); viewer.setFrames(result.frames); setStatus(`Extracted ${result.partials.length} partials`, 'info'); viewer.setPartials(result.partials); viewer.setKeepCount(getKeepCount()); + viewer.selectPartial(-1); } catch (err) { setStatus('Extraction error: ' + err.message, 'error'); console.error(err); } extractBtn.disabled = false; + autoSpreadAllBtn.disabled = false; }, 50); } - // Extract partials extractBtn.addEventListener('click', () => { if (!audioBuffer) return; runExtraction(); }); + autoSpreadAllBtn.addEventListener('click', () => { + if (!extractedPartials || !stftCache) return; + const fs = stftCache.fftSize; + const sr = audioBuffer.sampleRate; + const defaults = { decay_alpha: 0.1, jitter: 0.05, spread_above: 0.02, spread_below: 0.02 }; + for (const p of extractedPartials) { + const {spread_above, spread_below} = autodetectSpread(p, stftCache, fs, sr); + if (!p.replicas) p.replicas = { ...defaults }; + p.replicas.spread_above = spread_above; + p.replicas.spread_below = spread_below; + } + if (viewer) viewer.render(); + const sel = viewer ? viewer.selectedPartial : -1; + if (sel >= 0) editor.onPartialSelect(sel); + setStatus(`Auto-spread applied to ${extractedPartials.length} partials`, 'info'); + }); + threshold.addEventListener('change', () => { if (stftCache) runExtraction(); }); + freqWeightCb.addEventListener('change', () => { + if (stftCache) runExtraction(); + }); + function playAudioBuffer(buffer, statusMsg) { const startTime = audioContext.currentTime; currentSource = audioContext.createBufferSource(); @@ -384,14 +694,23 @@ setStatus('Synthesizing...', 'info'); const keepCount = getKeepCount(); - const partialsToUse = extractedPartials.slice(0, keepCount); - setStatus(`Synthesizing ${keepCount}/${extractedPartials.length} partials (${keepPct.value}%)...`, 'info'); + const partialsToUse = extractedPartials.slice(0, keepCount).filter(p => !p.muted); + setStatus(`Synthesizing ${partialsToUse.length}/${extractedPartials.length} partials (${keepPct.value}%)...`, 'info'); - const integratePhase = document.getElementById('integratePhase').checked; - const disableJitter = document.getElementById('disableJitter').checked; - const disableSpread = document.getElementById('disableSpread').checked; + const integratePhase = document.getElementById('integratePhase').checked; + const disableJitter = document.getElementById('disableJitter').checked; + const disableSpread = document.getElementById('disableSpread').checked; + const forceResonator = document.getElementById('forceResonator').checked; + const lpK1Raw = parseFloat(document.getElementById('lpK1').value); + const hpK2Raw = parseFloat(document.getElementById('hpK2').value); + const k1 = lpK1Raw < 1.0 ? lpK1Raw : null; + const k2 = hpK2Raw < 1.0 ? hpK2Raw : null; + const forceRGain = forceResonator && document.getElementById('forceRGain').checked; + const globalR = parseFloat(document.getElementById('globalR').value); + const globalGain = parseFloat(document.getElementById('globalGain').value); const pcm = synthesizeMQ(partialsToUse, audioBuffer.sampleRate, audioBuffer.duration, - integratePhase, {disableJitter, disableSpread}); + integratePhase, {disableJitter, disableSpread, forceResonator, + forceRGain, globalR, globalGain, k1, k2}); if (viewer) { viewer.setSynthStftCache(new STFTCache(pcm, audioBuffer.sampleRate, fftSize, parseInt(hopSize.value))); @@ -399,7 +718,7 @@ const synthBuffer = audioContext.createBuffer(1, pcm.length, audioBuffer.sampleRate); synthBuffer.getChannelData(0).set(pcm); - playAudioBuffer(synthBuffer, `Playing synthesized (${keepCount}/${extractedPartials.length} partials, ${keepPct.value}%)...`); + playAudioBuffer(synthBuffer, `Playing synthesized (${partialsToUse.length}/${extractedPartials.length} partials, ${keepPct.value}%)...`); } // Keyboard shortcuts @@ -422,9 +741,24 @@ viewer.showSynthFFT = !viewer.showSynthFFT; viewer.renderSpectrum(); } + } else if (e.code === 'KeyE') { + e.preventDefault(); + if (!extractBtn.disabled) extractBtn.click(); + } else if (e.code === 'Escape') { + if (viewer) viewer.selectPartial(-1); } }); + // Curve tab switching + document.querySelectorAll('.tab-btn').forEach(btn => { + btn.addEventListener('click', () => { + document.querySelectorAll('.tab-btn').forEach(b => b.classList.remove('active')); + btn.classList.add('active'); + document.querySelectorAll('.tab-pane').forEach(p => p.style.display = 'none'); + document.getElementById('tab' + btn.dataset.tab).style.display = ''; + }); + }); + // --- Test WAV peak validation --- function validateTestWAVPeaks(cache) { const SR = cache.sampleRate; diff --git a/tools/mq_editor/mq_extract.js b/tools/mq_editor/mq_extract.js index 8a0ea0e..a530960 100644 --- a/tools/mq_editor/mq_extract.js +++ b/tools/mq_editor/mq_extract.js @@ -3,18 +3,19 @@ // Extract partials from audio buffer function extractPartials(params, stftCache) { - const {fftSize, threshold, sampleRate} = params; + const {fftSize, threshold, sampleRate, freqWeight, prominence} = params; const numFrames = stftCache.getNumFrames(); const frames = []; for (let i = 0; i < numFrames; ++i) { const cachedFrame = stftCache.getFrameAtIndex(i); - const squaredAmp = stftCache.getSquaredAmplitude(cachedFrame.time); - const peaks = detectPeaks(squaredAmp, fftSize, sampleRate, threshold); + const squaredAmp = cachedFrame.squaredAmplitude; + const phase = cachedFrame.phase; + const peaks = detectPeaks(squaredAmp, phase, fftSize, sampleRate, threshold, freqWeight, prominence); frames.push({time: cachedFrame.time, peaks}); } - const partials = trackPartials(frames); + const partials = trackPartials(frames, params); // Second pass: extend partials leftward to recover onset frames expandPartialsLeft(partials, frames); @@ -27,12 +28,32 @@ function extractPartials(params, stftCache) { return {partials, frames}; } +// Helper to interpolate phase via quadratic formula on unwrapped neighbors. +// This provides a more accurate phase estimate at the sub-bin peak location. +function phaseInterp(p_minus, p_center, p_plus, p_frac) { + // unwrap neighbors relative to center + let dp_minus = p_minus - p_center; + while (dp_minus > Math.PI) dp_minus -= 2 * Math.PI; + while (dp_minus < -Math.PI) dp_minus += 2 * Math.PI; + + let dp_plus = p_plus - p_center; + while (dp_plus > Math.PI) dp_plus -= 2 * Math.PI; + while (dp_plus < -Math.PI) dp_plus += 2 * Math.PI; + + const p_interp = p_center + (dp_plus - dp_minus) * p_frac * 0.5 + (dp_plus + dp_minus) * p_frac * p_frac; + return p_interp; +} + // Detect spectral peaks via local maxima + parabolic interpolation // squaredAmp: pre-computed re*re+im*im per bin -function detectPeaks(squaredAmp, fftSize, sampleRate, thresholdDB) { +// phase: pre-computed atan2(im,re) per bin +// freqWeight: if true, weight by f before peak detection (f * Power(f)) +function detectPeaks(squaredAmp, phase, fftSize, sampleRate, thresholdDB, freqWeight, prominenceDB = 0) { const mag = new Float32Array(fftSize / 2); + const binHz = sampleRate / fftSize; for (let i = 0; i < fftSize / 2; ++i) { - mag[i] = 10 * Math.log10(Math.max(squaredAmp[i], 1e-20)); + const w = freqWeight ? (i * binHz) : 1.0; + mag[i] = 10 * Math.log10(Math.max(squaredAmp[i] * w, 1e-20)); } const peaks = []; @@ -41,23 +62,49 @@ function detectPeaks(squaredAmp, fftSize, sampleRate, thresholdDB) { mag[i] > mag[i-1] && mag[i] > mag[i-2] && mag[i] > mag[i+1] && mag[i] > mag[i+2]) { - // Parabolic interpolation for sub-bin accuracy + // Check prominence if requested + if (prominenceDB > 0) { + let minLeft = mag[i]; + for (let k = i - 1; k >= 0; --k) { + if (mag[k] > mag[i]) break; // Found higher peak + if (mag[k] < minLeft) minLeft = mag[k]; + } + + let minRight = mag[i]; + for (let k = i + 1; k < mag.length; ++k) { + if (mag[k] > mag[i]) break; // Found higher peak + if (mag[k] < minRight) minRight = mag[k]; + } + + const valley = Math.max(minLeft, minRight); + if (mag[i] - valley < prominenceDB) continue; + } + + // Parabolic interpolation for sub-bin accuracy on frequency, amplitude, and phase const alpha = mag[i-1]; const beta = mag[i]; const gamma = mag[i+1]; const p = 0.5 * (alpha - gamma) / (alpha - 2*beta + gamma); + + const p_phase = phaseInterp(phase[i-1], phase[i], phase[i+1], p); const freq = (i + p) * sampleRate / fftSize; const ampDB = beta - 0.25 * (alpha - gamma) * p; - peaks.push({freq, amp: Math.pow(10, ampDB / 20)}); + peaks.push({freq, amp: Math.pow(10, ampDB / 20), phase: p_phase}); } } return peaks; } -// Track partials across frames (birth/death/continuation) -function trackPartials(frames) { +// Helper to compute shortest angle difference (e.g., between -pi and pi) +function normalizeAngle(angle) { + return angle - 2 * Math.PI * Math.floor((angle + Math.PI) / (2 * Math.PI)); +} + +// Track partials across frames using phase coherence for robust matching. +function trackPartials(frames, params) { + const { sampleRate, hopSize } = params; const partials = []; const activePartials = []; const candidates = []; // pre-birth @@ -68,19 +115,37 @@ function trackPartials(frames) { const deathAge = 5; // frames without match before death const minLength = 10; // frames required to keep partial + // Weight phase error heavily in cost function, scaled by frequency. + // This makes phase deviation more significant for high-frequency partials. + const phaseErrorWeight = 2.0; + for (const frame of frames) { const matched = new Set(); - // Continue active partials + // --- Continue active partials --- for (const partial of activePartials) { const lastFreq = partial.freqs[partial.freqs.length - 1]; - const tol = Math.max(lastFreq * trackingRatio, minTrackingHz); - let bestIdx = -1, bestDist = Infinity; + const lastPhase = partial.phases[partial.phases.length - 1]; + const velocity = partial.velocity || 0; + const predictedFreq = lastFreq + velocity; + + // Predict phase for the current frame based on the last frame's frequency. + const phaseAdvance = 2 * Math.PI * lastFreq * hopSize / sampleRate; + const predictedPhase = lastPhase + phaseAdvance; + + const tol = Math.max(predictedFreq * trackingRatio, minTrackingHz); + let bestIdx = -1, bestCost = Infinity; + // Find the peak in the new frame with the lowest cost (freq + phase error). for (let i = 0; i < frame.peaks.length; ++i) { if (matched.has(i)) continue; - const dist = Math.abs(frame.peaks[i].freq - lastFreq); - if (dist < tol && dist < bestDist) { bestDist = dist; bestIdx = i; } + const pk = frame.peaks[i]; + const freqError = Math.abs(pk.freq - predictedFreq); + if (freqError > tol) continue; + + const phaseError = Math.abs(normalizeAngle(pk.phase - predictedPhase)); + const cost = freqError + phaseErrorWeight * phaseError * predictedFreq; + if (cost < bestCost) { bestCost = cost; bestIdx = i; } } if (bestIdx >= 0) { @@ -88,24 +153,38 @@ function trackPartials(frames) { partial.times.push(frame.time); partial.freqs.push(pk.freq); partial.amps.push(pk.amp); + partial.phases.push(pk.phase); partial.age = 0; + partial.velocity = pk.freq - lastFreq; matched.add(bestIdx); } else { partial.age++; } } - // Advance candidates + // --- Advance candidates --- for (let i = candidates.length - 1; i >= 0; --i) { const cand = candidates[i]; const lastFreq = cand.freqs[cand.freqs.length - 1]; - const tol = Math.max(lastFreq * trackingRatio, minTrackingHz); - let bestIdx = -1, bestDist = Infinity; + const lastPhase = cand.phases[cand.phases.length - 1]; + const velocity = cand.velocity || 0; + const predictedFreq = lastFreq + velocity; + + const phaseAdvance = 2 * Math.PI * lastFreq * hopSize / sampleRate; + const predictedPhase = lastPhase + phaseAdvance; + + const tol = Math.max(predictedFreq * trackingRatio, minTrackingHz); + let bestIdx = -1, bestCost = Infinity; for (let j = 0; j < frame.peaks.length; ++j) { if (matched.has(j)) continue; - const dist = Math.abs(frame.peaks[j].freq - lastFreq); - if (dist < tol && dist < bestDist) { bestDist = dist; bestIdx = j; } + const pk = frame.peaks[j]; + const freqError = Math.abs(pk.freq - predictedFreq); + if (freqError > tol) continue; + + const phaseError = Math.abs(normalizeAngle(pk.phase - predictedPhase)); + const cost = freqError + phaseErrorWeight * phaseError * predictedFreq; + if (cost < bestCost) { bestCost = cost; bestIdx = j; } } if (bestIdx >= 0) { @@ -113,24 +192,34 @@ function trackPartials(frames) { cand.times.push(frame.time); cand.freqs.push(pk.freq); cand.amps.push(pk.amp); + cand.phases.push(pk.phase); + cand.velocity = pk.freq - lastFreq; matched.add(bestIdx); + // "graduate" a candidate to a full partial if (cand.times.length >= birthPersistence) { activePartials.push(cand); candidates.splice(i, 1); } } else { - candidates.splice(i, 1); + candidates.splice(i, 1); // kill candidate } } - // Spawn candidates from unmatched peaks + // --- Spawn new candidates from unmatched peaks --- for (let i = 0; i < frame.peaks.length; ++i) { if (matched.has(i)) continue; const pk = frame.peaks[i]; - candidates.push({times: [frame.time], freqs: [pk.freq], amps: [pk.amp], age: 0}); + candidates.push({ + times: [frame.time], + freqs: [pk.freq], + amps: [pk.amp], + phases: [pk.phase], + age: 0, + velocity: 0 + }); } - // Kill aged-out partials + // --- Kill aged-out partials --- for (let i = activePartials.length - 1; i >= 0; --i) { if (activePartials[i].age > deathAge) { if (activePartials[i].times.length >= minLength) partials.push(activePartials[i]); @@ -139,7 +228,7 @@ function trackPartials(frames) { } } - // Collect remaining active partials + // --- Collect remaining active partials --- for (const partial of activePartials) { if (partial.times.length >= minLength) partials.push(partial); } @@ -158,6 +247,8 @@ function expandPartialsLeft(partials, frames) { for (let i = 0; i < frames.length; ++i) timeToIdx.set(frames[i].time, i); for (const partial of partials) { + if (!partial.phases) partial.phases = []; // Ensure old partials have phase array + let startIdx = timeToIdx.get(partial.times[0]); if (startIdx == null || startIdx === 0) continue; @@ -178,23 +269,130 @@ function expandPartialsLeft(partials, frames) { partial.times.unshift(frame.time); partial.freqs.unshift(pk.freq); partial.amps.unshift(pk.amp); + partial.phases.unshift(pk.phase); + } + } +} + +// Autodetect spread_above / spread_below from the spectrogram. +// For each (subsampled) STFT frame within the partial, measures the +// half-power (-3dB) width of the spectral peak above and below the center. +// spread = half_bandwidth / f0 (fractional). +function autodetectSpread(partial, stftCache, fftSize, sampleRate) { + const curve = partial.freqCurve; + if (!curve || !stftCache) return {spread_above: 0.02, spread_below: 0.02}; + + const numFrames = stftCache.getNumFrames(); + const binHz = sampleRate / fftSize; + const halfBins = fftSize / 2; + + let sumAbove = 0, sumBelow = 0, count = 0; + + const STEP = 4; + for (let fi = 0; fi < numFrames; fi += STEP) { + const frame = stftCache.getFrameAtIndex(fi); + if (!frame) continue; + const t = frame.time; + if (t < curve.t0 || t > curve.t3) continue; + + const f0 = evalBezier(curve, t); + if (f0 <= 0) continue; + + const sq = frame.squaredAmplitude; + if (!sq) continue; + + // Find peak bin in ±10% window + const binCenter = f0 / binHz; + const searchBins = Math.max(3, Math.round(f0 * 0.10 / binHz)); + const binLo = Math.max(1, Math.floor(binCenter - searchBins)); + const binHi = Math.min(halfBins - 2, Math.ceil(binCenter + searchBins)); + + let peakBin = binLo, peakVal = sq[binLo]; + for (let b = binLo + 1; b <= binHi; ++b) { + if (sq[b] > peakVal) { peakVal = sq[b]; peakBin = b; } } + + const halfPower = peakVal * 0.5; // -3dB in power + + // Walk above peak until half-power, interpolate crossing + let aboveBin = peakBin; + while (aboveBin < halfBins - 1 && sq[aboveBin] > halfPower) ++aboveBin; + const tA = aboveBin > peakBin && sq[aboveBin - 1] !== sq[aboveBin] + ? (halfPower - sq[aboveBin - 1]) / (sq[aboveBin] - sq[aboveBin - 1]) + : 0; + const widthAbove = (aboveBin - 1 + tA - peakBin) * binHz; + + // Walk below peak until half-power, interpolate crossing + let belowBin = peakBin; + while (belowBin > 1 && sq[belowBin] > halfPower) --belowBin; + const tB = belowBin < peakBin && sq[belowBin + 1] !== sq[belowBin] + ? (halfPower - sq[belowBin + 1]) / (sq[belowBin] - sq[belowBin + 1]) + : 0; + const widthBelow = (peakBin - belowBin - 1 + tB) * binHz; + + sumAbove += (widthAbove / f0) * (widthAbove / f0); + sumBelow += (widthBelow / f0) * (widthBelow / f0); + ++count; } + + const spread_above = count > 0 ? Math.sqrt(sumAbove / count) : 0.01; + const spread_below = count > 0 ? Math.sqrt(sumBelow / count) : 0.01; + return {spread_above, spread_below}; } -// Fit cubic bezier to trajectory using samples at ~1/3 and ~2/3 as control points +// Fit cubic bezier to trajectory using least-squares for inner control points function fitBezier(times, values) { const n = times.length - 1; const t0 = times[0], v0 = values[0]; const t3 = times[n], v3 = values[n]; const dt = t3 - t0; - if (dt <= 0 || n === 0) { - return {t0, v0, t1: t0, v1: v0, t2: t3, v2: v3, t3, v3}; + if (dt <= 1e-9 || n < 2) { + // Linear fallback for too few points or zero duration + return {t0, v0, t1: t0 + dt / 3, v1: v0 + (v3 - v0) / 3, t2: t0 + 2 * dt / 3, v2: v0 + 2 * (v3 - v0) / 3, t3, v3}; } - const v1 = values[Math.round(n / 3)]; - const v2 = values[Math.round(2 * n / 3)]; + // Least squares solve for v1, v2 + // Bezier: B(u) = (1-u)^3*v0 + 3(1-u)^2*u*v1 + 3(1-u)*u^2*v2 + u^3*v3 + // Target_i = val_i - (1-u)^3*v0 - u^3*v3 + // Model_i = A_i*v1 + B_i*v2 + // A_i = 3(1-u)^2*u + // B_i = 3(1-u)*u^2 + + let sA2 = 0, sB2 = 0, sAB = 0, sAT = 0, sBT = 0; + + for (let i = 0; i <= n; ++i) { + const u = (times[i] - t0) / dt; + const u2 = u * u; + const u3 = u2 * u; + const invU = 1.0 - u; + const invU2 = invU * invU; + const invU3 = invU2 * invU; + + const A = 3 * invU2 * u; + const B = 3 * invU * u2; + const target = values[i] - (invU3 * v0 + u3 * v3); + + sA2 += A * A; + sB2 += B * B; + sAB += A * B; + sAT += A * target; + sBT += B * target; + } + + const det = sA2 * sB2 - sAB * sAB; + let v1, v2; + + if (Math.abs(det) < 1e-9) { + // Fallback to simple 1/3, 2/3 heuristic if matrix is singular + const idx1 = Math.round(n / 3); + const idx2 = Math.round(2 * n / 3); + v1 = values[idx1]; + v2 = values[idx2]; + } else { + v1 = (sB2 * sAT - sAB * sBT) / det; + v2 = (sA2 * sBT - sAB * sAT) / det; + } return {t0, v0, t1: t0 + dt / 3, v1, t2: t0 + 2 * dt / 3, v2, t3, v3}; } diff --git a/tools/mq_editor/mq_synth.js b/tools/mq_editor/mq_synth.js index 1eec709..2d4cf1b 100644 --- a/tools/mq_editor/mq_synth.js +++ b/tools/mq_editor/mq_synth.js @@ -1,5 +1,5 @@ // MQ Synthesizer -// Replica oscillator bank for sinusoidal synthesis +// Replica oscillator bank for sinusoidal synthesis, plus two-pole resonator mode // Evaluate cubic bezier curve at time t function evalBezier(curve, t) { @@ -21,10 +21,13 @@ function randFloat(seed, min, max) { } // Synthesize audio from MQ partials -// partials: array of {freqCurve, ampCurve, replicas?} -// replicas: {offsets, decay_alpha, jitter, spread_above, spread_below} +// partials: array of {freqCurve, ampCurve, replicas?, resonator?} +// replicas: {offsets, decay_alpha, jitter, spread_above, spread_below} +// resonator: {enabled, r, gainComp} — two-pole resonator mode per partial // integratePhase: true = accumulate 2π*f/SR per sample (correct for varying freq) // false = 2π*f*t (simpler, only correct for constant freq) +// options.k1: LP coefficient in (0,1] — omit to bypass +// options.k2: HP coefficient in (0,1] — omit to bypass function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, options = {}) { const numSamples = Math.floor(sampleRate * duration); const pcm = new Float32Array(numSamples); @@ -43,27 +46,45 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt // Pre-build per-partial configs with fixed spread/jitter and phase accumulators const configs = []; for (let p = 0; p < partials.length; ++p) { - const rep = partials[p].replicas != null ? partials[p].replicas : defaultReplicas; - const offsets = rep.offsets != null ? rep.offsets : [1.0]; - const decay_alpha = rep.decay_alpha != null ? rep.decay_alpha : 0.0; - const jitter = rep.jitter != null ? rep.jitter : 0.0; - const spread_above = rep.spread_above != null ? rep.spread_above : 0.0; - const spread_below = rep.spread_below != null ? rep.spread_below : 0.0; + const partial = partials[p]; + const fc = partial.freqCurve; + const ac = partial.ampCurve; - const replicaData = []; - for (let r = 0; r < offsets.length; ++r) { - // Fixed per-replica spread (frequency detuning) and initial phase (jitter) - const spread = spreadMult * randFloat(p * 67890 + r * 999, -spread_below, spread_above); - const initPhase = randFloat(p * 67890 + r, 0.0, 1.0) * (jitter * jitterMult) * 2.0 * Math.PI; - replicaData.push({ratio: offsets[r], spread, phase: initPhase}); - } + if ((partial.resonator && partial.resonator.enabled) || options.forceResonator) { + // --- Two-pole resonator mode --- + // Driven by band-limited noise scaled by amp curve. + // r controls pole radius (bandwidth): r→1 = narrow, r→0 = wide. + // gainNorm = sqrt(1 - r²) normalises steady-state output power to ~A. + const res = partial.resonator || {}; + const r = options.forceRGain ? Math.min(0.9999, Math.max(0, options.globalR)) + : (res.r != null ? Math.min(0.9999, Math.max(0, res.r)) : 0.995); + const gainComp = options.forceRGain ? options.globalGain + : (res.gainComp != null ? res.gainComp : 1.0); + const gainNorm = Math.sqrt(Math.max(0, 1.0 - r * r)); + configs.push({ + mode: 'resonator', + fc, ac, + r, gainComp, gainNorm, + y1: 0.0, y2: 0.0, + noiseSeed: ((p * 1664525 + 1013904223) & 0xFFFFFFFF) >>> 0 + }); + } else { + // --- Sinusoidal (replica) mode --- + const rep = partial.replicas != null ? partial.replicas : defaultReplicas; + const offsets = rep.offsets != null ? rep.offsets : [1.0]; + const decay_alpha = rep.decay_alpha != null ? rep.decay_alpha : 0.0; + const jitter = rep.jitter != null ? rep.jitter : 0.0; + const spread_above = rep.spread_above != null ? rep.spread_above : 0.0; + const spread_below = rep.spread_below != null ? rep.spread_below : 0.0; - configs.push({ - fc: partials[p].freqCurve, - ac: partials[p].ampCurve, - decay_alpha, - replicaData - }); + const replicaData = []; + for (let r = 0; r < offsets.length; ++r) { + const spread = spreadMult * randFloat(p * 67890 + r * 999, -spread_below, spread_above); + const initPhase = randFloat(p * 67890 + r, 0.0, 1.0) * (jitter * jitterMult) * 2.0 * Math.PI; + replicaData.push({ratio: offsets[r], spread, phase: initPhase}); + } + configs.push({ mode: 'sinusoid', fc, ac, decay_alpha, replicaData }); + } } for (let i = 0; i < numSamples; ++i) { @@ -71,32 +92,77 @@ function synthesizeMQ(partials, sampleRate, duration, integratePhase = true, opt let sample = 0.0; for (let p = 0; p < configs.length; ++p) { - const {fc, ac, decay_alpha, replicaData} = configs[p]; - if (t < fc.t0 || t > fc.t3) continue; + const cfg = configs[p]; + const {fc, ac} = cfg; - const f0 = evalBezier(fc, t); - const A0 = evalBezier(ac, t); + if (cfg.mode === 'resonator') { + if (t < fc.t0 || t > fc.t3) { cfg.y1 = 0.0; cfg.y2 = 0.0; continue; } - for (let r = 0; r < replicaData.length; ++r) { - const rep = replicaData[r]; - const f = f0 * rep.ratio * (1.0 + rep.spread); - const A = A0 * Math.exp(-decay_alpha * Math.abs(f - f0)); + const f0 = evalBezier(fc, t); + const A = evalBezier(ac, t); + const omega = 2.0 * Math.PI * f0 / sampleRate; + const b1 = 2.0 * cfg.r * Math.cos(omega); - let phase; - if (integratePhase) { - rep.phase += 2.0 * Math.PI * f / sampleRate; - phase = rep.phase; - } else { - phase = 2.0 * Math.PI * f * t + rep.phase; - } + // LCG noise excitation (deterministic per-partial) + cfg.noiseSeed = (Math.imul(1664525, cfg.noiseSeed) + 1013904223) >>> 0; + const noise = cfg.noiseSeed / 0x100000000 * 2.0 - 1.0; + + const x = A * cfg.gainNorm * noise; + const y = b1 * cfg.y1 - cfg.r * cfg.r * cfg.y2 + x; + cfg.y2 = cfg.y1; + cfg.y1 = y; + sample += y * cfg.gainComp; + + } else { + if (t < fc.t0 || t > fc.t3) continue; + + const f0 = evalBezier(fc, t); + const A0 = evalBezier(ac, t); + const {decay_alpha, replicaData} = cfg; + + for (let r = 0; r < replicaData.length; ++r) { + const rep = replicaData[r]; + const f = f0 * rep.ratio * (1.0 + rep.spread); + const A = A0 * Math.exp(-decay_alpha * Math.abs(f - f0)); - sample += A * Math.sin(phase); + let phase; + if (integratePhase) { + rep.phase += 2.0 * Math.PI * f / sampleRate; + phase = rep.phase; + } else { + phase = 2.0 * Math.PI * f * t + rep.phase; + } + + sample += A * Math.sin(phase); + } } } pcm[i] = sample; } + // Post-synthesis filters (applied before normalization) + // LP: y[n] = k1*x[n] + (1-k1)*y[n-1] — options.k1 in (0,1], omit to bypass + // HP: y[n] = k2*(y[n-1] + x[n] - x[n-1]) — options.k2 in (0,1], omit to bypass + if (options.k1 != null) { + const k1 = Math.max(0, Math.min(1, options.k1)); + let y = 0.0; + for (let i = 0; i < numSamples; ++i) { + y = k1 * pcm[i] + (1.0 - k1) * y; + pcm[i] = y; + } + } + if (options.k2 != null) { + const k2 = Math.max(0, Math.min(1, options.k2)); + let y = 0.0, xp = 0.0; + for (let i = 0; i < numSamples; ++i) { + const x = pcm[i]; + y = k2 * (y + x - xp); + xp = x; + pcm[i] = y; + } + } + // Normalize let maxAbs = 0; for (let i = 0; i < numSamples; ++i) maxAbs = Math.max(maxAbs, Math.abs(pcm[i])); diff --git a/tools/mq_editor/viewer.js b/tools/mq_editor/viewer.js index 7f6e862..76c57e2 100644 --- a/tools/mq_editor/viewer.js +++ b/tools/mq_editor/viewer.js @@ -48,6 +48,12 @@ class SpectrogramViewer { this.showSynthFFT = false; // Toggle: false=original, true=synth this.synthStftCache = null; + // Selection and editing + this.selectedPartial = -1; + this.dragState = null; // {pointIndex: 0-3} + this.onPartialSelect = null; // callback(index) + this.onRender = null; // callback() called after each render (for synced panels) + // Setup event handlers this.setupMouseHandlers(); @@ -153,6 +159,47 @@ class SpectrogramViewer { return this.stftCache.getMagnitudeDB(time, freq); } + selectPartial(index) { + this.selectedPartial = index; + this.render(); + if (this.onPartialSelect) this.onPartialSelect(index); + } + + // Hit-test bezier curves: returns index of nearest partial within threshold + hitTestPartial(x, y) { + const THRESH = 10; + let bestIdx = -1, bestDist = THRESH; + for (let p = 0; p < this.partials.length; ++p) { + const curve = this.partials[p].freqCurve; + if (!curve) continue; + for (let i = 0; i <= 50; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / 50; + if (t < this.t_view_min || t > this.t_view_max) continue; + const f = evalBezier(curve, t); + if (f < this.freqStart || f > this.freqEnd) continue; + const px = this.timeToX(t), py = this.freqToY(f); + const dist = Math.hypot(px - x, py - y); + if (dist < bestDist) { bestDist = dist; bestIdx = p; } + } + } + return bestIdx; + } + + // Hit-test control points of a specific partial's freqCurve + hitTestControlPoint(x, y, partial) { + const curve = partial.freqCurve; + if (!curve) return -1; + const THRESH = 8; + for (let i = 0; i < 4; ++i) { + const t = curve['t' + i], v = curve['v' + i]; + if (t < this.t_view_min || t > this.t_view_max) continue; + if (v < this.freqStart || v > this.freqEnd) continue; + const px = this.timeToX(t), py = this.freqToY(v); + if (Math.hypot(px - x, py - y) <= THRESH) return i; + } + return -1; + } + // --- Render --- render() { @@ -162,6 +209,178 @@ class SpectrogramViewer { this.drawAxes(); this.drawPlayhead(); this.renderSpectrum(); + if (this.onRender) this.onRender(); + } + + renderPartials() { + for (let p = 0; p < this.partials.length; ++p) { + if (p === this.selectedPartial) continue; // draw selected last (on top) + this._renderPartial(p, this.partials[p], false); + } + if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { + this._renderPartial(this.selectedPartial, this.partials[this.selectedPartial], true); + } + this.ctx.globalAlpha = 1.0; + this.ctx.shadowBlur = 0; + } + + _renderPartial(p, partial, isSelected) { + const {ctx} = this; + const color = this.partialColor(p); + let alpha = isSelected ? 1.0 : (p < this.keepCount ? 1.0 : 0.12); + if (partial.muted && !isSelected) alpha = 0.15; + ctx.globalAlpha = alpha; + + // Raw trajectory + ctx.strokeStyle = color + '44'; + ctx.lineWidth = 1; + ctx.beginPath(); + let started = false; + for (let i = 0; i < partial.times.length; ++i) { + const t = partial.times[i]; + const f = partial.freqs[i]; + if (t < this.t_view_min || t > this.t_view_max) continue; + if (f < this.freqStart || f > this.freqEnd) continue; + const x = this.timeToX(t); + const y = this.freqToY(f); + if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); + } + if (started) ctx.stroke(); + + // Spread band (selected only) + if (isSelected && partial.freqCurve) { + this._renderSpreadBand(partial, color); + } + + // Bezier curve + if (partial.freqCurve) { + const curve = partial.freqCurve; + if (isSelected) { ctx.shadowColor = color; ctx.shadowBlur = 8; } + ctx.strokeStyle = color; + ctx.lineWidth = isSelected ? 3 : 2; + ctx.beginPath(); + started = false; + for (let i = 0; i <= 50; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / 50; + const freq = evalBezier(curve, t); + if (t < this.t_view_min || t > this.t_view_max) continue; + if (freq < this.freqStart || freq > this.freqEnd) continue; + const x = this.timeToX(t); + const y = this.freqToY(freq); + if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); + } + if (started) ctx.stroke(); + if (isSelected) ctx.shadowBlur = 0; + + ctx.fillStyle = color; + const cpR = isSelected ? 6 : 4; + this.drawControlPoint(curve.t0, curve.v0, cpR); + this.drawControlPoint(curve.t1, curve.v1, cpR); + this.drawControlPoint(curve.t2, curve.v2, cpR); + this.drawControlPoint(curve.t3, curve.v3, cpR); + } + } + + _renderSpreadBand(partial, color) { + const {ctx} = this; + const curve = partial.freqCurve; + const rep = partial.replicas || {}; + const sa = rep.spread_above != null ? rep.spread_above : 0.02; + const sb = rep.spread_below != null ? rep.spread_below : 0.02; + + const STEPS = 60; + const upper = [], lower = []; + for (let i = 0; i <= STEPS; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; + if (t < this.t_view_min - 0.01 || t > this.t_view_max + 0.01) continue; + const f = evalBezier(curve, t); + upper.push([this.timeToX(t), this.freqToY(f * (1 + sa))]); + lower.push([this.timeToX(t), this.freqToY(f * (1 - sb))]); + } + if (upper.length < 2) return; + + const savedAlpha = ctx.globalAlpha; + + // Outer soft fill + ctx.beginPath(); + ctx.moveTo(upper[0][0], upper[0][1]); + for (let i = 1; i < upper.length; ++i) ctx.lineTo(upper[i][0], upper[i][1]); + for (let i = lower.length - 1; i >= 0; --i) ctx.lineTo(lower[i][0], lower[i][1]); + ctx.closePath(); + ctx.fillStyle = color; + ctx.globalAlpha = 0.13; + ctx.fill(); + + // Dashed boundary lines + ctx.globalAlpha = 0.75; + ctx.strokeStyle = color; + ctx.lineWidth = 1.5; + ctx.setLineDash([4, 3]); + ctx.beginPath(); + ctx.moveTo(upper[0][0], upper[0][1]); + for (let i = 1; i < upper.length; ++i) ctx.lineTo(upper[i][0], upper[i][1]); + ctx.stroke(); + ctx.beginPath(); + ctx.moveTo(lower[0][0], lower[0][1]); + for (let i = 1; i < lower.length; ++i) ctx.lineTo(lower[i][0], lower[i][1]); + ctx.stroke(); + ctx.setLineDash([]); + + // 50% drop-off reference lines (dotted, dimmer) + const p5upper = [], p5lower = []; + for (let i = 0; i <= STEPS; ++i) { + const t = curve.t0 + (curve.t3 - curve.t0) * i / STEPS; + if (t < this.t_view_min - 0.01 || t > this.t_view_max + 0.01) continue; + const f = evalBezier(curve, t); + p5upper.push([this.timeToX(t), this.freqToY(f * 1.50)]); + p5lower.push([this.timeToX(t), this.freqToY(f * 0.50)]); + } + if (p5upper.length >= 2) { + ctx.globalAlpha = 0.55; + ctx.strokeStyle = color; + ctx.lineWidth = 1; + ctx.setLineDash([1, 5]); + ctx.beginPath(); + ctx.moveTo(p5upper[0][0], p5upper[0][1]); + for (let i = 1; i < p5upper.length; ++i) ctx.lineTo(p5upper[i][0], p5upper[i][1]); + ctx.stroke(); + ctx.beginPath(); + ctx.moveTo(p5lower[0][0], p5lower[0][1]); + for (let i = 1; i < p5lower.length; ++i) ctx.lineTo(p5lower[i][0], p5lower[i][1]); + ctx.stroke(); + ctx.setLineDash([]); + } + + ctx.globalAlpha = savedAlpha; + } + + renderPeaks() { + const {ctx, frames} = this; + if (!frames || frames.length === 0) return; + + ctx.fillStyle = '#fff'; + for (const frame of frames) { + const t = frame.time; + if (t < this.t_view_min || t > this.t_view_max) continue; + const x = this.timeToX(t); + for (const peak of frame.peaks) { + if (peak.freq < this.freqStart || peak.freq > this.freqEnd) continue; + ctx.fillRect(x - 1, this.freqToY(peak.freq) - 1, 3, 3); + } + } + } + + drawControlPoint(t, v, radius = 4) { + if (t < this.t_view_min || t > this.t_view_max) return; + if (v < this.freqStart || v > this.freqEnd) return; + const x = this.timeToX(t); + const y = this.freqToY(v); + this.ctx.beginPath(); + this.ctx.arc(x, y, radius, 0, 2 * Math.PI); + this.ctx.fill(); + this.ctx.strokeStyle = '#fff'; + this.ctx.lineWidth = 1; + this.ctx.stroke(); } drawMouseCursor(x) { @@ -243,118 +462,6 @@ class SpectrogramViewer { } } - renderPartials() { - const {ctx, partials} = this; - - for (let p = 0; p < partials.length; ++p) { - const partial = partials[p]; - const color = this.partialColor(p); - ctx.globalAlpha = p < this.keepCount ? 1.0 : 0.5; - - // Raw trajectory - ctx.strokeStyle = color + '44'; - ctx.lineWidth = 1; - ctx.beginPath(); - let started = false; - for (let i = 0; i < partial.times.length; ++i) { - const t = partial.times[i]; - const f = partial.freqs[i]; - if (t < this.t_view_min || t > this.t_view_max) continue; - if (f < this.freqStart || f > this.freqEnd) continue; - const x = this.timeToX(t); - const y = this.freqToY(f); - if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); - } - if (started) ctx.stroke(); - - // Bezier curve - if (partial.freqCurve) { - ctx.strokeStyle = color; - ctx.lineWidth = 2; - ctx.beginPath(); - const curve = partial.freqCurve; - started = false; - for (let i = 0; i <= 50; ++i) { - const t = curve.t0 + (curve.t3 - curve.t0) * i / 50; - const freq = evalBezier(curve, t); - if (t < this.t_view_min || t > this.t_view_max) continue; - if (freq < this.freqStart || freq > this.freqEnd) continue; - const x = this.timeToX(t); - const y = this.freqToY(freq); - if (!started) { ctx.moveTo(x, y); started = true; } else ctx.lineTo(x, y); - } - if (started) ctx.stroke(); - - ctx.fillStyle = color; - this.drawControlPoint(curve.t0, curve.v0); - this.drawControlPoint(curve.t1, curve.v1); - this.drawControlPoint(curve.t2, curve.v2); - this.drawControlPoint(curve.t3, curve.v3); - } - } - - ctx.globalAlpha = 1.0; - } - - renderPeaks() { - const {ctx, frames} = this; - if (!frames || frames.length === 0) return; - - ctx.fillStyle = '#fff'; - for (const frame of frames) { - const t = frame.time; - if (t < this.t_view_min || t > this.t_view_max) continue; - const x = this.timeToX(t); - for (const peak of frame.peaks) { - if (peak.freq < this.freqStart || peak.freq > this.freqEnd) continue; - ctx.fillRect(x - 1, this.freqToY(peak.freq) - 1, 3, 3); - } - } - } - - drawControlPoint(t, v) { - if (t < this.t_view_min || t > this.t_view_max) return; - if (v < this.freqStart || v > this.freqEnd) return; - const x = this.timeToX(t); - const y = this.freqToY(v); - this.ctx.beginPath(); - this.ctx.arc(x, y, 4, 0, 2 * Math.PI); - this.ctx.fill(); - this.ctx.strokeStyle = '#fff'; - this.ctx.lineWidth = 1; - this.ctx.stroke(); - } - - drawAxes() { - const {ctx, canvas} = this; - const width = canvas.width; - const height = canvas.height; - - ctx.strokeStyle = '#666'; - ctx.fillStyle = '#aaa'; - ctx.font = '11px monospace'; - ctx.lineWidth = 1; - - // Time axis - const timeDuration = this.t_view_max - this.t_view_min; - const timeStep = this.getAxisStep(timeDuration); - let t = Math.ceil(this.t_view_min / timeStep) * timeStep; - while (t <= this.t_view_max) { - const x = this.timeToX(t); - ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, height); ctx.stroke(); - ctx.fillText(t.toFixed(2) + 's', x + 2, height - 4); - t += timeStep; - } - - // Frequency axis (log-spaced ticks) - for (const f of [20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 16000]) { - if (f < this.freqStart || f > this.freqEnd) continue; - const y = this.freqToY(f); - ctx.beginPath(); ctx.moveTo(0, y); ctx.lineTo(width, y); ctx.stroke(); - ctx.fillText((f >= 1000 ? (f/1000).toFixed(0) + 'k' : f.toFixed(0)) + 'Hz', 2, y - 2); - } - } - renderSpectrum() { if (!this.spectrumCtx || !this.stftCache) return; @@ -386,11 +493,11 @@ class SpectrogramViewer { const bStart = Math.max(0, Math.floor(fStart / binWidth)); const bEnd = Math.min(numBins - 1, Math.ceil(fEnd / binWidth)); - let sum = 0, count = 0; - for (let b = bStart; b <= bEnd; ++b) { sum += squaredAmp[b]; ++count; } - if (count === 0) continue; + let maxSq = 0; + for (let b = bStart; b <= bEnd; ++b) { if (squaredAmp[b] > maxSq) maxSq = squaredAmp[b]; } + if (bStart > bEnd) continue; - const magDB = 10 * Math.log10(Math.max(sum / count, 1e-20)); + const magDB = 10 * Math.log10(Math.max(maxSq, 1e-20)); const barHeight = Math.round(this.normalizeDB(magDB, cache.maxDB) * height); if (barHeight === 0) continue; @@ -464,11 +571,44 @@ class SpectrogramViewer { setupMouseHandlers() { const {canvas, tooltip} = this; + canvas.addEventListener('mousedown', (e) => { + const rect = canvas.getBoundingClientRect(); + const x = e.clientX - rect.left; + const y = e.clientY - rect.top; + + // Check control point drag on selected partial + if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { + const ptIdx = this.hitTestControlPoint(x, y, this.partials[this.selectedPartial]); + if (ptIdx >= 0) { + this.dragState = { pointIndex: ptIdx }; + canvas.style.cursor = 'grabbing'; + e.preventDefault(); + return; + } + } + + // Otherwise: select partial by click + const idx = this.hitTestPartial(x, y); + this.selectPartial(idx); + }); + canvas.addEventListener('mousemove', (e) => { const rect = canvas.getBoundingClientRect(); const x = e.clientX - rect.left; const y = e.clientY - rect.top; + if (this.dragState) { + const t = Math.max(0, Math.min(this.t_max, this.canvasToTime(x))); + const v = Math.max(this.freqStart, Math.min(this.freqEnd, this.canvasToFreq(y))); + const partial = this.partials[this.selectedPartial]; + const i = this.dragState.pointIndex; + partial.freqCurve['t' + i] = t; + partial.freqCurve['v' + i] = v; + this.render(); + e.preventDefault(); + return; + } + this.mouseX = x; this.drawMouseCursor(x); @@ -481,6 +621,14 @@ class SpectrogramViewer { this.renderSpectrum(); } + // Cursor hint for control points + if (this.selectedPartial >= 0 && this.selectedPartial < this.partials.length) { + const ptIdx = this.hitTestControlPoint(x, y, this.partials[this.selectedPartial]); + canvas.style.cursor = ptIdx >= 0 ? 'grab' : 'crosshair'; + } else { + canvas.style.cursor = 'crosshair'; + } + tooltip.style.left = (e.clientX + 10) + 'px'; tooltip.style.top = (e.clientY + 10) + 'px'; tooltip.style.display = 'block'; @@ -493,6 +641,14 @@ class SpectrogramViewer { tooltip.style.display = 'none'; }); + canvas.addEventListener('mouseup', () => { + if (this.dragState) { + this.dragState = null; + canvas.style.cursor = 'crosshair'; + if (this.onPartialSelect) this.onPartialSelect(this.selectedPartial); + } + }); + canvas.addEventListener('wheel', (e) => { e.preventDefault(); const delta = e.deltaY !== 0 ? e.deltaY : e.deltaX; @@ -529,6 +685,36 @@ class SpectrogramViewer { for (const step of steps) { if (step >= targetStep) return step; } return steps[steps.length - 1]; } + + drawAxes() { + const {ctx, canvas} = this; + const width = canvas.width; + const height = canvas.height; + + ctx.strokeStyle = '#666'; + ctx.fillStyle = '#aaa'; + ctx.font = '11px monospace'; + ctx.lineWidth = 1; + + // Time axis + const timeDuration = this.t_view_max - this.t_view_min; + const timeStep = this.getAxisStep(timeDuration); + let t = Math.ceil(this.t_view_min / timeStep) * timeStep; + while (t <= this.t_view_max) { + const x = this.timeToX(t); + ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, height); ctx.stroke(); + ctx.fillText(t.toFixed(2) + 's', x + 2, height - 4); + t += timeStep; + } + + // Frequency axis (log-spaced ticks) + for (const f of [20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 16000]) { + if (f < this.freqStart || f > this.freqEnd) continue; + const y = this.freqToY(f); + ctx.beginPath(); ctx.moveTo(0, y); ctx.lineTo(width, y); ctx.stroke(); + ctx.fillText((f >= 1000 ? (f/1000).toFixed(0) + 'k' : f.toFixed(0)) + 'Hz', 2, y - 2); + } + } } // Bezier evaluation (shared utility) |