summaryrefslogtreecommitdiff
path: root/tools
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-06 13:36:19 +0100
committerskal <pascal.massimino@gmail.com>2026-02-06 13:36:19 +0100
commita0888c1afa8bf178b7a57d4e80373ad867a3474a (patch)
treeec7c4526d41e08c629c2e88115071b1614dbae34 /tools
parent3002553be5bf880ead27fb3e415bc97d484b43eb (diff)
feat(spectral_editor): Complete Phase 2 milestone - Full-featured web editor
MILESTONE: Spectral Brush Editor Phase 2 Complete (February 6, 2026) Phase 2 delivers a production-ready web-based editor for creating procedural audio by tracing spectrograms with parametric Bezier curves. This tool enables replacing 5KB .spec binary assets with ~100 bytes of C++ code (50-100× compression). Core Features Implemented: ======================== Audio I/O: - Load .wav and .spec files as reference spectrograms - Real-time audio preview (procedural vs original) - Live volume control with GainNode (updates during playback) - Export to procedural_params.txt (human-readable, re-editable format) - Generate C++ code (copy-paste ready for demo integration) Curve Editing: - Multi-curve support with individual colors and volumes - Bezier curve control points (frame, frequency, amplitude) - Drag-and-drop control point editing - Per-curve volume control (0-100%) - Right-click to delete control points - Curves only render within control point range (no spill) Profile System (All 3 types implemented): - Gaussian: exp(-(dist² / σ²)) - smooth harmonic falloff - Decaying Sinusoid: exp(-decay × dist) × cos(ω × dist) - metallic resonance - Noise: noise × exp(-(dist² / decay²)) - textured grit with decay envelope Visualization: - Log-scale frequency axis (20 Hz to 16 kHz) for better bass visibility - Logarithmic dB-scale intensity mapping (-60 dB to +40 dB range) - Reference opacity slider (0-100%) for mixing original/procedural views - Playhead indicator (red dashed line) during playback - Mouse crosshair with tooltip (frame number, frequency) - Control point info panel (frame, frequency, amplitude) Real-time Spectrum Viewer (NEW): - Always-visible bottom-right overlay (200×100px) - Shows frequency spectrum for frame under mouse (hover mode) - Shows current playback frame spectrum (playback mode) - Dual display: Reference (green) + Procedural (red) overlaid - dB-scale bar heights for accurate visualization - Frame number label (red during playback, gray when hovering) Rendering Architecture: - Destination-to-source pixel mapping (prevents gaps in log-scale) - Offscreen canvas compositing for proper alpha blending - Alpha channel for procedural intensity (pure colors, not dimmed) - Steeper dB falloff for procedural curves (-40 dB floor vs -60 dB reference) UI/UX: - Undo/Redo system (50-action history) - Keyboard shortcuts (1/2/Space for playback, Ctrl+Z/Ctrl+Shift+Z, Delete, Esc) - File load confirmation (warns about unsaved curves) - Automatic curve reset on new file load Technical Details: - DCT/IDCT implementation (JavaScript port matching C++ runtime) - Overlap-add synthesis with Hanning window - Web Audio API integration (32 kHz sample rate) - Zero external dependencies (pure HTML/CSS/JS) Files Modified: - tools/spectral_editor/script.js (~1730 lines, main implementation) - tools/spectral_editor/index.html (UI structure, spectrum viewer) - tools/spectral_editor/style.css (VSCode dark theme styling) - tools/spectral_editor/README.md (updated features, roadmap) Phase 3 TODO (Next): =================== - Effect combination system (noise + Gaussian modulation, layer compositing) - Improved C++ code testing (validation, edge cases) - Better frequency scale (mu-law or perceptual scale, less bass-heavy) - Pre-defined shape library (kick, snare, hi-hat templates) - Load procedural_params.txt back into editor (re-editing) - FFT-based DCT optimization (O(N log N) vs O(N²)) Integration: - Generate C++ code → Copy to src/audio/procedural_samples.cc - Add PROC() entry to assets/final/demo_assets.txt - Rebuild demo → Use AssetId::SOUND_PROC handoff(Claude): Phase 2 complete. Next: FFT implementation task for performance optimization. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'tools')
-rw-r--r--tools/spectral_editor/README.md73
-rw-r--r--tools/spectral_editor/index.html39
-rw-r--r--tools/spectral_editor/script.js638
-rw-r--r--tools/spectral_editor/style.css49
4 files changed, 737 insertions, 62 deletions
diff --git a/tools/spectral_editor/README.md b/tools/spectral_editor/README.md
index 221acb8..6bb3681 100644
--- a/tools/spectral_editor/README.md
+++ b/tools/spectral_editor/README.md
@@ -19,11 +19,25 @@ Replace large `.spec` binary assets with tiny procedural C++ code:
- Undo/Redo support (50-action history)
- Export to `procedural_params.txt` (re-editable)
- Generate C++ code (copy-paste ready)
+- **Live volume control** during playback
+- **Reference opacity slider** for mixing original/procedural views
+- **Per-curve volume control** for fine-tuning
### Profiles
-- **Gaussian:** Smooth harmonic falloff
-- **Decaying Sinusoid:** Resonant/metallic texture (coming soon)
-- **Noise:** Random texture/grit (coming soon)
+- **Gaussian:** Smooth harmonic falloff (fully implemented)
+- **Decaying Sinusoid:** Resonant/metallic texture (implemented)
+- **Noise:** Random texture/grit with decay envelope (implemented)
+
+### Visualization
+- **Log-scale frequency axis** (20 Hz to 16 kHz) for better bass visibility
+- **Logarithmic dB-scale intensity** for proper dynamic range display
+- **Playhead indicator** showing current playback position
+- **Mouse crosshair with tooltip** displaying frame and frequency
+- **Real-time spectrum viewer** (bottom-right):
+ - Shows frequency spectrum for frame under mouse (hover mode)
+ - Shows frequency spectrum for current playback frame (playback mode)
+ - Dual display: Reference (green) and Procedural (red) overlaid
+ - Always visible for instant feedback
## Quick Start
@@ -164,19 +178,52 @@ A spectral brush consists of:
3. Use overlap-add with Hanning window
4. Play via Web Audio API (32 kHz sample rate)
-## Limitations
+## Development Status
+
+### Phase 1: C++ Runtime ✅ COMPLETE
+- Spectral brush primitive (Bezier + profiles)
+- C++ implementation in `src/audio/spectral_brush.{h,cc}`
+- Integration with asset system
+
+### Phase 2: Web Editor ✅ COMPLETE (Milestone: February 6, 2026)
+- Full-featured web-based editor
+- Real-time audio preview and visualization
+- Log-scale frequency display with dB-scale intensity
+- All three profile types (Gaussian, Decaying Sinusoid, Noise)
+- Live volume control and reference opacity mixing
+- Real-time dual-spectrum viewer (reference + procedural)
+- Export to `procedural_params.txt` and C++ code
+
+### Phase 3: Advanced Features (TODO)
+
+**High Priority:**
+- **Effect Combination System:** How to combine effects (e.g., noise + Gaussian modulation)?
+ - Layer-based compositing (add/multiply/subtract)
+ - Profile modulation (noise modulated by Gaussian envelope)
+ - Multi-pass rendering pipeline
+
+- **Improved C++ Code Testing:**
+ - Verify generated code compiles correctly
+ - Test parameter ranges and edge cases
+ - Add validation warnings in editor
+
+- **Better Frequency Scale:**
+ - Current log-scale is too bass-heavy
+ - Investigate mu-law or similar perceptual scales
+ - Allow user-configurable frequency mapping
-### Phase 1 (Current)
-- Only Bezier + Gaussian profile implemented
-- Linear interpolation between control points
-- Single-layer spectrogram (no compositing yet)
+- **Pre-defined Shape Library:**
+ - Template curves for common sounds (kick, snare, hi-hat, bass)
+ - One-click insertion with adjustable parameters
+ - Save/load custom shape presets
-### Future Enhancements
+**Future Enhancements:**
- Cubic Bezier interpolation (smoother curves)
-- Decaying sinusoid and noise profiles
-- Composite profiles (add/subtract/multiply)
-- Multi-dimensional Bezier (vary decay, oscillation, etc.)
-- Frequency snapping (snap to musical notes)
+- Multi-dimensional Bezier (vary decay, oscillation over time)
+- Frequency snapping (snap to musical notes/scales)
+- Load `procedural_params.txt` back into editor (re-editing)
+- Frame cache optimization for faster rendering
+- FFT-based DCT optimization (O(N log N) vs O(N²))
## Troubleshooting
diff --git a/tools/spectral_editor/index.html b/tools/spectral_editor/index.html
index 52f1d8f..a9391dd 100644
--- a/tools/spectral_editor/index.html
+++ b/tools/spectral_editor/index.html
@@ -27,6 +27,10 @@
<p>Load a .wav or .spec file to begin</p>
<p class="hint">Click "Load .wav/.spec" button or press Ctrl+O</p>
</div>
+ <!-- Mini spectrum viewer (bottom-right overlay) -->
+ <div id="spectrumViewer" class="spectrum-viewer">
+ <canvas id="spectrumCanvas" width="200" height="100"></canvas>
+ </div>
</div>
<!-- Toolbar (right side, 20% width) -->
@@ -39,6 +43,22 @@
<span class="icon">×</span> Delete
</button>
<div id="curveList" class="curve-list"></div>
+
+ <h3>Selected Point</h3>
+ <div id="pointInfo" class="point-info">
+ <div class="info-row">
+ <span class="info-label">Frame:</span>
+ <span id="pointFrame" class="info-value">-</span>
+ </div>
+ <div class="info-row">
+ <span class="info-label">Frequency:</span>
+ <span id="pointFreq" class="info-value">-</span>
+ </div>
+ <div class="info-row">
+ <span class="info-label">Amplitude:</span>
+ <span id="pointAmp" class="info-value">-</span>
+ </div>
+ </div>
</div>
</div>
@@ -56,9 +76,20 @@
<label for="sigmaSlider" id="sigmaLabel">Sigma:</label>
<input type="range" id="sigmaSlider" class="slider" min="1" max="100" value="30" step="0.1">
<input type="number" id="sigmaValue" class="number-input" min="1" max="100" value="30" step="0.1">
+
+ <label for="curveVolumeSlider">Curve Vol:</label>
+ <input type="range" id="curveVolumeSlider" class="slider" min="0" max="100" value="100" step="1">
+ <input type="number" id="curveVolumeValue" class="number-input" min="0" max="100" value="100" step="1">
+ </div>
+
+ <!-- Middle section: Display controls -->
+ <div class="control-section">
+ <label for="refOpacitySlider">Ref Opacity:</label>
+ <input type="range" id="refOpacitySlider" class="slider" min="0" max="100" value="50" step="1">
+ <input type="number" id="refOpacityValue" class="number-input" min="0" max="100" value="50" step="1">
</div>
- <!-- Middle section: Curve selection -->
+ <!-- Curve selection -->
<div class="control-section">
<label for="curveSelect">Active Curve:</label>
<select id="curveSelect" class="select-input">
@@ -68,6 +99,12 @@
<!-- Right section: Playback controls -->
<div class="control-section playback-controls">
+ <label for="volumeSlider">Volume:</label>
+ <input type="range" id="volumeSlider" class="slider" min="0" max="100" value="100" step="1">
+ <input type="number" id="volumeValue" class="number-input" min="0" max="100" value="100" step="1">
+ </div>
+
+ <div class="control-section playback-controls">
<button id="playProceduralBtn" class="btn-playback" title="Play procedural sound (Key 1)">
<span class="icon">▶</span> <kbd>1</kbd> Procedural
</button>
diff --git a/tools/spectral_editor/script.js b/tools/spectral_editor/script.js
index 744ea97..48b0661 100644
--- a/tools/spectral_editor/script.js
+++ b/tools/spectral_editor/script.js
@@ -8,6 +8,11 @@
const SAMPLE_RATE = 32000;
const DCT_SIZE = 512;
+// Frequency range for log-scale display
+const FREQ_MIN = 20.0; // 20 Hz (lowest audible bass)
+const FREQ_MAX = 16000.0; // 16 kHz (Nyquist for 32kHz sample rate)
+const USE_LOG_SCALE = true; // Enable logarithmic frequency axis
+
const state = {
// Reference audio data
referenceSpectrogram: null, // Float32Array or null
@@ -30,6 +35,20 @@ const state = {
audioContext: null,
isPlaying: false,
currentSource: null,
+ currentGainNode: null, // Keep reference to gain node for live volume updates
+ playbackVolume: 1.0, // Global volume for playback (0.0-1.0, increased from 0.7)
+ referenceOpacity: 0.5, // Opacity for reference spectrogram (0.0-1.0, increased from 0.3)
+
+ // Playhead indicator
+ playbackStartTime: null,
+ playbackDuration: 0,
+ playbackCurrentFrame: 0,
+
+ // Mouse hover state
+ mouseX: -1,
+ mouseY: -1,
+ mouseFrame: 0,
+ mouseFreq: 0,
// Undo/Redo
history: [],
@@ -71,6 +90,10 @@ function initCanvas() {
canvas.addEventListener('mousemove', onCanvasMouseMove);
canvas.addEventListener('mouseup', onCanvasMouseUp);
canvas.addEventListener('contextmenu', onCanvasRightClick);
+
+ // Mouse hover handlers (for crosshair)
+ canvas.addEventListener('mousemove', onCanvasHover);
+ canvas.addEventListener('mouseleave', onCanvasLeave);
}
function initUI() {
@@ -90,8 +113,16 @@ function initUI() {
document.getElementById('profileType').addEventListener('change', onProfileChanged);
document.getElementById('sigmaSlider').addEventListener('input', onSigmaChanged);
document.getElementById('sigmaValue').addEventListener('input', onSigmaValueChanged);
+ document.getElementById('curveVolumeSlider').addEventListener('input', onCurveVolumeChanged);
+ document.getElementById('curveVolumeValue').addEventListener('input', onCurveVolumeValueChanged);
+
+ // Display controls
+ document.getElementById('refOpacitySlider').addEventListener('input', onRefOpacityChanged);
+ document.getElementById('refOpacityValue').addEventListener('input', onRefOpacityValueChanged);
// Playback controls
+ document.getElementById('volumeSlider').addEventListener('input', onVolumeChanged);
+ document.getElementById('volumeValue').addEventListener('input', onVolumeValueChanged);
document.getElementById('playProceduralBtn').addEventListener('click', () => playAudio('procedural'));
document.getElementById('playOriginalBtn').addEventListener('click', () => playAudio('original'));
document.getElementById('stopBtn').addEventListener('click', stopAudio);
@@ -194,6 +225,27 @@ function onFileSelected(e) {
const file = e.target.files[0];
if (!file) return;
+ // Check if there are unsaved curves
+ if (state.curves.length > 0) {
+ const confirmLoad = confirm(
+ 'You have unsaved curves. Loading a new file will reset all curves.\n\n' +
+ 'Do you want to save your work first?\n\n' +
+ 'Click "OK" to save, or "Cancel" to discard and continue loading.'
+ );
+
+ if (confirmLoad) {
+ // User wants to save first
+ saveProceduralParams();
+ // After saving, ask again if they want to proceed
+ const proceedLoad = confirm('File saved. Proceed with loading new file?');
+ if (!proceedLoad) {
+ // User changed their mind, reset file input
+ e.target.value = '';
+ return;
+ }
+ }
+ }
+
const fileName = file.name;
const fileExt = fileName.split('.').pop().toLowerCase();
@@ -347,10 +399,26 @@ function onReferenceLoaded(fileName) {
document.getElementById('canvasOverlay').classList.add('hidden');
document.getElementById('playOriginalBtn').disabled = false;
+ // Reset curves when loading new file
+ state.curves = [];
+ state.nextCurveId = 0;
+ state.selectedCurveId = null;
+ state.selectedControlPointIdx = null;
+
+ // Clear history
+ state.history = [];
+ state.historyIndex = -1;
+
+ // Reset mouse to frame 0
+ state.mouseFrame = 0;
+
// Adjust zoom to fit
state.pixelsPerFrame = Math.max(1.0, state.canvasWidth / state.referenceNumFrames);
+ updateCurveUI();
+ updateUndoRedoButtons();
render();
+ drawSpectrumViewer(); // Show initial spectrum
}
// ============================================================================
@@ -379,7 +447,8 @@ function addCurve() {
param1: 30.0, // sigma
param2: 0.0
},
- color: color
+ color: color,
+ volume: 1.0 // Per-curve volume multiplier (0.0-1.0)
};
state.curves.push(curve);
@@ -443,6 +512,7 @@ function updateCurveUI() {
state.selectedCurveId = curve.id;
state.selectedControlPointIdx = null;
updateCurveUI();
+ updatePointInfo();
render();
});
curveList.appendChild(div);
@@ -477,8 +547,43 @@ function updateCurveUI() {
document.getElementById('profileType').value = curve.profile.type;
document.getElementById('sigmaSlider').value = curve.profile.param1;
document.getElementById('sigmaValue').value = curve.profile.param1;
+
+ // Update curve volume slider
+ const volumePercent = Math.round(curve.volume * 100);
+ document.getElementById('curveVolumeSlider').value = volumePercent;
+ document.getElementById('curveVolumeValue').value = volumePercent;
}
}
+
+ // Update point info panel
+ updatePointInfo();
+}
+
+function updatePointInfo() {
+ const frameEl = document.getElementById('pointFrame');
+ const freqEl = document.getElementById('pointFreq');
+ const ampEl = document.getElementById('pointAmp');
+
+ if (state.selectedCurveId === null || state.selectedControlPointIdx === null) {
+ // No point selected
+ frameEl.textContent = '-';
+ freqEl.textContent = '-';
+ ampEl.textContent = '-';
+ return;
+ }
+
+ const curve = state.curves.find(c => c.id === state.selectedCurveId);
+ if (!curve || state.selectedControlPointIdx >= curve.controlPoints.length) {
+ frameEl.textContent = '-';
+ freqEl.textContent = '-';
+ ampEl.textContent = '-';
+ return;
+ }
+
+ const point = curve.controlPoints[state.selectedControlPointIdx];
+ frameEl.textContent = point.frame.toFixed(0);
+ freqEl.textContent = point.freqHz.toFixed(1) + ' Hz';
+ ampEl.textContent = point.amplitude.toFixed(3);
}
// ============================================================================
@@ -500,7 +605,7 @@ function onProfileChanged(e) {
} else if (curve.profile.type === 'decaying_sinusoid') {
label.textContent = 'Decay:';
} else if (curve.profile.type === 'noise') {
- label.textContent = 'Amplitude:';
+ label.textContent = 'Decay:'; // Changed from 'Amplitude:' to 'Decay:'
}
saveHistoryState('Change profile');
@@ -531,6 +636,62 @@ function onSigmaValueChanged(e) {
render();
}
+function onRefOpacityChanged(e) {
+ state.referenceOpacity = parseFloat(e.target.value) / 100.0; // Convert 0-100 to 0.0-1.0
+ document.getElementById('refOpacityValue').value = e.target.value;
+ render();
+}
+
+function onRefOpacityValueChanged(e) {
+ state.referenceOpacity = parseFloat(e.target.value) / 100.0;
+ document.getElementById('refOpacitySlider').value = e.target.value;
+ render();
+}
+
+function onVolumeChanged(e) {
+ state.playbackVolume = parseFloat(e.target.value) / 100.0; // Convert 0-100 to 0.0-1.0
+ document.getElementById('volumeValue').value = e.target.value;
+
+ // Update gain node if audio is currently playing
+ if (state.currentGainNode) {
+ state.currentGainNode.gain.value = state.playbackVolume;
+ }
+}
+
+function onVolumeValueChanged(e) {
+ state.playbackVolume = parseFloat(e.target.value) / 100.0;
+ document.getElementById('volumeSlider').value = e.target.value;
+
+ // Update gain node if audio is currently playing
+ if (state.currentGainNode) {
+ state.currentGainNode.gain.value = state.playbackVolume;
+ }
+}
+
+function onCurveVolumeChanged(e) {
+ if (state.selectedCurveId === null) return;
+
+ const curve = state.curves.find(c => c.id === state.selectedCurveId);
+ if (!curve) return;
+
+ curve.volume = parseFloat(e.target.value) / 100.0; // Convert 0-100 to 0.0-1.0
+ document.getElementById('curveVolumeValue').value = e.target.value;
+
+ render();
+}
+
+function onCurveVolumeValueChanged(e) {
+ if (state.selectedCurveId === null) return;
+
+ const curve = state.curves.find(c => c.id === state.selectedCurveId);
+ if (!curve) return;
+
+ curve.volume = parseFloat(e.target.value) / 100.0;
+ document.getElementById('curveVolumeSlider').value = e.target.value;
+
+ render();
+}
+
// ============================================================================
// Canvas Interaction
// ============================================================================
@@ -555,6 +716,7 @@ function onCanvasMouseDown(e) {
dragStartX = x;
dragStartY = y;
updateCurveUI();
+ updatePointInfo();
render();
} else if (state.selectedCurveId !== null) {
// Place new control point
@@ -568,6 +730,7 @@ function onCanvasMouseDown(e) {
saveHistoryState('Add control point');
updateCurveUI();
+ updatePointInfo();
render();
}
}
@@ -596,6 +759,9 @@ function onCanvasMouseMove(e) {
// Re-sort by frame
curve.controlPoints.sort((a, b) => a.frame - b.frame);
+ // Update point info panel in real-time
+ updatePointInfo();
+
render();
}
@@ -666,6 +832,30 @@ function deselectAll() {
state.selectedCurveId = null;
state.selectedControlPointIdx = null;
updateCurveUI();
+ updatePointInfo();
+ render();
+}
+
+function onCanvasHover(e) {
+ const rect = e.target.getBoundingClientRect();
+ state.mouseX = e.clientX - rect.left;
+ state.mouseY = e.clientY - rect.top;
+
+ // Convert to spectrogram coordinates
+ const coords = screenToSpectrogram(state.mouseX, state.mouseY);
+ state.mouseFrame = Math.floor(coords.frame);
+ state.mouseFreq = coords.freqHz;
+
+ // Only redraw if not dragging (avoid slowdown during drag)
+ if (!isDragging) {
+ render();
+ drawSpectrumViewer(); // Update spectrum viewer with frame under mouse
+ }
+}
+
+function onCanvasLeave(e) {
+ state.mouseX = -1;
+ state.mouseY = -1;
render();
}
@@ -675,23 +865,49 @@ function deselectAll() {
function screenToSpectrogram(screenX, screenY) {
const frame = Math.round(screenX / state.pixelsPerFrame);
- const bin = Math.round((state.canvasHeight - screenY) / state.pixelsPerBin);
- const freqHz = (bin / state.referenceDctSize) * (SAMPLE_RATE / 2);
+
+ let freqHz;
+ if (USE_LOG_SCALE) {
+ // Logarithmic frequency mapping
+ const logMin = Math.log10(FREQ_MIN);
+ const logMax = Math.log10(FREQ_MAX);
+ const normalizedY = 1.0 - (screenY / state.canvasHeight); // Flip Y (0 at bottom, 1 at top)
+ const logFreq = logMin + normalizedY * (logMax - logMin);
+ freqHz = Math.pow(10, logFreq);
+ } else {
+ // Linear frequency mapping (old behavior)
+ const bin = Math.round((state.canvasHeight - screenY) / state.pixelsPerBin);
+ freqHz = (bin / state.referenceDctSize) * (SAMPLE_RATE / 2);
+ }
// Amplitude from Y position (normalized 0-1, top = 1.0, bottom = 0.0)
const amplitude = 1.0 - (screenY / state.canvasHeight);
return {
frame: Math.max(0, frame),
- freqHz: Math.max(0, freqHz),
+ freqHz: Math.max(FREQ_MIN, Math.min(FREQ_MAX, freqHz)),
amplitude: Math.max(0, Math.min(1, amplitude))
};
}
function spectrogramToScreen(frame, freqHz) {
- const bin = (freqHz / (SAMPLE_RATE / 2)) * state.referenceDctSize;
const x = frame * state.pixelsPerFrame;
- const y = state.canvasHeight - (bin * state.pixelsPerBin);
+
+ let y;
+ if (USE_LOG_SCALE) {
+ // Logarithmic frequency mapping
+ const logMin = Math.log10(FREQ_MIN);
+ const logMax = Math.log10(FREQ_MAX);
+ const clampedFreq = Math.max(FREQ_MIN, Math.min(FREQ_MAX, freqHz));
+ const logFreq = Math.log10(clampedFreq);
+ const normalizedY = (logFreq - logMin) / (logMax - logMin);
+ y = state.canvasHeight * (1.0 - normalizedY); // Flip Y back to screen coords
+ } else {
+ // Linear frequency mapping (old behavior)
+ const bin = (freqHz / (SAMPLE_RATE / 2)) * state.referenceDctSize;
+ y = state.canvasHeight - (bin * state.pixelsPerBin);
+ }
+
return {x, y};
}
@@ -721,28 +937,116 @@ function render() {
drawProceduralSpectrogram(ctx);
}
+ // Draw frequency axis (log-scale grid and labels)
+ drawFrequencyAxis(ctx);
+
+ // Draw playhead indicator
+ drawPlayhead(ctx);
+
+ // Draw mouse crosshair and tooltip
+ drawCrosshair(ctx);
+
// Draw control points
drawControlPoints(ctx);
}
+function drawPlayhead(ctx) {
+ if (!state.isPlaying || state.playbackCurrentFrame < 0) return;
+
+ const x = state.playbackCurrentFrame * state.pixelsPerFrame;
+
+ // Draw vertical line
+ ctx.strokeStyle = '#ff3333'; // Bright red
+ ctx.lineWidth = 2;
+ ctx.setLineDash([5, 3]); // Dashed line
+ ctx.beginPath();
+ ctx.moveTo(x, 0);
+ ctx.lineTo(x, state.canvasHeight);
+ ctx.stroke();
+ ctx.setLineDash([]); // Reset to solid line
+}
+
+function drawCrosshair(ctx) {
+ if (state.mouseX < 0 || state.mouseY < 0) return;
+
+ // Draw vertical line
+ ctx.strokeStyle = 'rgba(255, 255, 255, 0.3)';
+ ctx.lineWidth = 1;
+ ctx.beginPath();
+ ctx.moveTo(state.mouseX, 0);
+ ctx.lineTo(state.mouseX, state.canvasHeight);
+ ctx.stroke();
+
+ // Draw tooltip
+ const frameText = `Frame: ${state.mouseFrame}`;
+ const freqText = `Freq: ${state.mouseFreq.toFixed(1)} Hz`;
+
+ ctx.font = '12px monospace';
+ const frameWidth = ctx.measureText(frameText).width;
+ const freqWidth = ctx.measureText(freqText).width;
+ const maxWidth = Math.max(frameWidth, freqWidth);
+
+ const tooltipX = state.mouseX + 10;
+ const tooltipY = state.mouseY - 40;
+ const tooltipWidth = maxWidth + 20;
+ const tooltipHeight = 40;
+
+ // Background
+ ctx.fillStyle = 'rgba(0, 0, 0, 0.8)';
+ ctx.fillRect(tooltipX, tooltipY, tooltipWidth, tooltipHeight);
+
+ // Border
+ ctx.strokeStyle = 'rgba(255, 255, 255, 0.3)';
+ ctx.lineWidth = 1;
+ ctx.strokeRect(tooltipX, tooltipY, tooltipWidth, tooltipHeight);
+
+ // Text
+ ctx.fillStyle = '#ffffff';
+ ctx.fillText(frameText, tooltipX + 10, tooltipY + 15);
+ ctx.fillText(freqText, tooltipX + 10, tooltipY + 30);
+}
+
function drawReferenceSpectrogram(ctx) {
- // Draw semi-transparent reference
- ctx.globalAlpha = 0.3;
+ // Create offscreen canvas for reference layer
+ const offscreen = document.createElement('canvas');
+ offscreen.width = state.canvasWidth;
+ offscreen.height = state.canvasHeight;
+ const offscreenCtx = offscreen.getContext('2d');
- const imgData = ctx.createImageData(state.canvasWidth, state.canvasHeight);
+ const imgData = offscreenCtx.createImageData(state.canvasWidth, state.canvasHeight);
- for (let frameIdx = 0; frameIdx < state.referenceNumFrames; frameIdx++) {
- const x = Math.floor(frameIdx * state.pixelsPerFrame);
- if (x >= state.canvasWidth) break;
+ // CORRECT MAPPING: Iterate over destination pixels → sample source bins
+ // This prevents gaps and overlaps
+ for (let screenY = 0; screenY < state.canvasHeight; screenY++) {
+ for (let screenX = 0; screenX < state.canvasWidth; screenX++) {
+ // Convert screen coordinates to spectrogram coordinates
+ const spectroCoords = screenToSpectrogram(screenX, screenY);
+ const frameIdx = Math.floor(spectroCoords.frame);
- for (let bin = 0; bin < state.referenceDctSize; bin++) {
- const y = state.canvasHeight - Math.floor(bin * state.pixelsPerBin);
- if (y < 0 || y >= state.canvasHeight) continue;
+ // Convert freqHz back to bin
+ const bin = Math.round((spectroCoords.freqHz / (SAMPLE_RATE / 2)) * state.referenceDctSize);
+ // Bounds check
+ if (frameIdx < 0 || frameIdx >= state.referenceNumFrames) continue;
+ if (bin < 0 || bin >= state.referenceDctSize) continue;
+
+ // Sample spectrogram
const specValue = state.referenceSpectrogram[frameIdx * state.referenceDctSize + bin];
- const intensity = Math.min(255, Math.abs(specValue) * 50); // Scale for visibility
- const pixelIdx = (y * state.canvasWidth + x) * 4;
+ // Logarithmic intensity mapping (dB scale)
+ // Maps wide dynamic range to visible range
+ const amplitude = Math.abs(specValue);
+ let intensity = 0;
+ if (amplitude > 0.0001) { // Noise floor
+ const dB = 20.0 * Math.log10(amplitude);
+ const dB_min = -60.0; // Noise floor (-60 dB)
+ const dB_max = 40.0; // Peak (40 dB headroom)
+ const normalized = (dB - dB_min) / (dB_max - dB_min);
+ intensity = Math.floor(Math.max(0, Math.min(255, normalized * 255)));
+ }
+
+ // Write pixel
+ const pixelIdx = (screenY * state.canvasWidth + screenX) * 4;
imgData.data[pixelIdx + 0] = intensity; // R
imgData.data[pixelIdx + 1] = intensity; // G
imgData.data[pixelIdx + 2] = intensity; // B
@@ -750,19 +1054,27 @@ function drawReferenceSpectrogram(ctx) {
}
}
- ctx.putImageData(imgData, 0, 0);
+ offscreenCtx.putImageData(imgData, 0, 0);
+
+ // Draw offscreen canvas with proper alpha blending
+ ctx.globalAlpha = state.referenceOpacity;
+ ctx.drawImage(offscreen, 0, 0);
ctx.globalAlpha = 1.0;
}
function drawProceduralSpectrogram(ctx) {
- // Draw each curve separately with its own color
+ // Draw each curve separately with its own color and volume
const numFrames = state.referenceNumFrames || 100;
- ctx.globalAlpha = 0.6;
-
state.curves.forEach(curve => {
if (curve.controlPoints.length === 0) return;
+ // Create offscreen canvas for this curve
+ const offscreen = document.createElement('canvas');
+ offscreen.width = state.canvasWidth;
+ offscreen.height = state.canvasHeight;
+ const offscreenCtx = offscreen.getContext('2d');
+
// Generate spectrogram for this curve only
const curveSpec = new Float32Array(state.referenceDctSize * numFrames);
drawCurveToSpectrogram(curve, curveSpec, state.referenceDctSize, numFrames);
@@ -770,30 +1082,53 @@ function drawProceduralSpectrogram(ctx) {
// Parse curve color (hex to RGB)
const color = hexToRgb(curve.color || '#0e639c');
- const imgData = ctx.createImageData(state.canvasWidth, state.canvasHeight);
+ const imgData = offscreenCtx.createImageData(state.canvasWidth, state.canvasHeight);
- for (let frameIdx = 0; frameIdx < numFrames; frameIdx++) {
- const x = Math.floor(frameIdx * state.pixelsPerFrame);
- if (x >= state.canvasWidth) break;
+ // CORRECT MAPPING: Iterate over destination pixels → sample source bins
+ for (let screenY = 0; screenY < state.canvasHeight; screenY++) {
+ for (let screenX = 0; screenX < state.canvasWidth; screenX++) {
+ // Convert screen coordinates to spectrogram coordinates
+ const spectroCoords = screenToSpectrogram(screenX, screenY);
+ const frameIdx = Math.floor(spectroCoords.frame);
- for (let bin = 0; bin < state.referenceDctSize; bin++) {
- const y = state.canvasHeight - Math.floor(bin * state.pixelsPerBin);
- if (y < 0 || y >= state.canvasHeight) continue;
+ // Convert freqHz back to bin
+ const bin = Math.round((spectroCoords.freqHz / (SAMPLE_RATE / 2)) * state.referenceDctSize);
+ // Bounds check
+ if (frameIdx < 0 || frameIdx >= numFrames) continue;
+ if (bin < 0 || bin >= state.referenceDctSize) continue;
+
+ // Sample spectrogram
const specValue = curveSpec[frameIdx * state.referenceDctSize + bin];
- const intensity = Math.min(1.0, Math.abs(specValue) / 10.0); // Normalize to 0-1
+
+ // Logarithmic intensity mapping with steeper falloff for procedural curves
+ const amplitude = Math.abs(specValue);
+ let intensity = 0.0;
+ if (amplitude > 0.001) { // Higher noise floor for cleaner visualization
+ const dB = 20.0 * Math.log10(amplitude);
+ const dB_min = -40.0; // Higher floor = steeper falloff (was -60)
+ const dB_max = 40.0; // Peak
+ const normalized = (dB - dB_min) / (dB_max - dB_min);
+ intensity = Math.max(0, Math.min(1.0, normalized)); // 0.0 to 1.0
+ }
if (intensity > 0.01) { // Only draw visible pixels
- const pixelIdx = (y * state.canvasWidth + x) * 4;
- imgData.data[pixelIdx + 0] = Math.floor(color.r * intensity);
- imgData.data[pixelIdx + 1] = Math.floor(color.g * intensity);
- imgData.data[pixelIdx + 2] = Math.floor(color.b * intensity);
- imgData.data[pixelIdx + 3] = 255;
+ const pixelIdx = (screenY * state.canvasWidth + screenX) * 4;
+ // Use constant color with alpha for intensity (pure colors)
+ imgData.data[pixelIdx + 0] = color.r;
+ imgData.data[pixelIdx + 1] = color.g;
+ imgData.data[pixelIdx + 2] = color.b;
+ imgData.data[pixelIdx + 3] = Math.floor(intensity * 255); // Alpha = intensity
}
}
}
- ctx.putImageData(imgData, 0, 0);
+ offscreenCtx.putImageData(imgData, 0, 0);
+
+ // Draw offscreen canvas with curve volume as opacity (blends properly)
+ const curveOpacity = 0.6 * curve.volume; // Base opacity × curve volume
+ ctx.globalAlpha = curveOpacity;
+ ctx.drawImage(offscreen, 0, 0);
});
ctx.globalAlpha = 1.0;
@@ -858,6 +1193,37 @@ function drawControlPoints(ctx) {
});
}
+function drawFrequencyAxis(ctx) {
+ if (!USE_LOG_SCALE) return; // Only draw axis in log-scale mode
+
+ // Standard musical frequencies to display
+ const frequencies = [20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 16000];
+
+ ctx.fillStyle = '#cccccc';
+ ctx.font = '11px monospace';
+ ctx.textAlign = 'right';
+ ctx.textBaseline = 'middle';
+
+ frequencies.forEach(freq => {
+ const screenPos = spectrogramToScreen(0, freq);
+ const y = screenPos.y;
+
+ if (y >= 0 && y <= state.canvasHeight) {
+ // Draw frequency label
+ const label = freq >= 1000 ? `${freq / 1000}k` : `${freq}`;
+ ctx.fillText(label, state.canvasWidth - 5, y);
+
+ // Draw subtle grid line
+ ctx.strokeStyle = 'rgba(255, 255, 255, 0.1)';
+ ctx.lineWidth = 1;
+ ctx.beginPath();
+ ctx.moveTo(0, y);
+ ctx.lineTo(state.canvasWidth - 40, y); // Leave space for label
+ ctx.stroke();
+ }
+ });
+}
+
// ============================================================================
// Procedural Spectrogram Generation
// ============================================================================
@@ -876,10 +1242,20 @@ function generateProceduralSpectrogram(numFrames) {
function drawCurveToSpectrogram(curve, spectrogram, dctSize, numFrames) {
if (curve.controlPoints.length === 0) return;
+ // Find the frame range covered by control points
+ const frames = curve.controlPoints.map(p => p.frame);
+ const minFrame = Math.max(0, Math.min(...frames)); // Clamp to valid range
+ const maxFrame = Math.min(numFrames - 1, Math.max(...frames));
+
// Amplitude scaling factor to match typical DCT coefficient magnitudes
- const AMPLITUDE_SCALE = 10.0;
+ // Increased from 10.0 to 50.0 for better audibility
+ const AMPLITUDE_SCALE = 50.0;
- for (let frame = 0; frame < numFrames; frame++) {
+ // Apply curve volume to the amplitude
+ const curveVolume = curve.volume || 1.0;
+
+ // Only iterate over the range where control points exist
+ for (let frame = minFrame; frame <= maxFrame; frame++) {
// Evaluate Bezier curve at this frame
const freqHz = evaluateBezierLinear(curve.controlPoints, frame, 'freqHz');
const amplitude = evaluateBezierLinear(curve.controlPoints, frame, 'amplitude');
@@ -893,7 +1269,7 @@ function drawCurveToSpectrogram(curve, spectrogram, dctSize, numFrames) {
const profileValue = evaluateProfile(curve.profile, dist);
const idx = frame * dctSize + bin;
- spectrogram[idx] += amplitude * profileValue * AMPLITUDE_SCALE;
+ spectrogram[idx] += amplitude * profileValue * AMPLITUDE_SCALE * curveVolume;
}
}
}
@@ -935,10 +1311,17 @@ function evaluateProfile(profile, distance) {
case 'noise': {
const amplitude = profile.param1;
- const seed = profile.param2 || 42;
- // Simple deterministic noise
- const hash = (seed + Math.floor(distance * 1000)) % 10000;
- return amplitude * (hash / 10000);
+ const decay = profile.param2 || 30.0; // Decay rate (like sigma for Gaussian)
+
+ // Deterministic noise based on distance
+ const seed = 1234;
+ const hash = Math.floor((seed + distance * 17.13) * 1000) % 10000;
+ const noise = (hash / 10000) * 2.0 - 1.0; // Random value: -1 to +1
+
+ // Apply exponential decay (like Gaussian)
+ const decayFactor = Math.exp(-(distance * distance) / (decay * decay));
+
+ return amplitude * noise * decayFactor;
}
default:
@@ -984,28 +1367,182 @@ function playAudio(source) {
const audioBuffer = state.audioContext.createBuffer(1, audioData.length, SAMPLE_RATE);
audioBuffer.getChannelData(0).set(audioData);
+ // Create gain node for volume control
+ const gainNode = state.audioContext.createGain();
+ gainNode.gain.value = state.playbackVolume;
+
// Play
const bufferSource = state.audioContext.createBufferSource();
bufferSource.buffer = audioBuffer;
- bufferSource.connect(state.audioContext.destination);
+ bufferSource.connect(gainNode);
+ gainNode.connect(state.audioContext.destination);
bufferSource.start();
state.currentSource = bufferSource;
+ state.currentGainNode = gainNode; // Store gain node for live volume updates
state.isPlaying = true;
+ // Start playhead animation
+ state.playbackStartTime = state.audioContext.currentTime;
+ state.playbackDuration = audioData.length / SAMPLE_RATE;
+ state.playbackCurrentFrame = 0;
+ updatePlayhead();
+
bufferSource.onended = () => {
state.isPlaying = false;
state.currentSource = null;
+ state.currentGainNode = null; // Clear gain node reference
+ state.playbackCurrentFrame = 0;
+ render(); // Clear playhead
};
- console.log('Playing audio:', audioData.length, 'samples');
+ console.log('Playing audio:', audioData.length, 'samples at volume', state.playbackVolume);
+}
+
+function updatePlayhead() {
+ if (!state.isPlaying) return;
+
+ // Calculate current playback position
+ const elapsed = state.audioContext.currentTime - state.playbackStartTime;
+ const progress = Math.min(1.0, elapsed / state.playbackDuration);
+ state.playbackCurrentFrame = progress * (state.referenceNumFrames || 100);
+
+ // Redraw with playhead
+ render();
+
+ // Update spectrum viewer
+ drawSpectrumViewer();
+
+ // Continue animation
+ requestAnimationFrame(updatePlayhead);
+}
+
+function drawSpectrumViewer() {
+ const viewer = document.getElementById('spectrumViewer');
+ const canvas = document.getElementById('spectrumCanvas');
+ const ctx = canvas.getContext('2d');
+
+ // Always show viewer (not just during playback)
+ viewer.classList.add('active');
+
+ // Determine which frame to display
+ let frameIdx;
+ if (state.isPlaying) {
+ frameIdx = Math.floor(state.playbackCurrentFrame);
+ } else {
+ // When not playing, show frame under mouse
+ frameIdx = state.mouseFrame;
+ }
+
+ if (frameIdx < 0 || frameIdx >= (state.referenceNumFrames || 100)) return;
+
+ // Clear canvas
+ ctx.fillStyle = '#1e1e1e';
+ ctx.fillRect(0, 0, canvas.width, canvas.height);
+
+ const numBars = 100; // Downsample to 100 bars for performance
+ const barWidth = canvas.width / numBars;
+
+ // Get reference spectrum (if available)
+ let refSpectrum = null;
+ if (state.referenceSpectrogram && frameIdx < state.referenceNumFrames) {
+ refSpectrum = new Float32Array(state.referenceDctSize);
+ for (let bin = 0; bin < state.referenceDctSize; bin++) {
+ refSpectrum[bin] = state.referenceSpectrogram[frameIdx * state.referenceDctSize + bin];
+ }
+ }
+
+ // Get procedural spectrum (if curves exist)
+ let procSpectrum = null;
+ if (state.curves.length > 0) {
+ const numFrames = state.referenceNumFrames || 100;
+ const fullProcSpec = new Float32Array(state.referenceDctSize * numFrames);
+ state.curves.forEach(curve => {
+ drawCurveToSpectrogram(curve, fullProcSpec, state.referenceDctSize, numFrames);
+ });
+
+ // Extract just this frame
+ procSpectrum = new Float32Array(state.referenceDctSize);
+ for (let bin = 0; bin < state.referenceDctSize; bin++) {
+ procSpectrum[bin] = fullProcSpec[frameIdx * state.referenceDctSize + bin];
+ }
+ }
+
+ // Draw spectrum bars (both reference and procedural overlaid)
+ for (let i = 0; i < numBars; i++) {
+ const binIdx = Math.floor((i / numBars) * state.referenceDctSize);
+
+ // Draw reference spectrum (green, behind)
+ if (refSpectrum) {
+ const amplitude = Math.abs(refSpectrum[binIdx]);
+ let height = 0;
+ if (amplitude > 0.0001) {
+ const dB = 20.0 * Math.log10(amplitude);
+ const dB_min = -60.0;
+ const dB_max = 40.0;
+ const normalized = (dB - dB_min) / (dB_max - dB_min);
+ height = Math.max(0, Math.min(canvas.height, normalized * canvas.height));
+ }
+
+ if (height > 0) {
+ const gradient = ctx.createLinearGradient(0, canvas.height - height, 0, canvas.height);
+ gradient.addColorStop(0, '#00ff00');
+ gradient.addColorStop(1, '#004400');
+ ctx.fillStyle = gradient;
+ ctx.fillRect(i * barWidth, canvas.height - height, barWidth - 1, height);
+ }
+ }
+
+ // Draw procedural spectrum (red, overlaid)
+ if (procSpectrum) {
+ const amplitude = Math.abs(procSpectrum[binIdx]);
+ let height = 0;
+ if (amplitude > 0.001) {
+ const dB = 20.0 * Math.log10(amplitude);
+ const dB_min = -40.0; // Same as procedural spectrogram rendering
+ const dB_max = 40.0;
+ const normalized = (dB - dB_min) / (dB_max - dB_min);
+ height = Math.max(0, Math.min(canvas.height, normalized * canvas.height));
+ }
+
+ if (height > 0) {
+ const gradient = ctx.createLinearGradient(0, canvas.height - height, 0, canvas.height);
+ gradient.addColorStop(0, '#ff5555'); // Bright red
+ gradient.addColorStop(1, '#550000'); // Dark red
+ ctx.fillStyle = gradient;
+ // Make it slightly transparent to see overlap
+ ctx.globalAlpha = 0.7;
+ ctx.fillRect(i * barWidth, canvas.height - height, barWidth - 1, height);
+ ctx.globalAlpha = 1.0;
+ }
+ }
+ }
+
+ // Draw frequency labels
+ ctx.fillStyle = '#888888';
+ ctx.font = '9px monospace';
+ ctx.textAlign = 'left';
+ ctx.fillText('20 Hz', 2, canvas.height - 2);
+ ctx.textAlign = 'right';
+ ctx.fillText('16 kHz', canvas.width - 2, canvas.height - 2);
+
+ // Draw frame number label (top-left)
+ ctx.textAlign = 'left';
+ ctx.fillStyle = state.isPlaying ? '#ff3333' : '#aaaaaa';
+ ctx.fillText(`Frame ${frameIdx}`, 2, 10);
}
function stopAudio() {
if (state.currentSource) {
- state.currentSource.stop();
+ try {
+ state.currentSource.stop();
+ state.currentSource.disconnect();
+ } catch (e) {
+ // Source may have already stopped naturally
+ }
state.currentSource = null;
}
+ state.currentGainNode = null; // Clear gain node reference
state.isPlaying = false;
}
@@ -1138,6 +1675,9 @@ function generateProceduralParamsText() {
}
text += '\n';
+ // Add curve volume
+ text += ` VOLUME ${curve.volume.toFixed(3)}\n`;
+
text += 'END_CURVE\n\n';
});
@@ -1162,7 +1702,7 @@ function generateCppCodeText() {
code += 'void gen_procedural(float* spec, int dct_size, int num_frames) {\n';
state.curves.forEach((curve, curveIdx) => {
- code += ` // Curve ${curveIdx}\n`;
+ code += ` // Curve ${curveIdx} (volume=${curve.volume.toFixed(3)})\n`;
code += ' {\n';
// Control points arrays
@@ -1175,8 +1715,10 @@ function generateCppCodeText() {
code += curve.controlPoints.map(p => `${p.freqHz.toFixed(1)}f`).join(', ');
code += '};\n';
+ // Apply curve volume to amplitudes
+ const curveVolume = curve.volume || 1.0;
code += ` const float amps[] = {`;
- code += curve.controlPoints.map(p => `${p.amplitude.toFixed(3)}f`).join(', ');
+ code += curve.controlPoints.map(p => `${(p.amplitude * curveVolume).toFixed(3)}f`).join(', ');
code += '};\n\n';
// Profile type
diff --git a/tools/spectral_editor/style.css b/tools/spectral_editor/style.css
index 36b4eb3..fa71d1d 100644
--- a/tools/spectral_editor/style.css
+++ b/tools/spectral_editor/style.css
@@ -87,6 +87,30 @@ header h1 {
display: none;
}
+/* Mini spectrum viewer (bottom-right overlay) */
+.spectrum-viewer {
+ position: absolute;
+ bottom: 10px;
+ right: 10px;
+ width: 200px;
+ height: 100px;
+ background: rgba(30, 30, 30, 0.9);
+ border: 1px solid #3e3e42;
+ border-radius: 3px;
+ display: block; /* Always visible */
+ pointer-events: none; /* Don't interfere with mouse events */
+}
+
+.spectrum-viewer.active {
+ display: block; /* Keep for backward compatibility */
+}
+
+#spectrumCanvas {
+ width: 100%;
+ height: 100%;
+ display: block;
+}
+
.canvas-overlay p {
font-size: 16px;
margin: 8px 0;
@@ -173,6 +197,31 @@ header h1 {
border-color: #0e639c;
}
+/* Point info panel */
+.point-info {
+ margin-top: 10px;
+ padding: 10px;
+ background: #2d2d30;
+ border-radius: 3px;
+ font-size: 12px;
+}
+
+.info-row {
+ display: flex;
+ justify-content: space-between;
+ padding: 4px 0;
+}
+
+.info-label {
+ color: #858585;
+ font-weight: 600;
+}
+
+.info-value {
+ color: #d4d4d4;
+ font-family: monospace;
+}
+
/* Control panel (bottom) */
.control-panel {
background: #252526;