summaryrefslogtreecommitdiff
path: root/tools/spectral_editor/README.md
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-06 13:36:19 +0100
committerskal <pascal.massimino@gmail.com>2026-02-06 13:36:19 +0100
commita0888c1afa8bf178b7a57d4e80373ad867a3474a (patch)
treeec7c4526d41e08c629c2e88115071b1614dbae34 /tools/spectral_editor/README.md
parent3002553be5bf880ead27fb3e415bc97d484b43eb (diff)
feat(spectral_editor): Complete Phase 2 milestone - Full-featured web editor
MILESTONE: Spectral Brush Editor Phase 2 Complete (February 6, 2026) Phase 2 delivers a production-ready web-based editor for creating procedural audio by tracing spectrograms with parametric Bezier curves. This tool enables replacing 5KB .spec binary assets with ~100 bytes of C++ code (50-100× compression). Core Features Implemented: ======================== Audio I/O: - Load .wav and .spec files as reference spectrograms - Real-time audio preview (procedural vs original) - Live volume control with GainNode (updates during playback) - Export to procedural_params.txt (human-readable, re-editable format) - Generate C++ code (copy-paste ready for demo integration) Curve Editing: - Multi-curve support with individual colors and volumes - Bezier curve control points (frame, frequency, amplitude) - Drag-and-drop control point editing - Per-curve volume control (0-100%) - Right-click to delete control points - Curves only render within control point range (no spill) Profile System (All 3 types implemented): - Gaussian: exp(-(dist² / σ²)) - smooth harmonic falloff - Decaying Sinusoid: exp(-decay × dist) × cos(ω × dist) - metallic resonance - Noise: noise × exp(-(dist² / decay²)) - textured grit with decay envelope Visualization: - Log-scale frequency axis (20 Hz to 16 kHz) for better bass visibility - Logarithmic dB-scale intensity mapping (-60 dB to +40 dB range) - Reference opacity slider (0-100%) for mixing original/procedural views - Playhead indicator (red dashed line) during playback - Mouse crosshair with tooltip (frame number, frequency) - Control point info panel (frame, frequency, amplitude) Real-time Spectrum Viewer (NEW): - Always-visible bottom-right overlay (200×100px) - Shows frequency spectrum for frame under mouse (hover mode) - Shows current playback frame spectrum (playback mode) - Dual display: Reference (green) + Procedural (red) overlaid - dB-scale bar heights for accurate visualization - Frame number label (red during playback, gray when hovering) Rendering Architecture: - Destination-to-source pixel mapping (prevents gaps in log-scale) - Offscreen canvas compositing for proper alpha blending - Alpha channel for procedural intensity (pure colors, not dimmed) - Steeper dB falloff for procedural curves (-40 dB floor vs -60 dB reference) UI/UX: - Undo/Redo system (50-action history) - Keyboard shortcuts (1/2/Space for playback, Ctrl+Z/Ctrl+Shift+Z, Delete, Esc) - File load confirmation (warns about unsaved curves) - Automatic curve reset on new file load Technical Details: - DCT/IDCT implementation (JavaScript port matching C++ runtime) - Overlap-add synthesis with Hanning window - Web Audio API integration (32 kHz sample rate) - Zero external dependencies (pure HTML/CSS/JS) Files Modified: - tools/spectral_editor/script.js (~1730 lines, main implementation) - tools/spectral_editor/index.html (UI structure, spectrum viewer) - tools/spectral_editor/style.css (VSCode dark theme styling) - tools/spectral_editor/README.md (updated features, roadmap) Phase 3 TODO (Next): =================== - Effect combination system (noise + Gaussian modulation, layer compositing) - Improved C++ code testing (validation, edge cases) - Better frequency scale (mu-law or perceptual scale, less bass-heavy) - Pre-defined shape library (kick, snare, hi-hat templates) - Load procedural_params.txt back into editor (re-editing) - FFT-based DCT optimization (O(N log N) vs O(N²)) Integration: - Generate C++ code → Copy to src/audio/procedural_samples.cc - Add PROC() entry to assets/final/demo_assets.txt - Rebuild demo → Use AssetId::SOUND_PROC handoff(Claude): Phase 2 complete. Next: FFT implementation task for performance optimization. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'tools/spectral_editor/README.md')
-rw-r--r--tools/spectral_editor/README.md73
1 files changed, 60 insertions, 13 deletions
diff --git a/tools/spectral_editor/README.md b/tools/spectral_editor/README.md
index 221acb8..6bb3681 100644
--- a/tools/spectral_editor/README.md
+++ b/tools/spectral_editor/README.md
@@ -19,11 +19,25 @@ Replace large `.spec` binary assets with tiny procedural C++ code:
- Undo/Redo support (50-action history)
- Export to `procedural_params.txt` (re-editable)
- Generate C++ code (copy-paste ready)
+- **Live volume control** during playback
+- **Reference opacity slider** for mixing original/procedural views
+- **Per-curve volume control** for fine-tuning
### Profiles
-- **Gaussian:** Smooth harmonic falloff
-- **Decaying Sinusoid:** Resonant/metallic texture (coming soon)
-- **Noise:** Random texture/grit (coming soon)
+- **Gaussian:** Smooth harmonic falloff (fully implemented)
+- **Decaying Sinusoid:** Resonant/metallic texture (implemented)
+- **Noise:** Random texture/grit with decay envelope (implemented)
+
+### Visualization
+- **Log-scale frequency axis** (20 Hz to 16 kHz) for better bass visibility
+- **Logarithmic dB-scale intensity** for proper dynamic range display
+- **Playhead indicator** showing current playback position
+- **Mouse crosshair with tooltip** displaying frame and frequency
+- **Real-time spectrum viewer** (bottom-right):
+ - Shows frequency spectrum for frame under mouse (hover mode)
+ - Shows frequency spectrum for current playback frame (playback mode)
+ - Dual display: Reference (green) and Procedural (red) overlaid
+ - Always visible for instant feedback
## Quick Start
@@ -164,19 +178,52 @@ A spectral brush consists of:
3. Use overlap-add with Hanning window
4. Play via Web Audio API (32 kHz sample rate)
-## Limitations
+## Development Status
+
+### Phase 1: C++ Runtime ✅ COMPLETE
+- Spectral brush primitive (Bezier + profiles)
+- C++ implementation in `src/audio/spectral_brush.{h,cc}`
+- Integration with asset system
+
+### Phase 2: Web Editor ✅ COMPLETE (Milestone: February 6, 2026)
+- Full-featured web-based editor
+- Real-time audio preview and visualization
+- Log-scale frequency display with dB-scale intensity
+- All three profile types (Gaussian, Decaying Sinusoid, Noise)
+- Live volume control and reference opacity mixing
+- Real-time dual-spectrum viewer (reference + procedural)
+- Export to `procedural_params.txt` and C++ code
+
+### Phase 3: Advanced Features (TODO)
+
+**High Priority:**
+- **Effect Combination System:** How to combine effects (e.g., noise + Gaussian modulation)?
+ - Layer-based compositing (add/multiply/subtract)
+ - Profile modulation (noise modulated by Gaussian envelope)
+ - Multi-pass rendering pipeline
+
+- **Improved C++ Code Testing:**
+ - Verify generated code compiles correctly
+ - Test parameter ranges and edge cases
+ - Add validation warnings in editor
+
+- **Better Frequency Scale:**
+ - Current log-scale is too bass-heavy
+ - Investigate mu-law or similar perceptual scales
+ - Allow user-configurable frequency mapping
-### Phase 1 (Current)
-- Only Bezier + Gaussian profile implemented
-- Linear interpolation between control points
-- Single-layer spectrogram (no compositing yet)
+- **Pre-defined Shape Library:**
+ - Template curves for common sounds (kick, snare, hi-hat, bass)
+ - One-click insertion with adjustable parameters
+ - Save/load custom shape presets
-### Future Enhancements
+**Future Enhancements:**
- Cubic Bezier interpolation (smoother curves)
-- Decaying sinusoid and noise profiles
-- Composite profiles (add/subtract/multiply)
-- Multi-dimensional Bezier (vary decay, oscillation, etc.)
-- Frequency snapping (snap to musical notes)
+- Multi-dimensional Bezier (vary decay, oscillation over time)
+- Frequency snapping (snap to musical notes/scales)
+- Load `procedural_params.txt` back into editor (re-editing)
+- Frame cache optimization for faster rendering
+- FFT-based DCT optimization (O(N log N) vs O(N²))
## Troubleshooting