diff options
Diffstat (limited to 'tools/spectral_editor/README.md')
| -rw-r--r-- | tools/spectral_editor/README.md | 221 |
1 files changed, 221 insertions, 0 deletions
diff --git a/tools/spectral_editor/README.md b/tools/spectral_editor/README.md new file mode 100644 index 0000000..221acb8 --- /dev/null +++ b/tools/spectral_editor/README.md @@ -0,0 +1,221 @@ +# Spectral Brush Editor + +A web-based tool for creating procedural audio by tracing spectrograms with parametric Bezier curves. + +## Purpose + +Replace large `.spec` binary assets with tiny procedural C++ code: +- **Before:** 5 KB binary `.spec` file +- **After:** ~100 bytes of C++ code calling `draw_bezier_curve()` + +**Compression ratio:** 50-100× + +## Features + +### Core Functionality +- Load `.wav` or `.spec` files as reference +- Trace spectrograms with Bezier curves + vertical profiles +- Real-time audio preview (procedural vs. original) +- Undo/Redo support (50-action history) +- Export to `procedural_params.txt` (re-editable) +- Generate C++ code (copy-paste ready) + +### Profiles +- **Gaussian:** Smooth harmonic falloff +- **Decaying Sinusoid:** Resonant/metallic texture (coming soon) +- **Noise:** Random texture/grit (coming soon) + +## Quick Start + +1. **Open the editor:** + ```bash + open tools/spectral_editor/index.html + ``` + (Or open in your browser via file:// protocol) + +2. **Load a reference sound:** + - Click "Load .wav/.spec" or press `Ctrl+O` + - Select a `.wav` or `.spec` file + +3. **Add a curve:** + - Click "Add Curve" button + - Click on canvas to place control points + - Drag control points to adjust frequency and amplitude + +4. **Adjust profile:** + - Use "Sigma" slider to control width + - Higher sigma = wider frequency spread + +5. **Preview audio:** + - Press `1` to play procedural sound + - Press `2` to play original .wav + - Press `Space` to stop + +6. **Export:** + - `Ctrl+S` → Save `procedural_params.txt` (re-editable) + - `Ctrl+Shift+S` → Generate C++ code + +## Keyboard Shortcuts + +| Key | Action | +|-----|--------| +| **1** | Play procedural sound | +| **2** | Play original .wav | +| **Space** | Stop playback | +| **Delete** | Delete selected control point | +| **Esc** | Deselect all | +| **Ctrl+Z** | Undo | +| **Ctrl+Shift+Z** | Redo | +| **Ctrl+S** | Save procedural_params.txt | +| **Ctrl+Shift+S** | Generate C++ code | +| **Ctrl+O** | Open file | +| **?** | Show help | + +## Mouse Controls + +- **Click** on canvas: Place control point +- **Drag** control point: Adjust position (frame, frequency, amplitude) +- **Right-click** control point: Delete + +## Workflow + +### Example: Create a Kick Drum + +1. Load a reference kick drum (e.g., `kick.wav`) +2. Add a curve +3. Place control points to trace the low-frequency punch: + - Point 1: Frame 0, ~200 Hz, amplitude 0.9 + - Point 2: Frame 20, ~80 Hz, amplitude 0.7 + - Point 3: Frame 100, ~50 Hz, amplitude 0.0 +4. Adjust sigma to ~30 (smooth falloff) +5. Press `1` to preview +6. Fine-tune control points +7. Export C++ code + +### Generated C++ Code Example + +```cpp +// Generated by Spectral Brush Editor +#include "audio/spectral_brush.h" + +void gen_kick_procedural(float* spec, int dct_size, int num_frames) { + // Curve 0 + { + const float frames[] = {0.0f, 20.0f, 100.0f}; + const float freqs[] = {200.0f, 80.0f, 50.0f}; + const float amps[] = {0.900f, 0.700f, 0.000f}; + + draw_bezier_curve(spec, dct_size, num_frames, + frames, freqs, amps, 3, + PROFILE_GAUSSIAN, 30.00f); + } +} + +// Usage in demo_assets.txt: +// KICK_PROC, PROC(gen_kick_procedural), NONE, "Procedural kick drum" +``` + +## File Formats + +### procedural_params.txt (Re-editable) + +Human-readable text format that can be loaded back into the editor: + +``` +# Spectral Brush Procedural Parameters +METADATA dct_size=512 num_frames=100 sample_rate=32000 + +CURVE bezier + CONTROL_POINT 0 200.0 0.900 + CONTROL_POINT 20 80.0 0.700 + CONTROL_POINT 100 50.0 0.000 + PROFILE gaussian sigma=30.0 +END_CURVE +``` + +### C++ Code (Ready to Compile) + +Generated code using the spectral_brush runtime API. Copy-paste into `src/audio/procedural_samples.cc`. + +## Technical Details + +### Spectral Brush Primitive + +A spectral brush consists of: + +1. **Central Curve** (Bezier): Traces a path through time-frequency space + - `{freq_bin, amplitude} = bezier(frame_number)` + - Control points: `(frame, freq_hz, amplitude)` + +2. **Vertical Profile**: Shapes the "brush stroke" around the central curve + - Gaussian: `exp(-(dist² / σ²))` + - Applied vertically at each frame + +### Coordinate System + +- **X-axis (Time):** Frame number (0 → num_frames) +- **Y-axis (Frequency):** Frequency in Hz (0 → 16 kHz for 32 kHz sample rate) +- **Amplitude:** Controlled by Y-position of control points (0.0-1.0) + +### Audio Synthesis + +1. Generate procedural spectrogram (DCT coefficients) +2. Apply IDCT to convert to time-domain audio +3. Use overlap-add with Hanning window +4. Play via Web Audio API (32 kHz sample rate) + +## Limitations + +### Phase 1 (Current) +- Only Bezier + Gaussian profile implemented +- Linear interpolation between control points +- Single-layer spectrogram (no compositing yet) + +### Future Enhancements +- Cubic Bezier interpolation (smoother curves) +- Decaying sinusoid and noise profiles +- Composite profiles (add/subtract/multiply) +- Multi-dimensional Bezier (vary decay, oscillation, etc.) +- Frequency snapping (snap to musical notes) + +## Troubleshooting + +**Q: Audio doesn't play** +- Check browser console for errors +- Ensure audio context initialized (some browsers require user interaction first) +- Try clicking canvas before pressing `1` or `2` + +**Q: Canvas is blank** +- Make sure you loaded a reference file (`.wav` or `.spec`) +- Check console for file loading errors + +**Q: Exported code doesn't compile** +- Ensure `spectral_brush.h/cc` is built and linked +- Verify `draw_bezier_curve()` function is available +- Check include paths in your build system + +**Q: Generated sound doesn't match original** +- Adjust sigma (profile width) +- Add more control points for finer detail +- Use multiple curves for complex sounds + +## Integration with Demo + +1. Generate C++ code from editor +2. Copy code into `src/audio/procedural_samples.cc` +3. Add entry to `assets/final/demo_assets.txt`: + ``` + SOUND_PROC, PROC(gen_procedural), NONE, "Procedural sound" + ``` +4. Rebuild demo +5. Use `AssetId::SOUND_PROC` in your code + +## Browser Compatibility + +- **Tested:** Chrome 90+, Firefox 88+, Edge 90+, Safari 14+ +- **Requirements:** Web Audio API support +- **Recommended:** Desktop browser (mobile support limited) + +## License + +Part of the 64k demo project. See project LICENSE. |
