diff options
Diffstat (limited to 'doc/archive/SPECTRAL_BRUSH_EDITOR.md')
| -rw-r--r-- | doc/archive/SPECTRAL_BRUSH_EDITOR.md | 195 |
1 files changed, 195 insertions, 0 deletions
diff --git a/doc/archive/SPECTRAL_BRUSH_EDITOR.md b/doc/archive/SPECTRAL_BRUSH_EDITOR.md new file mode 100644 index 0000000..a7d0e3a --- /dev/null +++ b/doc/archive/SPECTRAL_BRUSH_EDITOR.md @@ -0,0 +1,195 @@ +# Spectral Brush Editor (Task #5) + +## Concept + +Replace large `.spec` assets with procedural C++ code (50-100× compression). + +**Before:** 5 KB binary `.spec` file +**After:** ~100 bytes C++ code calling `draw_bezier_curve()` + +**Workflow:** +``` +.wav → Load in editor → Trace with Bezier curves → Export procedural_params.txt + C++ code +``` + +--- + +## Core Primitive: "Spectral Brush" + +### 1. Central Curve (Bezier) +Traces time-frequency path: `{freq_bin, amplitude} = bezier(frame_number)` + +**Example control points:** +```javascript +[ + {frame: 0, freq_hz: 200.0, amplitude: 0.9}, // Attack + {frame: 20, freq_hz: 80.0, amplitude: 0.7}, // Sustain + {frame: 100, freq_hz: 50.0, amplitude: 0.0} // Decay +] +``` + +### 2. Vertical Profile +Shapes "brush stroke" around curve at each frame. + +**Profile Types:** +- **Gaussian**: `exp(-(dist² / σ²))` - Musical tones, bass +- **Decaying Sinusoid**: `exp(-decay * dist) * cos(ω * dist)` - Metallic sounds +- **Noise**: `random(seed, dist) * amplitude` - Hi-hats, cymbals +- **Composite**: Combine multiple profiles (add/subtract/multiply) + +--- + +## File Formats + +### A. `procedural_params.txt` (Human-readable) +```text +METADATA dct_size=512 num_frames=100 sample_rate=32000 + +CURVE bezier + CONTROL_POINT 0 200.0 0.9 + CONTROL_POINT 20 80.0 0.7 + CONTROL_POINT 100 50.0 0.0 + PROFILE gaussian sigma=30.0 +END_CURVE +``` + +### B. C++ Code (Ready to compile) +```cpp +#include "audio/spectral_brush.h" + +void gen_kick_procedural(float* spec, int dct_size, int num_frames) { + const float frames[] = {0.0f, 20.0f, 100.0f}; + const float freqs[] = {200.0f, 80.0f, 50.0f}; + const float amps[] = {0.9f, 0.7f, 0.0f}; + + draw_bezier_curve(spec, dct_size, num_frames, + frames, freqs, amps, 3, + PROFILE_GAUSSIAN, 30.0f); +} +``` + +--- + +## C++ Runtime API + +### Files: `src/audio/spectral_brush.{h,cc}` + +**Key functions:** +```cpp +enum ProfileType { + PROFILE_GAUSSIAN, + PROFILE_DECAYING_SINUSOID, + PROFILE_NOISE +}; + +void draw_bezier_curve(float* spectrogram, int dct_size, int num_frames, + const float* control_frames, + const float* control_freqs_hz, + const float* control_amps, + int n_control_points, + ProfileType profile_type, + float profile_param1, + float profile_param2 = 0.0f); + +float evaluate_bezier_linear(const float* control_frames, + const float* control_values, + int n_points, + float frame); + +float evaluate_profile(ProfileType type, float distance, + float param1, float param2); +``` + +--- + +## Editor UI + +### Technology Stack +- HTML5 Canvas (visualization) +- Web Audio API (playback) +- Pure JavaScript (no dependencies) +- Reuse `dct.js` from existing editor + +### Key Features +- Dual-layer canvas (reference + procedural spectrograms) +- Drag control points to adjust curves +- Real-time spectrogram rendering +- Audio playback (keys 1/2 for procedural/original) +- Undo/Redo (Ctrl+Z, Ctrl+Shift+Z) +- Load .wav/.spec, save params, generate C++ code + +### Keyboard Shortcuts +| Key | Action | +|-----|--------| +| 1 | Play procedural | +| 2 | Play original | +| Space | Play/pause | +| Delete | Remove control point | +| Ctrl+Z | Undo | +| Ctrl+S | Save params | + +--- + +## Implementation Phases + +### Phase 1: C++ Runtime +**Files:** `src/audio/spectral_brush.{h,cc}`, `src/tests/test_spectral_brush.cc` + +**Tasks:** +- Define API (ProfileType, draw_bezier_curve, evaluate_profile) +- Implement linear Bezier interpolation +- Implement Gaussian profile +- Add unit tests +- **Deliverable:** Compiles, tests pass + +### Phase 2: Editor Core +**Files:** `tools/spectral_editor/{index.html, script.js, style.css, dct.js}` + +**Tasks:** +- HTML structure (canvas, controls, file input) +- Bezier curve editor (place/drag/delete control points) +- Dual-layer canvas rendering +- Real-time spectrogram generation +- Audio playback (IDCT → Web Audio API) +- Undo/Redo system +- **Deliverable:** Interactive editor, can trace .wav files + +### Phase 3: File I/O +**Tasks:** +- Load .wav (decode, STFT → spectrogram) +- Load .spec (binary parser) +- Save procedural_params.txt (text format) +- Generate C++ code (template) +- Load procedural_params.txt (re-editing) +- **Deliverable:** Full save/load cycle + +--- + +## Design Decisions + +- **Bezier:** Linear interpolation (Phase 1), cubic later +- **Profiles:** Gaussian only (Phase 1), others later +- **Parameters:** Soft UI limits, no enforced bounds +- **RNG:** Home-brew deterministic (small, repeatable) +- **Code gen:** Single function per sound (generic loader later) + +--- + +## Size Impact + +**Example: Kick drum** + +**Before (Binary):** +- 512 bins × 100 frames × 4 bytes = 200 KB uncompressed +- ~5 KB compressed (zlib) + +**After (Procedural):** +- 4 control points × 3 arrays × 4 floats = ~48 bytes data +- Function call overhead = ~20 bytes +- **Total: ~100 bytes** (50-100× reduction) + +**Trade-off:** Runtime CPU cost, acceptable for 64k demo. + +--- + +*See TODO.md for detailed implementation tasks.* |