| Age | Commit message (Collapse) | Author |
|---|
|
Implements a more robust partial tracking algorithm by using phase
coherence to validate and link spectral peaks across frames. This
significantly improves tracking accuracy, especially for crossing or
closely-spaced partials.
Key changes:
- : The STFT cache now computes and stores the phase for each
frequency bin alongside the magnitude.
- : The peak detection logic now interpolates the
phase for sub-bin accuracy.
- : The core tracking algorithm was replaced with a
phase-aware model. It predicts the expected phase for a partial in
the next frame and uses a cost function combining both frequency and
phase error to find the best match.
This implementation follows the design outlined in the new document
.
handoff(Gemini): Phase prediction implemented. Further tuning of the cost function weights may be beneficial.
|
|
- Implement Predictive Kinematic Tracking to improve partial tracking during fast glissandos and vibrato.
- Add Peak Prominence Pruning to filter out insignificant local maxima.
- Replace heuristic Bezier fitting with a Least-Squares solver for more accurate trajectories.
- Update UI to include a Prominence parameter input.
- Archive MQ_EXTRACTION_IMPROVEMENTS.md design document.
handoff(Gemini): implemented MQ extraction improvements (kinematic tracking, prominence pruning, least-squares bezier)
|
|
Add r (pole) and gain sliders with force r/gain checkbox in Synthesis
panel, visible when Resonator (all) is active. Restrict r range to
[0.75, 0.9999] for both global and per-partial sliders.
handoff(Claude): global resonator r/gain override controls added
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
Add k1/k2 range sliders for post-synthesis LP/HP filters. Display maps
coefficients to -3dB cutoff frequency using exact IIR formulas; shows
'bypass' at k=1.0. Uses audioBuffer.sampleRate when loaded, falls back
to 44100 Hz.
handoff(Claude): LP k1 and HP k2 sliders added to Synthesis panel with Hz display.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
Adds "Resonator (all)" checkbox that forces resonator synthesis for all
partials without modifying per-partial mode settings.
Fixes null deref when partial.resonator is undefined in forceResonator mode.
handoff(Gemini): resonator test-mode checkbox added; no per-partial data mutated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
Add first-order IIR low-pass (options.k1) and high-pass (options.k2)
filters applied globally after the synthesis loop and before normalization.
Both are bypassed when the key is absent; LP is applied before HP
(bandpass configuration when both active).
handoff(Claude): LP/HP filters added to synthesizeMQ, no UI yet.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
Each partial in the Synth tab now has a Sinusoid/Resonator toggle.
Resonator path: y[n] = 2r·cos(ω₀)·y1 − r²·y2 + A(t)·√(1−r²)·noise,
coefficients recomputed per-sample from the freq Bezier curve.
gainNorm=√(1−r²) normalises steady-state power; gainComp for trim.
UI: mode toggle buttons, r + gain jog sliders, RES badge in header.
Docs updated in tools/mq_editor/README.md.
handoff(Claude): resonator mode complete, coefficients translated from
spread params in README, ready for perceptual comparison testing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
auto-spread
- Add 'f·Power' checkbox: weights spectrum by f before peak detection (f·FFT_Power(f))
to accentuate high-frequency peaks; re-runs extraction on toggle
- Deselect partial after Extract Partials run
- Fix right panel not refreshing after Auto Spread All: re-call editor.onPartialSelect
handoff(Claude): mq_editor UX polish
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Map 'E' key to Extract Partials
- Move extractBtn inline style to CSS rule #extractBtn
- Hoist autoSpreadAllBtn DOM ref to top with other refs
- Drop unused PADY local var in _renderAmpEditor
- Remove redundant comment before extractBtn listener
- README: add E/Esc keys, add editor.js to architecture, mark Phase 3 done
handoff(Gemini): mq_editor UX polish + doc consolidated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
panel refresh after extract
- Clear extractedPartials and editor state when loading a new WAV
- After extract, refresh right panels (re-select if index still valid)
- Synth fields (decay, jitter, spread) get jog sliders: drag to nudge, spring-back on release
- Spread extension limit dashed line: alpha 0.4→0.75, lineWidth 1→1.5, dash [3,4]→[4,3]
handoff(Gemini): mq_editor UX polish — jog sliders, WAV reset, panel refresh, spread line visibility
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- autodetectSpread(): measures half-power (-3dB) peak width in spectrogram
to infer spread_above/below (replaces near-zero bezier residual approach)
- 'Auto' button per partial + 'Auto Spread All' toolbar button
- Spread panel inputs now trigger viewer refresh on change
- 50% drop-off dotted reference lines on spread band (selected partial only)
- Mini-spectrum: use max() instead of mean() over bins per pixel column,
fixing high-frequency amplitude mismatch between original and synthesis
handoff(Gemini): spread autodetection and mini-spectrum fixes done
|
|
- Freq/Amp curve params now in tabs to save vertical space
- Add Synth tab: per-partial decay, jitter, spread_above/below controls
- Visualize spread band on selected partial (filled band + dashed bounds)
- Larger fonts and right panel (14px body, 260px panel width)
- Non-kept partials at 0.12 alpha (was 0.5)
- Default threshold -20dB (was -60dB)
handoff(Claude): mq_editor UI/UX improvements
|
|
- Click-to-select partials on canvas (proximity hit test on bezier)
- Right panel: peak freq/amp, time range, freq/amp bezier text inputs, mute/delete
- Selected partial renders on top with glow + larger control points
- Draggable freq curve control points on main canvas (grab/grabbing cursor)
- Amplitude bezier editor: 120px canvas below spectrogram, time-synced
with main view zoom/scroll via viewer.onRender callback
- Amp edits live-affect synthesis (mq_synth.js already uses ampCurve)
- PartialEditor class in editor.js owns all editing logic; index.html
wires it with 5 calls (setViewer, setPartials, onPartialSelect, onRender,
onPartialDeleted)
handoff(Gemini): partial editing MVP complete. Next: freq curve drag polish
or export (.spec generation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Document UI layout (right panel, mini-spectrum color coding)
- Add keyboard shortcuts table
- Document viewer.js coordinate API
- Update algorithm section (backward expansion pass)
- Refresh implementation status checklist
handoff(Claude): docs updated
|
|
viewer.js:
- Add timeToX(), freqLogNorm(), normToFreq() coordinate primitives
- freqToY/canvasToFreq now delegate to freqLogNorm/normToFreq
- normalizeDB() replaces duplicated (magDB-(maxDB-80))/80 formula
- partialColor(p) replaces repeated color array
- All inline time/freq→pixel math replaced with API calls
index.html:
- getKeepCount() replaces 3 copies of the same calculation
- playAudioBuffer() replaces duplicated playback setup + RAF loop
handoff(Claude): refactor complete
|
|
- Right panel with synthesis checkboxes (integrate phase, disable jitter, disable spread)
- Style file chooser as button; show filename next to page title
- Second backward pass in extractPartials to recover partial onsets
- Cursor line drawn on overlay canvas (no full redraw on mousemove)
handoff(Claude): UI + algo improvements complete
|
|
audio
- Fix incoherent per-sample jitter/spread: seed by partial index only
- Fix || fallback for zero-valued params (use != null checks)
- Phase integration: accumulator (2π*f/SR per sample) replaces 2π*f*t
- Add 'Integrate phase' checkbox to toggle between modes
- Revert Catmull-Rom back to simple bezier (1/3, 2/3 sample points)
- Remove all debug logging, clean up trackPartials, fitBezier
handoff(Gemini): dual-sine test validates full MQ pipeline (extract→track→synth).
Next: real audio loading and partial detection improvements.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Fix mini-spectrum: log-scale frequency axis, per-pixel bin averaging,
red peak bars after extraction
- Fix key handlers swallowing digits in input fields
- Clamp hop size to min 64 to prevent hang
- Store maxDB in STFTCache, use it to normalize both mini-spectrum and
main spectrogram (fixes clipping at 0dB for pure tones)
- Add Test WAV console validation for 440/660Hz peaks
- Grayscale colormap for main spectrogram
handoff(Gemini): mq_editor display fixes complete, tests not affected
|
|
Generates 2s of 440+660Hz (A4+E5) at 32kHz without needing a file.
Refactors WAV load path into shared loadAudioBuffer() used by both
file input and the test button.
handoff(Gemini): test button wired, full UI pipeline testable without assets.
|
|
- tools/mq_editor/test_fft.html: browser test for fft.js (12 tests:
DC impulse, single tone, STFT magnitude, pairs, triplets)
- tools/gen_sine_440.py: generate 1s 440Hz WAV at 32kHz for manual testing
handoff(Gemini): FFT isolation tests added, all passing in browser.
|
|
Replace all inline linear freq→Y math with freqToY()/canvasToFreq() pair.
freqStart set to 20Hz (log-safe). Freq axis ticks now log-spaced.
handoff(Gemini): log scale done, all render paths use freqToY/canvasToFreq
|
|
handoff(Claude): removed full spectrum storage from STFTCache frames;
getFFT() deleted; viewers use squaredAmplitude + 10*log10 directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- evalBezier: guard dt<=0 to avoid NaN on degenerate curves
- fitBezier: replace nearest-neighbor control points with Catmull-Rom
tangents (Hermite->Bezier), curve now passes through endpoints
- key 'a': toggle mini-spectrum between original and synth FFT
handoff(Claude): bezier fix + synth FFT comparison
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Sort extracted partials by decreasing peak amplitude
- Add Keep% range slider (1-100%) to toolbar
- Viewer draws omitted partials at 50% opacity (live on slider input)
- synthesizeMQ uses only the top-N% partials
handoff(Claude): partial amplitude filtering complete
|
|
- STFTCache: cache squaredAmplitude (re²+im²) per frame, add getSquaredAmplitude(t)
- getMagnitudeDB uses cached sq amp (10*log10, no sqrt)
- detectPeaks uses squaredAmp from cache instead of recomputing FFT
- extractPartials: use cache frames, return {partials, frames}
- Press 'p' to toggle raw peak overlay in viewer
- threshold input: step=any, change event triggers re-extraction
- runExtraction() shared by button and threshold change
handoff(Claude): mq_partial peaks/cache refactor complete
|
|
|
|
Phase 2 - JS Synthesizer:
- Created mq_synth.js with replica oscillator bank
- Bezier curve evaluation (cubic De Casteljau algorithm)
- Replica synthesis: frequency spread, amplitude decay, phase jitter
- PCM buffer generation from extracted MQ partials
- Normalization to prevent clipping
- Key '1' plays synthesized audio, key '2' plays original
- Playback comparison with animated playhead
STFT Cache Optimization:
- Created STFTCache class in fft.js for pre-computed windowed FFT frames
- Clean interface: getFFT(t), getMagnitudeDB(t, freq), setHopSize()
- Pre-computes all frames on WAV load (eliminates redundant FFT calls)
- Dynamic cache update when hop size changes
- Shared across spectrogram, tooltip, and mini-spectrum viewer
- Significant performance improvement
Mini-Spectrum Viewer:
- Bottom-right overlay (200x100) matching spectral_editor style
- Real-time FFT display at playhead or mouse position
- 100-bar visualization with cyan-to-yellow gradient
- Updates during playback or mouse hover
Files:
- tools/mq_editor/mq_synth.js (new)
- tools/mq_editor/fft.js (STFTCache class added)
- tools/mq_editor/index.html (synthesis playback, cache integration)
- tools/mq_editor/viewer.js (cache-based rendering, spectrum viewer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Phase 1 deliverables complete:
- MQ extraction with improved tracking
- Spectrogram visualization with zoom/scroll
- Original WAV playback with playhead
- Ready for Phase 2 (JS synthesizer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Tracking improvements:
- Frequency-dependent threshold (5% of freq, min 20 Hz)
- Candidate system requiring 3-frame persistence before birth
- Extended death tolerance (5 frames) for robust trajectories
- Minimum 10-frame length filter for valid partials
- Result: cleaner, less scattered partial trajectories
Audio playback:
- Web Audio API integration for original WAV playback
- Play/Stop buttons with proper state management
- Animated red playhead bar during playback
- Keyboard shortcuts: '2' plays original, '1' reserved for synthesis
Visualization:
- Power law (gamma=0.3) for improved spectrogram contrast
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fixed FFT to 1024 bins for 31.25 Hz resolution (better bass analysis)
- Refactored view state to zoom_factor + t_center for cleaner pan/zoom
- Mousewheel scrolls horizontally, shift+mousewheel zooms (respects deltaX/Y)
- Spectrogram bins now fill complete time/freq buckets at all zoom levels
- Extended dB range to -80→0 dB (80 dB) for better high-amplitude granularity
- Added real-time intensity tooltip in dB
- 50% alpha on spectrogram to reduce clutter over partial trajectories
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implement McAulay-Quatieri sinusoidal analysis tool for audio compression.
New files:
- doc/SPECTRAL_BRUSH_2.md: Complete design doc (MQ algorithm, data format, synthesis, roadmap)
- tools/mq_editor/index.html: Web UI (file loader, params, canvas)
- tools/mq_editor/fft.js: Radix-2 Cooley-Tukey FFT (from spectral_editor)
- tools/mq_editor/mq_extract.js: MQ algorithm (peak detection, tracking, bezier fitting)
- tools/mq_editor/viewer.js: Visualization (spectrogram, partials, zoom, axes)
- tools/mq_editor/README.md: Usage and implementation status
Features:
- Load WAV → extract sinusoidal partials → fit cubic bezier curves
- Time-frequency spectrogram with hot colormap (0-16 kHz)
- Horizontal zoom (mousewheel) around mouse position
- Axis ticks with labels (time: seconds, freq: Hz/kHz)
- Mouse tooltip showing time/frequency coordinates
- Real-time adjustable MQ parameters (FFT size, hop, threshold)
Algorithm:
- STFT with Hann windows (2048 FFT, 512 hop)
- Peak detection with parabolic interpolation
- Birth/death/continuation tracking (50 Hz tolerance)
- Cubic bezier fitting (4 control points per trajectory)
Next: Phase 2 (JS synthesizer for audio preview)
handoff(Claude): MQ editor Phase 1 complete. Ready for synthesis implementation.
|
|
Replace _pad with noise field in UniformsSequenceParams, providing
effects with non-deterministic random values [0..1] updated each frame.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
When starting from a clean tree (where `src/generated/` does not exist), the build would fail with a "file not found" error for `generated/timeline.h`. This was due to an incorrect dependency graph where `gpu.cc` was compiled before its required header was generated.
This commit fixes the issue by making the dependency explicit:
- Modified `tools/seq_compiler.py` to explicitly generate `timeline.h` alongside `timeline.cc`.
- Updated `cmake/DemoCodegen.cmake` to declare both files as `OUTPUT`s of the timeline compilation step.
- Added a direct dependency from the `gpu` library target to the `generate_timeline` custom target in `cmake/DemoLibraries.cmake`.
- Refactored the generated file paths in `DemoCodegen.cmake` into a single `GENERATED_CODE` variable for improved clarity and future-proofing.
|
|
Refactors the timeline code generator (`tools/seq_compiler.py`) to use the `gpu_init_color_attachment` helper function from `src/gpu/gpu.h`.
This change removes platform-specific `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives from the generated C++ files (`timeline.cc`, `test_timeline.cc`) and centralizes the platform-aware initialization logic within the existing helper function.
The generated code is now cleaner and easier to maintain. Both native macOS and Windows cross-compilation builds are confirmed to be successful.
|
|
This commit fixes several issues that caused the Windows cross-compilation build (`scripts/build_win.sh`) to fail.
The root causes were platform-specific API differences in the wgpu-native library and incorrect dependency tracking in the CMake build system for generated code.
Changes:
- **`tools/seq_compiler.py`**: The timeline generator now wraps `depthSlice` assignments in `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives to handle API differences in `WGPURenderPassColorAttachment`.
- **`src/gpu/gpu.h`**: The `gpu_init_color_attachment` helper is now platform-aware, using a preprocessor guard for the `depthSlice` member.
- **`src/effects/*.cc`**: All effects are updated to use the new platform-aware helper or have explicit guards for `depthSlice`. Also, replaced `WGPUTexelCopyTextureInfo` with the cross-platform alias `GpuTextureCopyInfo` in `rotating_cube_effect.cc`.
- **`cmake`**: Added `tools/seq_compiler.py` as an explicit dependency to the `generate_timeline` and `generate_test_demo_timeline` custom commands. This ensures that changes to the generator script correctly trigger a rebuild of the generated C++ files.
- **`scripts/build_win.sh`**: Removed the erroneous attempt to build the `seq_compiler.py` script as a native executable.
With these changes, the Windows cross-compilation build now completes successfully.
|
|
Converts all static_cast<>, reinterpret_cast<> to C-style casts
per CODING_STYLE.md guidelines.
- Modified 12 files across gpu, 3d, util, tests, and tools
- All builds passing, 34/34 tests passing
- No functional changes, pure style cleanup
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Effects now accept start/end time parameters and automatically passthrough
when inactive. Implements buffer chain integrity via compile-time validation.
- Effect base class: dispatch_render() checks time bounds, auto-passthroughs
1:1 input/output effects outside [start, end] interval
- seq_compiler.py: validates producer/consumer lifespan constraints for
multi-output effects, adds --validate flag, always validates before codegen
- Updated all 9 effect classes and test fixtures to pass start/end times
- check_all.sh: includes timeline validation step
- Tests: 34/34 passing, demo runs successfully
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
|
|
- Removed unused v1 shader declarations (13 variables)
- Removed _v2 suffix from active shader names
- Moved shaders.{h,cc} from src/gpu to src/effects
- Updated all includes and build references
- All tests pass (34/34)
handoff(Claude): Cleaned up shader management, tests passing
|
|
Drastically reduce documentation verbosity while retaining essential info.
README.md (152→82 lines):
- Consolidated features into organized sections
- Single concise v2 format example
- Removed redundant explanations
- Quick start section upfront
ROADMAP.md (685→60 lines):
- Completed features: simple bullet list
- Future work: brief descriptions only
- Removed verbose implementation details
- Removed outdated sections
Net reduction: -851 lines (-90%)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updates all documentation to reflect Sequence V2 format support:
Timeline Editor (tools/timeline_editor/):
- README.md: Updated features list, file format examples with NODE
declarations and arrow syntax, usage instructions for node editing
- ROADMAP.md: Added completed item 1.0 "Sequence V2 Format Support"
Core Documentation (doc/):
- HOWTO.md: Updated timeline example to use v2 arrow syntax, added
NODE/buffer chain features to visual editor description
- SEQUENCE.md: Marked timeline editor graph visualization as completed
All examples now show v2 format:
EFFECT + ClassName input1 input2 -> output1 output2 start end
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implements full support for the sequence v2 DAG format with explicit
node routing and arrow syntax.
New features:
- timeline-format.js module for parsing/serializing v2 format
- NODE declarations with typed buffers (u8x4_norm, f32x4, etc.)
- Arrow syntax for effect routing: input1 input2 -> output1 output2
- Buffer chain visualization in properties panel and tooltips
- Node editor modal for adding/deleting node declarations
- Validation for undeclared node references (when NODEs explicit)
- Backward compatible with auto-inferred nodes
Files added:
- tools/timeline_editor/timeline-format.js (214 lines)
- tools/timeline_editor/test_format.html (automated tests)
- workspaces/test/timeline_v2_test.seq (test file with NODE declarations)
Files modified:
- tools/timeline_editor/index.html (~40 changes for v2 support)
All success criteria met. Round-trip tested with existing timelines.
handoff(Claude): Timeline editor now fully supports v2 format with
explicit node routing, NODE declarations, and buffer chain visualization.
Parser handles both explicit NODE declarations and auto-inferred nodes.
Validation only runs when explicit NODEs exist. Ready for production use.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- test_demo now uses workspaces/test/{timeline.seq,music.track}
- Removed tools/test_demo.{seq,track} (no longer used)
- Updated docs to reference workspace files
- Changes to workspaces/test/timeline.seq now trigger rebuild
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Prevents compilation errors when multiple sequences share the same name.
Compiler now appends _{index}_Sequence for unique class names.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Remove #if !defined(DEMO_CROSS_COMPILE_WIN32) guards
- platform.h included transitively handles depthSlice compatibility
- Regenerate timeline.cc with cleaned output
- Tests: 34/34 passing
|
|
- Remove tools/seq_compiler.cc (replaced by seq_compiler.py)
- Remove C++ seq_compiler build target from cmake/DemoTools.cmake
- Update documentation to remove Gantt chart mentions
- Keep seq_compiler.py (active Python compiler)
- All tests passing (34/34)
|
|
Remove END_DEMO keyword from timeline format. Demo duration now
calculated from max effect end time across all sequences. Sort
sequences by start time at compile time for deterministic ordering.
Changes:
- seq_compiler.py: Auto-calculate duration, sort sequences
- seq_compiler.cc: Remove END_DEMO parsing, sort by start time
- workspaces/test/timeline.seq: Remove END_DEMO directive
- Generated timeline.cc: Duration now 40.0f (was hardcoded)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- seq_compiler.py: Calculate beat_phase from beat_time (was hardcoded 0.0f)
- Refactor: Replace CommonPostProcessUniforms with UniformsSequenceParams
- Remove duplicate struct definition in post_process_helper.h
- Update all CNN effects and tests to use unified uniform struct
- Fixes FlashEffect showing solid white instead of flashing to beat
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
