| Age | Commit message (Collapse) | Author |
|---|
|
- Right panel with synthesis checkboxes (integrate phase, disable jitter, disable spread)
- Style file chooser as button; show filename next to page title
- Second backward pass in extractPartials to recover partial onsets
- Cursor line drawn on overlay canvas (no full redraw on mousemove)
handoff(Claude): UI + algo improvements complete
|
|
audio
- Fix incoherent per-sample jitter/spread: seed by partial index only
- Fix || fallback for zero-valued params (use != null checks)
- Phase integration: accumulator (2π*f/SR per sample) replaces 2π*f*t
- Add 'Integrate phase' checkbox to toggle between modes
- Revert Catmull-Rom back to simple bezier (1/3, 2/3 sample points)
- Remove all debug logging, clean up trackPartials, fitBezier
handoff(Gemini): dual-sine test validates full MQ pipeline (extract→track→synth).
Next: real audio loading and partial detection improvements.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Fix mini-spectrum: log-scale frequency axis, per-pixel bin averaging,
red peak bars after extraction
- Fix key handlers swallowing digits in input fields
- Clamp hop size to min 64 to prevent hang
- Store maxDB in STFTCache, use it to normalize both mini-spectrum and
main spectrogram (fixes clipping at 0dB for pure tones)
- Add Test WAV console validation for 440/660Hz peaks
- Grayscale colormap for main spectrogram
handoff(Gemini): mq_editor display fixes complete, tests not affected
|
|
Generates 2s of 440+660Hz (A4+E5) at 32kHz without needing a file.
Refactors WAV load path into shared loadAudioBuffer() used by both
file input and the test button.
handoff(Gemini): test button wired, full UI pipeline testable without assets.
|
|
- tools/mq_editor/test_fft.html: browser test for fft.js (12 tests:
DC impulse, single tone, STFT magnitude, pairs, triplets)
- tools/gen_sine_440.py: generate 1s 440Hz WAV at 32kHz for manual testing
handoff(Gemini): FFT isolation tests added, all passing in browser.
|
|
Replace all inline linear freq→Y math with freqToY()/canvasToFreq() pair.
freqStart set to 20Hz (log-safe). Freq axis ticks now log-spaced.
handoff(Gemini): log scale done, all render paths use freqToY/canvasToFreq
|
|
handoff(Claude): removed full spectrum storage from STFTCache frames;
getFFT() deleted; viewers use squaredAmplitude + 10*log10 directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- evalBezier: guard dt<=0 to avoid NaN on degenerate curves
- fitBezier: replace nearest-neighbor control points with Catmull-Rom
tangents (Hermite->Bezier), curve now passes through endpoints
- key 'a': toggle mini-spectrum between original and synth FFT
handoff(Claude): bezier fix + synth FFT comparison
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Sort extracted partials by decreasing peak amplitude
- Add Keep% range slider (1-100%) to toolbar
- Viewer draws omitted partials at 50% opacity (live on slider input)
- synthesizeMQ uses only the top-N% partials
handoff(Claude): partial amplitude filtering complete
|
|
- STFTCache: cache squaredAmplitude (re²+im²) per frame, add getSquaredAmplitude(t)
- getMagnitudeDB uses cached sq amp (10*log10, no sqrt)
- detectPeaks uses squaredAmp from cache instead of recomputing FFT
- extractPartials: use cache frames, return {partials, frames}
- Press 'p' to toggle raw peak overlay in viewer
- threshold input: step=any, change event triggers re-extraction
- runExtraction() shared by button and threshold change
handoff(Claude): mq_partial peaks/cache refactor complete
|
|
|
|
JS synthesizer with replica oscillator bank and STFT cache complete.
Ready for Phase 3 (editing UI).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Phase 2 - JS Synthesizer:
- Created mq_synth.js with replica oscillator bank
- Bezier curve evaluation (cubic De Casteljau algorithm)
- Replica synthesis: frequency spread, amplitude decay, phase jitter
- PCM buffer generation from extracted MQ partials
- Normalization to prevent clipping
- Key '1' plays synthesized audio, key '2' plays original
- Playback comparison with animated playhead
STFT Cache Optimization:
- Created STFTCache class in fft.js for pre-computed windowed FFT frames
- Clean interface: getFFT(t), getMagnitudeDB(t, freq), setHopSize()
- Pre-computes all frames on WAV load (eliminates redundant FFT calls)
- Dynamic cache update when hop size changes
- Shared across spectrogram, tooltip, and mini-spectrum viewer
- Significant performance improvement
Mini-Spectrum Viewer:
- Bottom-right overlay (200x100) matching spectral_editor style
- Real-time FFT display at playhead or mouse position
- 100-bar visualization with cyan-to-yellow gradient
- Updates during playback or mouse hover
Files:
- tools/mq_editor/mq_synth.js (new)
- tools/mq_editor/fft.js (STFTCache class added)
- tools/mq_editor/index.html (synthesis playback, cache integration)
- tools/mq_editor/viewer.js (cache-based rendering, spectrum viewer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Completed MQ extraction and visualization with improved tracking.
Ready for Phase 2 (JS synthesizer).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Phase 1 deliverables complete:
- MQ extraction with improved tracking
- Spectrogram visualization with zoom/scroll
- Original WAV playback with playhead
- Ready for Phase 2 (JS synthesizer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Tracking improvements:
- Frequency-dependent threshold (5% of freq, min 20 Hz)
- Candidate system requiring 3-frame persistence before birth
- Extended death tolerance (5 frames) for robust trajectories
- Minimum 10-frame length filter for valid partials
- Result: cleaner, less scattered partial trajectories
Audio playback:
- Web Audio API integration for original WAV playback
- Play/Stop buttons with proper state management
- Animated red playhead bar during playback
- Keyboard shortcuts: '2' plays original, '1' reserved for synthesis
Visualization:
- Power law (gamma=0.3) for improved spectrogram contrast
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fixed FFT to 1024 bins for 31.25 Hz resolution (better bass analysis)
- Refactored view state to zoom_factor + t_center for cleaner pan/zoom
- Mousewheel scrolls horizontally, shift+mousewheel zooms (respects deltaX/Y)
- Spectrogram bins now fill complete time/freq buckets at all zoom levels
- Extended dB range to -80→0 dB (80 dB) for better high-amplitude granularity
- Added real-time intensity tooltip in dB
- 50% alpha on spectrogram to reduce clutter over partial trajectories
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implement McAulay-Quatieri sinusoidal analysis tool for audio compression.
New files:
- doc/SPECTRAL_BRUSH_2.md: Complete design doc (MQ algorithm, data format, synthesis, roadmap)
- tools/mq_editor/index.html: Web UI (file loader, params, canvas)
- tools/mq_editor/fft.js: Radix-2 Cooley-Tukey FFT (from spectral_editor)
- tools/mq_editor/mq_extract.js: MQ algorithm (peak detection, tracking, bezier fitting)
- tools/mq_editor/viewer.js: Visualization (spectrogram, partials, zoom, axes)
- tools/mq_editor/README.md: Usage and implementation status
Features:
- Load WAV → extract sinusoidal partials → fit cubic bezier curves
- Time-frequency spectrogram with hot colormap (0-16 kHz)
- Horizontal zoom (mousewheel) around mouse position
- Axis ticks with labels (time: seconds, freq: Hz/kHz)
- Mouse tooltip showing time/frequency coordinates
- Real-time adjustable MQ parameters (FFT size, hop, threshold)
Algorithm:
- STFT with Hann windows (2048 FFT, 512 hop)
- Peak detection with parabolic interpolation
- Birth/death/continuation tracking (50 Hz tolerance)
- Cubic bezier fitting (4 control points per trajectory)
Next: Phase 2 (JS synthesizer for audio preview)
handoff(Claude): MQ editor Phase 1 complete. Ready for synthesis implementation.
|
|
Replace _pad with noise field in UniformsSequenceParams, providing
effects with non-deterministic random values [0..1] updated each frame.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Centralized uniforms_buffer_ initialization and updates to Effect base class:
- init_uniforms_buffer() now automatic in Effect::Effect()
- uniforms_buffer_.update() now automatic in dispatch_render()
- Removed redundant calls from all effect subclasses
- Updated effect.h comments to reflect automatic behavior
- Updated EFFECT_WORKFLOW.md templates
Benefits:
- 16 lines removed from effect implementations
- Consistent pattern enforced at compile time
- Reduced boilerplate for new effects
Tests: 34/34 passing
handoff(Claude): Effect base class now handles uniforms automatically
|
|
Adds audio intensity visualization for debugging audio-visual sync.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Introduces WGPUResource template with automatic release on destruction.
Reduces boilerplate in effect destructors and prevents resource leaks.
- set() for one-time initialization
- replace() for per-frame recreation
- Field ordering documented for dependency management
Converted 3 effects (Heptagon, Flash, Passthrough) and Effect base class.
All tests pass (34/34).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Eliminate ~100 lines of duplicated code across effect subclasses by moving common resource initialization to the base Effect class. Most effects repeatedly created uniforms buffers, samplers, and dummy textures with identical configurations.
Changes:
- Add shared members to Effect: uniforms_buffer_, sampler_, dummy_texture_*
- Add helpers: init_uniforms_buffer(), create_*_sampler(), create_dummy_scene_texture()
- Add gpu_create_*_sampler() and gpu_create_dummy_scene_texture() to gpu.h
- Move HEADLESS_RETURN_IF_NULL to Effect constructor
- Update 7 effects to use base class helpers (Flash, Heptagon, Passthrough, Placeholder, GaussianBlur, Particles, PeakMeter)
Benefits: Improved consistency, easier maintenance, reduced binary size.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
When starting from a clean tree (where `src/generated/` does not exist), the build would fail with a "file not found" error for `generated/timeline.h`. This was due to an incorrect dependency graph where `gpu.cc` was compiled before its required header was generated.
This commit fixes the issue by making the dependency explicit:
- Modified `tools/seq_compiler.py` to explicitly generate `timeline.h` alongside `timeline.cc`.
- Updated `cmake/DemoCodegen.cmake` to declare both files as `OUTPUT`s of the timeline compilation step.
- Added a direct dependency from the `gpu` library target to the `generate_timeline` custom target in `cmake/DemoLibraries.cmake`.
- Refactored the generated file paths in `DemoCodegen.cmake` into a single `GENERATED_CODE` variable for improved clarity and future-proofing.
|
|
Removed redundant and obvious comments from 7 GPU headers:
- post_process_helper.h: binding comments, stale UniformsSequenceParams note
- shader_composer.h: verbose Compose/VerifyIncludes descriptions
- uniform_helper.h: obvious method comments
- effect.h: redundant render/resize comments
- gpu.h: verbose struct/header comments, GPU perf placeholder
- sdf_effect.h: obvious method comments
- sequence.h: duplicate header, obvious API comments
Kept only non-obvious context (binding conventions, headless mode notes).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Removed outdated comments referencing old naming conventions and non-existent functions.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
Refactors the timeline code generator (`tools/seq_compiler.py`) to use the `gpu_init_color_attachment` helper function from `src/gpu/gpu.h`.
This change removes platform-specific `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives from the generated C++ files (`timeline.cc`, `test_timeline.cc`) and centralizes the platform-aware initialization logic within the existing helper function.
The generated code is now cleaner and easier to maintain. Both native macOS and Windows cross-compilation builds are confirmed to be successful.
|
|
This commit fixes several issues that caused the Windows cross-compilation build (`scripts/build_win.sh`) to fail.
The root causes were platform-specific API differences in the wgpu-native library and incorrect dependency tracking in the CMake build system for generated code.
Changes:
- **`tools/seq_compiler.py`**: The timeline generator now wraps `depthSlice` assignments in `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives to handle API differences in `WGPURenderPassColorAttachment`.
- **`src/gpu/gpu.h`**: The `gpu_init_color_attachment` helper is now platform-aware, using a preprocessor guard for the `depthSlice` member.
- **`src/effects/*.cc`**: All effects are updated to use the new platform-aware helper or have explicit guards for `depthSlice`. Also, replaced `WGPUTexelCopyTextureInfo` with the cross-platform alias `GpuTextureCopyInfo` in `rotating_cube_effect.cc`.
- **`cmake`**: Added `tools/seq_compiler.py` as an explicit dependency to the `generate_timeline` and `generate_test_demo_timeline` custom commands. This ensures that changes to the generator script correctly trigger a rebuild of the generated C++ files.
- **`scripts/build_win.sh`**: Removed the erroneous attempt to build the `seq_compiler.py` script as a native executable.
With these changes, the Windows cross-compilation build now completes successfully.
|
|
Refactors the SceneLoader class to a namespace. Since it only contained a single static utility function, a namespace is a more appropriate and idiomatic C++ construct. No functional changes.
|
|
|
|
Converts all static_cast<>, reinterpret_cast<> to C-style casts
per CODING_STYLE.md guidelines.
- Modified 12 files across gpu, 3d, util, tests, and tools
- All builds passing, 34/34 tests passing
- No functional changes, pure style cleanup
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Move platform-specific type definitions to gpu.h and establish coding rule
that platform ifdefs must be confined to gpu/platform layers.
- gpu.h: add GpuTextureCopyInfo, GpuTextureDataLayout type aliases
- effect.cc: use GpuTextureCopyInfo instead of platform ifdefs
- texture_manager.cc: use type aliases and label_view() helper
- CODING_STYLE.md: add platform-specific code section with rule
Tests: 34/34 passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Effects now accept start/end time parameters and automatically passthrough
when inactive. Implements buffer chain integrity via compile-time validation.
- Effect base class: dispatch_render() checks time bounds, auto-passthroughs
1:1 input/output effects outside [start, end] interval
- seq_compiler.py: validates producer/consumer lifespan constraints for
multi-output effects, adds --validate flag, always validates before codegen
- Updated all 9 effect classes and test fixtures to pass start/end times
- check_all.sh: includes timeline validation step
- Tests: 34/34 passing, demo runs successfully
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
|
|
|
|
- Removed unused v1 shader declarations (13 variables)
- Removed _v2 suffix from active shader names
- Moved shaders.{h,cc} from src/gpu to src/effects
- Updated all includes and build references
- All tests pass (34/34)
handoff(Claude): Cleaned up shader management, tests passing
|
|
Drastically reduce documentation verbosity while retaining essential info.
README.md (152→82 lines):
- Consolidated features into organized sections
- Single concise v2 format example
- Removed redundant explanations
- Quick start section upfront
ROADMAP.md (685→60 lines):
- Completed features: simple bullet list
- Future work: brief descriptions only
- Removed verbose implementation details
- Removed outdated sections
Net reduction: -851 lines (-90%)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updates all documentation to reflect Sequence V2 format support:
Timeline Editor (tools/timeline_editor/):
- README.md: Updated features list, file format examples with NODE
declarations and arrow syntax, usage instructions for node editing
- ROADMAP.md: Added completed item 1.0 "Sequence V2 Format Support"
Core Documentation (doc/):
- HOWTO.md: Updated timeline example to use v2 arrow syntax, added
NODE/buffer chain features to visual editor description
- SEQUENCE.md: Marked timeline editor graph visualization as completed
All examples now show v2 format:
EFFECT + ClassName input1 input2 -> output1 output2 start end
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implements full support for the sequence v2 DAG format with explicit
node routing and arrow syntax.
New features:
- timeline-format.js module for parsing/serializing v2 format
- NODE declarations with typed buffers (u8x4_norm, f32x4, etc.)
- Arrow syntax for effect routing: input1 input2 -> output1 output2
- Buffer chain visualization in properties panel and tooltips
- Node editor modal for adding/deleting node declarations
- Validation for undeclared node references (when NODEs explicit)
- Backward compatible with auto-inferred nodes
Files added:
- tools/timeline_editor/timeline-format.js (214 lines)
- tools/timeline_editor/test_format.html (automated tests)
- workspaces/test/timeline_v2_test.seq (test file with NODE declarations)
Files modified:
- tools/timeline_editor/index.html (~40 changes for v2 support)
All success criteria met. Round-trip tested with existing timelines.
handoff(Claude): Timeline editor now fully supports v2 format with
explicit node routing, NODE declarations, and buffer chain visualization.
Parser handles both explicit NODE declarations and auto-inferred nodes.
Validation only runs when explicit NODEs exist. Ready for production use.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Switches miniaudio from conservative to low_latency performance profile
to fix irregular beat timing on macOS. Conservative profile caused
uneven callback intervals, desynchronizing playback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Change draw call from 21 to 3 vertices (fullscreen triangle)
- Replace broken folding-based SDF with IQ's atan-based regular polygon formula
- Simplify test timeline to render Heptagon directly to sink
- Reduce heptagon radius from 0.5 to 0.3 for better visibility
The effect was not visible due to incorrect vertex count and broken SDF
returning negative values everywhere (showing only fill color).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Hybrid3D was calling Renderer3D::render() which creates its own command
encoder, bypassing the sequence system. Now uses renderer_.draw() with
the passed encoder.
Also adds texture blit support for RotatingCube compositing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fix shader struct to match C++ (add strength_audio, stretch)
- Increase default blur strength to 8.0 for visibility
- Add blur effect to test sequence for validation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- test_demo now uses workspaces/test/{timeline.seq,music.track}
- Removed tools/test_demo.{seq,track} (no longer used)
- Updated docs to reference workspace files
- Changes to workspaces/test/timeline.seq now trigger rebuild
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Prevents compilation errors when multiple sequences share the same name.
Compiler now appends _{index}_Sequence for unique class names.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Remove #if !defined(DEMO_CROSS_COMPILE_WIN32) guards
- platform.h included transitively handles depthSlice compatibility
- Regenerate timeline.cc with cleaned output
- Tests: 34/34 passing
|
|
- Remove tools/seq_compiler.cc (replaced by seq_compiler.py)
- Remove C++ seq_compiler build target from cmake/DemoTools.cmake
- Update documentation to remove Gantt chart mentions
- Keep seq_compiler.py (active Python compiler)
- All tests passing (34/34)
|
|
Remove END_DEMO keyword from timeline format. Demo duration now
calculated from max effect end time across all sequences. Sort
sequences by start time at compile time for deterministic ordering.
Changes:
- seq_compiler.py: Auto-calculate duration, sort sequences
- seq_compiler.cc: Remove END_DEMO parsing, sort by start time
- workspaces/test/timeline.seq: Remove END_DEMO directive
- Generated timeline.cc: Duration now 40.0f (was hardcoded)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
