| Age | Commit message (Collapse) | Author |
|---|
|
|
|
JS synthesizer with replica oscillator bank and STFT cache complete.
Ready for Phase 3 (editing UI).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Phase 2 - JS Synthesizer:
- Created mq_synth.js with replica oscillator bank
- Bezier curve evaluation (cubic De Casteljau algorithm)
- Replica synthesis: frequency spread, amplitude decay, phase jitter
- PCM buffer generation from extracted MQ partials
- Normalization to prevent clipping
- Key '1' plays synthesized audio, key '2' plays original
- Playback comparison with animated playhead
STFT Cache Optimization:
- Created STFTCache class in fft.js for pre-computed windowed FFT frames
- Clean interface: getFFT(t), getMagnitudeDB(t, freq), setHopSize()
- Pre-computes all frames on WAV load (eliminates redundant FFT calls)
- Dynamic cache update when hop size changes
- Shared across spectrogram, tooltip, and mini-spectrum viewer
- Significant performance improvement
Mini-Spectrum Viewer:
- Bottom-right overlay (200x100) matching spectral_editor style
- Real-time FFT display at playhead or mouse position
- 100-bar visualization with cyan-to-yellow gradient
- Updates during playback or mouse hover
Files:
- tools/mq_editor/mq_synth.js (new)
- tools/mq_editor/fft.js (STFTCache class added)
- tools/mq_editor/index.html (synthesis playback, cache integration)
- tools/mq_editor/viewer.js (cache-based rendering, spectrum viewer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Completed MQ extraction and visualization with improved tracking.
Ready for Phase 2 (JS synthesizer).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Phase 1 deliverables complete:
- MQ extraction with improved tracking
- Spectrogram visualization with zoom/scroll
- Original WAV playback with playhead
- Ready for Phase 2 (JS synthesizer)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Tracking improvements:
- Frequency-dependent threshold (5% of freq, min 20 Hz)
- Candidate system requiring 3-frame persistence before birth
- Extended death tolerance (5 frames) for robust trajectories
- Minimum 10-frame length filter for valid partials
- Result: cleaner, less scattered partial trajectories
Audio playback:
- Web Audio API integration for original WAV playback
- Play/Stop buttons with proper state management
- Animated red playhead bar during playback
- Keyboard shortcuts: '2' plays original, '1' reserved for synthesis
Visualization:
- Power law (gamma=0.3) for improved spectrogram contrast
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fixed FFT to 1024 bins for 31.25 Hz resolution (better bass analysis)
- Refactored view state to zoom_factor + t_center for cleaner pan/zoom
- Mousewheel scrolls horizontally, shift+mousewheel zooms (respects deltaX/Y)
- Spectrogram bins now fill complete time/freq buckets at all zoom levels
- Extended dB range to -80→0 dB (80 dB) for better high-amplitude granularity
- Added real-time intensity tooltip in dB
- 50% alpha on spectrogram to reduce clutter over partial trajectories
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implement McAulay-Quatieri sinusoidal analysis tool for audio compression.
New files:
- doc/SPECTRAL_BRUSH_2.md: Complete design doc (MQ algorithm, data format, synthesis, roadmap)
- tools/mq_editor/index.html: Web UI (file loader, params, canvas)
- tools/mq_editor/fft.js: Radix-2 Cooley-Tukey FFT (from spectral_editor)
- tools/mq_editor/mq_extract.js: MQ algorithm (peak detection, tracking, bezier fitting)
- tools/mq_editor/viewer.js: Visualization (spectrogram, partials, zoom, axes)
- tools/mq_editor/README.md: Usage and implementation status
Features:
- Load WAV → extract sinusoidal partials → fit cubic bezier curves
- Time-frequency spectrogram with hot colormap (0-16 kHz)
- Horizontal zoom (mousewheel) around mouse position
- Axis ticks with labels (time: seconds, freq: Hz/kHz)
- Mouse tooltip showing time/frequency coordinates
- Real-time adjustable MQ parameters (FFT size, hop, threshold)
Algorithm:
- STFT with Hann windows (2048 FFT, 512 hop)
- Peak detection with parabolic interpolation
- Birth/death/continuation tracking (50 Hz tolerance)
- Cubic bezier fitting (4 control points per trajectory)
Next: Phase 2 (JS synthesizer for audio preview)
handoff(Claude): MQ editor Phase 1 complete. Ready for synthesis implementation.
|
|
Replace _pad with noise field in UniformsSequenceParams, providing
effects with non-deterministic random values [0..1] updated each frame.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Centralized uniforms_buffer_ initialization and updates to Effect base class:
- init_uniforms_buffer() now automatic in Effect::Effect()
- uniforms_buffer_.update() now automatic in dispatch_render()
- Removed redundant calls from all effect subclasses
- Updated effect.h comments to reflect automatic behavior
- Updated EFFECT_WORKFLOW.md templates
Benefits:
- 16 lines removed from effect implementations
- Consistent pattern enforced at compile time
- Reduced boilerplate for new effects
Tests: 34/34 passing
handoff(Claude): Effect base class now handles uniforms automatically
|
|
Adds audio intensity visualization for debugging audio-visual sync.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Introduces WGPUResource template with automatic release on destruction.
Reduces boilerplate in effect destructors and prevents resource leaks.
- set() for one-time initialization
- replace() for per-frame recreation
- Field ordering documented for dependency management
Converted 3 effects (Heptagon, Flash, Passthrough) and Effect base class.
All tests pass (34/34).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Eliminate ~100 lines of duplicated code across effect subclasses by moving common resource initialization to the base Effect class. Most effects repeatedly created uniforms buffers, samplers, and dummy textures with identical configurations.
Changes:
- Add shared members to Effect: uniforms_buffer_, sampler_, dummy_texture_*
- Add helpers: init_uniforms_buffer(), create_*_sampler(), create_dummy_scene_texture()
- Add gpu_create_*_sampler() and gpu_create_dummy_scene_texture() to gpu.h
- Move HEADLESS_RETURN_IF_NULL to Effect constructor
- Update 7 effects to use base class helpers (Flash, Heptagon, Passthrough, Placeholder, GaussianBlur, Particles, PeakMeter)
Benefits: Improved consistency, easier maintenance, reduced binary size.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
When starting from a clean tree (where `src/generated/` does not exist), the build would fail with a "file not found" error for `generated/timeline.h`. This was due to an incorrect dependency graph where `gpu.cc` was compiled before its required header was generated.
This commit fixes the issue by making the dependency explicit:
- Modified `tools/seq_compiler.py` to explicitly generate `timeline.h` alongside `timeline.cc`.
- Updated `cmake/DemoCodegen.cmake` to declare both files as `OUTPUT`s of the timeline compilation step.
- Added a direct dependency from the `gpu` library target to the `generate_timeline` custom target in `cmake/DemoLibraries.cmake`.
- Refactored the generated file paths in `DemoCodegen.cmake` into a single `GENERATED_CODE` variable for improved clarity and future-proofing.
|
|
Removed redundant and obvious comments from 7 GPU headers:
- post_process_helper.h: binding comments, stale UniformsSequenceParams note
- shader_composer.h: verbose Compose/VerifyIncludes descriptions
- uniform_helper.h: obvious method comments
- effect.h: redundant render/resize comments
- gpu.h: verbose struct/header comments, GPU perf placeholder
- sdf_effect.h: obvious method comments
- sequence.h: duplicate header, obvious API comments
Kept only non-obvious context (binding conventions, headless mode notes).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Removed outdated comments referencing old naming conventions and non-existent functions.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
Refactors the timeline code generator (`tools/seq_compiler.py`) to use the `gpu_init_color_attachment` helper function from `src/gpu/gpu.h`.
This change removes platform-specific `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives from the generated C++ files (`timeline.cc`, `test_timeline.cc`) and centralizes the platform-aware initialization logic within the existing helper function.
The generated code is now cleaner and easier to maintain. Both native macOS and Windows cross-compilation builds are confirmed to be successful.
|
|
This commit fixes several issues that caused the Windows cross-compilation build (`scripts/build_win.sh`) to fail.
The root causes were platform-specific API differences in the wgpu-native library and incorrect dependency tracking in the CMake build system for generated code.
Changes:
- **`tools/seq_compiler.py`**: The timeline generator now wraps `depthSlice` assignments in `#if !defined(DEMO_CROSS_COMPILE_WIN32)` directives to handle API differences in `WGPURenderPassColorAttachment`.
- **`src/gpu/gpu.h`**: The `gpu_init_color_attachment` helper is now platform-aware, using a preprocessor guard for the `depthSlice` member.
- **`src/effects/*.cc`**: All effects are updated to use the new platform-aware helper or have explicit guards for `depthSlice`. Also, replaced `WGPUTexelCopyTextureInfo` with the cross-platform alias `GpuTextureCopyInfo` in `rotating_cube_effect.cc`.
- **`cmake`**: Added `tools/seq_compiler.py` as an explicit dependency to the `generate_timeline` and `generate_test_demo_timeline` custom commands. This ensures that changes to the generator script correctly trigger a rebuild of the generated C++ files.
- **`scripts/build_win.sh`**: Removed the erroneous attempt to build the `seq_compiler.py` script as a native executable.
With these changes, the Windows cross-compilation build now completes successfully.
|
|
Refactors the SceneLoader class to a namespace. Since it only contained a single static utility function, a namespace is a more appropriate and idiomatic C++ construct. No functional changes.
|
|
|
|
Converts all static_cast<>, reinterpret_cast<> to C-style casts
per CODING_STYLE.md guidelines.
- Modified 12 files across gpu, 3d, util, tests, and tools
- All builds passing, 34/34 tests passing
- No functional changes, pure style cleanup
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Move platform-specific type definitions to gpu.h and establish coding rule
that platform ifdefs must be confined to gpu/platform layers.
- gpu.h: add GpuTextureCopyInfo, GpuTextureDataLayout type aliases
- effect.cc: use GpuTextureCopyInfo instead of platform ifdefs
- texture_manager.cc: use type aliases and label_view() helper
- CODING_STYLE.md: add platform-specific code section with rule
Tests: 34/34 passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Effects now accept start/end time parameters and automatically passthrough
when inactive. Implements buffer chain integrity via compile-time validation.
- Effect base class: dispatch_render() checks time bounds, auto-passthroughs
1:1 input/output effects outside [start, end] interval
- seq_compiler.py: validates producer/consumer lifespan constraints for
multi-output effects, adds --validate flag, always validates before codegen
- Updated all 9 effect classes and test fixtures to pass start/end times
- check_all.sh: includes timeline validation step
- Tests: 34/34 passing, demo runs successfully
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
|
|
|
|
- Removed unused v1 shader declarations (13 variables)
- Removed _v2 suffix from active shader names
- Moved shaders.{h,cc} from src/gpu to src/effects
- Updated all includes and build references
- All tests pass (34/34)
handoff(Claude): Cleaned up shader management, tests passing
|
|
Drastically reduce documentation verbosity while retaining essential info.
README.md (152→82 lines):
- Consolidated features into organized sections
- Single concise v2 format example
- Removed redundant explanations
- Quick start section upfront
ROADMAP.md (685→60 lines):
- Completed features: simple bullet list
- Future work: brief descriptions only
- Removed verbose implementation details
- Removed outdated sections
Net reduction: -851 lines (-90%)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updates all documentation to reflect Sequence V2 format support:
Timeline Editor (tools/timeline_editor/):
- README.md: Updated features list, file format examples with NODE
declarations and arrow syntax, usage instructions for node editing
- ROADMAP.md: Added completed item 1.0 "Sequence V2 Format Support"
Core Documentation (doc/):
- HOWTO.md: Updated timeline example to use v2 arrow syntax, added
NODE/buffer chain features to visual editor description
- SEQUENCE.md: Marked timeline editor graph visualization as completed
All examples now show v2 format:
EFFECT + ClassName input1 input2 -> output1 output2 start end
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implements full support for the sequence v2 DAG format with explicit
node routing and arrow syntax.
New features:
- timeline-format.js module for parsing/serializing v2 format
- NODE declarations with typed buffers (u8x4_norm, f32x4, etc.)
- Arrow syntax for effect routing: input1 input2 -> output1 output2
- Buffer chain visualization in properties panel and tooltips
- Node editor modal for adding/deleting node declarations
- Validation for undeclared node references (when NODEs explicit)
- Backward compatible with auto-inferred nodes
Files added:
- tools/timeline_editor/timeline-format.js (214 lines)
- tools/timeline_editor/test_format.html (automated tests)
- workspaces/test/timeline_v2_test.seq (test file with NODE declarations)
Files modified:
- tools/timeline_editor/index.html (~40 changes for v2 support)
All success criteria met. Round-trip tested with existing timelines.
handoff(Claude): Timeline editor now fully supports v2 format with
explicit node routing, NODE declarations, and buffer chain visualization.
Parser handles both explicit NODE declarations and auto-inferred nodes.
Validation only runs when explicit NODEs exist. Ready for production use.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Switches miniaudio from conservative to low_latency performance profile
to fix irregular beat timing on macOS. Conservative profile caused
uneven callback intervals, desynchronizing playback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Change draw call from 21 to 3 vertices (fullscreen triangle)
- Replace broken folding-based SDF with IQ's atan-based regular polygon formula
- Simplify test timeline to render Heptagon directly to sink
- Reduce heptagon radius from 0.5 to 0.3 for better visibility
The effect was not visible due to incorrect vertex count and broken SDF
returning negative values everywhere (showing only fill color).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Hybrid3D was calling Renderer3D::render() which creates its own command
encoder, bypassing the sequence system. Now uses renderer_.draw() with
the passed encoder.
Also adds texture blit support for RotatingCube compositing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fix shader struct to match C++ (add strength_audio, stretch)
- Increase default blur strength to 8.0 for visibility
- Add blur effect to test sequence for validation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- test_demo now uses workspaces/test/{timeline.seq,music.track}
- Removed tools/test_demo.{seq,track} (no longer used)
- Updated docs to reference workspace files
- Changes to workspaces/test/timeline.seq now trigger rebuild
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Prevents compilation errors when multiple sequences share the same name.
Compiler now appends _{index}_Sequence for unique class names.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Remove #if !defined(DEMO_CROSS_COMPILE_WIN32) guards
- platform.h included transitively handles depthSlice compatibility
- Regenerate timeline.cc with cleaned output
- Tests: 34/34 passing
|
|
- Remove tools/seq_compiler.cc (replaced by seq_compiler.py)
- Remove C++ seq_compiler build target from cmake/DemoTools.cmake
- Update documentation to remove Gantt chart mentions
- Keep seq_compiler.py (active Python compiler)
- All tests passing (34/34)
|
|
Remove END_DEMO keyword from timeline format. Demo duration now
calculated from max effect end time across all sequences. Sort
sequences by start time at compile time for deterministic ordering.
Changes:
- seq_compiler.py: Auto-calculate duration, sort sequences
- seq_compiler.cc: Remove END_DEMO parsing, sort by start time
- workspaces/test/timeline.seq: Remove END_DEMO directive
- Generated timeline.cc: Duration now 40.0f (was hardcoded)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Update effect class definitions in headers and implementations to match
timeline.seq naming convention. All tests passing (34/34).
Classes renamed:
- PassthroughEffect → Passthrough
- GaussianBlurEffect → GaussianBlur
- PlaceholderEffect → Placeholder
- HeptagonEffect → Heptagon
- ParticlesEffect → Particles
- RotatingCubeEffect → RotatingCube
- Hybrid3DEffect → Hybrid3D
- FlashEffect → Flash
- PeakMeterEffect → PeakMeter
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Remove 530 lines of redundant content, archive dated docs, compact CNN
training sections, fix inconsistencies (effect count, test status).
Improves maintainability and reduces context load for AI agents.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Ports PeakMeterEffect to v2 Effect system with proper DAG routing.
Red horizontal bar overlay displays audio_intensity for visual debugging
of audio-visual synchronization.
Changes:
- New: src/effects/peak_meter_effect.{h,cc} - v2 implementation
- Timeline: FlashEffect -> flash_out -> PeakMeterEffect -> sink
- Build: Added to COMMON_GPU_EFFECTS and demo_effects.h
- Test: Added to test_demo_effects.cc (9/9 effects pass)
- Cleanup: Removed old disabled PeakMeterEffect code from test_demo.cc
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- seq_compiler.py: Calculate beat_phase from beat_time (was hardcoded 0.0f)
- Refactor: Replace CommonPostProcessUniforms with UniformsSequenceParams
- Remove duplicate struct definition in post_process_helper.h
- Update all CNN effects and tests to use unified uniform struct
- Fixes FlashEffect showing solid white instead of flashing to beat
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
RotatingCubeEffect was failing with "Unable to find entry point 'vs_main'"
because shader was hardcoded to empty string. Load from ASSET_SHADER_ROTATING_CUBE_V2.
Fixes DemoEffectsTest (34/34 tests now pass).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
WGPU_DEPTH_SLICE_UNDEFINED is defined unconditionally via platform.h
(native headers or Win32 macro). Remove redundant conditional compilation
from 7 effects and gpu.h helper.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
FlashEffect requires flash.wgsl shader in main workspace.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- FlashEffect: Beat-synchronized white flash using ShaderComposer
- Loads shader from assets (flash.wgsl) with sequence_uniforms include
- Uses pow(1.0 - beat_phase, 4.0) for sharp flash at beat start
- Updated test_demo.seq to use FlashEffect (was HeptagonEffect)
- Added FlashEffect to test suite (test_demo_effects.cc)
- Made cnn_test conditional on main workspace (fixes build error)
- Flash intensity: 1.0 at beat start, fades to 0.0 by beat end
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
