| Age | Commit message (Collapse) | Author |
|---|
|
Added note for future enhancement: mix salient + random samples.
Rationale:
- Salient point detection focuses on edges/corners
- Random samples improve generalization across entire image
- Prevents overfitting to only high-gradient regions
Proposed implementation:
- Default: 90% salient points, 10% random samples
- Configurable: --random-sample-percent parameter
- Example: 64 patches = 58 salient + 6 random
Location: train_cnn_v2.py
- TODO in _detect_salient_points() method
- TODO in argument parser
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Salient point detection on original images with patch extraction.
Changes:
- Added PatchDataset class (harris/fast/shi-tomasi/gradient detectors)
- Detects salient points on ORIGINAL images (no resize)
- Extracts 32×32 patches around salient points
- Default: 64 patches/image, harris detector
- Batch size: 16 (512 patches per batch)
Training modes:
1. Patch-based (default): --patch-size 32 --patches-per-image 64 --detector harris
2. Full-image (option): --full-image --image-size 256
Benefits:
- Focuses training on interesting regions
- Handles variable image sizes naturally
- Matches CNN v1 workflow
- Better convergence with limited data (8 images → 512 patches)
Script updated:
- train_cnn_v2_full.sh: Patch-based by default
- Configuration exposed for easy switching
Example:
./scripts/train_cnn_v2_full.sh # Patch-based
# Edit script: uncomment FULL_IMAGE for resize mode
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training script now resizes all images to fixed size before batching.
Issue: RuntimeError when batching variable-sized images
- Images had different dimensions (376x626 vs 344x361)
- PyTorch DataLoader requires uniform tensor sizes for batching
Solution:
- Add --image-size parameter (default: 256)
- Resize all images to target_size using LANCZOS interpolation
- Preserves aspect ratio independent training
Changes:
- train_cnn_v2.py: ImagePairDataset now resizes to fixed dimensions
- train_cnn_v2_full.sh: Added IMAGE_SIZE=256 configuration
Tested: 8 image pairs, variable sizes → uniform 256×256 batches
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated project status to reflect CNN v2 implementation completion.
Changes:
- TODO.md: Marked Task #85 as [READY FOR TRAINING]
- All 5 phases complete
- Infrastructure ready for model training and integration
- PROJECT_CONTEXT.md: Updated Effects section
- Added CNN v2 parametric static features reference
- Added CNN_V2.md to technical documentation list
Status summary:
✅ Phase 1: Static features shader (8×f16 packed, 3 mip levels)
✅ Phase 2: C++ effect class (CNNv2Effect)
✅ Phase 3: Training pipeline (train_cnn_v2.py, export)
✅ Phase 4: Validation tooling (validate_cnn_v2.sh)
✅ Phase 5: Render pipeline (compute passes, bind groups)
Next steps: Train model, generate layer shaders, demo integration
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Complete multi-pass compute execution for CNNv2Effect.
Implementation:
- Layer texture creation (ping-pong buffers for intermediate results)
- Static features compute pipeline with bind group layout
- Bind group creation with 5 bindings (input mips + depth + output)
- compute() override for multi-pass execution
- Static features pass with proper workgroup dispatch
Architecture:
- Static features: 8×f16 packed as 4×u32 (RGBD + UV + sin + bias)
- Layer buffers: 2×RGBA32Uint textures (8 channels f16 each)
- Input mips: 3 levels (0, 1, 2) for multi-scale features
- Workgroup size: 8×8 threads
Status:
- Static features compute pass functional
- Layer pipeline infrastructure ready
- All 36/36 tests passing
Next: Layer shader integration, multi-layer execution
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Infrastructure for enhanced CNN post-processing with 7D feature input.
Phase 1: Shaders
- Static features compute (RGBD + UV + sin10_x + bias → 8×f16)
- Layer template (convolution skeleton, packing/unpacking)
- 3 mip level support for multi-scale features
Phase 2: C++ Effect
- CNNv2Effect class (multi-pass architecture)
- Texture management (static features, layer buffers)
- Build integration (CMakeLists, assets, tests)
Phase 3: Training Pipeline
- train_cnn_v2.py: PyTorch model with static feature concatenation
- export_cnn_v2_shader.py: f32→f16 quantization, WGSL generation
- Configurable architecture (kernels, channels)
Phase 4: Validation
- validate_cnn_v2.sh: End-to-end pipeline
- Checkpoint → shaders → build → test images
Tests: 36/36 passing
Next: Complete render pipeline implementation (bind groups, multi-pass)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Design document for CNN v2 with enhanced feature inputs:
- 7D static features: RGBD + UV + sin encoding + bias
- Per-layer configurable kernels (1×1, 3×3, 5×5)
- Float16 weight storage (~6.4 KB vs 3.2 KB)
- Multi-pass architecture with static feature compute
Implementation plan:
1. Static features compute shader (RGBD + UV + sin + bias)
2. C++ effect class (CNNv2Effect)
3. Training pipeline (train_cnn_v2.py, export_cnn_v2_shader.py)
4. Validation tooling (validate_cnn_v2.sh)
Files:
- doc/CNN_V2.md: Complete technical design (architecture, training, export)
- scripts/validate_cnn_v2.sh: End-to-end validation script
- TODO.md: Add CNN v2 as Priority 2 task
- doc/HOWTO.md: Add CNN v2 validation usage
Target: <10 KB for 64k demo constraint
handoff(Claude): CNN v2 design ready for implementation
|
|
Fixes:
- Sequence dragging with scroll offset
- Double-click collapse/expand (DOM recreation issue)
- Collapsed sequence dragging (removed stopPropagation)
Features:
- Quantize dropdown (Off, 1/32→1 beat) replaces snap-to-beat checkbox
- Works in both beat and second display modes
- Hotkeys: 0=Off, 1=1beat, 2=1/2, 3=1/4, 4=1/8, 5=1/16, 6=1/32
- Separate "Show Beats" toggle for display vs snap behavior
Technical:
- Track dragMoved state to avoid unnecessary DOM recreation
- Preserve dblclick detection by deferring renderTimeline()
- Quantization applies to sequences and effects uniformly
handoff(Claude): timeline editor quantize + drag fixes complete
|
|
Remove test snippets (a/b) that belong in test workspace only.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
|
|
§
|
|
Reduced file size from 1899 to 823 lines (57% reduction) while improving
maintainability and user experience.
CSS improvements:
- Added CSS variables for colors, spacing, and border radius
- Consolidated duplicate button/input/label styles
- Added missing .zoom-controls class definition
- Reduced CSS from ~510 to ~100 lines
JavaScript refactoring:
- Centralized global state into single `state` object
- Created `dom` object to cache all element references
- Removed all inline event handlers (onclick, oninput)
- Replaced with proper addEventListener pattern
- Fixed missing playbackControls reference (bug fix)
- Reduced JS from ~1320 to ~660 lines
UX improvements:
- Playback indicators (red bars) now always visible, start at 0s
- During playback, highlight current sequence green (no expand/collapse reflow)
- Smooth scrolling follows playback indicator (10% interpolation at 40% viewport)
- Moved "Show Beats" checkbox inline with BPM controls
- Fixed playback controls layout (time left of button, proper gap/alignment)
- Error messages now logged to console as well as UI
No functional regressions - all features work identically.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Add red bar playback indicator on waveform (synced with timeline)
- Fix playback continuation after double-click seek (async/await)
- Improve stopPlayback() to preserve jump positions
- Add error handling to startPlayback()
- Update waveform click-to-seek to match double-click behavior
- Sync waveform indicator scroll with timeline
- Display time in both seconds and beats on seek
- Update documentation with new features
handoff(Claude): Timeline editor now has dual playback indicators and seamless seeking.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Update test status: 34/36 (94%) → 36/36 (100%)
- Add timeline editor to PROJECT_CONTEXT.md
- Fix broken BEAT_TIMING_SUMMARY.md references → doc/BEAT_TIMING.md
- Consolidate duplicate entries in README.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
- Add render/scene_query_mode to known placeholders in VerifyIncludes
- Remove warning for duplicate auxiliary texture registration (valid for multiple CNNEffect stacks)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Audio playback controls (play/pause, spacebar shortcut)
- Red playback indicator with auto-scroll (middle third viewport)
- Auto-expand active sequence during playback, collapse previous
- Click waveform to seek
- Sticky header: waveform + timeline ticks stay at top
- Sequences confined to separate scrollable container below header
- Document known bugs: zoom sync, positioning, reflow issues
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Timeline editor now stores all times internally as beats (not seconds),
aligning with the project's beat-based timing system. Added BPM slider
for tempo control. Serializes to beats (default format) and displays
beats primarily with seconds in tooltips.
Changes:
- parseTime() returns beats (converts 's' suffix to beats)
- serializeSeqFile() outputs beats (bare numbers)
- Timeline markers show beats (4-beat/bar increments)
- BPM slider (60-200) for tempo editing
- Snap-to-beat rounds to nearest beat
- Audio waveform aligned to beats
- showBeats enabled by default
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- wav_dump_backend: Fix data_size double-counting channels (line 126)
- test_wav_dump: Assert on data_size validation instead of warning
- main: Add SIGINT/SIGTERM handlers to finalize WAV on Ctrl+C
- Guard signal handler code with DEMO_HEADLESS ifdef
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated all affected documentation files:
- UNIFORM_BUFFER_GUIDELINES.md: New CommonUniforms example
- ARCHITECTURE.md: Beat-based timing section
- EFFECT_WORKFLOW.md: Available uniforms reference
- CONTRIBUTING.md: Updated uniform buffer checklist
handoff(Claude): Beat-based timing system fully implemented and documented.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
The custom render() signature didn't match PostProcessEffect::render(),
so it was never called. The base class method was used instead, which
didn't update uniforms with the peak value.
Fixed by:
- Using correct override signature: render(pass, uniforms)
- Calling PostProcessEffect::render() to handle standard rendering
- Removed unused custom parameters (time, beat, peak_value, aspect_ratio)
- Added override keyword to update_bind_group()
Peak meter bar now displays correctly with audio_intensity.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Instead of duplicating the uniform structure definition, PeakMeterEffect
now uses ShaderComposer to include the common_uniforms snippet, ensuring
the struct definition always matches the canonical version.
Changes:
- Added shader_composer.h include
- Use ShaderComposer::Get().Compose() to prepend common_uniforms
- Changed 'Uniforms' → 'CommonUniforms' in shader
- Removed duplicate struct definition
This ensures consistency across all shaders and eliminates potential
drift between duplicate definitions.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
The embedded shader in PeakMeterEffect was using the old uniform
structure with _pad0/_pad1 instead of the new beat_time/beat_phase
fields, causing the peak meter bar to not display correctly.
Updated to match CommonPostProcessUniforms structure:
- Removed _pad0, _pad1
- Added beat_time, beat_phase
- Moved _pad to end
Peak meter visualization now works correctly in test_demo.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Added comprehensive doc/BEAT_TIMING.md user guide
- Updated BEAT_TIMING_SUMMARY.md with verification results
- Updated PROJECT_CONTEXT.md to highlight timing system
- Updated README.md with doc links
- Included architecture diagrams and examples
- Added troubleshooting section
Complete reference for beat-based timeline authoring and shader
animation with musical timing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fixed particle_spray_compute.wgsl (uniforms.beat → uniforms.beat_phase)
- Fixed ellipse.wgsl (uniforms.beat → uniforms.beat_phase)
- Applied to all workspace and asset directories
Resolves shader compilation error on demo64k startup.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
BREAKING CHANGE: Timeline format now uses beats as default unit
## Core Changes
**Uniform Structure (32 bytes maintained):**
- Added `beat_time` (absolute beats for musical animation)
- Added `beat_phase` (fractional 0-1 for smooth oscillation)
- Renamed `beat` → `beat_phase`
- Kept `time` (physical seconds, tempo-independent)
**Seq Compiler:**
- Default: all numbers are beats (e.g., `5`, `16.5`)
- Explicit seconds: `2.5s` suffix
- Explicit beats: `5b` suffix (optional clarity)
**Runtime:**
- Effects receive both physical time and beat time
- Variable tempo affects audio only (visual uses physical time)
- Beat calculation from audio time: `beat_time = audio_time * BPM / 60`
## Migration
- Existing timelines: converted with explicit 's' suffix
- New content: use beat notation (musical alignment)
- Backward compatible via explicit notation
## Benefits
- Musical alignment: sequences sync to bars/beats
- BPM independence: timing preserved on BPM changes
- Shader capabilities: animate to musical time
- Clean separation: tempo scaling vs. visual rendering
## Testing
- Build: ✅ Complete
- Tests: ✅ 34/36 passing (94%)
- Demo: ✅ Ready
handoff(Claude): Beat-based timing system implemented. Variable tempo
only affects audio sample triggering. Visual effects use physical_time
(constant) and beat_time (musical). Shaders can now animate to beats.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
+misc
|
|
- Sequences start collapsed by default for better overview
- Move controls to header for vertical space savings
- Remove "Pixels per second" label (redundant with zoom %)
- Move properties panel to bottom left
- Update collapse arrows for vertical layout (▼/▲)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Sticky time markers stay visible when scrolling
- Faint vertical grid lines aligned with ticks for better alignment
- Collapsible sequences via double-click (35px collapsed state)
- Updated all references from demo.seq to timeline.seq
- Consolidated and tightened documentation
- Fixed _collapsed initialization bug
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Comprehensive analysis of single-pass CNN shader architecture:
- Full flatten (3 layers): 544 bytes/thread register pressure - NOT recommended
- Partial flatten (layers 1+2): 288 bytes/thread - marginal benefit
- Current multi-pass: Optimal for GPU occupancy and maintainability
Recommendation: Keep current 3-pass architecture.
Alternative size optimizations: weight quantization, kernel reduction.
handoff(Claude): CNN flatten analysis documented
|
|
- Fix stale comments: RGBD→RGB (not grayscale)
- Clarify shape transformations in inference
- Add CNN_BIAS_FIX_2026-02.md consolidating recent fixes
- Include regenerated weights with 5x5 kernel for layer 0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Fix bias division bug: divide by num_positions to compensate for
shader loop accumulation (affects all layers)
- train_cnn.py: Save RGBA output preserving alpha channel from input
- Add --debug-hex flag to both tools for pixel-level debugging
- Remove sRGB/linear_png debug code from cnn_test
- Regenerate weights with corrected bias export
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
SamplerCache singleton never released samplers, causing device to retain
references at shutdown. Add clear() method and call before fixture cleanup.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Release queue reference after submit in texture_readback
- Add final wgpuDevicePoll before cleanup to sync GPU work
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
Simplify coordinate initialization by generating [-1,1] range directly
instead of [0,1] then normalizing. Mathematically equivalent, clearer.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Match training forward pass: compute grayscale from original [0,1] RGB
before normalization, then normalize gray to [-1,1].
Previously computed gray from normalized [-1,1] RGB in generated shader,
creating mismatch with train.py which does:
gray = 0.2126*R + 0.7152*G + 0.0722*B # [0,1]
gray = (gray - 0.5) * 2.0 # [-1,1]
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Add cnn_conv1x1 to shader composer registration
- Add VerifyIncludes() to detect missing snippet registrations
- STRIP_ALL-protected verification warns about unregistered includes
- Fixes cnn_test runtime failure loading cnn_layer.wgsl
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add EFFECT_WORKFLOW.md to Tier 2, update Tier 3/4 references, refresh state snapshot.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Summary of fixes:
1. MainSequence: resize() before init() (effect.cc:179-180, 189-190)
2. Auxiliary textures: Register in init() using width_/height_
3. Renderer3D effects: Add initialized_ flag guard
4. RotatingCubeEffect: Fix hardcoded vec2(1280,720) → u.resolution
Audit: No other hardcoded resolutions in effects.
All 36 tests pass. Ready for handoff.
handoff(Claude): Fixed auxiliary texture initialization order bug.
Main change: resize() called before init() in MainSequence.
Added guards for Renderer3D effects. Fixed hardcoded dimensions.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Hardcoded vec2(1280.0f, 720.0f) → u.resolution
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Document the additional fix required for effects with Renderer3D members.
Explains why initialized_ flag is needed instead of ctx_.device check.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
ctx_.device exists before init() but Renderer3D not initialized yet.
Changed guard from !ctx_.device to !initialized_ flag.
Set initialized_ = true after renderer_.init() in both effects.
All 36 tests pass. Demo runs without crash.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Root cause: After swapping init/resize order, effects with Renderer3D crashed
because resize() called before init() tried to use uninitialized GPU resources.
Changes:
- Add guards in FlashCubeEffect::resize() and Hybrid3DEffect::resize() to
check ctx_.device before calling renderer_.resize()
- Remove lazy initialization remnants from CircleMaskEffect and CNNEffect
- Register auxiliary textures directly in init() (width_/height_ already set)
- Remove ensure_texture() methods and texture_initialized_ flags
All 36 tests passing. Demo runs without crashes.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Documents the "half resolution" bug, root cause analysis, and
solution decision (resize before init vs lazy initialization).
Key points:
- Problem: Auxiliary textures created with default dimensions
- Root cause: init() called before resize()
- Solution: Swap order (resize → init) for 2-line fix
- Rejected: Lazy initialization (too complex, cascade effects)
Includes implementation details and guidelines for new effects.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Simpler solution than lazy initialization: effects need correct
dimensions during init() to register auxiliary textures.
Changed initialization order in MainSequence:
- resize() sets width_/height_ FIRST
- init() can then use correct dimensions
Reverted lazy initialization complexity. One-line fix.
Tests: All 36 tests passing, demo runs without error
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Prevents init/resize ordering bug and avoids unnecessary reallocation.
Changes:
- Auxiliary textures created on first use (compute/update_bind_group)
- Added ensure_texture() methods to defer registration until resize()
- Added early return in resize() if dimensions unchanged
- Removed texture registration from init() methods
Benefits:
- No reallocation on window resize if dimensions match
- Texture created with correct dimensions from start
- Memory saved if effect never renders
Tests: All 36 tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Auxiliary textures were created during init() using default dimensions
(1280x720) before resize() was called with actual window size. This
caused compute shaders to receive uniforms with correct resolution but
render to wrong-sized textures.
Changes:
- Add MainSequence::resize_auxiliary_texture() to recreate textures
- Override resize() in CircleMaskEffect to resize circle_mask texture
- Override resize() in CNNEffect to resize captured_frame texture
- Bind groups are recreated with new texture views after resize
Tests: All 36 tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
