| Age | Commit message (Collapse) | Author |
|---|
|
Binary format v2 includes mip_level in header (20 bytes, was 16).
Effect reads mip_level and passes to static features shader via uniform.
Shader samples from correct mip texture based on mip_level.
Changes:
- export_cnn_v2_weights.py: Header v2 with mip_level field
- cnn_v2_effect.h: Add StaticFeatureParams, mip_level member, params buffer
- cnn_v2_effect.cc: Read mip_level from weights, create/bind params buffer, update per-frame
- cnn_v2_static.wgsl: Accept params uniform, sample from selected mip level
Binary format v2:
- Header: 20 bytes (magic, version=2, num_layers, total_weights, mip_level)
- Backward compatible: v1 weights load with mip_level=0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training pipeline now accepts --mip-level flag (0-3) and passes to train_cnn_v2.py.
Compatible with all existing modes (train, validate, export-only).
Changes:
- Add --mip-level argument parsing (default: 0)
- Pass MIP_LEVEL to training command
- Display mip level in config output
- Update help text with examples
Usage: ./scripts/train_cnn_v2_full.sh --mip-level 1
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Export scripts now read mip_level from checkpoint config and display it.
Shader generator includes mip level in generated comments.
Changes:
- export_cnn_v2_weights.py: Read mip_level, print in config
- export_cnn_v2_shader.py: Read mip_level, pass to shader gen, add to comments
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add mip level control for p0-p3 features (0=original, 1=half, 2=quarter, 3=eighth).
Uses pyrDown/pyrUp for proper Gaussian filtering during mip generation.
Changes:
- compute_static_features(): Accept mip_level param, generate mip via cv2 pyramid
- PatchDataset/ImagePairDataset: Pass mip_level to feature computation
- CLI: Add --mip-level arg with choices [0,1,2,3]
- Save mip_level in checkpoint config for tracking
- Doc updates: HOWTO.md and CNN_V2.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Refactoring:
- Extract FULLSCREEN_QUAD_VS shader (reused in mipmap, display, layer viz)
- Add helper methods: getDimensions(), setVideoControlsEnabled()
- Add section headers and improve code organization (~40 lines saved)
- Move Mip Level selector to bottom of left sidebar
- Remove "Features (p0-p3)" panel header
Features:
- Add video loop support (continuous playback)
Documentation:
- Update CNN_V2_WEB_TOOL.md with latest changes
- Document refactoring benefits and code organization
- Update UI layout section with current structure
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add dropdown menu in left panel to select mip levels 0-2 for parametric features (p0-p3/RGBD). Uses trilinear filtering for smooth downsampling at higher mip levels.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Layer 0 now uses clamp [0,1] in both training and inference (was using ReLU in shaders).
- index.html: Add is_layer_0 flag to LayerParams, handle Layer 0 separately
- export_cnn_v2_shader.py: Generate correct activation for Layer 0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Change Depth control from number input to slider (0-1 range)
- Move video controls to floating overlay at top of canvas
- Remove View mode indicator from header (shortcuts still work)
- Remove scrollbar from Layer Visualization panel
- Fix layer viz flickering during video playback
- Fix video controls responsiveness during playback
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Features:
- Video file support (MP4, WebM, etc.) via drag-and-drop
- Play/Pause button with non-realtime playback (drops frames if CNN slow)
- Frame-by-frame navigation (◄/► step buttons)
- Unified image/video processing through same CNN pipeline
- Audio muted (video frames only)
Optimizations:
- Layer visualization updates only on pause/seek (~5-10ms saved per frame)
Architecture:
- copyExternalImageToTexture() works with both ImageBitmap and HTMLVideoElement
- Video loading: wait for metadata → seek to frame 0 → wait for readyState≥2 (decoded)
- Playback loop: requestAnimationFrame with isProcessing guard prevents overlapping inference
- Controls always visible, disabled for images
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
UI Changes:
- Three-panel layout: left (weights), center (canvas), right (activations)
- Left sidebar: clickable weights drop zone, weights info, kernel visualization
- Right sidebar: 4 small activation views + large 4× zoom view
- Controls moved to header (inline with title)
Weights Visualization:
- Dedicated panel in left sidebar with layer buttons
- 1 pixel per weight (was 20px)
- All input channels horizontal, output channels stacked vertically
- Renders to separate canvas (not in activation grid)
Activation Viewer:
- 4 channels in horizontal row (was 2×2 grid)
- Mouse-driven zoom view below (32×32 area at 4× magnification)
- Zoom shows all 4 channels in 2×2 quadrant layout
- Removed activations/weights mode toggle
State Preservation:
- Blend changes preserve selected layer/channel
- Fixed activation view reset bug
Documentation:
- Updated README with new layout and feature descriptions
- Marked implemented features (weights viz, layer viewer)
- Updated size estimates (~22 KB total)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Fixed validation error where staticTex was used for both storage write
(in static compute pass) and texture read (in CNN bind group) within
same command encoder. Now uses layerTextures[0] for reading, which is
the copy destination and safe for read-only access.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Align layer naming with codebase: Layer 0/1/2 (not Layer 1/2/3)
- Split static features: Static 0-3 (p0-p3) and Static 4-7 (uv,sin,bias)
- Fix Layer 2 not appearing: removed isOutput filter from layerOutputs
- Fix canvas context switching: force clear before recreation
- Disable static buttons in weights mode
- Add ASCII pipeline diagram to CNN_V2.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Added SHADER_SNIPPET_A and SHADER_SNIPPET_B entries to test assets
config to resolve missing AssetId compile error in test_shader_composer.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Allows exporting weights from a checkpoint without training or validation.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Specifies sample offset (shift trigger left) and humanization (per-note timing/volume variation) for realistic playback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Fixes test_assets.cc compilation by adding missing test asset IDs and
procedural generators. Test-specific code is protected with DEMO_STRIP_ALL
to exclude from release builds.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- tracker_compiler: Sort events by time before C++ generation (required
for runtime early-exit optimization)
- tracker.cc: Add FATAL_CHECK validating sorted events at init
- Add --check mode: Validate .track file without compiling
- Add --sanitize mode: Rewrite .track with sorted events and normalized
formatting
- Fix parser: Skip indented comment lines in patterns
All audio tests passing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Converted track.md drum notation to .track format and integrated as main music.
165 BPM high-energy pattern with syncopated kicks, 16th note hi-hats, and break.
- Add workspaces/main/pop_punk_drums.track (3 patterns, 4-bar sequence)
- Add workspaces/main/track.md (notation reference)
- Update workspace.cfg to use pop_punk_drums.track
- Update BPM to 165
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated CNN_V2.md to document that:
- Model outputs 4 channels (RGBA)
- Training targets preserve alpha from target images
- Loss function compares all 4 channels
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Changed target loading from RGB to RGBA to preserve transparency.
Model learns to predict alpha channel from target image instead of
constant 1.0 padding.
Before: Target padded with alpha=1.0
After: Target uses actual alpha from image (or 1.0 if no alpha)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Changes:
- KERNEL_SIZES: Use comma-separated values (3,3,3 not 3 3 3)
- Remove --channels (no longer exists in uniform 12D→4D architecture)
- Add --num-layers parameter
- Use export_cnn_v2_weights.py (storage buffer) instead of export_cnn_v2_shader.py
- Fix duplicate export: only export in step 2 (training) or validation mode
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated comments to clarify that per-layer kernel sizes are supported.
Code already handles this correctly via LayerInfo.kernel_size field.
Changes:
- cnn_v2_effect.h: Add comment about per-layer kernel sizes
- cnn_v2_compute.wgsl: Clarify LayerParams provides per-layer config
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training:
- train_cnn_v2.py: Accept --kernel-sizes as comma-separated list
- CNNv2 model: Per-layer kernel sizes (e.g., [1,3,5])
- Single value replicates across layers (e.g., "3" → [3,3,3])
Export:
- export_cnn_v2_weights.py: Backward compatible with old checkpoints
- Handles both kernel_size (old) and kernel_sizes (new) format
Documentation:
- CNN_V2.md: Updated code examples and config format
- HOWTO.md: Updated training examples to show comma-separated syntax
Binary format: Already supports per-layer kernel sizes (no changes)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
**Architecture changes:**
- Static features (8D): p0-p3 (parametric) + uv_x, uv_y, sin(10×uv_x), bias
- Input RGBD (4D): fed separately to all layers
- All layers: uniform 12D→4D (4 prev/input + 8 static → 4 output)
- Bias integrated in static features (bias=False in PyTorch)
**Weight calculations:**
- 3 layers × (12 × 3×3 × 4) = 1296 weights
- f16: 2.6 KB (vs old variable arch: ~6.4 KB)
**Updated files:**
*Training (Python):*
- train_cnn_v2.py: Uniform model, takes input_rgbd + static_features
- export_cnn_v2_weights.py: Binary export for storage buffers
- export_cnn_v2_shader.py: Per-layer shader export (debugging)
*Shaders (WGSL):*
- cnn_v2_static.wgsl: p0-p3 parametric features (mips/gradients)
- cnn_v2_compute.wgsl: 12D input, 4D output, vec4 packing
*Tools:*
- HTML tool (cnn_v2_test): Updated for 12D→4D, layer visualization
*Docs:*
- CNN_V2.md: Updated architecture, training, validation sections
- HOWTO.md: Reference HTML tool for validation
*Removed:*
- validate_cnn_v2.sh: Obsolete (used CNN v1 tool)
All code consistent with bias=False (bias in static features as 1.0).
handoff(Claude): CNN v2 architecture finalized and documented
|
|
- Rename 'Static (L0)' → 'Static' (clearer, less confusing)
- Update channel labels: 'R/G/B/D' → 'Ch0 (R)/Ch1 (G)/Ch2 (B)/Ch3 (D)'
- Add 'Layer' prefix in weights table for consistency
- Document layer indexing: Static + Layer 1,2,3... (UI) ↔ weights.layers[0,1,2...]
- Add explanatory notes about 7D input and 4-of-8 channel display
- Create doc/CNN_V2_BINARY_FORMAT.md with complete .bin specification
- Cross-reference spec in CNN_V2.md and CNN_V2_WEB_TOOL.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Features:
- Right sidebar with Layer Visualization (top) and Weights Info (collapsible, bottom)
- Activations mode: 4-channel grayscale views per layer (Static L0 + CNN layers)
- Weights mode: Kernel visualization with 2D canvas rendering
- Mode tabs to switch between activation and weight inspection
- Per-layer texture storage (separate from ping-pong compute buffers)
- Debug shader modes (UV gradient, raw packed data, unpacked f16)
- Comprehensive logging for diagnostics
Architecture:
- Persistent layerTextures[] for visualization (one per layer)
- Separate computeTextures[] for CNN ping-pong
- copyTextureToTexture after each layer pass
- Canvas recreation on mode switch (2D vs WebGPU context)
- Weight parsing with f16 unpacking and min/max calculation
Known Issues:
- Layer activations show black (texture data empty despite copies)
- Weight kernels not displaying (2D canvas renders not visible)
- Debug mode 10 (UV gradient) works, confirming texture access OK
- Root cause: likely GPU command ordering or texture usage flags
Documentation:
- Added doc/CNN_V2_WEB_TOOL.md with full status, architecture, debug steps
- Detailed issue tracking with investigation notes and next steps
Status: Infrastructure complete, debugging data flow issues.
handoff(Claude): Layer viz black due to empty textures despite copyTextureToTexture.
Weight viz black despite correct canvas setup. Both issues need GPU pipeline audit.
|
|
Implements single-file HTML tool for rapid CNN weight validation:
Features:
- Drag-drop PNG images (whole window) and .bin weights
- Real-time WebGPU compute pipeline (static features + N layers)
- Data-driven execution (reads layer count from binary)
- View modes: CNN output / Original / Diff (×10)
- Blend slider (0.0-1.0) for effect strength
- Console log with timestamps
- Keyboard shortcuts: SPACE (original), D (diff)
Architecture:
- Embedded WGSL shaders (static + compute + display)
- Binary parser for .bin format (header + layer info + f16 weights)
- Persistent textures for view mode switching
- Absolute weight offset calculation (header + layer info skip)
Implementation notes:
- Weight offsets in binary are relative to weights section
- JavaScript precalculates absolute offsets: headerOffsetU32 * 2 + offset
- Matches C++ shader behavior (simple get_weight without offset param)
- Ping-pong textures for multi-layer processing
TODO:
- Side panel: .bin metadata, weight statistics, validation
- Layer inspection: R/G/B/A plane split, intermediate outputs
- Activation heatmaps for debugging
Files:
- tools/cnn_v2_test/index.html (24 KB, 730 lines)
- tools/cnn_v2_test/README.md (usage guide, troubleshooting)
handoff(Claude): CNN v2 HTML testing tool complete, documented TODOs for future enhancements
|
|
Consolidate repeated timeline/resource analysis code to improve
maintainability and reduce duplication.
seq_compiler.cc changes:
- Extract timeline analysis (max time, sorting) into analyze_timeline()
- Extract sequence end calculation into get_sequence_end()
- Reduces ~45 lines of duplicate code
tracker_compiler.cc changes:
- Extract resource analysis into ResourceAnalysis struct
- Consolidate sample counting and recommendations
- Reduces ~75 lines of duplicate code
Both tools verified with successful builds.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Extract repeated logic into focused helper functions:
- ParseProceduralParams() - eliminates 2× duplicate parameter parsing
- ParseProceduralFunction() - unifies PROC() and PROC_GPU() handling
- ProcessMeshFile() - encapsulates 164-line mesh processing
- ProcessImageFile() - encapsulates image loading
Reduces 598→568 lines (-5%), improves readability, preserves behavior.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
EOF
|
|
Updated docs to reflect February 13, 2026 changes:
- doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md: Complete summary
- doc/WORKSPACE_SYSTEM.md: Current structure, workspace.cfg format
- doc/SHADER_REUSE_INVESTIGATION.md: Implementation status
- PROJECT_CONTEXT.md: Workspace and shader system updates
Key changes documented:
- src/app/ application structure
- workspaces/{music,weights,obj,shaders}/ layout
- common/shaders/ shared shader system
- Eliminated 36 duplicate shaders
- Asset packer path normalization
handoff(Claude): Documentation updated for hierarchy cleanup
|
|
Eliminates 36 duplicate shader files across workspaces.
Structure:
- common/shaders/{math,render,compute}/ - Shared utilities (20 files)
- workspaces/*/shaders/ - Workspace-specific only
Changes:
- Created common/shaders/ with math, render, compute subdirectories
- Moved 20 common shaders from workspaces to common/
- Removed duplicates from test workspace
- Updated assets.txt: ../../common/shaders/ references
- Enhanced asset_packer.cc: filesystem path normalization for ../ resolution
Implementation: Option 1 from SHADER_REUSE_INVESTIGATION.md
- Single source of truth for common code
- Workspace references via relative paths
- Path normalization in asset packer
handoff(Claude): Common shader directory implemented
|
|
Analyzed 36 duplicate common shaders across workspaces.
Documented 5 approaches with tradeoffs:
1. Shared common/ directory
2. Symbolic links
3. Build-time sync
4. Asset system extension
5. Status quo + documentation
See doc/SHADER_REUSE_INVESTIGATION.md for full analysis.
handoff(Claude): Shader reuse investigation complete
|
|
Updated asset_dirs and shader_dirs to reflect reorganization:
- Removed legacy assets/ and ../common/ references
- Added new directories: music/, weights/, obj/
- Simplified shader_dirs to just shaders/
handoff(Claude): workspace.cfg files updated
|
|
Each workspace now has a weights/ directory to store binary weight files
from CNN training (e.g., cnn_v2_weights.bin).
Changes:
- Created workspaces/{main,test}/weights/
- Moved cnn_v2_weights.bin → workspaces/main/weights/
- Updated assets.txt reference
- Updated training scripts and export tool paths
handoff(Claude): Workspace weights/ directories added
|
|
Workspace structure now:
- workspaces/{main,test}/obj/ (3D models)
- workspaces/{main,test}/shaders/ (WGSL shaders)
- workspaces/{main,test}/music/ (audio samples)
Changes:
- Moved workspaces/*/assets/music/ → workspaces/*/music/
- Updated assets.txt paths (assets/music/ → music/)
- Moved test_demo.{seq,track} to tools/
- Moved assets/originals/ → tools/originals/
- Removed assets/common/ (legacy, duplicated in workspaces)
- Removed assets/final/ (legacy, superseded by workspaces)
- Updated hot-reload paths in main.cc
- Updated CMake references for test_demo and validation
- Updated gen_spectrograms.sh paths
handoff(Claude): Workspace reorganization complete
|
|
Moved main.cc, stub_main.cc, and test_demo.cc from src/ to src/app/
for better organization. Updated cmake/DemoExecutables.cmake paths.
handoff(Claude): App files reorganized into src/app/ directory
|
|
- Add --cnn-version <1|2> flag to select between CNN v1 and v2
- Implement beat_phase modulation for dynamic blend in both CNN effects
- Fix CNN v2 per-layer uniform buffer sharing (each layer needs own buffer)
- Fix CNN v2 y-axis orientation to match render pass convention
- Add Scene1Effect as base visual layer to test_demo timeline
- Reorganize CNN v2 shaders into cnn_v2/ subdirectory
- Update asset paths and documentation for new shader organization
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
|
|
|
|
FATAL_CHECK triggers when condition is TRUE (error case).
Inverted equality checks: magic/version == correct_value
would fatal when weights were valid.
Changed to != checks to fail on invalid data.
|
|
Use container's getBoundingClientRect instead of timeline's. Timeline can
scroll off-screen with negative left values. Container stays visible and
provides reliable viewport coordinates. Fixes double-click seek and drag.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Read actual canvas offset from style.left instead of assuming scrollLeft.
Canvas is positioned with negative left offset, so we subtract it to get
correct beat position.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Use helper functions (beatsToTime, timeToBeats) consistently in click
handlers. Fixes red cursor jumping to wrong position during seek.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Factorize common patterns: POST_PROCESS_EFFECTS constant, helper functions
(beatsToTime, timeToBeats, beatRange, detectConflicts). Reduce verbosity
with modern JS features (nullish coalescing, optional chaining, Object.assign).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Detects same-priority post-process effects (within and cross-sequence).
CPU load bar (10px, pastel colors) shows conflicts. Matches seq_compiler.cc
logic: warns when 2+ post-process effects share priority, regardless of
time overlap.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Features:
- CPU load bar: Color-coded (green→yellow→red) effect density visualization
- Overlays under waveform to save space, always visible
- Constant load (1.0) per active effect, 0.1 beat resolution
- Add Effect button: Create new effects in selected sequence
- Delete buttons in properties panel for quick access
- Timeline favicon (green bars SVG)
Fixes:
- Handle drag no longer jumps on mousedown (offset tracking)
- Sequence name input accepts numbers (explicit inputmode)
- Start Time label corrected (beats, not seconds)
Updated timeline.seq with beat-based timing adjustments.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated:
- HOWTO.md: Complete pipeline, storage buffer, --validate mode
- TODO.md: Mark CNN v2 complete, add QAT TODO
- PROJECT_CONTEXT.md: Update Effects status
- CNN_V2.md: Mark complete, add storage buffer notes
- train_cnn_v2_full.sh: Add --help message
All documentation now reflects:
- Storage buffer architecture
- Binary weight format
- Live training progress
- Validation-only mode
- 8-bit quantization TODO
|
|
Usage:
./train_cnn_v2_full.sh --validate [checkpoint.pth]
Skips training and weight export, uses existing weights.
Validates all input images with latest (or specified) checkpoint.
Example:
./train_cnn_v2_full.sh --validate checkpoints/checkpoint_epoch_50.pth
|
