| Age | Commit message (Collapse) | Author |
|---|
|
Adds --mix flag to blend input channels with static features:
- p0+p4 → p0 (RGBA + UV.x)
- p1+p5 → p1 (RGBA + UV.y)
- p2+p6 → p2 (RGBA + sin encoding)
- p3+p7 → p3 (RGBA + bias)
Useful for debugging static feature contribution in CNN v2.
Updated doc/CNN_V2_DEBUG_TOOLS.md with --mix usage examples.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
currentLayerIdx indexes layerOutputs array (0=Static Features, 1=Layer 0).
Filename should use layer number, not array index.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Root cause: Binary format is [header:20B][layer_info:20B×N][weights].
Both cnn_test and CNNv2Effect uploaded entire file to weights_buffer,
but shader reads weights_buffer[0] expecting first weight, not header.
Fix: Skip header + layer_info when uploading to GPU buffer.
- cnn_test.cc: Calculate weights_offset, upload only weights section
- cnn_v2_effect.cc: Same fix for runtime effect
Before: layer_0 output showed [R, uv_x, uv_y, black] (wrong channels)
After: layer_0 output shows [R, G, B, D] (correct identity mapping)
Tests: 34/36 passing (2 unrelated failures)
|
|
When using --weights option:
- Layer count and kernel sizes loaded from binary header
- Warnings shown if --layers or --cnn-version specified
- Help text clarifies precedence order
- Binary weights always take precedence over CLI args
Updated documentation:
- doc/CNN_TEST_TOOL.md: Usage examples with --weights
- doc/HOWTO.md: Runtime weight loading example
handoff(Claude): cnn_test --weights config override
|
|
Enables testing different CNN v2 weight files without rebuilding. Automatically forces CNN v2 when --weights is specified, with warning if --cnn-version conflicts.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
All layers now use scale 1.0, shader clamps values >1.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Layer 0 output is clamped [0,1], does not need 0.5 dimming.
Middle layers (ReLU) keep 0.5 scale for values >1.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add identity weight generator and composited layer save for debugging
HTML/C++ output differences.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Cast depth array to float32 when provided, preventing torch Double/Float
dtype mismatch during forward pass.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training changes:
- Changed p3 default depth from 0.0 to 1.0 (far plane semantics)
- Extract depth from target alpha channel in both datasets
- Consistent alpha-as-depth across training/validation
Test tool enhancements (cnn_test):
- Added load_depth_from_alpha() for R32Float depth texture
- Fixed bind group layout for UnfilterableFloat sampling
- Added --save-intermediates with per-channel grayscale composites
- Each layer saved as 4x wide PNG (p0-p3 stacked horizontally)
- Global layers_composite.png for vertical layer stack overview
Investigation notes:
- Static features p4-p7 ARE computed and bound correctly
- Sin_20_y pattern visibility difference between tools under investigation
- Binary weights timestamp (Feb 13 20:36) vs HTML tool (Feb 13 22:12)
- Next: Update HTML tool with canonical binary weights
handoff(Claude): HTML tool weights update pending - base64 encoded
canonical weights ready in /tmp/weights_b64.txt for line 392 replacement.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training changes (train_cnn_v2.py):
- p3 now uses target image alpha channel (depth proxy for 2D images)
- Default changed from 0.0 → 1.0 (far plane semantics)
- Both PatchDataset and ImagePairDataset updated
Test tools (cnn_test.cc):
- New load_depth_from_alpha() extracts PNG alpha → p3 texture
- Fixed bind group layout: use UnfilterableFloat for R32Float depth
- Added --save-intermediates support for CNN v2:
* Each layer_N.png shows 4 channels horizontally (1812×345 grayscale)
* layers_composite.png stacks all layers vertically (1812×1380)
* static_features.png shows 4 feature channels horizontally
- Per-channel visualization enables debugging layer-by-layer differences
HTML tool (index.html):
- Extract alpha channel from input image → depth texture
- Matches training data distribution for validation
Note: Current weights trained with p3=0 are now mismatched. Both tools
use p3=alpha consistently, so outputs remain comparable for debugging.
Retrain required for optimal quality.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add documentation for DEFAULT_WEIGHTS_B64 constant:
- Current config: 4 layers, mip_level=2
- Update procedure: base64 encode and replace
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Replaces v1 weights (3 layers) with v2 weights from workspaces/main/weights/cnn_v2_weights.bin:
- 4 layers: 3×3, 5×5, 3×3, 3×3
- 2496 f16 weights
- mip_level=2
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Changes:
- Static shader: Point sampler (nearest filter) instead of linear
- Mip handling: Use textureSampleLevel with point sampler (fixes coordinate scaling)
- Save PNG: GPU readback via staging buffer (WebGPU canvas lacks toBlob support)
- Depth binding: Use input texture as depth (matches C++ simplification)
- Header offset: Version-aware calculation (v1=4, v2=5 u32)
Known issue: Output still differs from cnn_test (color tones). Root cause TBD.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Root cause: HTML tool was producing incorrect output vs cnn_test due to:
1. Linear filtering: textureSampleLevel() with sampler blurred p0-p3 features
2. Header offset bug: Used 4 u32 instead of 5 u32 for version 2 binary format
Changes:
- Static shader: Replace textureSampleLevel (linear) with textureLoad (point)
- Bind group: Use 3 separate mip views instead of sampler
- Header offset: Account for version-specific header size (v1=4, v2=5 u32)
- Add version field to weights object for correct offset calculation
- Add savePNG button for convenience
Result: HTML output now matches cnn_test output exactly.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Improve drop zone visibility with larger borders, bold blue text, and
brighter hover states for better user guidance.
Replace hover-based zoom with click-to-preview: clicking any of the
4 small channel views displays it large below. Active channel
highlighted with white border for clear visual feedback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Add helper functions: export_weights(), find_latest_checkpoint(), build_target()
- Eliminate duplicate export logic (3 instances → 1 function)
- Eliminate duplicate checkpoint finding (2 instances → 1 function)
- Consolidate build commands (4 instances → 1 function)
- Simplify optional flags with inline command substitution
- Fix validation mode: correct cnn_test argument order (positional args before --cnn-version)
- 30 fewer lines, improved readability
handoff(Claude): Refactored CNN v2 training script, fixed validation bug
|
|
Implement full CNN v2 support for offline validation:
- Add --cnn-version flag (1=render pipeline, 2=compute shader)
- Load binary weights from storage buffer (~3-5 KB)
- Static features compute pass (7D: RGBD + UV + sin + bias)
- Dynamic layer count from binary header
- RGBA32Uint texture readback with f16→u8 conversion
- Custom f16 decoder (handles denormals, infinity, NaN)
Status:
- CNN v1: Produces incorrect output (all white)
- CNN v2: ✅ Fully functional, matches CNNv2Effect
Updated docs:
- doc/CNN_TEST_TOOL.md: Architecture, usage, validation workflow
- doc/HOWTO.md: Recommend v2 for validation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
computation
Add option to compute loss on grayscale (Y = 0.299*R + 0.587*G + 0.114*B) instead of full RGBA channels. Useful for training models that prioritize luminance accuracy over color accuracy.
Changes:
- training/train_cnn_v2.py: Add --grayscale-loss flag and grayscale conversion in loss computation
- scripts/train_cnn_v2_full.sh: Add --grayscale-loss parameter support
- doc/CNN_V2.md: Document grayscale loss in training configuration and checkpoint format
- doc/HOWTO.md: Add usage examples for --grayscale-loss flag
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Fix two issues causing validation errors in test_demo:
1. Remove redundant pipeline creation without layout (static_pipeline_)
2. Change vec3<u32> to 3× u32 fields in StaticFeatureParams struct
WGSL vec3<u32> aligns to 16 bytes (std140), making struct 32 bytes,
while C++ struct was 16 bytes. Explicit fields ensure consistent layout.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Expose all hardcoded parameters in train_cnn_v2_full.sh:
- Training: epochs, batch-size, checkpoint-every, kernel-sizes, num-layers, mip-level
- Patches: patch-size, patches-per-image, detector, full-image, image-size
- Directories: input, target, checkpoint-dir, validation-dir
Update --help with organized sections (modes, training, patches, directories).
Update doc/HOWTO.md with usage examples.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Update positional encoding to use vertical coordinate at higher frequency.
Changes:
- train_cnn_v2.py: sin10_x → sin20_y (computed from uv_y)
- cnn_v2_static.wgsl: sin10_x → sin20_y (computed from uv_y)
- index.html: sin10_x → sin20_y (STATIC_SHADER)
- CNN_V2.md: Update feature descriptions and examples
- CNN_V2_BINARY_FORMAT.md: Update static features documentation
Feature vector: [p0, p1, p2, p3, uv_x, uv_y, sin20_y, bias]
Rationale: Higher frequency (20 vs 10) + vertical axis provides better
spatial discrimination for position encoding.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Document future enhancement for arbitrary feature vector layouts.
Proposed feature descriptor in binary format v3:
- Specify feature types, sources, and ordering
- Enable runtime experimentation without shader recompilation
- Examples: [R,G,B,dx,dy,uv_x,bias] or [mip1.r,mip2.g,laplacian,uv_x,sin20_x,bias]
Added TODOs in:
- CNN_V2_BINARY_FORMAT.md: Detailed proposal with struct layout
- CNN_V2.md: Future extensions section
- train_cnn_v2.py: compute_static_features() docstring
- cnn_v2_static.wgsl: Shader header comment
- cnn_v2_effect.cc: Version check comment
Current limitation: Hardcoded [p0,p1,p2,p3,uv_x,uv_y,sin10_x,bias] layout.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated documentation to reflect binary format v2 with mip_level field.
Changes:
- CNN_V2_BINARY_FORMAT.md: Document v2 (20-byte header) with mip_level, v1 backward compat
- CNN_V2_WEB_TOOL.md: Document auto-detection of mip_level, UI updates
- CNN_V2.md: Update overview with mip-level feature, training pipeline
Binary format v2:
- Header: 20 bytes (was 16)
- New field: mip_level (u32) at offset 0x10
- Backward compatible: v1 loaders treat as mip_level=0
Documentation complete for full mip-level pipeline integration.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Parse v2 header (20 bytes) and read mip_level field.
Display mip_level in metadata panel, set UI dropdown on load.
Changes:
- parseWeights(): Handle v1 (16-byte) and v2 (20-byte) headers
- Read mip_level from header[4] for version 2
- Return mipLevel in parsed weights object
- updateWeightsPanel(): Display mip level in metadata
- loadWeights(): Set this.mipLevel and update UI dropdown
Backward compatible: v1 weights → mipLevel=0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Binary format v2 includes mip_level in header (20 bytes, was 16).
Effect reads mip_level and passes to static features shader via uniform.
Shader samples from correct mip texture based on mip_level.
Changes:
- export_cnn_v2_weights.py: Header v2 with mip_level field
- cnn_v2_effect.h: Add StaticFeatureParams, mip_level member, params buffer
- cnn_v2_effect.cc: Read mip_level from weights, create/bind params buffer, update per-frame
- cnn_v2_static.wgsl: Accept params uniform, sample from selected mip level
Binary format v2:
- Header: 20 bytes (magic, version=2, num_layers, total_weights, mip_level)
- Backward compatible: v1 weights load with mip_level=0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training pipeline now accepts --mip-level flag (0-3) and passes to train_cnn_v2.py.
Compatible with all existing modes (train, validate, export-only).
Changes:
- Add --mip-level argument parsing (default: 0)
- Pass MIP_LEVEL to training command
- Display mip level in config output
- Update help text with examples
Usage: ./scripts/train_cnn_v2_full.sh --mip-level 1
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Export scripts now read mip_level from checkpoint config and display it.
Shader generator includes mip level in generated comments.
Changes:
- export_cnn_v2_weights.py: Read mip_level, print in config
- export_cnn_v2_shader.py: Read mip_level, pass to shader gen, add to comments
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add mip level control for p0-p3 features (0=original, 1=half, 2=quarter, 3=eighth).
Uses pyrDown/pyrUp for proper Gaussian filtering during mip generation.
Changes:
- compute_static_features(): Accept mip_level param, generate mip via cv2 pyramid
- PatchDataset/ImagePairDataset: Pass mip_level to feature computation
- CLI: Add --mip-level arg with choices [0,1,2,3]
- Save mip_level in checkpoint config for tracking
- Doc updates: HOWTO.md and CNN_V2.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Refactoring:
- Extract FULLSCREEN_QUAD_VS shader (reused in mipmap, display, layer viz)
- Add helper methods: getDimensions(), setVideoControlsEnabled()
- Add section headers and improve code organization (~40 lines saved)
- Move Mip Level selector to bottom of left sidebar
- Remove "Features (p0-p3)" panel header
Features:
- Add video loop support (continuous playback)
Documentation:
- Update CNN_V2_WEB_TOOL.md with latest changes
- Document refactoring benefits and code organization
- Update UI layout section with current structure
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Add dropdown menu in left panel to select mip levels 0-2 for parametric features (p0-p3/RGBD). Uses trilinear filtering for smooth downsampling at higher mip levels.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Layer 0 now uses clamp [0,1] in both training and inference (was using ReLU in shaders).
- index.html: Add is_layer_0 flag to LayerParams, handle Layer 0 separately
- export_cnn_v2_shader.py: Generate correct activation for Layer 0
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Change Depth control from number input to slider (0-1 range)
- Move video controls to floating overlay at top of canvas
- Remove View mode indicator from header (shortcuts still work)
- Remove scrollbar from Layer Visualization panel
- Fix layer viz flickering during video playback
- Fix video controls responsiveness during playback
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Features:
- Video file support (MP4, WebM, etc.) via drag-and-drop
- Play/Pause button with non-realtime playback (drops frames if CNN slow)
- Frame-by-frame navigation (◄/► step buttons)
- Unified image/video processing through same CNN pipeline
- Audio muted (video frames only)
Optimizations:
- Layer visualization updates only on pause/seek (~5-10ms saved per frame)
Architecture:
- copyExternalImageToTexture() works with both ImageBitmap and HTMLVideoElement
- Video loading: wait for metadata → seek to frame 0 → wait for readyState≥2 (decoded)
- Playback loop: requestAnimationFrame with isProcessing guard prevents overlapping inference
- Controls always visible, disabled for images
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
UI Changes:
- Three-panel layout: left (weights), center (canvas), right (activations)
- Left sidebar: clickable weights drop zone, weights info, kernel visualization
- Right sidebar: 4 small activation views + large 4× zoom view
- Controls moved to header (inline with title)
Weights Visualization:
- Dedicated panel in left sidebar with layer buttons
- 1 pixel per weight (was 20px)
- All input channels horizontal, output channels stacked vertically
- Renders to separate canvas (not in activation grid)
Activation Viewer:
- 4 channels in horizontal row (was 2×2 grid)
- Mouse-driven zoom view below (32×32 area at 4× magnification)
- Zoom shows all 4 channels in 2×2 quadrant layout
- Removed activations/weights mode toggle
State Preservation:
- Blend changes preserve selected layer/channel
- Fixed activation view reset bug
Documentation:
- Updated README with new layout and feature descriptions
- Marked implemented features (weights viz, layer viewer)
- Updated size estimates (~22 KB total)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Fixed validation error where staticTex was used for both storage write
(in static compute pass) and texture read (in CNN bind group) within
same command encoder. Now uses layerTextures[0] for reading, which is
the copy destination and safe for read-only access.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Align layer naming with codebase: Layer 0/1/2 (not Layer 1/2/3)
- Split static features: Static 0-3 (p0-p3) and Static 4-7 (uv,sin,bias)
- Fix Layer 2 not appearing: removed isOutput filter from layerOutputs
- Fix canvas context switching: force clear before recreation
- Disable static buttons in weights mode
- Add ASCII pipeline diagram to CNN_V2.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Added SHADER_SNIPPET_A and SHADER_SNIPPET_B entries to test assets
config to resolve missing AssetId compile error in test_shader_composer.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Allows exporting weights from a checkpoint without training or validation.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Specifies sample offset (shift trigger left) and humanization (per-note timing/volume variation) for realistic playback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Fixes test_assets.cc compilation by adding missing test asset IDs and
procedural generators. Test-specific code is protected with DEMO_STRIP_ALL
to exclude from release builds.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- tracker_compiler: Sort events by time before C++ generation (required
for runtime early-exit optimization)
- tracker.cc: Add FATAL_CHECK validating sorted events at init
- Add --check mode: Validate .track file without compiling
- Add --sanitize mode: Rewrite .track with sorted events and normalized
formatting
- Fix parser: Skip indented comment lines in patterns
All audio tests passing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Converted track.md drum notation to .track format and integrated as main music.
165 BPM high-energy pattern with syncopated kicks, 16th note hi-hats, and break.
- Add workspaces/main/pop_punk_drums.track (3 patterns, 4-bar sequence)
- Add workspaces/main/track.md (notation reference)
- Update workspace.cfg to use pop_punk_drums.track
- Update BPM to 165
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated CNN_V2.md to document that:
- Model outputs 4 channels (RGBA)
- Training targets preserve alpha from target images
- Loss function compares all 4 channels
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Changed target loading from RGB to RGBA to preserve transparency.
Model learns to predict alpha channel from target image instead of
constant 1.0 padding.
Before: Target padded with alpha=1.0
After: Target uses actual alpha from image (or 1.0 if no alpha)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Changes:
- KERNEL_SIZES: Use comma-separated values (3,3,3 not 3 3 3)
- Remove --channels (no longer exists in uniform 12D→4D architecture)
- Add --num-layers parameter
- Use export_cnn_v2_weights.py (storage buffer) instead of export_cnn_v2_shader.py
- Fix duplicate export: only export in step 2 (training) or validation mode
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Updated comments to clarify that per-layer kernel sizes are supported.
Code already handles this correctly via LayerInfo.kernel_size field.
Changes:
- cnn_v2_effect.h: Add comment about per-layer kernel sizes
- cnn_v2_compute.wgsl: Clarify LayerParams provides per-layer config
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Training:
- train_cnn_v2.py: Accept --kernel-sizes as comma-separated list
- CNNv2 model: Per-layer kernel sizes (e.g., [1,3,5])
- Single value replicates across layers (e.g., "3" → [3,3,3])
Export:
- export_cnn_v2_weights.py: Backward compatible with old checkpoints
- Handles both kernel_size (old) and kernel_sizes (new) format
Documentation:
- CNN_V2.md: Updated code examples and config format
- HOWTO.md: Updated training examples to show comma-separated syntax
Binary format: Already supports per-layer kernel sizes (no changes)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
