summaryrefslogtreecommitdiff
path: root/doc/CNN_V2.md
AgeCommit message (Collapse)Author
16 hoursrefactor(gpu): Relocate effects to src/effects and streamline includesskal
This refactoring improves the project's structure by decoupling visual effects from the core GPU module. All effect implementations have been moved from to a new top-level directory. Shared utilities like , , and have been consolidated into the parent directory. - **Motivation**: To create a clearer separation of concerns, making the codebase easier to navigate and maintain. This move treats effects as a distinct layer that depends on the core GPU module, rather than being embedded within it. - **Changes**: - Created new directory. - Moved all effect source files (, ) to . - Moved shared helpers (, , ) to . - Updated and to reflect the new file locations for all build targets. - Corrected all directives across the entire codebase (, , ) to point to the new paths. - Updated all markdown documentation ( files) to ensure file paths and architectural descriptions are accurate. - Fixed several compiler errors related to incorrect enum casting () that were exposed during cross-compilation for Windows. - **Verification**: - The entire project builds successfully for both native and Windows cross-compilation targets. - All 34 tests pass (Usage ctest [options]). - The --- Running Native Build & Tests --- Configuring with all options enabled (tests + tools)... -- -- Build Configuration: -- DEMO_SIZE_OPT: ON -- DEMO_STRIP_ALL: ON -- DEMO_FINAL_STRIP: OFF -- DEMO_STRIP_EXTERNAL_LIBS: OFF -- DEMO_BUILD_TESTS: ON -- DEMO_BUILD_TOOLS: ON -- DEMO_ENABLE_COVERAGE: OFF -- DEMO_ENABLE_DEBUG_LOGS: OFF -- DEMO_HEADLESS: OFF -- DEMO_WORKSPACE: main -- -- Loaded workspace: Main Demo -- Timeline: timeline.seq -- Music: pop_punk_drums.track -- Assets: assets.txt -- Using workspace: main -- Configuring done (0.0s) -- Generating done (0.1s) -- Build files have been written to: /Users/skal/demo/build Building all targets (demo, tests, and tools)... [ 0%] Built target validate_uniforms_script [ 1%] Built target procedural [ 2%] Validating uniform buffer sizes and alignments... [ 3%] Built target tracker_compiler [ 4%] Built target test_3d [ 4%] Built target test_maths [ 4%] Built target seq_compiler [ 4%] Built target tracker_compiler_host [ 5%] Built target asset_packer [ 5%] Built target test_procedural [ 6%] Compiling demo sequence from workspace main... [ 6%] Built target generate_tracker_music [ 6%] Built target generate_test_demo_music [ 6%] Compiling test_demo sequence... Using BPM: 90 Successfully generated timeline with 16 sequences. Using BPM: 120 Demo end time: 16.000000s Successfully generated timeline with 1 sequences. [ 6%] Built target generate_test_demo_timeline [ 6%] Built target generate_timeline Validation Warning for 'CommonPostProcessUniforms': Matching WGSL struct not found. Validation OK for 'FadeParams': Size 16 matches C++ expected size. Validation OK for 'ThemeModulationParams': Size 16 matches C++ expected size. Validation OK for 'GaussianBlurParams': Size 8 matches C++ expected size. Validation OK for 'DistortParams': Size 8 matches C++ expected size. Validation OK for 'CircleMaskParams': Size 16 matches C++ expected size. [ 6%] Built target generate_test_assets [ 7%] Built target generate_demo_assets [ 7%] Built target validate_uniforms [ 8%] Built target util [ 10%] Built target test_assets [ 11%] Built target test_shader_assets [ 12%] Built target test_file_watcher [ 15%] Built target 3d [ 21%] Built target test_platform [ 22%] Built target audio [ 23%] Built target test_window [ 26%] Built target test_fft [ 27%] Built target test_synth [ 27%] Built target test_spectral_brush [ 27%] Built target test_physics [ 28%] Built target test_dct [ 31%] Building CXX object CMakeFiles/gpu.dir/src/gpu/effect.cc.o [ 30%] Built target test_mock_backend [ 32%] Built target test_scene_loader [ 33%] Built target test_audio_backend [ 34%] Built target test_audio_gen [ 36%] Built target test_silent_backend [ 39%] Built target test_jittered_audio [ 39%] Building CXX object CMakeFiles/gpu.dir/src/effects/heptagon_effect.cc.o [ 42%] Built target test_wav_dump [ 44%] Built target test_tracker_timing [ 44%] Building CXX object CMakeFiles/gpu.dir/src/effects/particles_effect.cc.o [ 45%] Building CXX object CMakeFiles/gpu.dir/src/effects/passthrough_effect.cc.o [ 47%] Built target test_variable_tempo [ 50%] Built target test_audio_engine [ 52%] Built target test_tracker [ 52%] Building CXX object CMakeFiles/gpu.dir/src/effects/moving_ellipse_effect.cc.o [ 52%] Building CXX object CMakeFiles/gpu.dir/src/effects/particle_spray_effect.cc.o [ 52%] Building CXX object CMakeFiles/gpu.dir/src/effects/gaussian_blur_effect.cc.o [ 54%] Built target test_spectool [ 55%] Building CXX object CMakeFiles/gpu.dir/src/effects/solarize_effect.cc.o [ 55%] Building CXX object CMakeFiles/gpu.dir/src/effects/scene1_effect.cc.o [ 55%] Building CXX object CMakeFiles/gpu.dir/src/effects/chroma_aberration_effect.cc.o [ 55%] Building CXX object CMakeFiles/gpu.dir/src/gpu/shaders.cc.o [ 57%] Building CXX object CMakeFiles/gpu.dir/src/effects/vignette_effect.cc.o [ 57%] Building CXX object CMakeFiles/gpu.dir/src/gpu/post_process_helper.cc.o [ 57%] Linking CXX static library libgpu.a [ 60%] Built target gpu [ 60%] Linking CXX executable test_uniform_helper [ 60%] Linking CXX executable test_shader_composer [ 60%] Building CXX object CMakeFiles/test_sequence.dir/src/tests/assets/test_sequence.cc.o [ 61%] Linking CXX executable test_noise_functions [ 62%] Linking CXX executable test_shader_compilation [ 62%] Building CXX object CMakeFiles/test_demo.dir/src/app/test_demo.cc.o [ 62%] Building CXX object CMakeFiles/demo64k.dir/src/app/main.cc.o [ 62%] Building CXX object CMakeFiles/test_3d_render.dir/src/generated/timeline.cc.o [ 63%] Built target test_uniform_helper [ 64%] Built target test_shader_composer [ 64%] Building CXX object CMakeFiles/test_3d_physics.dir/src/generated/timeline.cc.o [ 65%] Built target test_noise_functions [ 66%] Built target test_shader_compilation [ 67%] Building CXX object CMakeFiles/test_mesh.dir/src/generated/timeline.cc.o [ 67%] Building CXX object CMakeFiles/test_effect_base.dir/src/tests/gpu/test_effect_base.cc.o [ 67%] Building CXX object CMakeFiles/test_demo_effects.dir/src/tests/gpu/test_demo_effects.cc.o [ 67%] Building CXX object CMakeFiles/test_sequence.dir/src/generated/timeline.cc.o [ 68%] Building CXX object CMakeFiles/test_demo.dir/src/generated/test_demo_timeline.cc.o [ 68%] Building CXX object CMakeFiles/demo64k.dir/src/generated/timeline.cc.o [ 68%] Linking CXX executable test_3d_render [ 68%] Building CXX object CMakeFiles/test_effect_base.dir/src/generated/timeline.cc.o [ 68%] Linking CXX executable test_3d_physics [ 68%] Linking CXX executable test_mesh [ 71%] Built target test_3d_render [ 71%] Building CXX object CMakeFiles/test_post_process_helper.dir/src/tests/gpu/test_post_process_helper.cc.o [ 72%] Building CXX object CMakeFiles/test_demo_effects.dir/src/generated/timeline.cc.o [ 72%] Linking CXX executable test_demo [ 75%] Built target test_3d_physics [ 77%] Built target test_mesh [ 77%] Linking CXX executable test_texture_manager [ 78%] Linking CXX executable test_sequence [ 78%] Linking CXX executable test_gpu_procedural [ 80%] Built target test_demo [ 81%] Linking CXX executable test_gpu_composite [ 81%] Linking CXX executable demo64k [ 83%] Built target test_sequence [ 85%] Built target test_texture_manager [ 86%] Built target test_gpu_procedural [ 86%] Linking CXX executable test_post_process_helper [ 86%] Linking CXX executable test_effect_base [ 87%] Built target test_gpu_composite [ 90%] Built target demo64k [ 92%] Built target test_post_process_helper [ 96%] Built target test_effect_base [ 96%] Linking CXX executable test_demo_effects [100%] Built target test_demo_effects Running test suite... Test project /Users/skal/demo/build Start 1: HammingWindowTest 1/34 Test #1: HammingWindowTest ................ Passed 0.00 sec Start 2: MathUtilsTest 2/34 Test #2: MathUtilsTest .................... Passed 0.00 sec Start 3: FileWatcherTest 3/34 Test #3: FileWatcherTest .................. Passed 0.00 sec Start 4: SynthEngineTest 4/34 Test #4: SynthEngineTest .................. Passed 0.00 sec Start 5: DctTest 5/34 Test #5: DctTest .......................... Passed 0.00 sec Start 6: FftTest 6/34 Test #6: FftTest .......................... Passed 0.01 sec Start 7: SpectralBrushTest 7/34 Test #7: SpectralBrushTest ................ Passed 0.01 sec Start 8: AudioGenTest 8/34 Test #8: AudioGenTest ..................... Passed 0.00 sec Start 9: AudioBackendTest 9/34 Test #9: AudioBackendTest ................. Passed 0.00 sec Start 10: SilentBackendTest 10/34 Test #10: SilentBackendTest ................ Passed 0.00 sec Start 11: MockAudioBackendTest 11/34 Test #11: MockAudioBackendTest ............. Passed 0.00 sec Start 12: WavDumpBackendTest 12/34 Test #12: WavDumpBackendTest ............... Passed 0.00 sec Start 13: JitteredAudioBackendTest 13/34 Test #13: JitteredAudioBackendTest ......... Passed 0.00 sec Start 14: TrackerTimingTest 14/34 Test #14: TrackerTimingTest ................ Passed 0.00 sec Start 15: VariableTempoTest 15/34 Test #15: VariableTempoTest ................ Passed 0.00 sec Start 16: TrackerSystemTest 16/34 Test #16: TrackerSystemTest ................ Passed 0.01 sec Start 17: AudioEngineTest 17/34 Test #17: AudioEngineTest .................. Passed 0.00 sec Start 18: ShaderAssetValidation 18/34 Test #18: ShaderAssetValidation ............ Passed 0.00 sec Start 19: ShaderCompilationTest 19/34 Test #19: ShaderCompilationTest ............ Passed 0.02 sec Start 20: NoiseFunctionsTest 20/34 Test #20: NoiseFunctionsTest ............... Passed 0.01 sec Start 21: UniformHelperTest 21/34 Test #21: UniformHelperTest ................ Passed 0.00 sec Start 22: AssetManagerTest 22/34 Test #22: AssetManagerTest ................. Passed 0.01 sec Start 23: SequenceSystemTest 23/34 Test #23: SequenceSystemTest ............... Passed 0.01 sec Start 24: ProceduralGenTest 24/34 Test #24: ProceduralGenTest ................ Passed 0.01 sec Start 25: PhysicsTest 25/34 Test #25: PhysicsTest ...................... Passed 0.01 sec Start 26: ThreeDSystemTest 26/34 Test #26: ThreeDSystemTest ................. Passed 0.00 sec Start 27: ShaderComposerTest 27/34 Test #27: ShaderComposerTest ............... Passed 0.01 sec Start 28: SceneLoaderTest 28/34 Test #28: SceneLoaderTest .................. Passed 0.01 sec Start 29: EffectBaseTest 29/34 Test #29: EffectBaseTest ................... Passed 0.04 sec Start 30: DemoEffectsTest 30/34 Test #30: DemoEffectsTest .................. Passed 0.03 sec Start 31: PostProcessHelperTest 31/34 Test #31: PostProcessHelperTest ............ Passed 0.02 sec Start 32: TextureManagerTest 32/34 Test #32: TextureManagerTest ............... Passed 0.02 sec Start 33: GpuProceduralTest 33/34 Test #33: GpuProceduralTest ................ Passed 0.18 sec Start 34: GpuCompositeTest 34/34 Test #34: GpuCompositeTest ................. Passed 0.20 sec 100% tests passed, 0 tests failed out of 34 Label Time Summary: 3d = 0.01 sec*proc (3 tests) assets = 0.02 sec*proc (2 tests) audio = 0.07 sec*proc (15 tests) gpu = 0.54 sec*proc (11 tests) util = 0.01 sec*proc (3 tests) Total Test time (real) = 0.67 sec Verifying tools compile... [ 9%] Built target procedural [ 18%] Built target tracker_compiler_host [ 18%] Built target tracker_compiler [ 18%] Built target generate_tracker_music [ 18%] Built target asset_packer [ 27%] Built target generate_demo_assets [ 27%] Built target generate_test_assets [ 36%] Built target util [ 81%] Built target audio [100%] Built target test_spectool --- Running Windows Cross-Compilation Build --- Building native tools... -- -- Build Configuration: -- DEMO_SIZE_OPT: OFF -- DEMO_STRIP_ALL: OFF -- DEMO_FINAL_STRIP: OFF -- DEMO_STRIP_EXTERNAL_LIBS: OFF -- DEMO_BUILD_TESTS: OFF -- DEMO_BUILD_TOOLS: OFF -- DEMO_ENABLE_COVERAGE: OFF -- DEMO_ENABLE_DEBUG_LOGS: OFF -- DEMO_HEADLESS: OFF -- DEMO_WORKSPACE: main -- -- Loaded workspace: Main Demo -- Timeline: timeline.seq -- Music: pop_punk_drums.track -- Assets: assets.txt -- Using workspace: main -- Configuring done (0.0s) -- Generating done (0.0s) -- Build files have been written to: /Users/skal/demo/build_native [ 50%] Built target procedural [100%] Built target asset_packer [100%] Built target seq_compiler [100%] Built target tracker_compiler_host Cross-compiling for Windows... -- -- Build Configuration: -- DEMO_SIZE_OPT: ON -- DEMO_STRIP_ALL: ON -- DEMO_FINAL_STRIP: OFF -- DEMO_STRIP_EXTERNAL_LIBS: OFF -- DEMO_BUILD_TESTS: OFF -- DEMO_BUILD_TOOLS: OFF -- DEMO_ENABLE_COVERAGE: OFF -- DEMO_ENABLE_DEBUG_LOGS: OFF -- DEMO_HEADLESS: OFF -- DEMO_WORKSPACE: main -- -- Loaded workspace: Main Demo -- Timeline: timeline.seq -- Music: pop_punk_drums.track -- Assets: assets.txt -- Using workspace: main -- Configuring done (0.0s) -- Generating done (0.0s) -- Build files have been written to: /Users/skal/demo/build_win [ 2%] Built target validate_uniforms_script [ 2%] Built target generate_timeline [ 4%] Built target generate_test_demo_timeline [ 4%] Built target generate_demo_assets [ 4%] Built target generate_test_assets [ 6%] Built target procedural [ 9%] Built target tracker_compiler_host [ 10%] Validating uniform buffer sizes and alignments... [ 11%] Built target generate_tracker_music [ 13%] Built target generate_test_demo_music [ 16%] Built target util [ 28%] Built target 3d [ 45%] Built target audio [ 49%] Building CXX object CMakeFiles/gpu.dir/src/effects/heptagon_effect.cc.obj [ 52%] Building CXX object CMakeFiles/gpu.dir/src/effects/gaussian_blur_effect.cc.obj [ 54%] Building CXX object CMakeFiles/gpu.dir/src/effects/particles_effect.cc.obj [ 54%] Building CXX object CMakeFiles/gpu.dir/src/effects/moving_ellipse_effect.cc.obj [ 54%] Building CXX object CMakeFiles/gpu.dir/src/gpu/effect.cc.obj [ 54%] Building CXX object CMakeFiles/gpu.dir/src/effects/passthrough_effect.cc.obj [ 54%] Building CXX object CMakeFiles/gpu.dir/src/effects/particle_spray_effect.cc.obj Validation Warning for 'CommonPostProcessUniforms': Matching WGSL struct not found. Validation OK for 'FadeParams': Size 16 matches C++ expected size. Validation OK for 'ThemeModulationParams': Size 16 matches C++ expected size. Validation OK for 'GaussianBlurParams': Size 8 matches C++ expected size. Validation OK for 'DistortParams': Size 8 matches C++ expected size. Validation OK for 'CircleMaskParams': Size 16 matches C++ expected size. [ 54%] Built target validate_uniforms [ 55%] Building CXX object CMakeFiles/gpu.dir/src/effects/solarize_effect.cc.obj [ 57%] Building CXX object CMakeFiles/gpu.dir/src/effects/scene1_effect.cc.obj [ 57%] Building CXX object CMakeFiles/gpu.dir/src/effects/chroma_aberration_effect.cc.obj [ 58%] Building CXX object CMakeFiles/gpu.dir/src/effects/vignette_effect.cc.obj [ 59%] Building CXX object CMakeFiles/gpu.dir/src/gpu/post_process_helper.cc.obj [ 60%] Building CXX object CMakeFiles/gpu.dir/src/gpu/shaders.cc.obj [ 62%] Linking CXX static library libgpu.a [ 77%] Built target gpu [ 79%] Building CXX object CMakeFiles/demo64k.dir/src/app/main.cc.obj [ 79%] Building CXX object CMakeFiles/test_demo.dir/src/app/test_demo.cc.obj [ 80%] Building CXX object CMakeFiles/demo64k.dir/src/generated/timeline.cc.obj [ 81%] Building CXX object CMakeFiles/test_demo.dir/src/generated/test_demo_timeline.cc.obj [ 82%] Linking CXX executable test_demo.exe [ 90%] Built target test_demo [ 91%] Linking CXX executable demo64k.exe [100%] Built target demo64k Copying MinGW DLLs... Crunching build_win/demo64k.exe... Ultimate Packer for eXecutables Copyright (C) 1996 - 2026 UPX 5.1.0 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 7th 2026 File size Ratio Format Name -------------------- ------ ----------- ----------- 7036416 -> 4680704 66.52% win64/pe demo64k_packed.exe Packed 1 file. ------------------------------------------------ Size Report: -rwxr-xr-x 1 skal 89939 6.7M Feb 14 14:55 build_win/demo64k.exe -rwxr-xr-x 1 skal 89939 6.7M Feb 14 14:55 build_win/demo64k_stripped.exe -rwxr-xr-x 1 skal 89939 4.5M Feb 14 14:55 build_win/demo64k_packed.exe ------------------------------------------------ Top 20 Largest Symbols (from unstripped): ------------------------------------------------ Build complete. Output: build_win/demo64k.exe All checks passed successfully. script completes without errors. This change streamlines the project's architecture without altering any functionality.
29 hoursUpdate docs: CNN v2 sigmoid activation summaryskal
- PROJECT_CONTEXT.md: Updated Effects section (sigmoid, stable training) - TODO.md: Added sigmoid activation to CNN v2 status - CNN_V2.md: Streamlined (removed outdated issues, updated code examples) handoff(Claude): Documentation synchronized with sigmoid implementation.
29 hoursReplace hard clamp with sigmoid activation in CNN v2skal
Fixes training collapse where p1/p2 channels saturate due to gradient blocking at clamp boundaries. Sigmoid provides smooth [0,1] mapping with continuous gradients. Changes: - Layer 0: clamp(x, 0, 1) → sigmoid(x) - Final layer: clamp(x, 0, 1) → sigmoid(x) - Middle layers: ReLU unchanged (already stable) Updated files: - training/train_cnn_v2.py: PyTorch model activations - workspaces/main/shaders/cnn_v2/cnn_v2_compute.wgsl: WGSL shader - tools/cnn_v2_test/index.html: HTML validation tool - doc/CNN_V2.md: Documentation Validation: - Build clean (no shader errors) - 34/36 tests pass (2 unrelated script tests fail) - 10-epoch training: loss 0.153 → 0.088 (good convergence) - cnn_test processes images successfully Breaking change: Old checkpoints trained with clamp() incompatible. Retrain from scratch required. handoff(Claude): CNN v2 sigmoid activation implemented and validated.
30 hoursFix CNN v2 static feature channel mapping (p4-p7 → channels 8-11)skal
Fixed bug in gen_identity_weights.py --p47 mode: static features p4-p7 (uv_x, uv_y, sin20_y, bias) are at input channels 8-11, not 4-7. Weight tensor layout: - Channels 0-3: Previous layer output (4D RGBA) - Channels 4-11: Static features (8D: p0-p7) Static features: - p0-p3 (channels 4-7): RGB+D from mip level - p4-p7 (channels 8-11): uv_x, uv_y, sin20_y, bias Updated: - training/gen_identity_weights.py: Change weights[i,i+4] to weights[i,i+8] - workspaces/main/weights/mix_p47.bin: Regenerated (not in repo) - doc/CNN_V2.md: Add Input Channel Mapping section with full layout table Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
32 hoursCNN v2: Alpha channel depth handling and layer visualizationskal
Training changes: - Changed p3 default depth from 0.0 to 1.0 (far plane semantics) - Extract depth from target alpha channel in both datasets - Consistent alpha-as-depth across training/validation Test tool enhancements (cnn_test): - Added load_depth_from_alpha() for R32Float depth texture - Fixed bind group layout for UnfilterableFloat sampling - Added --save-intermediates with per-channel grayscale composites - Each layer saved as 4x wide PNG (p0-p3 stacked horizontally) - Global layers_composite.png for vertical layer stack overview Investigation notes: - Static features p4-p7 ARE computed and bound correctly - Sin_20_y pattern visibility difference between tools under investigation - Binary weights timestamp (Feb 13 20:36) vs HTML tool (Feb 13 22:12) - Next: Update HTML tool with canonical binary weights handoff(Claude): HTML tool weights update pending - base64 encoded canonical weights ready in /tmp/weights_b64.txt for line 392 replacement. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
37 hoursCNN v2 training: Add --grayscale-loss option for luminance-based loss ↵skal
computation Add option to compute loss on grayscale (Y = 0.299*R + 0.587*G + 0.114*B) instead of full RGBA channels. Useful for training models that prioritize luminance accuracy over color accuracy. Changes: - training/train_cnn_v2.py: Add --grayscale-loss flag and grayscale conversion in loss computation - scripts/train_cnn_v2_full.sh: Add --grayscale-loss parameter support - doc/CNN_V2.md: Document grayscale loss in training configuration and checkpoint format - doc/HOWTO.md: Add usage examples for --grayscale-loss flag Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
38 hoursCNN v2: Change feature #6 from sin(10*x) to sin(20*y)skal
Update positional encoding to use vertical coordinate at higher frequency. Changes: - train_cnn_v2.py: sin10_x → sin20_y (computed from uv_y) - cnn_v2_static.wgsl: sin10_x → sin20_y (computed from uv_y) - index.html: sin10_x → sin20_y (STATIC_SHADER) - CNN_V2.md: Update feature descriptions and examples - CNN_V2_BINARY_FORMAT.md: Update static features documentation Feature vector: [p0, p1, p2, p3, uv_x, uv_y, sin20_y, bias] Rationale: Higher frequency (20 vs 10) + vertical axis provides better spatial discrimination for position encoding. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
38 hoursCNN v2: Add TODO for flexible feature layout in binary format v3skal
Document future enhancement for arbitrary feature vector layouts. Proposed feature descriptor in binary format v3: - Specify feature types, sources, and ordering - Enable runtime experimentation without shader recompilation - Examples: [R,G,B,dx,dy,uv_x,bias] or [mip1.r,mip2.g,laplacian,uv_x,sin20_x,bias] Added TODOs in: - CNN_V2_BINARY_FORMAT.md: Detailed proposal with struct layout - CNN_V2.md: Future extensions section - train_cnn_v2.py: compute_static_features() docstring - cnn_v2_static.wgsl: Shader header comment - cnn_v2_effect.cc: Version check comment Current limitation: Hardcoded [p0,p1,p2,p3,uv_x,uv_y,sin10_x,bias] layout. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
38 hoursDoc: Update CNN v2 docs for binary format v2 and mip-level supportskal
Updated documentation to reflect binary format v2 with mip_level field. Changes: - CNN_V2_BINARY_FORMAT.md: Document v2 (20-byte header) with mip_level, v1 backward compat - CNN_V2_WEB_TOOL.md: Document auto-detection of mip_level, UI updates - CNN_V2.md: Update overview with mip-level feature, training pipeline Binary format v2: - Header: 20 bytes (was 16) - New field: mip_level (u32) at offset 0x10 - Backward compatible: v1 loaders treat as mip_level=0 Documentation complete for full mip-level pipeline integration. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
38 hoursCNN v2: Add --mip-level option for parametric featuresskal
Add mip level control for p0-p3 features (0=original, 1=half, 2=quarter, 3=eighth). Uses pyrDown/pyrUp for proper Gaussian filtering during mip generation. Changes: - compute_static_features(): Accept mip_level param, generate mip via cv2 pyramid - PatchDataset/ImagePairDataset: Pass mip_level to feature computation - CLI: Add --mip-level arg with choices [0,1,2,3] - Save mip_level in checkpoint config for tracking - Doc updates: HOWTO.md and CNN_V2.md Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
40 hoursCNN v2 web tool: Fix layer naming and visualization bugsskal
- Align layer naming with codebase: Layer 0/1/2 (not Layer 1/2/3) - Split static features: Static 0-3 (p0-p3) and Static 4-7 (uv,sin,bias) - Fix Layer 2 not appearing: removed isOutput filter from layerOutputs - Fix canvas context switching: force clear before recreation - Disable static buttons in weights mode - Add ASCII pipeline diagram to CNN_V2.md Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
42 hoursDoc: Clarify CNN v2 training uses RGBA targetsskal
Updated CNN_V2.md to document that: - Model outputs 4 channels (RGBA) - Training targets preserve alpha from target images - Loss function compares all 4 channels Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
42 hoursCNN v2: Restore per-layer kernel sizes supportskal
Training: - train_cnn_v2.py: Accept --kernel-sizes as comma-separated list - CNNv2 model: Per-layer kernel sizes (e.g., [1,3,5]) - Single value replicates across layers (e.g., "3" → [3,3,3]) Export: - export_cnn_v2_weights.py: Backward compatible with old checkpoints - Handles both kernel_size (old) and kernel_sizes (new) format Documentation: - CNN_V2.md: Updated code examples and config format - HOWTO.md: Updated training examples to show comma-separated syntax Binary format: Already supports per-layer kernel sizes (no changes) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
43 hoursCNN v2: Refactor to uniform 12D→4D architectureskal
**Architecture changes:** - Static features (8D): p0-p3 (parametric) + uv_x, uv_y, sin(10×uv_x), bias - Input RGBD (4D): fed separately to all layers - All layers: uniform 12D→4D (4 prev/input + 8 static → 4 output) - Bias integrated in static features (bias=False in PyTorch) **Weight calculations:** - 3 layers × (12 × 3×3 × 4) = 1296 weights - f16: 2.6 KB (vs old variable arch: ~6.4 KB) **Updated files:** *Training (Python):* - train_cnn_v2.py: Uniform model, takes input_rgbd + static_features - export_cnn_v2_weights.py: Binary export for storage buffers - export_cnn_v2_shader.py: Per-layer shader export (debugging) *Shaders (WGSL):* - cnn_v2_static.wgsl: p0-p3 parametric features (mips/gradients) - cnn_v2_compute.wgsl: 12D input, 4D output, vec4 packing *Tools:* - HTML tool (cnn_v2_test): Updated for 12D→4D, layer visualization *Docs:* - CNN_V2.md: Updated architecture, training, validation sections - HOWTO.md: Reference HTML tool for validation *Removed:* - validate_cnn_v2.sh: Obsolete (used CNN v1 tool) All code consistent with bias=False (bias in static features as 1.0). handoff(Claude): CNN v2 architecture finalized and documented
43 hoursCNN v2 Web Tool: Unify layer terminology and add binary format specskal
- Rename 'Static (L0)' → 'Static' (clearer, less confusing) - Update channel labels: 'R/G/B/D' → 'Ch0 (R)/Ch1 (G)/Ch2 (B)/Ch3 (D)' - Add 'Layer' prefix in weights table for consistency - Document layer indexing: Static + Layer 1,2,3... (UI) ↔ weights.layers[0,1,2...] - Add explanatory notes about 7D input and 4-of-8 channel display - Create doc/CNN_V2_BINARY_FORMAT.md with complete .bin specification - Cross-reference spec in CNN_V2.md and CNN_V2_WEB_TOOL.md Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
3 daystest_demo: Add beat-synchronized CNN post-processing with version selectionskal
- Add --cnn-version <1|2> flag to select between CNN v1 and v2 - Implement beat_phase modulation for dynamic blend in both CNN effects - Fix CNN v2 per-layer uniform buffer sharing (each layer needs own buffer) - Fix CNN v2 y-axis orientation to match render pass convention - Add Scene1Effect as base visual layer to test_demo timeline - Reorganize CNN v2 shaders into cnn_v2/ subdirectory - Update asset paths and documentation for new shader organization Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
3 daysUpdate docs and help messages for CNN v2 completionskal
Updated: - HOWTO.md: Complete pipeline, storage buffer, --validate mode - TODO.md: Mark CNN v2 complete, add QAT TODO - PROJECT_CONTEXT.md: Update Effects status - CNN_V2.md: Mark complete, add storage buffer notes - train_cnn_v2_full.sh: Add --help message All documentation now reflects: - Storage buffer architecture - Binary weight format - Live training progress - Validation-only mode - 8-bit quantization TODO
3 daysCNN v2: parametric static features - design docskal
Design document for CNN v2 with enhanced feature inputs: - 7D static features: RGBD + UV + sin encoding + bias - Per-layer configurable kernels (1×1, 3×3, 5×5) - Float16 weight storage (~6.4 KB vs 3.2 KB) - Multi-pass architecture with static feature compute Implementation plan: 1. Static features compute shader (RGBD + UV + sin + bias) 2. C++ effect class (CNNv2Effect) 3. Training pipeline (train_cnn_v2.py, export_cnn_v2_shader.py) 4. Validation tooling (validate_cnn_v2.sh) Files: - doc/CNN_V2.md: Complete technical design (architecture, training, export) - scripts/validate_cnn_v2.sh: End-to-end validation script - TODO.md: Add CNN v2 as Priority 2 task - doc/HOWTO.md: Add CNN v2 validation usage Target: <10 KB for 64k demo constraint handoff(Claude): CNN v2 design ready for implementation
generated by cgit v1.2.3 (git 2.51.0) at 2026-02-15 06:08:50 +0000