diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-12 12:23:17 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-12 12:23:17 +0100 |
| commit | 9dd123e1d25325f09167134e98c04fd6d7e91970 (patch) | |
| tree | 887656ec247850c3796152665d6dbe617ea1bf70 | |
| parent | dbbabc0b2f819e5e2e550ed20c03806dfd089252 (diff) | |
Update docs and help messages for CNN v2 completion
Updated:
- HOWTO.md: Complete pipeline, storage buffer, --validate mode
- TODO.md: Mark CNN v2 complete, add QAT TODO
- PROJECT_CONTEXT.md: Update Effects status
- CNN_V2.md: Mark complete, add storage buffer notes
- train_cnn_v2_full.sh: Add --help message
All documentation now reflects:
- Storage buffer architecture
- Binary weight format
- Live training progress
- Validation-only mode
- 8-bit quantization TODO
| -rw-r--r-- | PROJECT_CONTEXT.md | 2 | ||||
| -rw-r--r-- | TODO.md | 36 | ||||
| -rw-r--r-- | doc/CNN_V2.md | 7 | ||||
| -rw-r--r-- | doc/HOWTO.md | 52 | ||||
| -rwxr-xr-x | scripts/train_cnn_v2_full.sh | 18 | ||||
| -rw-r--r-- | validation_results/img_000_output.png | bin | 0 -> 198761 bytes | |||
| -rw-r--r-- | validation_results/img_001_output.png | bin | 0 -> 83763 bytes | |||
| -rw-r--r-- | validation_results/img_002_output.png | bin | 0 -> 103619 bytes | |||
| -rw-r--r-- | validation_results/img_003_output.png | bin | 0 -> 90164 bytes | |||
| -rw-r--r-- | validation_results/img_004_output.png | bin | 0 -> 145202 bytes | |||
| -rw-r--r-- | validation_results/img_005_output.png | bin | 0 -> 40745 bytes | |||
| -rw-r--r-- | validation_results/img_006_output.png | bin | 0 -> 290518 bytes | |||
| -rw-r--r-- | validation_results/img_007_output.png | bin | 0 -> 216602 bytes |
13 files changed, 75 insertions, 40 deletions
diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index b889b2e..2ed2b09 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -36,7 +36,7 @@ - **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. Comprehensive tests. - **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition. - **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline. -- **Effects:** CNN post-processing foundation (3-layer architecture, modular snippets). CNNEffect validated in demo. CNN v2 with parametric static features (7D input: RGBD + UV + sin encoding) ready for training. +- **Effects:** CNN post-processing foundation (3-layer architecture, modular snippets). CNNEffect validated in demo. CNN v2 complete: storage buffer architecture, binary weights (~3.2 KB), dynamic layer count, patch-based training (100 epochs, 3×3 kernels). TODO: 8-bit quantization. - **Tools:** CNN test tool (readback works, output incorrect - under investigation). Texture readback utility functional. Timeline editor (web-based, beat-aligned, audio playback). - **Build:** Asset dependency tracking. Size measurement. Hot-reload (debug-only). - **Testing:** **36/36 passing (100%)** @@ -24,33 +24,33 @@ Self-contained workspaces for parallel demo development. --- -## Priority 2: CNN v2 - Parametric Static Features (Task #85) [READY FOR TRAINING] +## Priority 2: CNN v2 - Parametric Static Features (Task #85) [COMPLETE] Enhanced CNN post-processing with multi-dimensional feature inputs. **Design:** `doc/CNN_V2.md` **Status:** -- ✅ Phase 1: Static features shader (RGBD + UV + sin encoding + bias → 8×f16, 3 mip levels) -- ✅ Phase 2: C++ effect class (CNNv2Effect skeleton, multi-pass architecture) -- ✅ Phase 3: Training pipeline (`train_cnn_v2.py`, `export_cnn_v2_shader.py`) -- ✅ Phase 4: Validation tooling (`scripts/validate_cnn_v2.sh`) -- ✅ Phase 5: Render pipeline (compute passes, bind groups, texture management) +- ✅ Phase 1-5: All implementation phases complete +- ✅ Storage buffer architecture (dynamic layer count support) +- ✅ Binary weight format (header + layer info + f16 weights) +- ✅ Training pipeline with live progress display +- ✅ Complete validation tooling (`train_cnn_v2_full.sh --validate`) +- ✅ All tests passing (36/36) -**Implementation complete:** -- Static features compute pass functional -- Multi-pass architecture ready -- Layer shader integration structure in place -- All tests passing (36/36) +**Features:** +- 7D static feature input (RGBD + UV + sin encoding + bias) +- Storage buffer weights (~3.2 KB for 3-layer, 8→4→4 config) +- Dynamic layer count (not hardcoded) +- Single compute shader with per-layer params +- Patch-based training (harris detector, 32×32 patches) +- Fast training config: 100 epochs, 3×3 kernels -**Next:** Train model, generate layer shaders, integrate into demo +**Performance:** +- Storage buffer overhead: ~10-20% vs constants (negligible @ 60fps) +- Target achieved: <10 KB for 64k demo constraint -**Key improvements over v1:** -- 7D static feature input (vs 4D RGB) -- Per-layer configurable kernels (1×1, 3×3, 5×5) -- Float16 weight storage (~6.4 KB) - -**Target:** <10 KB for 64k demo constraint +**TODO:** 8-bit quantization for 2× size reduction (~1.6 KB). Requires QAT. --- diff --git a/doc/CNN_V2.md b/doc/CNN_V2.md index b3b6587..9407934 100644 --- a/doc/CNN_V2.md +++ b/doc/CNN_V2.md @@ -13,10 +13,13 @@ CNN v2 extends the original CNN post-processing effect with parametric static fe - Multi-frequency position encoding (NeRF-style) - Per-layer configurable kernel sizes (1×1, 3×3, 5×5) - Variable channel counts per layer -- Float16 weight storage (GPU-optimized) +- Float16 weight storage (~3.2 KB for 3-layer model) - Bias integrated as static feature dimension +- Storage buffer architecture (dynamic layer count) +- Binary weight format for runtime loading -**Status:** Design complete, ready for implementation +**Status:** ✅ Complete. Training pipeline functional, validation tools ready. +**TODO:** 8-bit quantization with QAT for 2× size reduction (~1.6 KB) --- diff --git a/doc/HOWTO.md b/doc/HOWTO.md index 7be5246..1ae1d94 100644 --- a/doc/HOWTO.md +++ b/doc/HOWTO.md @@ -132,36 +132,52 @@ Processes entire image with sliding window (matches WGSL): ### CNN v2 Training -Enhanced CNN with parametric static features (7D input: RGBD + UV + sin encoding + bias): +Enhanced CNN with parametric static features (7D input: RGBD + UV + sin encoding + bias). +**Complete Pipeline** (recommended): ```bash -# Train CNN v2 with default config (1×1, 3×3, 5×5 kernels, 16→8→4 channels) +# Train → Export → Build → Validate +./scripts/train_cnn_v2_full.sh +``` + +Config: 100 epochs, 3×3 kernels, 8→4→4 channels, patch-based (harris detector). +- Live progress with single-line update +- Validates all input images on final epoch +- Exports binary weights (storage buffer architecture) + +**Validation Only** (skip training): +```bash +# Use latest checkpoint +./scripts/train_cnn_v2_full.sh --validate + +# Use specific checkpoint +./scripts/train_cnn_v2_full.sh --validate checkpoints/checkpoint_epoch_50.pth +``` + +**Manual Training:** +```bash +# Default config ./training/train_cnn_v2.py \ - --input training/input/ --target training/output/ \ - --epochs 5000 --batch-size 16 \ - --checkpoint-every 1000 + --input training/input/ --target training/target_2/ \ + --epochs 100 --batch-size 16 --checkpoint-every 5 -# Custom architecture (smaller for size optimization) +# Custom architecture ./training/train_cnn_v2.py \ - --input training/input/ --target training/output/ \ - --kernel-sizes 1 3 3 --channels 8 4 4 \ + --input training/input/ --target training/target_2/ \ + --kernel-sizes 1 3 5 --channels 16 8 4 \ --epochs 5000 --batch-size 16 ``` -**Export shaders:** +**Export Binary Weights:** ```bash -./training/export_cnn_v2_shader.py checkpoints/checkpoint_epoch_5000.pth \ - --output-dir workspaces/main/shaders +./training/export_cnn_v2_weights.py checkpoints/checkpoint_epoch_100.pth \ + --output-weights workspaces/main/cnn_v2_weights.bin ``` -Generates `cnn_v2_layer_0.wgsl`, `cnn_v2_layer_1.wgsl`, `cnn_v2_layer_2.wgsl` with f16 weights. +Generates binary format: header + layer info + f16 weights (~3.2 KB for 3-layer model). +Storage buffer architecture allows dynamic layer count. -### CNN v2 Validation - -End-to-end testing: checkpoint → shaders → build → test images → results - -```bash -./scripts/validate_cnn_v2.sh checkpoints/checkpoint_epoch_5000.pth +**TODO:** 8-bit quantization for 2× size reduction (~1.6 KB). Requires quantization-aware training (QAT). # Options: # -i DIR Test images directory (default: training/validation) diff --git a/scripts/train_cnn_v2_full.sh b/scripts/train_cnn_v2_full.sh index b3ece8a..fc9355a 100755 --- a/scripts/train_cnn_v2_full.sh +++ b/scripts/train_cnn_v2_full.sh @@ -1,7 +1,18 @@ #!/bin/bash # Complete CNN v2 Training Pipeline # Train → Export → Build → Validate -# Usage: ./train_cnn_v2_full.sh [--validate [checkpoint.pth]] +# Usage: ./train_cnn_v2_full.sh [OPTIONS] +# +# OPTIONS: +# (none) Run complete pipeline: train → export → build → validate +# --validate Validate only (skip training, use existing weights) +# --validate CHECKPOINT Validate with specific checkpoint file +# --help Show this help message +# +# Examples: +# ./train_cnn_v2_full.sh +# ./train_cnn_v2_full.sh --validate +# ./train_cnn_v2_full.sh --validate checkpoints/checkpoint_epoch_50.pth set -e @@ -12,6 +23,11 @@ cd "$PROJECT_ROOT" VALIDATE_ONLY=false VALIDATE_CHECKPOINT="" +if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then + head -20 "$0" | grep "^#" | grep -v "^#!/" | sed 's/^# *//' + exit 0 +fi + if [ "$1" = "--validate" ]; then VALIDATE_ONLY=true if [ -n "$2" ]; then diff --git a/validation_results/img_000_output.png b/validation_results/img_000_output.png Binary files differnew file mode 100644 index 0000000..ea63a31 --- /dev/null +++ b/validation_results/img_000_output.png diff --git a/validation_results/img_001_output.png b/validation_results/img_001_output.png Binary files differnew file mode 100644 index 0000000..8c5a6e0 --- /dev/null +++ b/validation_results/img_001_output.png diff --git a/validation_results/img_002_output.png b/validation_results/img_002_output.png Binary files differnew file mode 100644 index 0000000..2044840 --- /dev/null +++ b/validation_results/img_002_output.png diff --git a/validation_results/img_003_output.png b/validation_results/img_003_output.png Binary files differnew file mode 100644 index 0000000..9caff40 --- /dev/null +++ b/validation_results/img_003_output.png diff --git a/validation_results/img_004_output.png b/validation_results/img_004_output.png Binary files differnew file mode 100644 index 0000000..b9fa2da --- /dev/null +++ b/validation_results/img_004_output.png diff --git a/validation_results/img_005_output.png b/validation_results/img_005_output.png Binary files differnew file mode 100644 index 0000000..0b43ab0 --- /dev/null +++ b/validation_results/img_005_output.png diff --git a/validation_results/img_006_output.png b/validation_results/img_006_output.png Binary files differnew file mode 100644 index 0000000..26885d3 --- /dev/null +++ b/validation_results/img_006_output.png diff --git a/validation_results/img_007_output.png b/validation_results/img_007_output.png Binary files differnew file mode 100644 index 0000000..2a3a411 --- /dev/null +++ b/validation_results/img_007_output.png |
