diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-10 22:54:38 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-10 22:54:38 +0100 |
| commit | 2adcf1bac1ec651861930eb2af00641eb23f6ef1 (patch) | |
| tree | 2208b753b68783be5a26906b5bc1690c23a267d0 /doc | |
| parent | 58f276378735e0b51f4d1517a844357e45e376a7 (diff) | |
docs: Update CNN training documentation with patch extraction
Streamlined and updated all training docs with new patch-based approach.
Changes:
- HOWTO.md: Updated training section with patch/full-image examples
- CNN_EFFECT.md: Streamlined training workflow, added detector info
- training/README.md: Complete rewrite with detector comparison table
New sections:
- Detector comparison (harris, fast, shi-tomasi, gradient)
- Practical examples for different use cases
- Tips for patch size and batch size selection
- Benefits of patch-based training
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/CNN_EFFECT.md | 71 | ||||
| -rw-r--r-- | doc/HOWTO.md | 33 |
2 files changed, 51 insertions, 53 deletions
diff --git a/doc/CNN_EFFECT.md b/doc/CNN_EFFECT.md index 22cf985..06065b1 100644 --- a/doc/CNN_EFFECT.md +++ b/doc/CNN_EFFECT.md @@ -98,73 +98,54 @@ workspaces/main/shaders/cnn/ ### 1. Prepare Training Data -Collect input/target image pairs: -- **Input:** RGBA (RGB + depth as alpha channel, D=1/z) -- **Target:** Grayscale stylized output - -```bash -training/input/img_000.png # RGBA render (RGB + depth) +Input/target image pairs: +``` +training/input/img_000.png # RGBA (RGB + alpha) training/output/img_000.png # Grayscale target ``` -**Note:** Input images must be RGBA where alpha = inverse depth (1/z) +**Note:** Alpha channel can be depth (1/z) or constant (255). Network learns from RGB primarily. ### 2. Train Network +**Patch-based (Recommended)** - Preserves natural pixel scale: ```bash python3 training/train_cnn.py \ - --input training/input \ - --target training/output \ - --layers 1 \ - --kernel-sizes 3 \ - --epochs 500 \ - --checkpoint-every 50 + --input training/input --target training/output \ + --patch-size 32 --patches-per-image 64 --detector harris \ + --layers 3 --kernel-sizes 3,5,3 \ + --epochs 5000 --batch-size 16 --checkpoint-every 1000 ``` -**Multi-layer example (3 layers with varying kernel sizes):** +**Detectors:** `harris` (corners), `fast` (features), `shi-tomasi` (corners), `gradient` (edges) + +**Full-image (Legacy)** - Resizes to 256×256: ```bash python3 training/train_cnn.py \ - --input training/input \ - --target training/output \ - --layers 3 \ - --kernel-sizes 3,5,3 \ - --epochs 1000 \ - --checkpoint-every 100 + --input training/input --target training/output \ + --layers 3 --kernel-sizes 3,5,3 \ + --epochs 10000 --batch-size 8 --checkpoint-every 1000 ``` -**Note:** Training script auto-generates: -- `cnn_weights_generated.wgsl` - weight arrays for all layers -- `cnn_layer.wgsl` - shader with layer switches and original input binding +**Auto-generates:** +- `cnn_weights_generated.wgsl` - Weight arrays +- `cnn_layer.wgsl` - Layer shader -**Resume from checkpoint:** -```bash -python3 training/train_cnn.py \ - --input training/input \ - --target training/output \ - --resume training/checkpoints/checkpoint_epoch_200.pth -``` +### 3. Export & Validate -**Export WGSL from checkpoint (no training):** ```bash -python3 training/train_cnn.py \ - --export-only training/checkpoints/checkpoint_epoch_200.pth \ - --output workspaces/main/shaders/cnn/cnn_weights_generated.wgsl -``` +# Export shaders +./training/train_cnn.py --export-only checkpoints/checkpoint_epoch_5000.pth -**Generate ground truth (for shader validation):** -```bash -python3 training/train_cnn.py \ - --infer training/input/img_000.png \ - --export-only training/checkpoints/checkpoint_epoch_200.pth \ - --output training/ground_truth.png +# Generate ground truth +./training/train_cnn.py --infer input.png \ + --export-only checkpoints/checkpoint_epoch_5000.pth --output ground_truth.png ``` -### 3. Rebuild Demo +### 4. Rebuild Demo -Training script auto-generates both `cnn_weights_generated.wgsl` and `cnn_layer.wgsl`: ```bash -cmake --build build -j4 -./build/demo64k +cmake --build build -j4 && ./build/demo64k ``` --- diff --git a/doc/HOWTO.md b/doc/HOWTO.md index 5ea6afd..ba550bb 100644 --- a/doc/HOWTO.md +++ b/doc/HOWTO.md @@ -88,23 +88,40 @@ make run_util_tests # Utility tests ## Training +### Patch-Based (Recommended) +Extracts patches at salient points, preserves natural pixel scale: ```bash -./training/train_cnn.py --layers 3 --kernel_sizes 3,5,3 --epochs 10000 --batch_size 8 --input training/input/ --target training/output/ --checkpoint-every 1000 +# Train with 32×32 patches at detected corners/edges +./training/train_cnn.py \ + --input training/input/ --target training/output/ \ + --patch-size 32 --patches-per-image 64 --detector harris \ + --layers 3 --kernel_sizes 3,5,3 --epochs 5000 --batch_size 16 \ + --checkpoint-every 1000 ``` -Generate shaders from checkpoint: +**Detectors:** `harris` (default), `fast`, `shi-tomasi`, `gradient` + +### Full-Image (Legacy) +Resizes to 256×256 (distorts scale): ```bash -./training/train_cnn.py --export-only training/checkpoints/checkpoint_epoch_7000.pth +./training/train_cnn.py \ + --input training/input/ --target training/output/ \ + --layers 3 --kernel_sizes 3,5,3 --epochs 10000 --batch_size 8 \ + --checkpoint-every 1000 ``` -Generate ground truth (for shader validation): +### Export & Validation ```bash -./training/train_cnn.py --infer input.png --export-only checkpoints/checkpoint_epoch_7000.pth --output ground_truth.png +# Generate shaders from checkpoint +./training/train_cnn.py --export-only checkpoints/checkpoint_epoch_5000.pth + +# Generate ground truth for comparison +./training/train_cnn.py --infer input.png \ + --export-only checkpoints/checkpoint_epoch_5000.pth \ + --output ground_truth.png ``` -**Note:** Kernel sizes must match shader functions: -- 3×3 kernel → `cnn_conv3x3_7to4` (36 weights: 9 pos × 4 channels) -- 5×5 kernel → `cnn_conv5x5_7to4` (100 weights: 25 pos × 4 channels) +**Kernel sizes:** 3×3 (36 weights), 5×5 (100 weights), 7×7 (196 weights) --- |
