summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-10 22:54:38 +0100
committerskal <pascal.massimino@gmail.com>2026-02-10 22:54:38 +0100
commit2adcf1bac1ec651861930eb2af00641eb23f6ef1 (patch)
tree2208b753b68783be5a26906b5bc1690c23a267d0 /doc
parent58f276378735e0b51f4d1517a844357e45e376a7 (diff)
docs: Update CNN training documentation with patch extraction
Streamlined and updated all training docs with new patch-based approach. Changes: - HOWTO.md: Updated training section with patch/full-image examples - CNN_EFFECT.md: Streamlined training workflow, added detector info - training/README.md: Complete rewrite with detector comparison table New sections: - Detector comparison (harris, fast, shi-tomasi, gradient) - Practical examples for different use cases - Tips for patch size and batch size selection - Benefits of patch-based training Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'doc')
-rw-r--r--doc/CNN_EFFECT.md71
-rw-r--r--doc/HOWTO.md33
2 files changed, 51 insertions, 53 deletions
diff --git a/doc/CNN_EFFECT.md b/doc/CNN_EFFECT.md
index 22cf985..06065b1 100644
--- a/doc/CNN_EFFECT.md
+++ b/doc/CNN_EFFECT.md
@@ -98,73 +98,54 @@ workspaces/main/shaders/cnn/
### 1. Prepare Training Data
-Collect input/target image pairs:
-- **Input:** RGBA (RGB + depth as alpha channel, D=1/z)
-- **Target:** Grayscale stylized output
-
-```bash
-training/input/img_000.png # RGBA render (RGB + depth)
+Input/target image pairs:
+```
+training/input/img_000.png # RGBA (RGB + alpha)
training/output/img_000.png # Grayscale target
```
-**Note:** Input images must be RGBA where alpha = inverse depth (1/z)
+**Note:** Alpha channel can be depth (1/z) or constant (255). Network learns from RGB primarily.
### 2. Train Network
+**Patch-based (Recommended)** - Preserves natural pixel scale:
```bash
python3 training/train_cnn.py \
- --input training/input \
- --target training/output \
- --layers 1 \
- --kernel-sizes 3 \
- --epochs 500 \
- --checkpoint-every 50
+ --input training/input --target training/output \
+ --patch-size 32 --patches-per-image 64 --detector harris \
+ --layers 3 --kernel-sizes 3,5,3 \
+ --epochs 5000 --batch-size 16 --checkpoint-every 1000
```
-**Multi-layer example (3 layers with varying kernel sizes):**
+**Detectors:** `harris` (corners), `fast` (features), `shi-tomasi` (corners), `gradient` (edges)
+
+**Full-image (Legacy)** - Resizes to 256×256:
```bash
python3 training/train_cnn.py \
- --input training/input \
- --target training/output \
- --layers 3 \
- --kernel-sizes 3,5,3 \
- --epochs 1000 \
- --checkpoint-every 100
+ --input training/input --target training/output \
+ --layers 3 --kernel-sizes 3,5,3 \
+ --epochs 10000 --batch-size 8 --checkpoint-every 1000
```
-**Note:** Training script auto-generates:
-- `cnn_weights_generated.wgsl` - weight arrays for all layers
-- `cnn_layer.wgsl` - shader with layer switches and original input binding
+**Auto-generates:**
+- `cnn_weights_generated.wgsl` - Weight arrays
+- `cnn_layer.wgsl` - Layer shader
-**Resume from checkpoint:**
-```bash
-python3 training/train_cnn.py \
- --input training/input \
- --target training/output \
- --resume training/checkpoints/checkpoint_epoch_200.pth
-```
+### 3. Export & Validate
-**Export WGSL from checkpoint (no training):**
```bash
-python3 training/train_cnn.py \
- --export-only training/checkpoints/checkpoint_epoch_200.pth \
- --output workspaces/main/shaders/cnn/cnn_weights_generated.wgsl
-```
+# Export shaders
+./training/train_cnn.py --export-only checkpoints/checkpoint_epoch_5000.pth
-**Generate ground truth (for shader validation):**
-```bash
-python3 training/train_cnn.py \
- --infer training/input/img_000.png \
- --export-only training/checkpoints/checkpoint_epoch_200.pth \
- --output training/ground_truth.png
+# Generate ground truth
+./training/train_cnn.py --infer input.png \
+ --export-only checkpoints/checkpoint_epoch_5000.pth --output ground_truth.png
```
-### 3. Rebuild Demo
+### 4. Rebuild Demo
-Training script auto-generates both `cnn_weights_generated.wgsl` and `cnn_layer.wgsl`:
```bash
-cmake --build build -j4
-./build/demo64k
+cmake --build build -j4 && ./build/demo64k
```
---
diff --git a/doc/HOWTO.md b/doc/HOWTO.md
index 5ea6afd..ba550bb 100644
--- a/doc/HOWTO.md
+++ b/doc/HOWTO.md
@@ -88,23 +88,40 @@ make run_util_tests # Utility tests
## Training
+### Patch-Based (Recommended)
+Extracts patches at salient points, preserves natural pixel scale:
```bash
-./training/train_cnn.py --layers 3 --kernel_sizes 3,5,3 --epochs 10000 --batch_size 8 --input training/input/ --target training/output/ --checkpoint-every 1000
+# Train with 32×32 patches at detected corners/edges
+./training/train_cnn.py \
+ --input training/input/ --target training/output/ \
+ --patch-size 32 --patches-per-image 64 --detector harris \
+ --layers 3 --kernel_sizes 3,5,3 --epochs 5000 --batch_size 16 \
+ --checkpoint-every 1000
```
-Generate shaders from checkpoint:
+**Detectors:** `harris` (default), `fast`, `shi-tomasi`, `gradient`
+
+### Full-Image (Legacy)
+Resizes to 256×256 (distorts scale):
```bash
-./training/train_cnn.py --export-only training/checkpoints/checkpoint_epoch_7000.pth
+./training/train_cnn.py \
+ --input training/input/ --target training/output/ \
+ --layers 3 --kernel_sizes 3,5,3 --epochs 10000 --batch_size 8 \
+ --checkpoint-every 1000
```
-Generate ground truth (for shader validation):
+### Export & Validation
```bash
-./training/train_cnn.py --infer input.png --export-only checkpoints/checkpoint_epoch_7000.pth --output ground_truth.png
+# Generate shaders from checkpoint
+./training/train_cnn.py --export-only checkpoints/checkpoint_epoch_5000.pth
+
+# Generate ground truth for comparison
+./training/train_cnn.py --infer input.png \
+ --export-only checkpoints/checkpoint_epoch_5000.pth \
+ --output ground_truth.png
```
-**Note:** Kernel sizes must match shader functions:
-- 3×3 kernel → `cnn_conv3x3_7to4` (36 weights: 9 pos × 4 channels)
-- 5×5 kernel → `cnn_conv5x5_7to4` (100 weights: 25 pos × 4 channels)
+**Kernel sizes:** 3×3 (36 weights), 5×5 (100 weights), 7×7 (196 weights)
---