diff options
Diffstat (limited to 'training')
| -rw-r--r-- | training/README.md | 233 |
1 files changed, 124 insertions, 109 deletions
diff --git a/training/README.md b/training/README.md index 08379ee..0a46718 100644 --- a/training/README.md +++ b/training/README.md @@ -1,167 +1,182 @@ -# Image Style Processor +# CNN Training Tools -A comprehensive Python script that applies artistic hand-drawn and futuristic effects to images. +Tools for training and preparing data for the CNN post-processing effect. -## Requirements +--- -- Python 3 -- OpenCV (cv2) -- NumPy +## train_cnn.py + +PyTorch-based training script for image-to-image stylization. + +### Basic Usage -Install dependencies: ```bash -pip install opencv-python numpy +python3 train_cnn.py --input <input_dir> --target <target_dir> [options] ``` -## Usage +### Examples +**Single layer, 3×3 kernel:** ```bash -python3 image_style_processor.py <input_directory> <output_directory> <style> +python3 train_cnn.py --input training/input --target training/output \ + --layers 1 --kernel-sizes 3 --epochs 500 ``` -### Arguments +**Multi-layer, mixed kernels:** +```bash +python3 train_cnn.py --input training/input --target training/output \ + --layers 3 --kernel-sizes 3,5,3 --epochs 1000 +``` -- `input_directory`: Directory containing your input images (PNG, JPG, JPEG) -- `output_directory`: Directory where processed images will be saved (created if doesn't exist) -- `style`: The artistic style to apply (see below) +**With checkpointing:** +```bash +python3 train_cnn.py --input training/input --target training/output \ + --epochs 500 --checkpoint-every 50 +``` -## Available Styles +**Resume from checkpoint:** +```bash +python3 train_cnn.py --input training/input --target training/output \ + --resume training/checkpoints/checkpoint_epoch_200.pth +``` + +### Options -### Sketch Styles +| Option | Default | Description | +|--------|---------|-------------| +| `--input` | *required* | Input image directory | +| `--target` | *required* | Target image directory | +| `--layers` | 1 | Number of CNN layers | +| `--kernel-sizes` | 3 | Comma-separated kernel sizes (auto-repeats if single value) | +| `--epochs` | 100 | Training epochs | +| `--batch-size` | 4 | Batch size | +| `--learning-rate` | 0.001 | Learning rate | +| `--output` | `workspaces/main/shaders/cnn/cnn_weights_generated.wgsl` | Output WGSL file | +| `--checkpoint-every` | 0 | Save checkpoint every N epochs (0=disabled) | +| `--checkpoint-dir` | `training/checkpoints` | Checkpoint directory | +| `--resume` | None | Resume from checkpoint file | -1. **pencil_sketch** - Dense cross-hatching with progressive layers in shadows - - Best for: Detailed technical drawings, architectural scenes - - Features: Clean line art, 5 layers of cross-hatching, strong shadow definition +### Architecture -2. **ink_drawing** - Bold black outlines with comic book aesthetic - - Best for: Graphic novel style, high contrast scenes - - Features: Bold outlines, posterized tones, minimal shading +- **Layer 0:** `CoordConv2d` - accepts (x,y) patch center + 3×3 RGBA samples +- **Layers 1+:** Standard `Conv2d` - 3×3 RGBA samples only +- **Activation:** Tanh between layers +- **Output:** Residual connection (30% stylization blend) -3. **charcoal_pastel** - Dramatic contrasts with soft, smudged textures - - Best for: Portraits, dramatic landscapes - - Features: Soft blending, grainy texture, highlighted areas +### Requirements -4. **conte_crayon** - Directional strokes following image contours - - Best for: Figure studies, natural forms - - Features: Stroke direction follows gradients, cross-hatching in dark areas +```bash +pip install torch torchvision pillow +``` -5. **gesture_sketch** - Loose, quick observational sketch style - - Best for: Quick studies, energetic compositions - - Features: Randomized line wobble, sparse suggestion lines +--- -### Futuristic Styles +## image_style_processor.py -6. **circuit_board** - Tech blueprint with circuit paths and geometric patterns - - Best for: Sci-fi imagery, technological themes - - Features: Multi-layer circuit paths, connection nodes, technical grid overlay +Generates stylized target images from raw renders. -7. **glitch_art** - Digital corruption with scan line shifts and pixel sorting - - Best for: Cyberpunk aesthetics, digital art - - Features: Horizontal scan artifacts, block displacement, pixel sorting, noise strips +### Usage -8. **wireframe_topo** - Topographic contour lines with holographic grid - - Best for: Landscape, abstract patterns, sci-fi hologram effect - - Features: 20 contour levels, scan lines, measurement markers, grid overlay +```bash +python3 image_style_processor.py <input_dir> <output_dir> <style> +``` -9. **data_mosaic** - Voronoi geometric fragmentation with angular cells - - Best for: Abstract art, geometric compositions - - Features: 200 Voronoi cells, posterized tones, embedded geometric patterns +### Available Styles -10. **holographic_scan** - CRT/hologram display with scanlines and HUD elements - - Best for: Retro-futuristic, heads-up display aesthetic - - Features: Scanlines, interference patterns, glitch effects, corner brackets, crosshair +**Sketch:** +- `pencil_sketch` - Dense cross-hatching +- `ink_drawing` - Bold outlines, comic style +- `charcoal_pastel` - Soft, dramatic contrasts +- `conte_crayon` - Directional strokes +- `gesture_sketch` - Loose, energetic lines -## Examples +**Futuristic:** +- `circuit_board` - Tech blueprint +- `glitch_art` - Digital corruption +- `wireframe_topo` - Topographic contours +- `data_mosaic` - Voronoi fragmentation +- `holographic_scan` - CRT/HUD aesthetic -### Sketch Effects +### Examples -Process images with pencil sketch: ```bash -python3 image_style_processor.py ./photos ./output pencil_sketch -``` +# Generate pencil sketch targets +python3 image_style_processor.py input/ output/ pencil_sketch -Apply ink drawing style: -```bash -python3 image_style_processor.py ./input ./sketches ink_drawing +# Generate glitch art targets +python3 image_style_processor.py input/ output/ glitch_art ``` -Create charcoal effect: +### Requirements + ```bash -python3 image_style_processor.py ./images ./results charcoal_pastel +pip install opencv-python numpy ``` -### Futuristic Effects +--- + +## Workflow + +### 1. Render Raw Frames -Apply circuit board style: +Generate raw 3D renders as input: ```bash -python3 image_style_processor.py ./photos ./output circuit_board +./build/demo64k --headless --duration 5 --output training/input/ ``` -Create glitch art: +### 2. Generate Stylized Targets + +Apply artistic style: ```bash -python3 image_style_processor.py ./input ./glitched glitch_art +python3 training/image_style_processor.py training/input/ training/output/ pencil_sketch ``` -Apply holographic effect: +### 3. Train CNN + +Train network to reproduce the style: ```bash -python3 image_style_processor.py ./images ./holo holographic_scan +python3 training/train_cnn.py \ + --input training/input \ + --target training/output \ + --epochs 500 \ + --checkpoint-every 50 ``` -## Output +### 4. Rebuild Demo -- Processed images are saved to the output directory with **the same filename** as the input -- Supported input formats: PNG, JPG, JPEG (case-insensitive) -- Output format: PNG (preserves quality) -- Original images are never modified +Weights auto-exported to `cnn_weights_generated.wgsl`: +```bash +cmake --build build -j4 +./build/demo64k +``` -## Style Comparison +--- -### Sketch Styles -- **pencil_sketch**: Most detailed, traditional drawing look -- **ink_drawing**: Boldest, most graphic/comic-like -- **charcoal_pastel**: Softest, most artistic/painterly -- **conte_crayon**: Most directional, follows contours -- **gesture_sketch**: Loosest, most expressive +## Tips -### Futuristic Styles -- **circuit_board**: Cleanest, most technical/blueprint-like -- **glitch_art**: Most chaotic, digital corruption aesthetic -- **wireframe_topo**: Most structured, topographic/hologram feel -- **data_mosaic**: Most geometric, fragmented cells -- **holographic_scan**: Most retro-futuristic, HUD/CRT display +- **Training data:** 10-50 image pairs recommended +- **Resolution:** 256×256 (auto-resized during training) +- **Checkpoints:** Save every 50-100 epochs for long runs +- **Loss plateaus:** Try lower learning rate (0.0001) or more layers +- **Residual connection:** Prevents catastrophic divergence (input always blended in) -## Tips +--- -- Images are automatically converted to grayscale before processing -- All styles work best with high-resolution images (300+ DPI recommended) -- Processing time varies by style: - - Fast: ink_drawing, glitch_art, holographic_scan - - Medium: charcoal_pastel, gesture_sketch, circuit_board, wireframe_topo - - Slow: pencil_sketch, conte_crayon, data_mosaic (due to intensive computation) -- For batch processing large collections, consider processing in smaller batches -- Randomized styles (glitch_art, gesture_sketch, data_mosaic) will produce slightly different results each run +## Coordinate-Aware Layer 0 -## Technical Notes +Layer 0 receives normalized (x,y) patch center coordinates, enabling position-dependent effects: -### Randomization -Some styles use randomization for natural variation: -- **glitch_art**: Random scan line shifts, block positions -- **gesture_sketch**: Random line wobble, stroke placement -- **data_mosaic**: Random Voronoi cell centers -- **circuit_board**: Random pattern placement in dark regions -- **holographic_scan**: Random glitch line positions +- **Vignetting:** Darker edges +- **Radial gradients:** Center-focused stylization +- **Corner effects:** Edge-specific treatments -### Processing Details -- **pencil_sketch**: Uses 5-level progressive cross-hatching algorithm -- **conte_crayon**: Follows Sobel gradients for directional strokes -- **wireframe_topo**: Generates 20 brightness-based contour levels -- **data_mosaic**: Creates 200 Voronoi cells via nearest-neighbor algorithm -- **holographic_scan**: Applies scanline patterns and interference waves +Training coordinate grid is auto-generated during forward pass. No manual intervention needed. -## License +Size impact: +32B coord weights (kernel-agnostic). -Free to use and modify for any purpose. +--- -## Version +## References -Version 1.0 - Complete collection of 10 artistic styles (5 sketch + 5 futuristic) +- **CNN Effect Documentation:** `doc/CNN_EFFECT.md` +- **Training Architecture:** See `train_cnn.py` (CoordConv2d class) |