| Age | Commit message (Collapse) | Author |
|---|
|
**Training changes:**
- Final layer now outputs [0,1] directly with torch.clamp()
- Removed denormalization step (was converting [-1,1] to [0,1])
- Network learns [0,1] output natively
**Shader generation fixes:**
- Layer 0 uses _src variant (5 params, normalizes [0,1] input internally)
- Removed pre-normalization of input texture (handled by _src)
- Final layer blending: gray_out already [0,1], no denormalization needed
- Added generate_conv_src_function() for all kernel sizes
- Auto-generates _src variants when exporting (skips if exists)
**Cleanup:**
- Removed obsolete 4-channel functions from cnn_conv5x5.wgsl
- Keep only 7-channel variants (_7to4, _7to1, _7to4_src)
**Normalization flow:**
[0,1] texture → _src normalizes to [-1,1] → tanh [-1,1] → ... → final conv [0,1] clipped
handoff(Claude): CNN normalization pipeline fixed and consistent with training
|
|
|
|
Normalize textures once in fs_main instead of in every conv function.
Keep all intermediate layers in [-1,1] range, denormalize only for final display.
Changes:
- train_cnn.py: Generator normalizes input once, keeps [-1,1] between layers
- cnn_conv*.wgsl: Remove texture normalization (already [-1,1])
- cnn_layer.wgsl: Regenerated with new normalization flow
- CNN_EFFECT.md: Updated documentation
Eliminates redundant [0,1]↔[-1,1] conversions, reducing shader complexity.
handoff(Claude): CNN normalization optimized, all tests passing (35/36).
|
|
Upgrade CNN architecture to process RGBD input, output grayscale, with
7-channel layer inputs (RGBD + UV coords + grayscale).
Architecture changes:
- Inner layers: Conv2d(7→4) output RGBD
- Final layer: Conv2d(7→1) output grayscale
- All inputs normalized to [-1,1] for tanh activation
- Removed CoordConv2d in favor of unified 7-channel input
Training (train_cnn.py):
- SimpleCNN: 7→4 (inner), 7→1 (final) architecture
- Forward: Normalize RGBD/coords/gray to [-1,1]
- Weight export: array<array<f32, 8>, 36> (inner), array<f32, 8>, 9> (final)
- Dataset: Load RGBA (RGBD) input
Shaders (cnn_conv3x3.wgsl):
- Added cnn_conv3x3_7to4: 7-channel input → RGBD output
- Added cnn_conv3x3_7to1: 7-channel input → grayscale output
- Both normalize inputs and use flattened weight arrays
Documentation:
- CNN_EFFECT.md: Updated architecture, training, weight format
- CNN_RGBD_GRAYSCALE_SUMMARY.md: Implementation summary
- HOWTO.md: Added training command example
Next: Train with RGBD input data
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
- Implement CoordConv2d custom layer accepting (x,y) patch center
- Split layer 0 weights: rgba_weights (9x mat4x4) + coord_weights (mat2x4)
- Add *_with_coord() functions to 3x3/5x5/7x7 convolution shaders
- Update training script to generate coordinate grid and export split weights
- Regenerate placeholder weights with new format
Size impact: +32B coord weights + ~100B shader code = +132B total
All 36 tests passing (100%)
handoff(Claude): CNN coordinate awareness implemented, ready for training
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
Implements multi-layer convolutional neural network shader for stylized
post-processing of 3D rendered scenes:
**Core Components:**
- CNNEffect: C++ effect class with single-layer rendering (expandable to multi-pass)
- Modular WGSL snippets: cnn_activation, cnn_conv3x3/5x5/7x7, cnn_weights_generated
- Placeholder identity-like weights for initial testing (to be replaced by trained weights)
**Architecture:**
- Flexible kernel sizes (3×3, 5×5, 7×7) via separate snippet files
- ShaderComposer integration (#include resolution)
- Residual connections (input + processed output)
- Supports parallel convolutions (design ready, single conv implemented)
**Size Impact:**
- ~3-4 KB shader code (snippets + main shader)
- ~2-4 KB weights (depends on network architecture when trained)
- Total: ~5-8 KB (acceptable for 64k demo)
**Testing:**
- CNNEffect added to test_demo_effects.cc
- 36/36 tests passing (100%)
**Next Steps:**
- Training script (scripts/train_cnn.py) to generate real weights
- Multi-layer rendering with ping-pong textures
- Weight quantization for size optimization
handoff(Claude): CNN effect foundation complete, ready for training integration
|
