diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-10 08:08:25 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-10 08:08:25 +0100 |
| commit | 75af266889b61b5722d842a1a1eb23f79bc06a85 (patch) | |
| tree | c2ddb094eb24e47826ca8fd029e0abbf16b5d16b /doc/CNN.md | |
| parent | 47397444b30b0f461b1633297a68300179586fda (diff) | |
docs: Add CNN effect documentation and update project status
**New Documentation:**
- `doc/CNN_EFFECT.md` (223 lines): Comprehensive implementation guide
- Architecture overview (file structure, shader composition)
- Usage examples (C++ API, timeline integration)
- Training workflow (planned)
- Implementation details (convolution signatures, weight storage)
- Size budget breakdown (~5-8 KB total)
- Testing and troubleshooting
**Updated Documentation:**
- `doc/CNN.md`: Added implementation status section
- Completed items (✅ modular shaders, C++ class, tests)
- Pending items (⏳ training script, multi-layer, quantization)
- Size impact summary
- `PROJECT_CONTEXT.md`:
- Added "Effects: CNN post-processing foundation" to Current Status
- Added `CNN_EFFECT.md` to Technical Reference list
**Summary:**
CNN effect foundation complete with modular WGSL architecture, ready for
training script integration. All tests passing (36/36). ~5-8 KB footprint.
handoff(Claude): Documentation complete for CNN effect implementation
Diffstat (limited to 'doc/CNN.md')
| -rw-r--r-- | doc/CNN.md | 40 |
1 files changed, 34 insertions, 6 deletions
@@ -1,11 +1,15 @@ # Convolutional Neural Net Shader (CNN) post-processing +**Status:** ✅ Foundation implemented (single-layer, expandable to multi-pass) + ## Idea Have the input 3d scene be processed by a multi-layer CNN trained on the side. Input: some rendered scene. Output: 'stylized' scene with CNN post-processing. +**See `doc/CNN_EFFECT.md` for implementation details, usage, and API reference.** + ## Shader implementation ### input / output @@ -36,16 +40,40 @@ we need 3 or 4 layer ? Several different shaders for each layer. Ping-pong for input/output texture buffer between each layers? -## Training +## Implementation Status + +**Completed:** +- ✅ Modular WGSL shader architecture (6 snippet files) +- ✅ CNNEffect C++ class (single-layer rendering) +- ✅ ShaderComposer integration (#include resolution) +- ✅ Asset registration (7 new shader assets) +- ✅ Test coverage (test_demo_effects.cc) +- ✅ Placeholder identity weights for testing + +**Size:** ~3-4 KB shader code + ~2-4 KB weights = **5-8 KB total** + +**Pending:** +- ⏳ Training script (`scripts/train_cnn.py`) to generate real weights +- ⏳ Multi-layer rendering with ping-pong textures +- ⏳ Weight quantization for size optimization + +--- + +## Training (To Be Implemented) The layer weight/bias data are hard-coded in the shaders. -Need training with external python script. -File: CNN.py contains an example of what the training script could be. -Just an example, doesn't match our requirement yet. +Training workflow: + +1. Prepare image pairs (before: raw render, after: target style) +2. Run `python scripts/train_cnn.py --input scene.png --target stylized.png` +3. Script generates `cnn_weights_generated.wgsl` +4. Rebuild: `cmake --build build -j4` + +**Reference:** File `CNN.py` contains training example (needs adaptation). Need a repository of reference image pairs (before/after) for training and validation. -Each input image is randomly sampled into 3x3 patch of (r,g,b,1/z) input samples. +Each input image is randomly sampled into 3×3 patch of (r,g,b,1/z) input samples. And trained to match the (r,g,b,a) output. -Training generates the .wgsl code for layers' shaders, and the c++ code for the post-processing 'Effect'. +Training generates the .wgsl code for layers' shaders. |
