summaryrefslogtreecommitdiff
path: root/doc/CNN.md
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-10 08:08:25 +0100
committerskal <pascal.massimino@gmail.com>2026-02-10 08:08:25 +0100
commit75af266889b61b5722d842a1a1eb23f79bc06a85 (patch)
treec2ddb094eb24e47826ca8fd029e0abbf16b5d16b /doc/CNN.md
parent47397444b30b0f461b1633297a68300179586fda (diff)
docs: Add CNN effect documentation and update project status
**New Documentation:** - `doc/CNN_EFFECT.md` (223 lines): Comprehensive implementation guide - Architecture overview (file structure, shader composition) - Usage examples (C++ API, timeline integration) - Training workflow (planned) - Implementation details (convolution signatures, weight storage) - Size budget breakdown (~5-8 KB total) - Testing and troubleshooting **Updated Documentation:** - `doc/CNN.md`: Added implementation status section - Completed items (✅ modular shaders, C++ class, tests) - Pending items (⏳ training script, multi-layer, quantization) - Size impact summary - `PROJECT_CONTEXT.md`: - Added "Effects: CNN post-processing foundation" to Current Status - Added `CNN_EFFECT.md` to Technical Reference list **Summary:** CNN effect foundation complete with modular WGSL architecture, ready for training script integration. All tests passing (36/36). ~5-8 KB footprint. handoff(Claude): Documentation complete for CNN effect implementation
Diffstat (limited to 'doc/CNN.md')
-rw-r--r--doc/CNN.md40
1 files changed, 34 insertions, 6 deletions
diff --git a/doc/CNN.md b/doc/CNN.md
index 8bf2860..2dc3362 100644
--- a/doc/CNN.md
+++ b/doc/CNN.md
@@ -1,11 +1,15 @@
# Convolutional Neural Net Shader (CNN) post-processing
+**Status:** ✅ Foundation implemented (single-layer, expandable to multi-pass)
+
## Idea
Have the input 3d scene be processed by a multi-layer CNN trained on the side.
Input: some rendered scene.
Output: 'stylized' scene with CNN post-processing.
+**See `doc/CNN_EFFECT.md` for implementation details, usage, and API reference.**
+
## Shader implementation
### input / output
@@ -36,16 +40,40 @@ we need 3 or 4 layer ?
Several different shaders for each layer.
Ping-pong for input/output texture buffer between each layers?
-## Training
+## Implementation Status
+
+**Completed:**
+- ✅ Modular WGSL shader architecture (6 snippet files)
+- ✅ CNNEffect C++ class (single-layer rendering)
+- ✅ ShaderComposer integration (#include resolution)
+- ✅ Asset registration (7 new shader assets)
+- ✅ Test coverage (test_demo_effects.cc)
+- ✅ Placeholder identity weights for testing
+
+**Size:** ~3-4 KB shader code + ~2-4 KB weights = **5-8 KB total**
+
+**Pending:**
+- ⏳ Training script (`scripts/train_cnn.py`) to generate real weights
+- ⏳ Multi-layer rendering with ping-pong textures
+- ⏳ Weight quantization for size optimization
+
+---
+
+## Training (To Be Implemented)
The layer weight/bias data are hard-coded in the shaders.
-Need training with external python script.
-File: CNN.py contains an example of what the training script could be.
-Just an example, doesn't match our requirement yet.
+Training workflow:
+
+1. Prepare image pairs (before: raw render, after: target style)
+2. Run `python scripts/train_cnn.py --input scene.png --target stylized.png`
+3. Script generates `cnn_weights_generated.wgsl`
+4. Rebuild: `cmake --build build -j4`
+
+**Reference:** File `CNN.py` contains training example (needs adaptation).
Need a repository of reference image pairs (before/after) for training and validation.
-Each input image is randomly sampled into 3x3 patch of (r,g,b,1/z) input samples.
+Each input image is randomly sampled into 3×3 patch of (r,g,b,1/z) input samples.
And trained to match the (r,g,b,a) output.
-Training generates the .wgsl code for layers' shaders, and the c++ code for the post-processing 'Effect'.
+Training generates the .wgsl code for layers' shaders.