| Age | Commit message (Collapse) | Author |
|---|
|
- assemble_features() accepts optional prev ndarray (None → zeros)
- _load_sample() loads prev.png if present, else None
- __getitem__ slices/resizes prev alongside other channels
- TODO.md: note that cnn_v3/tools/shaders.js hardcodes prev=0 in both
pack shaders while C++ gbuf_pack.wgsl reads a real prev_cnn texture
handoff(Gemini): prev.png now used in training when present; web tool gap documented in TODO.md
|
|
architecture PNG
- Replace 1×1 pointwise bottleneck with Conv(8→8, 3×3, dilation=2):
effective RF grows from ~13px to ~29px at ¼res (~+1 KB weights)
- Add Sobel edge loss in training (--edge-loss-weight, default 0.1)
- Add FiLM 2-phase training: freeze MLP for warmup epochs then
unfreeze at lr×0.1 (--film-warmup-epochs, default 50)
- Update weight layout: BN 72→584 f16, total 1964→2476 f16 (4952 B)
- Cascade offsets in C++ effect, JS tool, export/gen_test_vectors scripts
- Regenerate test_vectors.h (1238 u32); parity max_err=9.77e-04
- Generate dark-theme U-Net+FiLM architecture PNG (gen_architecture_png.py)
- Replace ASCII art in CNN_V3.md and HOW_TO_CNN.md with PNG embed
handoff(Gemini): bottleneck dilation + Sobel loss + FiLM warmup landed.
Next: run first real training pass (see cnn_v3/docs/HOWTO.md §3).
|
|
- train_cnn_v3.py: --single-sample <dir> implies --full-image + --batch-size 1
- cnn_v3_utils.py: CNNv3Dataset accepts single_sample= kwarg (explicit override)
- HOWTO.md: document --single-sample workflow, fix pack_photo_sample.py usage (--target required)
- HOW_TO_CNN.md: fix GBufferEffect seq input (prev_cnn→source), fix binary name (demo→demo64k), add --resume to flag table, remove stale "pack without target" block
handoff(Gemini): --single-sample <dir> added to train_cnn_v3.py; docs audited and corrected
|
|
- cnn_v3/training/infer_cnn_v3.py: PyTorch inference tool; simple mode
(single PNG, zeroed geometry) and full mode (sample directory); supports
--identity-film (γ=1 β=0) to match C++ default, --cond for FiLM MLP,
--blend, --debug-hex for pixel comparison
- tools/cnn_test.cc: full rewrite, v3 only; packs 20-channel features
on CPU (training format: [0,1] oct normals, pyrdown mip), uploads to
GPU, runs CNNv3Effect, reads back RGBA16Float, saves PNG; --sample-dir
for full G-buffer input, --weights for .bin override, --debug-hex
- cmake/DemoTests.cmake: add cnn_v3/src include path, drop unused
offscreen_render_target.cc from cnn_test sources
- cnn_v3/docs/HOWTO.md: new §10 documenting both tools, comparison
workflow, and feature-format convention (training vs runtime)
handoff(Gemini): cnn_test + infer_cnn_v3.py ready for parity testing.
Run both with --identity-film / --debug-hex on same image to compare.
|
|
- cnn_v3/tools/weights.js: new file — base64-encoded cnn_v3_weights.bin +
cnn_v3_film_mlp.bin; loaded at startup so the tool works without dropping files
- tester.js: preload() falls back to embedded weights.js constants when fetch
fails; logs "Loaded embedded" vs "Preloaded" to distinguish the two paths
- index.html: load weights.js before tester.js
- export_cnn_v3_weights.py: add --html / --html-output flags that call
update_weights_js() to regenerate weights.js after a training run
- HOW_TO_CNN.md: update pipeline diagram, §3 export commands, §7 HTML tool
section (file table, workflow, weights.js description), Appendix A
handoff(Gemini): weights.js now the canonical source for HTML tool defaults;
regenerate with `uv run export_cnn_v3_weights.py <ckpt> --output ... --html`
|
|
|
|
|
|
|
|
Replace raw shadow (ch18) with dif = max(0,dot(normal,KEY_LIGHT))*shadow
across all layers. Channel count stays 20, weight shapes unchanged.
- gbuf_pack.wgsl: t1.z = pack4x8unorm(mip2.g, mip2.b, dif, transp); t1.w = 0u
- gbuf_deferred.wgsl: read dif from unpack4x8unorm(t1.z).z
- gbuf_view.wgsl: revert to 4×5 grid, ch18=dif label, ch19=trns label
- tools/shaders.js: FULL_PACK_SHADER adds oct_decode + computes dif
- cnn_v3_utils.py: assemble_features() computes dif on-the-fly via oct_decode
- docs: CNN_V3.md, HOWTO.md, HOW_TO_CNN.md, GBUF_DIF_MIGRATION.md updated
handoff(Gemini): shadow→dif migration done, ready for first training pass
|
|
|
|
- Add WEIGHTS_CNN_V3 and WEIGHTS_CNN_V3_FILM_MLP to workspaces/main/assets.txt
- Add opencv-python and pillow to export_cnn_v3_weights.py uv inline deps
- Update HOW_TO_CNN.md §3 export target → workspaces/main/weights/
- Update HOW_TO_CNN.md §4 weight loading → SafeGetAsset (asset system)
handoff(Gemini): cnn_v3 weight assets registered; export and C++ load path documented
|
|
|
|
https://colorifyai.art/photo-to-sketch/#playground with 'Ink Sketch' does a small zooming
and resize
|
|
- --patch-search-window N: at dataset init, find per-patch (dx,dy) in
[-N,N]² that minimises grayscale MSE between source albedo and target;
result cached so __getitem__ pays only a list-lookup per sample.
- --resume [CKPT]: restore model + Adam state from a checkpoint; omit
path to auto-select the latest in --checkpoint-dir.
- Ctrl-C (SIGINT) finishes the current batch, then saves a checkpoint
before exiting; finally-block guarded so no spurious epoch-0 save.
- Review: remove unused sd variable, lift patch_idx out of duplicate
computation, move _LUMA to Constants block, update module docstring.
handoff(Gemini): cnn_v3/training updated — no C++ or test changes.
|
|
|
|
handoff(Gemini): train_cnn_v3.py now has uv script metadata block
(torch, torchvision, numpy, pillow, opencv-python). HOW_TO_CNN §2
Prerequisites updated with uv quick-start alternative.
|
|
handoff(Gemini): CNNv3Dataset now loads all samples once in __init__ into
self._cache; __getitem__ reads from cache instead of reloading PNGs each
call. Eliminates N×patches_per_image file loads per epoch.
|
|
|
|
target.png can have a different resolution than albedo.png in simple
samples; patch slicing into the smaller target produced 0×0 tensors,
crashing torch.stack in the DataLoader collate.
handoff(Gemini): target resized in _load_sample (LANCZOS) + note in HOW_TO_CNN §1c.
|
|
handoff(Gemini): added render + batch-pack example commands at end of section 1b
|
|
blender_export.py:
- Replace broken compositor FileOutput approach with native OPEN_EXR_MULTILAYER
render output; all enabled passes included automatically, no socket wiring needed
- Suppress Fra:/Mem: render spam via os.dup2 fd redirect; per-frame progress
printed to stderr via render_post handler
pack_blender_sample.py:
- get_pass_r: try .R/.X/.Y/.Z/'' suffixes + aliases param for Depth→Z fallback
- combined_rgba loaded once via ("Combined","Image") loop; shared by transp+target
- Remove unused sys import
HOW_TO_CNN.md: update channel table to native EXR naming (Depth.Z, IndexOB.X,
Shadow.X), fix example command, note Shadow defaults to 255 when absent
handoff(Gemini): blender pipeline now produces correct multilayer EXR with all
G-buffer passes; pack script handles native channel naming
|
|
- Use bpy.app.version for version detection instead of attribute sniffing
- Blender 5.0.x: warn that per-pass compositor routing is broken (Combined only);
compositing_node_group path kept ready for when Blender fixes this upstream
- Remove all DEBUG prints and failed use_nodes=True experiment
- configure_scene() returns only discard_dir (compositor always configured)
- Move _SOCKET_ALIASES to module level; simplify slots/None fallback
handoff(Gemini): blender_export.py stable for Blender 4.5 LTS; Blender 5.x
path is forward-compatible but produces Combined-only output until upstream fix.
|
|
RenderLayer sockets
- Activate compositor in Blender 5.0+ by relying on compositing_node_group
assignment (no use_nodes needed, avoids deprecation warning)
- Document full CompositorNodeRLayers output socket list for Blender 5.0.1
- Clean up SOCKET_ALIASES to match confirmed socket names
|
|
after render
|
|
|
|
|
|
- Prefer file_output_items over file_slots; use explicit is-None checks
so empty collections do not fall through to the legacy attribute.
- Clear out_node.file_name so multilayer EXR frames are named 0001.exr
instead of file_name0001.exr.
handoff(Gemini): blender_export.py now produces frames/0001.exr on Blender 5.0.1.
|
|
|
|
- compositor: use compositing_node_group (Blender 5+) / node_tree (<=4.x)
- file output: use file_output_items.new(type, name) (5+) / file_slots (older)
- file output: use directory attr (5+) / base_path (older)
- suppress default PNG output via mkdtemp + shutil.rmtree after render
- link passes by name instead of positional index
- add TODO for Shadow socket name variance across blend files
- clean up: extract helpers, PASS_SOCKETS constant with socket types
handoff(Gemini): blender_export.py now works on Blender 5.0.1
|
|
Fixes KeyError when blend file uses a non-default view layer name.
Adds --view-layer NAME arg; pass '?' to list available layers.
Defaults to index 0 with a clear error if the name is not found.
handoff(Gemini): blender_export.py view layer selection now robust
|
|
- pack_photo_sample.py: --target now required (no albedo fallback)
- gen_sample.py: bash wrapper with positional args (input target output_dir)
- input/photo7.jpg: copy of photo2 (second style target)
- target_1: photo2_1_out→photo2_out, photo2_2_out→photo7_out
- dataset/simple/sample_001..007: 7 packed photo/target pairs
handoff(Gemini): training data ready; next step is train_cnn_v3.py run
|
|
- pack_photo_sample.py: --target now required (no albedo fallback)
- gen_sample.py: bash wrapper with positional args (input target output_dir)
- input/photo7.jpg: copy of photo2 (second style target)
- target_1: photo2_1_out→photo2_out, photo2_2_out→photo7_out
- dataset/simple/sample_001..007: 7 packed photo/target pairs
handoff(Gemini): training data ready; next step is train_cnn_v3.py run
|
|
- pack_photo_sample.py: --target now required (no albedo fallback)
- gen_sample: bash wrapper with positional args (input target output_dir)
- input/photo7.jpg: copy of photo2 (second style target)
- target_1: photo2_1_out→photo2_out, photo2_2_out→photo7_out
- dataset/simple/sample_001..007: 7 packed photo/target pairs
handoff(Gemini): training data ready; next step is train_cnn_v3.py run
|
|
C++:
- cnn_v3_effect.cc: fix declare_nodes comment (output node declared by caller)
- cnn_v3_effect.cc: add TODO(phase-7) marker for FiLM MLP replacement
WGSL:
- cnn_v3_bottleneck.wgsl: consolidate _pad fields onto one line, explain why
array<u32,3> is invalid in uniform address space
- cnn_v3_enc0.wgsl: fix "12xu8" → "12ch u8norm" in header comment
- cnn_v3_dec0.wgsl: clarify parity note (sigmoid after FiLM+ReLU, not raw conv)
- cnn_v3_common.wgsl: clarify unpack_8ch pack layout (low/high 16 bits)
Python:
- cnn_v3_utils.py: replace PIL-based _upsample_nearest (uint8 round-trip) with
pure numpy index arithmetic; rename _resize_rgb → _resize_img (handles any
channel count); add comment on normal zero-pad workaround
- export_cnn_v3_weights.py: add cross-ref to cnn_v3_effect.cc constants;
clarify weight count comments with Conv notation
Test:
- test_cnn_v3_parity.cc: enc0/dec1 layer failures now return 0 (were print-only)
handoff(Gemini): CNN v3 review complete, 36/36 tests passing.
|
|
- export_cnn_v3_weights.py: .pth → cnn_v3_weights.bin (f16 packed u32) + cnn_v3_film_mlp.bin (f32)
- HOW_TO_CNN.md: full pipeline playbook (data collection, training, export, C++ wiring, parity, HTML tool)
- TODO.md: mark export script done
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- train_cnn_v3.py: CNNv3 U-Net+FiLM model, training loop, CLI
- cnn_v3_utils.py: image I/O, pyrdown, depth_gradient, assemble_features,
apply_channel_dropout, detect_salient_points, CNNv3Dataset
- Patch-based training (default 64×64) with salient-point extraction
(harris/shi-tomasi/fast/gradient/random detectors, pre-cached at init)
- Channel dropout for geometric/context/temporal channels
- Random FiLM conditioning per sample for joint MLP+U-Net training
- docs: HOWTO.md §3 updated with commands and flag reference
- TODO.md: Phase 6 marked done, export script noted as next step
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- Add test_cnn_v3_parity.cc: zero_weights + random_weights tests
- Add gen_test_vectors.py: PyTorch reference implementation for enc0/enc1/bn/dec1/dec0
- Add test_vectors.h: generated C header with enc0, dec1, output expected values
- Fix declare_nodes(): intermediate textures at fractional resolutions (W/2, W/4)
using new NodeRegistry::default_width()/default_height() getters
- Add layer-by-layer readback (enc0, dec1) for regression coverage
- Final parity: enc0 max_err=1.95e-3, dec1 max_err=1.95e-3, out max_err=4.88e-4
handoff(Claude): CNN v3 parity done. Next: train_cnn_v3.py (FiLM MLP training).
|
|
G-buffer (Phase 1):
- Add NodeTypes GBUF_ALBEDO/DEPTH32/R8/RGBA32UINT to NodeRegistry
- GBufferEffect: MRT raster pass (albedo+normal_mat+depth) + pack compute
- Shaders: gbuf_raster.wgsl (MRT), gbuf_pack.wgsl (feature packing, 32B/px)
- Shadow/SDF passes stubbed (placeholder textures), CMake integration deferred
Training infrastructure (Phase 2):
- blender_export.py: headless EXR export with all G-buffer render passes
- pack_blender_sample.py: EXR → per-channel PNGs (oct-normals, 1/z depth)
- pack_photo_sample.py: photo → zero-filled G-buffer sample layout
handoff(Gemini): G-buffer phases 3-5 remain (U-Net shaders, CNNv3Effect, parity)
|
|
|
|
|
|
Initialize CNN v3 subdirectory with training pipeline layout:
- docs/, scripts/, shaders/, src/, tools/, weights/ for organization
- training/input/ with sample images
- training/target_1/, target_2/ for multi-style training
- README.md documenting structure
Training images tracked in repo for easy collaboration.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
