summaryrefslogtreecommitdiff
path: root/cnn_v3/docs
AgeCommit message (Collapse)Author
3 hoursfeat(cnn_v3): Phase 4 — type-aware SDF in shadow passskal
dfWithID() in gbuf_shadow.wgsl now branches on obj.params.x (ObjectType) instead of using sdBox for everything: 0=CUBE → sdBox(lp, vec3(1)) 1=SPHERE → sdSphere(lp, 1.0) 2=PLANE → sdPlane(lp, vec3(0,1,0), obj.params.y) 3=TORUS → sdTorus(lp, vec2(0.8, 0.2)) 36/36 tests pass.
3 hoursfeat(cnn_v3): GBufferEffect Pass 2 — SDF shadow raymarchingskal
Implements gbuf_shadow.wgsl: fullscreen render pass that reads depth from Pass 1, reconstructs world-space positions, evaluates a proxy-box SDF for each object (via inv_model), computes soft shadows for both directional lights using shadowWithStoredDistance(), and writes shadow factor to the RGBA8Unorm node_shadow_ target consumed by gbuf_pack.wgsl. Bind layout: B0=GlobalUniforms, B1=ObjectsBuffer (storage-read), B2=texture_depth_2d, B3=GBufLightsUniforms. Sky fragments (depth=1.0) are output as 1.0 (fully lit). Falls back to clear(1.0) if pipeline is not ready. 36/36 tests pass. handoff(Gemini): Pass 2 done. Pass 3 (transparency) still TODO. Phase 4 (type-aware SDF) optional after visual validation.
4 hoursfeat(cnn_v3): GBufferEffect internal scene + GBufViewEffect debug wiringskal
GBufferEffect: - set_scene() now owns Scene/Camera internally; no external pointers needed - 20 randomly rotating cubes (xorshift32 seed, axis-angle animation) - 4 pumping spheres (radius = base_r * (1 + audio_intensity * 0.8)) - Camera at (0,2.5,6) looking at origin; aspect updated per-frame - GBufLightsUniforms: 2 directional lights (warm key + cool fill) - object_type written to ObjectData.params.x (ready for SDF shadow) - shadow/transp nodes cleared via zero-draw render passes (placeholder) - bilinear sampler cached via create_linear_sampler() / sampler_.get() - dead placeholder textures removed GBufViewEffect: - gbuf_view.wgsl: all channels now fully grayscale (removed color tint) - seq_compiler.py: GBufViewEffect added to CLASS_TO_HEADER - timeline.seq: cnn_v3_test uses GBufViewEffect -> sink for debug view Docs: HOWTO.md §1 updated with set_scene() description + §1b implementation plan for Pass 2 SDF shadow (shader spec, bind layout, C++ additions) handoff(Gemini): GBufferEffect has internal scene, 36/36 tests green. Next: implement Pass 2 shadow (gbuf_shadow.wgsl) per §1b plan in HOWTO.md.
7 hoursfix(cnn_v3/tools): remove unused sampler binding from FULL_PACK_SHADERskal
WebGPU auto-reflects the BGL from the shader; a declared-but-unused sampler binding is omitted from the layout, causing CreateBindGroup to reject it. Removed binding 6 (sampler) entirely — all reads use textureLoad(). Renumbered f0/f1 from 7/8 to 6/7 to match.
7 hoursfeat(cnn_v3): add G-buffer visualizer + web sample loader (Phase 7)skal
C++ GBufViewEffect: renders all 20 feature channels from feat_tex0/feat_tex1 in a 4×5 tiled grid. Custom BGL with WGPUTextureSampleType_Uint; bind group rebuilt per frame via wgpuRenderPipelineGetBindGroupLayout. Web tool: "Load sample directory" button — webkitdirectory picker, FULL_PACK_SHADER compute (matches gbuf_pack.wgsl packing), runFromFeat() skips photo-pack step, computePSNR() readback + comparison vs target.png side-by-side. 36/36 tests pass. Docs updated: HOWTO.md §9, README, PROJECT_CONTEXT, TODO, COMPLETED. handoff(Gemini): CNN v3 Phase 7 done. Next: run train_cnn_v3.py (see HOWTO §3).
10 hoursfeat(cnn_v3): add weight assets to assets.txt, update HOW_TO_CNN export docsskal
- Add WEIGHTS_CNN_V3 and WEIGHTS_CNN_V3_FILM_MLP to workspaces/main/assets.txt - Add opencv-python and pillow to export_cnn_v3_weights.py uv inline deps - Update HOW_TO_CNN.md §3 export target → workspaces/main/weights/ - Update HOW_TO_CNN.md §4 weight loading → SafeGetAsset (asset system) handoff(Gemini): cnn_v3 weight assets registered; export and C++ load path documented
12 hoursdocs(cnn_v3): add uv inline deps to train_cnn_v3.py + HOW_TO_CNN noteskal
handoff(Gemini): train_cnn_v3.py now has uv script metadata block (torch, torchvision, numpy, pillow, opencv-python). HOW_TO_CNN §2 Prerequisites updated with uv quick-start alternative.
12 hoursdocs(cnn_v3): add Windows 10 + CUDA training section to HOW_TO_CNN §2skal
12 hoursfix(cnn_v3): resize target to albedo dims when sizes differskal
target.png can have a different resolution than albedo.png in simple samples; patch slicing into the smaller target produced 0×0 tensors, crashing torch.stack in the DataLoader collate. handoff(Gemini): target resized in _load_sample (LANCZOS) + note in HOW_TO_CNN §1c.
12 hoursdocs(cnn_v3): add full Old House example to HOW_TO_CNN §1bskal
handoff(Gemini): added render + batch-pack example commands at end of section 1b
13 hoursfix(cnn_v3): native OPEN_EXR_MULTILAYER + quiet render + flexible channel namesskal
blender_export.py: - Replace broken compositor FileOutput approach with native OPEN_EXR_MULTILAYER render output; all enabled passes included automatically, no socket wiring needed - Suppress Fra:/Mem: render spam via os.dup2 fd redirect; per-frame progress printed to stderr via render_post handler pack_blender_sample.py: - get_pass_r: try .R/.X/.Y/.Z/'' suffixes + aliases param for Depth→Z fallback - combined_rgba loaded once via ("Combined","Image") loop; shared by transp+target - Remove unused sys import HOW_TO_CNN.md: update channel table to native EXR naming (Depth.Z, IndexOB.X, Shadow.X), fix example command, note Shadow defaults to 255 when absent handoff(Gemini): blender pipeline now produces correct multilayer EXR with all G-buffer passes; pack script handles native channel naming
13 hoursdocs(cnn_v3): blender4 alias + Blender 4.5 LTS requirement for training dataskal
14 hoursdocs(cnn_v3): clarify --output is a base dir, not a frame_### patternskal
14 hoursdocs(cnn_v3): update HOW_TO_CNN for Blender 5.x compatibilityskal
15 hoursfix(cnn_v3): blender_export --view-layer flag + fallback to layer[0]skal
Fixes KeyError when blend file uses a non-default view layer name. Adds --view-layer NAME arg; pass '?' to list available layers. Defaults to index 0 with a clear error if the name is not found. handoff(Gemini): blender_export.py view layer selection now robust
15 hoursfeat(cnn_v3): gen_sample tool + 7 simple training samplesskal
- pack_photo_sample.py: --target now required (no albedo fallback) - gen_sample.py: bash wrapper with positional args (input target output_dir) - input/photo7.jpg: copy of photo2 (second style target) - target_1: photo2_1_out→photo2_out, photo2_2_out→photo7_out - dataset/simple/sample_001..007: 7 packed photo/target pairs handoff(Gemini): training data ready; next step is train_cnn_v3.py run
36 hoursfeat(cnn_v3): HTML WebGPU tool (index.html + shaders.js + tester.js)skal
3-file tool, 939 lines total. Implements full U-Net+FiLM inference in the browser: Pack→Enc0→Enc1→Bottleneck→Dec1→Dec0 compute passes, layer visualisation (Feat/Enc0/Enc1/BN/Dec1/Output), FiLM MLP sliders, drag-drop weights + image/video, Save PNG, diff/blend view modes. HOW_TO_CNN.md §7 updated to reflect tool is implemented. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
36 hoursfeat(cnn_v3): export script + HOW_TO_CNN.md playbookskal
- export_cnn_v3_weights.py: .pth → cnn_v3_weights.bin (f16 packed u32) + cnn_v3_film_mlp.bin (f32) - HOW_TO_CNN.md: full pipeline playbook (data collection, training, export, C++ wiring, parity, HTML tool) - TODO.md: mark export script done Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
37 hoursfeat(cnn_v3): Phase 6 — training script (train_cnn_v3.py + cnn_v3_utils.py)skal
- train_cnn_v3.py: CNNv3 U-Net+FiLM model, training loop, CLI - cnn_v3_utils.py: image I/O, pyrdown, depth_gradient, assemble_features, apply_channel_dropout, detect_salient_points, CNNv3Dataset - Patch-based training (default 64×64) with salient-point extraction (harris/shi-tomasi/fast/gradient/random detectors, pre-cached at init) - Channel dropout for geometric/context/temporal channels - Random FiLM conditioning per sample for joint MLP+U-Net training - docs: HOWTO.md §3 updated with commands and flag reference - TODO.md: Phase 6 marked done, export script noted as next step Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
37 hoursdocs(cnn_v3): update CNN_V3.md + HOWTO.md to reflect Phases 1-5 completeskal
- CNN_V3.md: status line, architecture channel counts (8/16→4/8), FiLM MLP output count (96→40 params), size budget table (real implemented values) - HOWTO.md: Phase status table (5→done, add phase 6 training TODO), sections 3-5 rewritten to reflect what exists vs what is still planned
38 hoursfeat(cnn_v3): Phase 4 complete — CNNv3Effect C++ + FiLM uniform uploadskal
- cnn_v3/src/cnn_v3_effect.{h,cc}: full Effect subclass with 5 compute passes (enc0→enc1→bottleneck→dec1→dec0), shared weights storage buffer, per-pass uniform buffers, set_film_params() API - Fixed WGSL/C++ struct alignment: vec3u has align=16, so CnnV3Params4ch is 64 bytes and CnnV3ParamsEnc1 is 96 bytes (not 48/80) - Weight offsets computed as explicit formulas (e.g. 20*4*9+4) for clarity - Registered in CMake, shaders.h/cc, demo_effects.h, test_demo_effects.cc - 35/35 tests pass handoff(Gemini): CNN v3 Phase 5 next — parity validation (Python ref vs WGSL)
38 hoursfeat(cnn_v3): Phase 3 complete — WGSL U-Net inference shadersskal
5 compute shaders + cnn_v3/common snippet: enc0: Conv(20→4,3×3) + FiLM + ReLU full-res enc1: AvgPool + Conv(4→8,3×3) + FiLM + ReLU half-res bottleneck: AvgPool + Conv(8→8,1×1) + ReLU quarter-res dec1: NearestUp + cat(enc1) + Conv(16→4) + FiLM half-res dec0: NearestUp + cat(enc0) + Conv(8→4) + FiLM + Sigmoid full-res Parity rules: zero-pad conv, AvgPool down, NearestUp, FiLM after conv+bias, skip=concat, OIHW weights+bias layout. Matches PyTorch train_cnn_v3.py forward() exactly. Registered in workspaces/main/assets.txt + src/effects/shaders.cc. Weight layout + Params struct documented in cnn_v3/docs/HOWTO.md §7. Next: Phase 4 — C++ CNNv3Effect + FiLM uniform upload. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 daysfeat(cnn_v3): Phase 1 complete - GBufferEffect integrated + HOWTO playbookskal
- Wire GBufferEffect into demo build: assets.txt, DemoSourceLists.cmake, demo_effects.h, shaders.h/cc. ShaderComposer::Compose() applied to gbuf_raster.wgsl (resolves #include "common_uniforms"). - Add GBufferEffect construction test. 35/35 passing. - Write cnn_v3/docs/HOWTO.md: G-buffer wiring, training data prep, training plan, per-pixel validation workflow, phase status table, troubleshooting guide. - Add project hooks: remind to update HOWTO.md on cnn_v3/ edits; warn on direct str_view(*_wgsl) usage bypassing ShaderComposer. - Update PROJECT_CONTEXT.md and TODO.md: Phase 1 done, Phase 3 (WGSL U-Net shaders) is next active. handoff(Gemini): CNN v3 Phase 3 is next - WGSL enc/dec/bottleneck/FiLM shaders in cnn_v3/shaders/. See cnn_v3/docs/CNN_V3.md Architecture section and cnn_v3/docs/HOWTO.md section 3 for spec. GBufferEffect outputs feat_tex0 + feat_tex1 (rgba32uint, 20ch, 32 bytes/pixel). C++ CNNv3Effect (Phase 4) takes those as input nodes.
3 daysdocs(cnn_v3): full design doc — U-Net + FiLM architecture planskal
- CNN_V3.md: complete design document - U-Net enc_channels=[4,8], ~5 KB f16 weights - FiLM conditioning (5D → γ/β per level, CPU-side MLP) - 20-channel feature buffer, 32 bytes/pixel: two rgba32uint textures - feat_tex0: albedo.rgb, normal.xy, depth, depth_grad.xy (f16) - feat_tex1: mat_id, prev.rgb, mip1.rgb, mip2.rgb, shadow, transp (u8) - 4-pass G-buffer: raster MRT + SDF compute + lighting + pack - Per-pixel parity framework: PyTorch / HTML WebGPU / C++ WebGPU (≤1/255) - Training pipelines: Blender full G-buffer + photo-only (channel dropout) - train_cnn_v3_full.sh spec (modelled on v2 script) - HTML tool adaptation plan from cnn_v2/tools/cnn_v2_test/index.html - Binary format v3 header spec - 8-phase ordered implementation checklist - TODO.md: add CNN v3 U-Net+FiLM future task with phases - cnn_v3/README.md: update status to design phase handoff(Gemini): CNN v3 design complete. Phase 0 (stub G-buffer) unblocks all other phases — one compute shader writing feat_tex0+feat_tex1 with synthetic values from the current framebuffer. See cnn_v3/docs/CNN_V3.md Implementation Checklist.
generated by cgit v1.2.3 (git 2.51.0) at 2026-03-22 21:55:06 +0000