diff options
Diffstat (limited to 'doc/MASKING_SYSTEM.md')
| -rw-r--r-- | doc/MASKING_SYSTEM.md | 247 |
1 files changed, 66 insertions, 181 deletions
diff --git a/doc/MASKING_SYSTEM.md b/doc/MASKING_SYSTEM.md index d468d48..86d6a74 100644 --- a/doc/MASKING_SYSTEM.md +++ b/doc/MASKING_SYSTEM.md @@ -1,240 +1,125 @@ # Auxiliary Texture Masking System -## Overview +## Purpose -The auxiliary texture masking system allows effects to share textures within a single frame render. Primary use case: **screen-space partitioning** where multiple effects render to complementary regions of the framebuffer. +Share textures between effects within a single frame for **screen-space partitioning** (split-screen, portals, picture-in-picture). -## Use Case +## Concept -**Problem:** Render two different 3D scenes to different regions of the screen (split-screen, portals, picture-in-picture). +- Effect1: Generate mask (1 = region A, 0 = region B) +- Effect1: Render scene A where mask = 1 +- Effect2: Reuse mask, render scene B where mask = 0 +- Both render to same framebuffer -**Solution:** -- Effect1 generates a mask (1 = Effect1's region, 0 = Effect2's region) -- Effect1 renders scene A where mask = 1 -- Effect2 reuses the mask and renders scene B where mask = 0 -- Both render to the same framebuffer in the same frame +## Design Choice: Mask Texture vs Stencil Buffer -## Architecture Choice: Mask Texture vs Stencil Buffer +**Chosen: Mask Texture** -### Option 1: Stencil Buffer (NOT CHOSEN) -**Pros:** Hardware-accelerated, fast early-rejection -**Cons:** 8-bit limitation, complex pipeline config, hard to debug - -### Option 2: Mask Texture (CHOSEN) -**Pros:** -- Flexible (soft edges, gradients, any format) -- Debuggable (visualize mask as texture) -- Reusable (multiple effects can read same mask) +Pros: +- Flexible (soft edges, gradients) +- Debuggable (visualize as texture) +- Reusable (multiple effects read same mask) - Simple pipeline setup -**Cons:** -- Requires auxiliary texture (~4 MB for 1280x720 RGBA8) +Cons: +- Memory cost (~4 MB for 1280x720 RGBA8) - Fragment shader discard (slightly slower than stencil) -**Verdict:** Mask texture flexibility and debuggability outweigh performance cost. +**Not Chosen: Stencil Buffer** -## Implementation +Pros: Hardware-accelerated, fast early-rejection +Cons: 8-bit limitation, complex pipeline, hard to debug -### MainSequence Auxiliary Texture Registry +## API Reference ```cpp class MainSequence { public: - // Register a named auxiliary texture (call once in effect init) + // Register once in effect init void register_auxiliary_texture(const char* name, int width, int height); - // Get texture view for reading/writing (call every frame) + // Get view every frame WGPUTextureView get_auxiliary_view(const char* name); - - private: - struct AuxiliaryTexture { - WGPUTexture texture; - WGPUTextureView view; - int width, height; - }; - std::map<std::string, AuxiliaryTexture> auxiliary_textures_; }; ``` -### Effect Lifecycle - -``` -Init Phase (once): - Effect1.init(): register_auxiliary_texture("mask_1", width, height) - Effect2.init(): (no registration - reuses Effect1's mask) - -Compute Phase (every frame): - Effect1.compute(): Generate mask to get_auxiliary_view("mask_1") - -Scene Pass (every frame, shared render pass): - Effect1.render(): Sample mask, discard if < 0.5, render scene A - Effect2.render(): Sample mask, discard if > 0.5, render scene B -``` - -### Render Flow Diagram - -``` -Frame N: -┌───────────────────────────────────────────────────────────┐ -│ Compute Phase: │ -│ Effect1.compute() │ -│ └─ Generate mask → auxiliary_textures_["mask_1"] │ -│ │ -├───────────────────────────────────────────────────────────┤ -│ Scene Pass (all effects share framebuffer A + depth): │ -│ Effect1.render() [priority 5] │ -│ ├─ Sample auxiliary_textures_["mask_1"] │ -│ ├─ Discard fragments where mask < 0.5 │ -│ └─ Render 3D scene A → framebuffer A │ -│ │ -│ Effect2.render() [priority 10] │ -│ ├─ Sample auxiliary_textures_["mask_1"] │ -│ ├─ Discard fragments where mask > 0.5 │ -│ └─ Render 3D scene B → framebuffer A │ -│ │ -│ Result: framebuffer A contains both scenes, partitioned │ -├───────────────────────────────────────────────────────────┤ -│ Post-Process Chain: │ -│ A ⟷ B ⟷ Screen │ -└───────────────────────────────────────────────────────────┘ -``` - -## Example: Circular Portal Effect - -### Effect1: Render Scene A (inside portal) +## Usage Pattern ```cpp -class PortalSceneEffect : public Effect { - public: - PortalSceneEffect(const GpuContext& ctx) : Effect(ctx) {} - - void init(MainSequence* demo) override { - demo_ = demo; - demo->register_auxiliary_texture("portal_mask", width_, height_); - // ... create pipelines - } - - void compute(WGPUCommandEncoder encoder, ...) override { - // Generate circular mask (portal region) - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... render fullscreen quad with circular mask shader - } - - void render(WGPURenderPassEncoder pass, ...) override { - // Render 3D scene, discard outside portal - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... bind mask, render scene with mask test - } -}; -``` - -### Effect2: Render Scene B (outside portal) +// Effect1 (mask generator) +void init(MainSequence* demo) { + demo->register_auxiliary_texture("portal_mask", width, height); +} -```cpp -class OutsideSceneEffect : public Effect { - public: - OutsideSceneEffect(const GpuContext& ctx) : Effect(ctx) {} +void compute(WGPUCommandEncoder encoder, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Generate mask to texture +} - void init(MainSequence* demo) override { - demo_ = demo; - // Don't register - reuse PortalSceneEffect's mask - } +void render(WGPURenderPassEncoder pass, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Sample mask, discard where mask < 0.5, render scene A +} - void render(WGPURenderPassEncoder pass, ...) override { - // Render 3D scene, discard inside portal - WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); - // ... bind mask, render scene with inverted mask test - } -}; +// Effect2 (mask consumer) +void render(WGPURenderPassEncoder pass, ...) { + WGPUTextureView mask = demo_->get_auxiliary_view("portal_mask"); + // Sample mask, discard where mask > 0.5, render scene B +} ``` -### Mask Generation Shader +## Shader Example +**Mask Generation:** ```wgsl -// portal_mask.wgsl -@group(0) @binding(0) var<uniform> uniforms: MaskUniforms; - @fragment fn fs_main(@builtin(position) p: vec4<f32>) -> @location(0) vec4<f32> { let uv = p.xy / uniforms.resolution; - let center = vec2<f32>(0.5, 0.5); - let radius = 0.3; - - let dist = length(uv - center); - let mask = f32(dist < radius); // 1.0 inside circle, 0.0 outside - - return vec4<f32>(mask, mask, mask, 1.0); + let dist = length(uv - vec2(0.5)); + let mask = f32(dist < 0.3); // Circular portal + return vec4<f32>(mask); } ``` -### Scene Rendering with Mask - +**Masked Rendering:** ```wgsl -// scene_with_mask.wgsl -@group(0) @binding(2) var mask_sampler: sampler; @group(0) @binding(3) var mask_texture: texture_2d<f32>; @fragment fn fs_main(in: VertexOutput) -> @location(0) vec4<f32> { - // Sample mask - let screen_uv = in.position.xy / uniforms.resolution; - let mask_value = textureSample(mask_texture, mask_sampler, screen_uv).r; - - // Effect1: Discard outside portal (mask = 0) - if (mask_value < 0.5) { - discard; - } + let uv = in.position.xy / uniforms.resolution; + let mask = textureSample(mask_texture, mask_sampler, uv).r; - // Effect2: Invert test - discard inside portal (mask = 1) - // if (mask_value > 0.5) { discard; } + if (mask < 0.5) { discard; } // Effect1: discard outside + // if (mask > 0.5) { discard; } // Effect2: discard inside - // Render scene return compute_scene_color(in); } ``` ## Memory Impact -Each auxiliary texture: **width × height × 4 bytes** -- 1280×720 RGBA8: ~3.7 MB -- 1920×1080 RGBA8: ~8.3 MB +Per texture: **width × height × 4 bytes** +- 1280×720: ~3.7 MB +- 1920×1080: ~8.3 MB -For 2-3 masks: 10-25 MB total (acceptable overhead). +Typical usage: 2-3 masks = 10-25 MB (acceptable overhead) ## Use Cases -1. **Split-screen**: Vertical/horizontal partition -2. **Portals**: Circular/arbitrary shape windows to other scenes -3. **Picture-in-picture**: Small viewport in corner -4. **Masked transitions**: Wipe effects between scenes -5. **Shadow maps**: Pre-generated in compute, used in render -6. **Reflection probes**: Generated once, reused by multiple objects - -## Alternatives Considered - -### Effect-Owned Texture (No MainSequence changes) -```cpp -auto effect1 = std::make_shared<Effect1>(...); -auto effect2 = std::make_shared<Effect2>(...); -effect2->set_mask_source(effect1->get_mask_view()); -``` - -**Pros:** No MainSequence changes -**Cons:** Manual wiring, effects coupled, less flexible - -**Verdict:** Not chosen. Registry approach is cleaner and more maintainable. - -## Future Extensions - -- **Multi-channel masks**: RGBA mask for 4 independent regions -- **Mipmap support**: For hierarchical queries -- **Compression**: Quantize masks to R8 (1 byte per pixel) -- **Enum-based lookup**: Replace string keys for size optimization +1. Split-screen (vertical/horizontal) +2. Portals (arbitrary shape windows) +3. Picture-in-picture +4. Masked transitions (wipe effects) +5. Shadow maps (pre-generated in compute) +6. Reflection probes ## Size Impact -- MainSequence changes: ~100 lines (~500 bytes code) -- std::map usage: ~1 KB overhead (low priority for CRT removal) -- Runtime memory: 4-8 MB per mask (acceptable) +- Code: ~100 lines (~500 bytes) +- Runtime memory: 4-8 MB per mask ---- +## Future Extensions -*Document created: February 8, 2026* +- Multi-channel masks (RGBA = 4 regions) +- Mipmap support +- R8 compression (1 byte/pixel) +- Enum-based lookup (replace string keys) |