diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/HOWTO.md | 38 | ||||
| -rw-r--r-- | doc/MASKING_SYSTEM.md | 240 |
2 files changed, 278 insertions, 0 deletions
diff --git a/doc/HOWTO.md b/doc/HOWTO.md index 3a1aee2..55580ba 100644 --- a/doc/HOWTO.md +++ b/doc/HOWTO.md @@ -158,6 +158,44 @@ void test_example() { For low-level synth-only tests, you can still call `synth_init()` directly. +## Auxiliary Texture Masking + +The project supports inter-effect texture sharing via the auxiliary texture registry. This allows effects to generate textures (masks, shadow maps, etc.) and make them available to other effects within the same frame. + +**Common use case:** Screen-space partitioning where different scenes render to complementary regions. + +```cpp +class MaskGeneratorEffect : public Effect { + void init(MainSequence* demo) override { + demo_ = demo; + // Register a named auxiliary texture + demo->register_auxiliary_texture("my_mask", width_, height_); + } + + void compute(WGPUCommandEncoder encoder, ...) override { + // Generate mask to auxiliary texture + WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); + // ... render mask + } + + void render(WGPURenderPassEncoder pass, ...) override { + // Use mask in scene rendering + WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); + // ... render scene with mask + } +}; + +class MaskUserEffect : public Effect { + void render(WGPURenderPassEncoder pass, ...) override { + // Reuse mask from MaskGeneratorEffect + WGPUTextureView mask_view = demo_->get_auxiliary_view("my_mask"); + // ... render scene with inverted mask + } +}; +``` + +See `doc/MASKING_SYSTEM.md` for detailed architecture and examples. + ## Debugging ### Seeking / Fast-Forward diff --git a/doc/MASKING_SYSTEM.md b/doc/MASKING_SYSTEM.md new file mode 100644 index 0000000..d468d48 --- /dev/null +++ b/doc/MASKING_SYSTEM.md @@ -0,0 +1,240 @@ +# Auxiliary Texture Masking System + +## Overview + +The auxiliary texture masking system allows effects to share textures within a single frame render. Primary use case: **screen-space partitioning** where multiple effects render to complementary regions of the framebuffer. + +## Use Case + +**Problem:** Render two different 3D scenes to different regions of the screen (split-screen, portals, picture-in-picture). + +**Solution:** +- Effect1 generates a mask (1 = Effect1's region, 0 = Effect2's region) +- Effect1 renders scene A where mask = 1 +- Effect2 reuses the mask and renders scene B where mask = 0 +- Both render to the same framebuffer in the same frame + +## Architecture Choice: Mask Texture vs Stencil Buffer + +### Option 1: Stencil Buffer (NOT CHOSEN) +**Pros:** Hardware-accelerated, fast early-rejection +**Cons:** 8-bit limitation, complex pipeline config, hard to debug + +### Option 2: Mask Texture (CHOSEN) +**Pros:** +- Flexible (soft edges, gradients, any format) +- Debuggable (visualize mask as texture) +- Reusable (multiple effects can read same mask) +- Simple pipeline setup + +**Cons:** +- Requires auxiliary texture (~4 MB for 1280x720 RGBA8) +- Fragment shader discard (slightly slower than stencil) + +**Verdict:** Mask texture flexibility and debuggability outweigh performance cost. + +## Implementation + +### MainSequence Auxiliary Texture Registry + +```cpp +class MainSequence { + public: + // Register a named auxiliary texture (call once in effect init) + void register_auxiliary_texture(const char* name, int width, int height); + + // Get texture view for reading/writing (call every frame) + WGPUTextureView get_auxiliary_view(const char* name); + + private: + struct AuxiliaryTexture { + WGPUTexture texture; + WGPUTextureView view; + int width, height; + }; + std::map<std::string, AuxiliaryTexture> auxiliary_textures_; +}; +``` + +### Effect Lifecycle + +``` +Init Phase (once): + Effect1.init(): register_auxiliary_texture("mask_1", width, height) + Effect2.init(): (no registration - reuses Effect1's mask) + +Compute Phase (every frame): + Effect1.compute(): Generate mask to get_auxiliary_view("mask_1") + +Scene Pass (every frame, shared render pass): + Effect1.render(): Sample mask, discard if < 0.5, render scene A + Effect2.render(): Sample mask, discard if > 0.5, render scene B +``` + +### Render Flow Diagram + +``` +Frame N: +┌───────────────────────────────────────────────────────────┐ +│ Compute Phase: │ +│ Effect1.compute() │ +│ └─ Generate mask → auxiliary_textures_["mask_1"] │ +│ │ +├───────────────────────────────────────────────────────────┤ +│ Scene Pass (all effects share framebuffer A + depth): │ +│ Effect1.render() [priority 5] │ +│ ├─ Sample auxiliary_textures_["mask_1"] │ +│ ├─ Discard fragments where mask < 0.5 │ +│ └─ Render 3D scene A → framebuffer A │ +│ │ +│ Effect2.render() [priority 10] │ +│ ├─ Sample auxiliary_textures_["mask_1"] │ +│ ├─ Discard fragments where mask > 0.5 │ +│ └─ Render 3D scene B → framebuffer A │ +│ │ +│ Result: framebuffer A contains both scenes, partitioned │ +├───────────────────────────────────────────────────────────┤ +│ Post-Process Chain: │ +│ A ⟷ B ⟷ Screen │ +└───────────────────────────────────────────────────────────┘ +``` + +## Example: Circular Portal Effect + +### Effect1: Render Scene A (inside portal) + +```cpp +class PortalSceneEffect : public Effect { + public: + PortalSceneEffect(const GpuContext& ctx) : Effect(ctx) {} + + void init(MainSequence* demo) override { + demo_ = demo; + demo->register_auxiliary_texture("portal_mask", width_, height_); + // ... create pipelines + } + + void compute(WGPUCommandEncoder encoder, ...) override { + // Generate circular mask (portal region) + WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); + // ... render fullscreen quad with circular mask shader + } + + void render(WGPURenderPassEncoder pass, ...) override { + // Render 3D scene, discard outside portal + WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); + // ... bind mask, render scene with mask test + } +}; +``` + +### Effect2: Render Scene B (outside portal) + +```cpp +class OutsideSceneEffect : public Effect { + public: + OutsideSceneEffect(const GpuContext& ctx) : Effect(ctx) {} + + void init(MainSequence* demo) override { + demo_ = demo; + // Don't register - reuse PortalSceneEffect's mask + } + + void render(WGPURenderPassEncoder pass, ...) override { + // Render 3D scene, discard inside portal + WGPUTextureView mask_view = demo_->get_auxiliary_view("portal_mask"); + // ... bind mask, render scene with inverted mask test + } +}; +``` + +### Mask Generation Shader + +```wgsl +// portal_mask.wgsl +@group(0) @binding(0) var<uniform> uniforms: MaskUniforms; + +@fragment fn fs_main(@builtin(position) p: vec4<f32>) -> @location(0) vec4<f32> { + let uv = p.xy / uniforms.resolution; + let center = vec2<f32>(0.5, 0.5); + let radius = 0.3; + + let dist = length(uv - center); + let mask = f32(dist < radius); // 1.0 inside circle, 0.0 outside + + return vec4<f32>(mask, mask, mask, 1.0); +} +``` + +### Scene Rendering with Mask + +```wgsl +// scene_with_mask.wgsl +@group(0) @binding(2) var mask_sampler: sampler; +@group(0) @binding(3) var mask_texture: texture_2d<f32>; + +@fragment fn fs_main(in: VertexOutput) -> @location(0) vec4<f32> { + // Sample mask + let screen_uv = in.position.xy / uniforms.resolution; + let mask_value = textureSample(mask_texture, mask_sampler, screen_uv).r; + + // Effect1: Discard outside portal (mask = 0) + if (mask_value < 0.5) { + discard; + } + + // Effect2: Invert test - discard inside portal (mask = 1) + // if (mask_value > 0.5) { discard; } + + // Render scene + return compute_scene_color(in); +} +``` + +## Memory Impact + +Each auxiliary texture: **width × height × 4 bytes** +- 1280×720 RGBA8: ~3.7 MB +- 1920×1080 RGBA8: ~8.3 MB + +For 2-3 masks: 10-25 MB total (acceptable overhead). + +## Use Cases + +1. **Split-screen**: Vertical/horizontal partition +2. **Portals**: Circular/arbitrary shape windows to other scenes +3. **Picture-in-picture**: Small viewport in corner +4. **Masked transitions**: Wipe effects between scenes +5. **Shadow maps**: Pre-generated in compute, used in render +6. **Reflection probes**: Generated once, reused by multiple objects + +## Alternatives Considered + +### Effect-Owned Texture (No MainSequence changes) +```cpp +auto effect1 = std::make_shared<Effect1>(...); +auto effect2 = std::make_shared<Effect2>(...); +effect2->set_mask_source(effect1->get_mask_view()); +``` + +**Pros:** No MainSequence changes +**Cons:** Manual wiring, effects coupled, less flexible + +**Verdict:** Not chosen. Registry approach is cleaner and more maintainable. + +## Future Extensions + +- **Multi-channel masks**: RGBA mask for 4 independent regions +- **Mipmap support**: For hierarchical queries +- **Compression**: Quantize masks to R8 (1 byte per pixel) +- **Enum-based lookup**: Replace string keys for size optimization + +## Size Impact + +- MainSequence changes: ~100 lines (~500 bytes code) +- std::map usage: ~1 KB overhead (low priority for CRT removal) +- Runtime memory: 4-8 MB per mask (acceptable) + +--- + +*Document created: February 8, 2026* |