diff options
Diffstat (limited to 'doc/SHADER_PARAMETRIZATION_PLAN.md')
| -rw-r--r-- | doc/SHADER_PARAMETRIZATION_PLAN.md | 596 |
1 files changed, 0 insertions, 596 deletions
diff --git a/doc/SHADER_PARAMETRIZATION_PLAN.md b/doc/SHADER_PARAMETRIZATION_PLAN.md deleted file mode 100644 index f5afa27..0000000 --- a/doc/SHADER_PARAMETRIZATION_PLAN.md +++ /dev/null @@ -1,596 +0,0 @@ -# Shader Parametrization Plan - -## Problem Statement - -**Current limitations:** -1. Effect parameters are hardcoded in shader code (e.g., FlashEffect always uses white color) -2. No way to pass parameters from .seq file to effect constructors -3. Each effect reimplements uniform buffer management -4. Uniform structures are manually sized and written -5. Cannot easily vary effect behavior without creating new effect classes - -**Use cases:** -- FlashEffect with configurable color (red, blue, green) -- ChromaAberrationEffect with configurable strength multiplier -- GaussianBlurEffect with configurable kernel size -- Any effect with user-defined parameters - ---- - -## Current Architecture Analysis - -### Effect Construction (timeline.cc) -```cpp -// Generated by seq_compiler from assets/demo.seq -seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); -seq->add_effect(std::make_shared<ChromaAberrationEffect>(ctx), 0, 6, 2); -``` - -**Issue:** Only `GpuContext` passed to constructor, no custom parameters. - -### Effect Rendering (flash_effect.cc) -```cpp -void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, - float intensity, float aspect_ratio) { - // Hardcoded uniform values - float uniforms[4] = {flash_intensity_, intensity, 0.0f, 0.0f}; - wgpuQueueWriteBuffer(ctx_.queue, uniforms_.buffer, 0, uniforms, sizeof(uniforms)); - // ... -} -``` - -**Issue:** Parameters hardcoded, uniform writing manual and repetitive. - -### Shader Parameters (flash_effect.cc:43-44) -```wgsl -let white = vec3<f32>(1.0, 1.0, 1.0); // Hardcoded color -let green = vec3<f32>(0.0, 1.0, 0.0); // Hardcoded alternative -var flashed = mix(color.rgb, green, uniforms.intensity); -``` - -**Issue:** Flash colors baked into shader code, not parameterizable. - -### Chromatic Aberration (chroma_aberration.wgsl:25) -```wgsl -let off = 0.02 * uniforms.intensity; // Hardcoded strength multiplier -``` - -**Issue:** Strength multiplier (0.02) is hardcoded, cannot be configured per instance. - ---- - -## Proposed Solution - -### Design Principles -1. **Constructor-time parameters:** Base/initial values (base colors, multipliers, modes) -2. **Per-frame computation:** Parameters computed in `render()` based on time/beat/intensity -3. **Uniform management:** Centralized helper to reduce boilerplate -4. **.seq file syntax:** Extend to support base parameter values -5. **Size-conscious:** Minimal overhead (this is a 64k demo) - -**IMPORTANT:** Parameters are **computed dynamically every frame**, not set once at construction. Constructor params provide base values, `render()` computes animated values. - ---- - -## Implementation Plan - -### Phase 1: Uniform Management Helper - -**Goal:** Reduce boilerplate for uniform buffer creation and updates. - -**New file:** `src/gpu/uniform_helper.h` - -```cpp -#pragma once -#include "gpu/gpu.h" -#include <cstring> - -// Generic uniform buffer helper -template <typename T> -class UniformBuffer { - public: - UniformBuffer() = default; - - void init(WGPUDevice device) { - buffer_ = gpu_create_buffer(device, sizeof(T), - WGPUBufferUsage_Uniform | WGPUBufferUsage_CopyDst); - } - - void update(WGPUQueue queue, const T& data) { - wgpuQueueWriteBuffer(queue, buffer_.buffer, 0, &data, sizeof(T)); - } - - GpuBuffer& get() { return buffer_; } - - private: - GpuBuffer buffer_; -}; -``` - -**Benefits:** -- Type-safe uniform updates -- Automatic size calculation -- Reduces manual `wgpuQueueWriteBuffer` calls - -**Size impact:** ~200 bytes (template instantiation minimal) - ---- - -### Phase 2: Effect Parameter Structs - -**Goal:** Define typed parameter structures for configurable effects. - -**Example:** `src/gpu/effects/flash_effect.h` - -```cpp -#pragma once -#include "gpu/effect.h" -#include "gpu/uniform_helper.h" - -struct FlashEffectParams { - float color[3] = {1.0f, 1.0f, 1.0f}; // Default: white - float decay_rate = 0.98f; // Default: fast decay - float trigger_threshold = 0.7f; // Default: trigger on strong beats -}; - -struct FlashUniforms { - float flash_intensity; - float intensity; - float color[3]; // Added: configurable color - float _pad; -}; - -class FlashEffect : public PostProcessEffect { - public: - FlashEffect(const GpuContext& ctx, const FlashEffectParams& params = {}); - void render(WGPURenderPassEncoder pass, float time, float beat, - float intensity, float aspect_ratio) override; - void update_bind_group(WGPUTextureView input_view) override; - - private: - FlashEffectParams params_; - UniformBuffer<FlashUniforms> uniforms_; - float flash_intensity_ = 0.0f; -}; -``` - -**Benefits:** -- Clear separation: constructor params vs runtime state -- Default values for backward compatibility -- Type safety - -**Size impact:** ~50 bytes per effect (struct storage) - ---- - -### Phase 3: Update Shaders to Use Parameters - -**Goal:** Make shaders accept parameterized values. - -**Example:** `assets/final/shaders/flash.wgsl` - -```wgsl -struct Uniforms { - flash_intensity: f32, - intensity: f32, - flash_color: vec3<f32>, // Added: parameterized color - _pad: f32, -}; - -@group(0) @binding(2) var<uniform> uniforms: Uniforms; - -@fragment -fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> { - let color = textureSample(inputTexture, inputSampler, input.uv); - - // Use uniform color instead of hardcoded white - var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity); - - return vec4<f32>(flashed, color.a); -} -``` - -**Benefits:** -- Flexible shader behavior without recompilation -- Single shader supports multiple color variants - -**Size impact:** +12 bytes per uniform struct (3 floats for color) - ---- - -### Phase 4: Update Effect Implementations - -**Goal:** Use new parameter system in effect constructors and render methods. - -**Example:** `src/gpu/effects/flash_effect.cc` - -```cpp -FlashEffect::FlashEffect(const GpuContext& ctx, const FlashEffectParams& params) - : PostProcessEffect(ctx), params_(params) { - - // Shader code (updated to use flash_color) - const char* shader_code = R"( - // ... (shader from Phase 3) - )"; - - pipeline_ = create_post_process_pipeline(ctx_.device, ctx_.format, shader_code); - uniforms_.init(ctx_.device); -} - -void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, - float intensity, float aspect_ratio) { - (void)aspect_ratio; - - // Trigger flash based on configured threshold - if (intensity > params_.trigger_threshold && flash_intensity_ < 0.2f) { - flash_intensity_ = 0.8f; - } - - // Decay based on configured rate - flash_intensity_ *= params_.decay_rate; - - // *** PER-FRAME PARAMETER COMPUTATION *** - // Animate color based on time and beat - float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f)); - float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f)); - float b = params_.color[2] * (1.0f + 0.3f * beat); // Pulse with beat - - // Update uniforms with computed (animated) values - FlashUniforms u = { - .flash_intensity = flash_intensity_, - .intensity = intensity, - .color = {r, g, b}, // Time-dependent, computed every frame - ._pad = 0.0f - }; - uniforms_.update(ctx_.queue, u); - - wgpuRenderPassEncoderSetPipeline(pass, pipeline_); - wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr); - wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0); -} -``` - -**Benefits:** -- Clean separation of parameters and logic -- Easy to test different configurations -- Type-safe uniform updates - ---- - -### Phase 5: Extend .seq File Format - -**Goal:** Allow parameter passing in sequence files. - -**Proposed syntax:** `assets/demo.seq` - -``` -# Old syntax (still supported) -EFFECT + FlashEffect 0.0 1.0 - -# New syntax with parameters -EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 decay=0.95 threshold=0.6 - -# Or JSON-like syntax -EFFECT + FlashEffect 0.0 1.0 {color: [1.0, 0.0, 0.0], decay: 0.95} - -# Multiple instances with different colors -SEQUENCE 0 0 - EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 # Red flash - EFFECT + FlashEffect 2.0 3.0 color=0.0,1.0,0.0 # Green flash - EFFECT + FlashEffect 4.0 5.0 color=0.0,0.0,1.0 # Blue flash -``` - -**Benefits:** -- Artist-friendly configuration -- No code changes for parameter tuning -- Multiple instances with different settings - -**Size impact:** ~100 bytes (parser extension) - ---- - -### Phase 6: Update seq_compiler - -**Goal:** Parse effect parameters and generate constructor calls. - -**File:** `tools/seq_compiler.cc` - -**Generated code:** `src/generated/timeline.cc` - -```cpp -// Before (no parameters) -seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); - -// After (with parameters) -{ - FlashEffectParams p; - p.color[0] = 1.0f; p.color[1] = 0.0f; p.color[2] = 0.0f; // Red - p.decay_rate = 0.95f; - p.trigger_threshold = 0.6f; - seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0); -} -``` - -**Benefits:** -- Compile-time parameter validation -- Type-safe parameter passing -- Zero runtime overhead (parameters baked into binary) - ---- - -## Additional Use Cases - -### Chromatic Aberration Strength - -**Params struct:** -```cpp -struct ChromaAberrationParams { - float strength_multiplier = 0.02f; // Default -}; -``` - -**Shader update:** -```wgsl -struct Uniforms { - time: f32, - intensity: f32, - strength_multiplier: f32, // Added - // ... -}; - -let off = uniforms.strength_multiplier * uniforms.intensity; // Parameterized -``` - -**Usage in .seq:** -``` -EFFECT + ChromaAberrationEffect 0 6 strength=0.05 # Stronger aberration -``` - ---- - -### Gaussian Blur Kernel Size - -**Params struct:** -```cpp -struct GaussianBlurParams { - int kernel_radius = 5; // Default: 5-pixel radius - float sigma = 2.0f; // Default: standard deviation -}; -``` - -**Shader update:** -```wgsl -struct Uniforms { - kernel_radius: i32, - sigma: f32, - // ... -}; - -// Use uniforms.kernel_radius in blur loop -for (var i = -uniforms.kernel_radius; i <= uniforms.kernel_radius; i++) { - // ... -} -``` - ---- - -## Testing Strategy - -### Unit Tests - -**File:** `src/tests/test_effect_params.cc` - -```cpp -void test_flash_effect_default_params() { - // Test default white color - FlashEffectParams params; - assert(params.color[0] == 1.0f && params.color[1] == 1.0f && params.color[2] == 1.0f); -} - -void test_flash_effect_custom_color() { - // Test red flash - FlashEffectParams params; - params.color[0] = 1.0f; - params.color[1] = 0.0f; - params.color[2] = 0.0f; - - GpuContext ctx = /* ... */; - FlashEffect effect(ctx, params); - // Verify effect uses red color -} -``` - -### Integration Tests - -**Verify:** -- .seq file parser handles parameters correctly -- Generated timeline.cc compiles -- Effects render with correct parameters -- Backward compatibility (effects without parameters still work) - ---- - -## Size Budget - -**Estimated size impact:** -- UniformHelper template: ~200 bytes -- Param structs (5 effects × 50 bytes): ~250 bytes -- seq_compiler parser: ~100 bytes -- Generated code overhead: ~50 bytes per parameterized effect -- **Total: ~600-800 bytes** - -**Savings:** -- Reduced boilerplate in effect implementations: ~300 bytes -- Single shader for multiple variants: ~1KB (avoid duplicate shaders) -- **Net impact: ~400 bytes or neutral** - ---- - -## Migration Path - -### Step 1: Add UniformHelper (backward compatible) -- No changes to existing effects -- Test with new helper - -### Step 2: Update FlashEffect (pilot) -- Add FlashEffectParams struct -- Update constructor to accept params (default values = current behavior) -- Update shader to use parameterized color -- Verify backward compatibility (no .seq changes yet) - -### Step 3: Extend .seq syntax -- Update seq_compiler to parse parameters -- Test with FlashEffect only -- Verify generated code compiles - -### Step 4: Migrate other effects -- ChromaAberrationEffect -- GaussianBlurEffect -- DistortEffect -- etc. - -### Step 5: Update demo.seq -- Add parameter specifications for desired effects -- Test visual output - ---- - -## Alternative Approaches Considered - -### Approach A: Virtual parameter interface -**Idea:** Add `virtual void set_param(const char* name, float value)` to Effect base class. - -**Pros:** Very flexible -**Cons:** -- String lookups at runtime (slow, size-costly) -- No type safety -- Poor compile-time validation - -**Verdict:** ❌ Rejected (size and performance concerns) - ---- - -### Approach B: Macro-based parameter system -**Idea:** Use preprocessor macros to generate param structs and parsers. - -**Pros:** Low code duplication -**Cons:** -- Hard to debug -- Poor IDE support -- Opaque error messages - -**Verdict:** ❌ Rejected (maintainability) - ---- - -### Approach C: Runtime parameter animation -**Idea:** Allow parameters to change over time via curves. - -**Pros:** Very flexible (animate flash color over time) -**Cons:** -- Significantly more complex -- Higher runtime cost -- Larger binary size - -**Verdict:** ⚠️ Future work (post-MVP) - ---- - -## Recommended Implementation Order - -1. ✅ **This document** - Design review and approval -2. **Phase 1** - UniformHelper template (~1 hour) -3. **Phase 2** - FlashEffectParams struct (~30 min) -4. **Phase 3** - Update flash.wgsl shader (~30 min) -5. **Phase 4** - Update FlashEffect implementation (~1 hour) -6. **Phase 5** - Extend .seq syntax (~1 hour) -7. **Phase 6** - Update seq_compiler (~2 hours) -8. **Testing** - Unit + integration tests (~1 hour) -9. **Migration** - Apply to 3-5 more effects (~3 hours) - -**Total effort:** ~10-12 hours - ---- - -## Open Questions - -1. **Parameter validation:** Should seq_compiler validate parameter ranges (e.g., color components in [0, 1])? - - Proposal: Yes, with warnings (not errors) for out-of-range values - -2. **Parameter naming:** Use C++ names (`color`) or shader names (`flash_color`)? - - Proposal: C++ names (more natural in .seq files) - -3. **Backward compatibility:** Keep old constructors `Effect(ctx)` or deprecate? - - Proposal: Keep for backward compatibility, use default params - -4. **Parameter persistence:** Store params in SequenceItem for debugging? - - Proposal: No (save size), params only in generated code - ---- - -## Success Criteria - -✅ FlashEffect supports configurable color (red, green, blue, etc.) -✅ ChromaAberrationEffect supports configurable strength -✅ .seq file syntax allows parameter specification -✅ No changes required to effects that don't use parameters -✅ Size impact < 1KB -✅ All existing tests pass -✅ New tests for parameter system pass - ---- - -## Per-Frame Parameter Computation Patterns - -### Pattern 1: Time-based animation -```cpp -void render(..., float time, ...) { - float r = params_.base_color[0] * (0.5f + 0.5f * sinf(time)); - float g = params_.base_color[1] * (0.5f + 0.5f * cosf(time * 1.3f)); - float b = params_.base_color[2]; - // Use r, g, b in uniforms -} -``` - -### Pattern 2: Beat synchronization -```cpp -void render(..., float beat, ...) { - float strength = params_.base_strength * (1.0f + 0.5f * beat); - // Pulses with audio beat -} -``` - -### Pattern 3: Intensity modulation -```cpp -void render(..., float intensity, ...) { - float alpha = params_.base_alpha * intensity; - // Fades with audio intensity -} -``` - -### Pattern 4: Combined animation -```cpp -void render(..., float time, float beat, float intensity, ...) { - float r = params_.color[0] * (0.8f + 0.2f * sinf(time)); // Oscillate - float g = params_.color[1] * (1.0f + 0.3f * beat); // Beat pulse - float b = params_.color[2] * intensity; // Audio reactive - // Complex, multi-factor animation -} -``` - -**Key principle:** `render()` is called every frame - compute parameters dynamically here, not in constructor. - ---- - -## Next Steps - -**Immediate:** -1. Review this document with team/user -2. Get approval on approach -3. Start Phase 1 implementation - -**Future enhancements (post-MVP):** -- Parameter curve system (keyframe animation) -- Parameter validation in seq_compiler -- Visual parameter editor tool -- More effects with parameters (all post-process effects) |