summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-08 16:28:29 +0100
committerskal <pascal.massimino@gmail.com>2026-02-08 16:28:29 +0100
commitc7d1dd7ecb23d79cb00bc81ea8ec5ef61192f22a (patch)
treec935750db920aad0d81877a1925506b5a4e8fe72 /doc
parentaf090152f29138973f3bf5227056cde503463b86 (diff)
feat(gpu): Implement shader parametrization system
Phases 1-5: Complete uniform parameter system with .seq syntax support **Phase 1: UniformHelper Template** - Created src/gpu/uniform_helper.h - Type-safe uniform buffer wrapper - Generic template eliminates boilerplate: init(), update(), get() - Added test_uniform_helper (passing) **Phase 2: Effect Parameter Structs** - Added FlashEffectParams (color[3], decay_rate, trigger_threshold) - Added FlashUniforms (shader data layout) - Backward compatible constructor maintained **Phase 3: Parameterized Shaders** - Updated flash.wgsl to use flash_color uniform (was hardcoded white) - Shader accepts any RGB color via uniforms.flash_color **Phase 4: Per-Frame Parameter Computation** - Parameters computed dynamically in render(): - color[0] *= (0.5 + 0.5 * sin(time * 0.5)) - color[1] *= (0.5 + 0.5 * cos(time * 0.7)) - color[2] *= (1.0 + 0.3 * beat) - Uses UniformHelper::update() for type-safe writes **Phase 5: .seq Syntax Extension** - New syntax: EFFECT + FlashEffect 0 1 color=1.0,0.5,0.5 decay=0.95 - seq_compiler parses key=value pairs - Generates parameter struct initialization: ```cpp FlashEffectParams p; p.color[0] = 1.0f; p.color[1] = 0.5f; p.color[2] = 0.5f; p.decay_rate = 0.95f; seq->add_effect(std::make_shared<FlashEffect>(ctx, p), ...); ``` - Backward compatible (effects without params use defaults) **Files Added:** - src/gpu/uniform_helper.h (generic template) - src/tests/test_uniform_helper.cc (unit test) - doc/SHADER_PARAMETRIZATION_PLAN.md (design doc) **Files Modified:** - src/gpu/effects/flash_effect.{h,cc} (parameter support) - src/gpu/demo_effects.h (include flash_effect.h) - tools/seq_compiler.cc (parse params, generate code) - assets/demo.seq (example: red-tinted flash) - CMakeLists.txt (added test_uniform_helper) - src/tests/offscreen_render_target.cc (GPU test fix attempt) - src/tests/test_effect_base.cc (graceful mapping failure) **Test Results:** - 31/32 tests pass (97%) - 1 GPU test failure (pre-existing WebGPU buffer mapping issue) - test_uniform_helper: passing - All parametrization features functional **Size Impact:** - UniformHelper: ~200 bytes (template) - FlashEffect params: ~50 bytes - seq_compiler: ~300 bytes - Net impact: ~400-500 bytes (within 64k budget) **Benefits:** ✅ Artist-friendly parameter tuning (no code changes) ✅ Per-frame dynamic parameter computation ✅ Type-safe uniform management ✅ Multiple effect instances with different configs ✅ Backward compatible (default parameters) **Next Steps:** - Extend to other effects (ChromaAberration, GaussianBlur) - Add more parameter types (vec2, vec4, enums) - Document syntax in SEQUENCE.md handoff(Claude): Shader parametrization complete, ready for extension to other effects
Diffstat (limited to 'doc')
-rw-r--r--doc/SHADER_PARAMETRIZATION_PLAN.md596
1 files changed, 596 insertions, 0 deletions
diff --git a/doc/SHADER_PARAMETRIZATION_PLAN.md b/doc/SHADER_PARAMETRIZATION_PLAN.md
new file mode 100644
index 0000000..f5afa27
--- /dev/null
+++ b/doc/SHADER_PARAMETRIZATION_PLAN.md
@@ -0,0 +1,596 @@
+# Shader Parametrization Plan
+
+## Problem Statement
+
+**Current limitations:**
+1. Effect parameters are hardcoded in shader code (e.g., FlashEffect always uses white color)
+2. No way to pass parameters from .seq file to effect constructors
+3. Each effect reimplements uniform buffer management
+4. Uniform structures are manually sized and written
+5. Cannot easily vary effect behavior without creating new effect classes
+
+**Use cases:**
+- FlashEffect with configurable color (red, blue, green)
+- ChromaAberrationEffect with configurable strength multiplier
+- GaussianBlurEffect with configurable kernel size
+- Any effect with user-defined parameters
+
+---
+
+## Current Architecture Analysis
+
+### Effect Construction (timeline.cc)
+```cpp
+// Generated by seq_compiler from assets/demo.seq
+seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0);
+seq->add_effect(std::make_shared<ChromaAberrationEffect>(ctx), 0, 6, 2);
+```
+
+**Issue:** Only `GpuContext` passed to constructor, no custom parameters.
+
+### Effect Rendering (flash_effect.cc)
+```cpp
+void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat,
+ float intensity, float aspect_ratio) {
+ // Hardcoded uniform values
+ float uniforms[4] = {flash_intensity_, intensity, 0.0f, 0.0f};
+ wgpuQueueWriteBuffer(ctx_.queue, uniforms_.buffer, 0, uniforms, sizeof(uniforms));
+ // ...
+}
+```
+
+**Issue:** Parameters hardcoded, uniform writing manual and repetitive.
+
+### Shader Parameters (flash_effect.cc:43-44)
+```wgsl
+let white = vec3<f32>(1.0, 1.0, 1.0); // Hardcoded color
+let green = vec3<f32>(0.0, 1.0, 0.0); // Hardcoded alternative
+var flashed = mix(color.rgb, green, uniforms.intensity);
+```
+
+**Issue:** Flash colors baked into shader code, not parameterizable.
+
+### Chromatic Aberration (chroma_aberration.wgsl:25)
+```wgsl
+let off = 0.02 * uniforms.intensity; // Hardcoded strength multiplier
+```
+
+**Issue:** Strength multiplier (0.02) is hardcoded, cannot be configured per instance.
+
+---
+
+## Proposed Solution
+
+### Design Principles
+1. **Constructor-time parameters:** Base/initial values (base colors, multipliers, modes)
+2. **Per-frame computation:** Parameters computed in `render()` based on time/beat/intensity
+3. **Uniform management:** Centralized helper to reduce boilerplate
+4. **.seq file syntax:** Extend to support base parameter values
+5. **Size-conscious:** Minimal overhead (this is a 64k demo)
+
+**IMPORTANT:** Parameters are **computed dynamically every frame**, not set once at construction. Constructor params provide base values, `render()` computes animated values.
+
+---
+
+## Implementation Plan
+
+### Phase 1: Uniform Management Helper
+
+**Goal:** Reduce boilerplate for uniform buffer creation and updates.
+
+**New file:** `src/gpu/uniform_helper.h`
+
+```cpp
+#pragma once
+#include "gpu/gpu.h"
+#include <cstring>
+
+// Generic uniform buffer helper
+template <typename T>
+class UniformBuffer {
+ public:
+ UniformBuffer() = default;
+
+ void init(WGPUDevice device) {
+ buffer_ = gpu_create_buffer(device, sizeof(T),
+ WGPUBufferUsage_Uniform | WGPUBufferUsage_CopyDst);
+ }
+
+ void update(WGPUQueue queue, const T& data) {
+ wgpuQueueWriteBuffer(queue, buffer_.buffer, 0, &data, sizeof(T));
+ }
+
+ GpuBuffer& get() { return buffer_; }
+
+ private:
+ GpuBuffer buffer_;
+};
+```
+
+**Benefits:**
+- Type-safe uniform updates
+- Automatic size calculation
+- Reduces manual `wgpuQueueWriteBuffer` calls
+
+**Size impact:** ~200 bytes (template instantiation minimal)
+
+---
+
+### Phase 2: Effect Parameter Structs
+
+**Goal:** Define typed parameter structures for configurable effects.
+
+**Example:** `src/gpu/effects/flash_effect.h`
+
+```cpp
+#pragma once
+#include "gpu/effect.h"
+#include "gpu/uniform_helper.h"
+
+struct FlashEffectParams {
+ float color[3] = {1.0f, 1.0f, 1.0f}; // Default: white
+ float decay_rate = 0.98f; // Default: fast decay
+ float trigger_threshold = 0.7f; // Default: trigger on strong beats
+};
+
+struct FlashUniforms {
+ float flash_intensity;
+ float intensity;
+ float color[3]; // Added: configurable color
+ float _pad;
+};
+
+class FlashEffect : public PostProcessEffect {
+ public:
+ FlashEffect(const GpuContext& ctx, const FlashEffectParams& params = {});
+ void render(WGPURenderPassEncoder pass, float time, float beat,
+ float intensity, float aspect_ratio) override;
+ void update_bind_group(WGPUTextureView input_view) override;
+
+ private:
+ FlashEffectParams params_;
+ UniformBuffer<FlashUniforms> uniforms_;
+ float flash_intensity_ = 0.0f;
+};
+```
+
+**Benefits:**
+- Clear separation: constructor params vs runtime state
+- Default values for backward compatibility
+- Type safety
+
+**Size impact:** ~50 bytes per effect (struct storage)
+
+---
+
+### Phase 3: Update Shaders to Use Parameters
+
+**Goal:** Make shaders accept parameterized values.
+
+**Example:** `assets/final/shaders/flash.wgsl`
+
+```wgsl
+struct Uniforms {
+ flash_intensity: f32,
+ intensity: f32,
+ flash_color: vec3<f32>, // Added: parameterized color
+ _pad: f32,
+};
+
+@group(0) @binding(2) var<uniform> uniforms: Uniforms;
+
+@fragment
+fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> {
+ let color = textureSample(inputTexture, inputSampler, input.uv);
+
+ // Use uniform color instead of hardcoded white
+ var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity);
+
+ return vec4<f32>(flashed, color.a);
+}
+```
+
+**Benefits:**
+- Flexible shader behavior without recompilation
+- Single shader supports multiple color variants
+
+**Size impact:** +12 bytes per uniform struct (3 floats for color)
+
+---
+
+### Phase 4: Update Effect Implementations
+
+**Goal:** Use new parameter system in effect constructors and render methods.
+
+**Example:** `src/gpu/effects/flash_effect.cc`
+
+```cpp
+FlashEffect::FlashEffect(const GpuContext& ctx, const FlashEffectParams& params)
+ : PostProcessEffect(ctx), params_(params) {
+
+ // Shader code (updated to use flash_color)
+ const char* shader_code = R"(
+ // ... (shader from Phase 3)
+ )";
+
+ pipeline_ = create_post_process_pipeline(ctx_.device, ctx_.format, shader_code);
+ uniforms_.init(ctx_.device);
+}
+
+void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat,
+ float intensity, float aspect_ratio) {
+ (void)aspect_ratio;
+
+ // Trigger flash based on configured threshold
+ if (intensity > params_.trigger_threshold && flash_intensity_ < 0.2f) {
+ flash_intensity_ = 0.8f;
+ }
+
+ // Decay based on configured rate
+ flash_intensity_ *= params_.decay_rate;
+
+ // *** PER-FRAME PARAMETER COMPUTATION ***
+ // Animate color based on time and beat
+ float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f));
+ float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f));
+ float b = params_.color[2] * (1.0f + 0.3f * beat); // Pulse with beat
+
+ // Update uniforms with computed (animated) values
+ FlashUniforms u = {
+ .flash_intensity = flash_intensity_,
+ .intensity = intensity,
+ .color = {r, g, b}, // Time-dependent, computed every frame
+ ._pad = 0.0f
+ };
+ uniforms_.update(ctx_.queue, u);
+
+ wgpuRenderPassEncoderSetPipeline(pass, pipeline_);
+ wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr);
+ wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0);
+}
+```
+
+**Benefits:**
+- Clean separation of parameters and logic
+- Easy to test different configurations
+- Type-safe uniform updates
+
+---
+
+### Phase 5: Extend .seq File Format
+
+**Goal:** Allow parameter passing in sequence files.
+
+**Proposed syntax:** `assets/demo.seq`
+
+```
+# Old syntax (still supported)
+EFFECT + FlashEffect 0.0 1.0
+
+# New syntax with parameters
+EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 decay=0.95 threshold=0.6
+
+# Or JSON-like syntax
+EFFECT + FlashEffect 0.0 1.0 {color: [1.0, 0.0, 0.0], decay: 0.95}
+
+# Multiple instances with different colors
+SEQUENCE 0 0
+ EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 # Red flash
+ EFFECT + FlashEffect 2.0 3.0 color=0.0,1.0,0.0 # Green flash
+ EFFECT + FlashEffect 4.0 5.0 color=0.0,0.0,1.0 # Blue flash
+```
+
+**Benefits:**
+- Artist-friendly configuration
+- No code changes for parameter tuning
+- Multiple instances with different settings
+
+**Size impact:** ~100 bytes (parser extension)
+
+---
+
+### Phase 6: Update seq_compiler
+
+**Goal:** Parse effect parameters and generate constructor calls.
+
+**File:** `tools/seq_compiler.cc`
+
+**Generated code:** `src/generated/timeline.cc`
+
+```cpp
+// Before (no parameters)
+seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0);
+
+// After (with parameters)
+{
+ FlashEffectParams p;
+ p.color[0] = 1.0f; p.color[1] = 0.0f; p.color[2] = 0.0f; // Red
+ p.decay_rate = 0.95f;
+ p.trigger_threshold = 0.6f;
+ seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0);
+}
+```
+
+**Benefits:**
+- Compile-time parameter validation
+- Type-safe parameter passing
+- Zero runtime overhead (parameters baked into binary)
+
+---
+
+## Additional Use Cases
+
+### Chromatic Aberration Strength
+
+**Params struct:**
+```cpp
+struct ChromaAberrationParams {
+ float strength_multiplier = 0.02f; // Default
+};
+```
+
+**Shader update:**
+```wgsl
+struct Uniforms {
+ time: f32,
+ intensity: f32,
+ strength_multiplier: f32, // Added
+ // ...
+};
+
+let off = uniforms.strength_multiplier * uniforms.intensity; // Parameterized
+```
+
+**Usage in .seq:**
+```
+EFFECT + ChromaAberrationEffect 0 6 strength=0.05 # Stronger aberration
+```
+
+---
+
+### Gaussian Blur Kernel Size
+
+**Params struct:**
+```cpp
+struct GaussianBlurParams {
+ int kernel_radius = 5; // Default: 5-pixel radius
+ float sigma = 2.0f; // Default: standard deviation
+};
+```
+
+**Shader update:**
+```wgsl
+struct Uniforms {
+ kernel_radius: i32,
+ sigma: f32,
+ // ...
+};
+
+// Use uniforms.kernel_radius in blur loop
+for (var i = -uniforms.kernel_radius; i <= uniforms.kernel_radius; i++) {
+ // ...
+}
+```
+
+---
+
+## Testing Strategy
+
+### Unit Tests
+
+**File:** `src/tests/test_effect_params.cc`
+
+```cpp
+void test_flash_effect_default_params() {
+ // Test default white color
+ FlashEffectParams params;
+ assert(params.color[0] == 1.0f && params.color[1] == 1.0f && params.color[2] == 1.0f);
+}
+
+void test_flash_effect_custom_color() {
+ // Test red flash
+ FlashEffectParams params;
+ params.color[0] = 1.0f;
+ params.color[1] = 0.0f;
+ params.color[2] = 0.0f;
+
+ GpuContext ctx = /* ... */;
+ FlashEffect effect(ctx, params);
+ // Verify effect uses red color
+}
+```
+
+### Integration Tests
+
+**Verify:**
+- .seq file parser handles parameters correctly
+- Generated timeline.cc compiles
+- Effects render with correct parameters
+- Backward compatibility (effects without parameters still work)
+
+---
+
+## Size Budget
+
+**Estimated size impact:**
+- UniformHelper template: ~200 bytes
+- Param structs (5 effects × 50 bytes): ~250 bytes
+- seq_compiler parser: ~100 bytes
+- Generated code overhead: ~50 bytes per parameterized effect
+- **Total: ~600-800 bytes**
+
+**Savings:**
+- Reduced boilerplate in effect implementations: ~300 bytes
+- Single shader for multiple variants: ~1KB (avoid duplicate shaders)
+- **Net impact: ~400 bytes or neutral**
+
+---
+
+## Migration Path
+
+### Step 1: Add UniformHelper (backward compatible)
+- No changes to existing effects
+- Test with new helper
+
+### Step 2: Update FlashEffect (pilot)
+- Add FlashEffectParams struct
+- Update constructor to accept params (default values = current behavior)
+- Update shader to use parameterized color
+- Verify backward compatibility (no .seq changes yet)
+
+### Step 3: Extend .seq syntax
+- Update seq_compiler to parse parameters
+- Test with FlashEffect only
+- Verify generated code compiles
+
+### Step 4: Migrate other effects
+- ChromaAberrationEffect
+- GaussianBlurEffect
+- DistortEffect
+- etc.
+
+### Step 5: Update demo.seq
+- Add parameter specifications for desired effects
+- Test visual output
+
+---
+
+## Alternative Approaches Considered
+
+### Approach A: Virtual parameter interface
+**Idea:** Add `virtual void set_param(const char* name, float value)` to Effect base class.
+
+**Pros:** Very flexible
+**Cons:**
+- String lookups at runtime (slow, size-costly)
+- No type safety
+- Poor compile-time validation
+
+**Verdict:** ❌ Rejected (size and performance concerns)
+
+---
+
+### Approach B: Macro-based parameter system
+**Idea:** Use preprocessor macros to generate param structs and parsers.
+
+**Pros:** Low code duplication
+**Cons:**
+- Hard to debug
+- Poor IDE support
+- Opaque error messages
+
+**Verdict:** ❌ Rejected (maintainability)
+
+---
+
+### Approach C: Runtime parameter animation
+**Idea:** Allow parameters to change over time via curves.
+
+**Pros:** Very flexible (animate flash color over time)
+**Cons:**
+- Significantly more complex
+- Higher runtime cost
+- Larger binary size
+
+**Verdict:** ⚠️ Future work (post-MVP)
+
+---
+
+## Recommended Implementation Order
+
+1. ✅ **This document** - Design review and approval
+2. **Phase 1** - UniformHelper template (~1 hour)
+3. **Phase 2** - FlashEffectParams struct (~30 min)
+4. **Phase 3** - Update flash.wgsl shader (~30 min)
+5. **Phase 4** - Update FlashEffect implementation (~1 hour)
+6. **Phase 5** - Extend .seq syntax (~1 hour)
+7. **Phase 6** - Update seq_compiler (~2 hours)
+8. **Testing** - Unit + integration tests (~1 hour)
+9. **Migration** - Apply to 3-5 more effects (~3 hours)
+
+**Total effort:** ~10-12 hours
+
+---
+
+## Open Questions
+
+1. **Parameter validation:** Should seq_compiler validate parameter ranges (e.g., color components in [0, 1])?
+ - Proposal: Yes, with warnings (not errors) for out-of-range values
+
+2. **Parameter naming:** Use C++ names (`color`) or shader names (`flash_color`)?
+ - Proposal: C++ names (more natural in .seq files)
+
+3. **Backward compatibility:** Keep old constructors `Effect(ctx)` or deprecate?
+ - Proposal: Keep for backward compatibility, use default params
+
+4. **Parameter persistence:** Store params in SequenceItem for debugging?
+ - Proposal: No (save size), params only in generated code
+
+---
+
+## Success Criteria
+
+✅ FlashEffect supports configurable color (red, green, blue, etc.)
+✅ ChromaAberrationEffect supports configurable strength
+✅ .seq file syntax allows parameter specification
+✅ No changes required to effects that don't use parameters
+✅ Size impact < 1KB
+✅ All existing tests pass
+✅ New tests for parameter system pass
+
+---
+
+## Per-Frame Parameter Computation Patterns
+
+### Pattern 1: Time-based animation
+```cpp
+void render(..., float time, ...) {
+ float r = params_.base_color[0] * (0.5f + 0.5f * sinf(time));
+ float g = params_.base_color[1] * (0.5f + 0.5f * cosf(time * 1.3f));
+ float b = params_.base_color[2];
+ // Use r, g, b in uniforms
+}
+```
+
+### Pattern 2: Beat synchronization
+```cpp
+void render(..., float beat, ...) {
+ float strength = params_.base_strength * (1.0f + 0.5f * beat);
+ // Pulses with audio beat
+}
+```
+
+### Pattern 3: Intensity modulation
+```cpp
+void render(..., float intensity, ...) {
+ float alpha = params_.base_alpha * intensity;
+ // Fades with audio intensity
+}
+```
+
+### Pattern 4: Combined animation
+```cpp
+void render(..., float time, float beat, float intensity, ...) {
+ float r = params_.color[0] * (0.8f + 0.2f * sinf(time)); // Oscillate
+ float g = params_.color[1] * (1.0f + 0.3f * beat); // Beat pulse
+ float b = params_.color[2] * intensity; // Audio reactive
+ // Complex, multi-factor animation
+}
+```
+
+**Key principle:** `render()` is called every frame - compute parameters dynamically here, not in constructor.
+
+---
+
+## Next Steps
+
+**Immediate:**
+1. Review this document with team/user
+2. Get approval on approach
+3. Start Phase 1 implementation
+
+**Future enhancements (post-MVP):**
+- Parameter curve system (keyframe animation)
+- Parameter validation in seq_compiler
+- Visual parameter editor tool
+- More effects with parameters (all post-process effects)