diff options
| author | skal <pascal.massimino@gmail.com> | 2026-02-08 21:28:40 +0100 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-02-08 21:28:40 +0100 |
| commit | d3a609fad91744c45f6bc59b625a26f8870e271d (patch) | |
| tree | 002b224a216f0710cb6df9823570814628b05203 /doc/archive/SHADER_PARAMETRIZATION_PLAN.md | |
| parent | f6324b0b5d65aef6e713e8b902a6b689659dd27f (diff) | |
docs: Archive historical documentation (26 files → doc/archive/)
Moved completed/historical docs to doc/archive/ for cleaner context:
Archived (26 files):
- Analysis docs: variable tempo, audio architecture, build optimization
- Handoff docs: 6 agent handoff documents
- Debug reports: shadows, peak meter, timing fixes
- Task summaries and planning docs
Kept (16 files):
- Essential: AI_RULES, HOWTO, CONTRIBUTING, CONTEXT_MAINTENANCE
- Active subsystems: 3D, ASSET_SYSTEM, TRACKER, SEQUENCE
- Current work: MASKING_SYSTEM, SPECTRAL_BRUSH_EDITOR
Updated COMPLETED.md with archive index for easy reference.
Diffstat (limited to 'doc/archive/SHADER_PARAMETRIZATION_PLAN.md')
| -rw-r--r-- | doc/archive/SHADER_PARAMETRIZATION_PLAN.md | 596 |
1 files changed, 596 insertions, 0 deletions
diff --git a/doc/archive/SHADER_PARAMETRIZATION_PLAN.md b/doc/archive/SHADER_PARAMETRIZATION_PLAN.md new file mode 100644 index 0000000..f5afa27 --- /dev/null +++ b/doc/archive/SHADER_PARAMETRIZATION_PLAN.md @@ -0,0 +1,596 @@ +# Shader Parametrization Plan + +## Problem Statement + +**Current limitations:** +1. Effect parameters are hardcoded in shader code (e.g., FlashEffect always uses white color) +2. No way to pass parameters from .seq file to effect constructors +3. Each effect reimplements uniform buffer management +4. Uniform structures are manually sized and written +5. Cannot easily vary effect behavior without creating new effect classes + +**Use cases:** +- FlashEffect with configurable color (red, blue, green) +- ChromaAberrationEffect with configurable strength multiplier +- GaussianBlurEffect with configurable kernel size +- Any effect with user-defined parameters + +--- + +## Current Architecture Analysis + +### Effect Construction (timeline.cc) +```cpp +// Generated by seq_compiler from assets/demo.seq +seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); +seq->add_effect(std::make_shared<ChromaAberrationEffect>(ctx), 0, 6, 2); +``` + +**Issue:** Only `GpuContext` passed to constructor, no custom parameters. + +### Effect Rendering (flash_effect.cc) +```cpp +void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) { + // Hardcoded uniform values + float uniforms[4] = {flash_intensity_, intensity, 0.0f, 0.0f}; + wgpuQueueWriteBuffer(ctx_.queue, uniforms_.buffer, 0, uniforms, sizeof(uniforms)); + // ... +} +``` + +**Issue:** Parameters hardcoded, uniform writing manual and repetitive. + +### Shader Parameters (flash_effect.cc:43-44) +```wgsl +let white = vec3<f32>(1.0, 1.0, 1.0); // Hardcoded color +let green = vec3<f32>(0.0, 1.0, 0.0); // Hardcoded alternative +var flashed = mix(color.rgb, green, uniforms.intensity); +``` + +**Issue:** Flash colors baked into shader code, not parameterizable. + +### Chromatic Aberration (chroma_aberration.wgsl:25) +```wgsl +let off = 0.02 * uniforms.intensity; // Hardcoded strength multiplier +``` + +**Issue:** Strength multiplier (0.02) is hardcoded, cannot be configured per instance. + +--- + +## Proposed Solution + +### Design Principles +1. **Constructor-time parameters:** Base/initial values (base colors, multipliers, modes) +2. **Per-frame computation:** Parameters computed in `render()` based on time/beat/intensity +3. **Uniform management:** Centralized helper to reduce boilerplate +4. **.seq file syntax:** Extend to support base parameter values +5. **Size-conscious:** Minimal overhead (this is a 64k demo) + +**IMPORTANT:** Parameters are **computed dynamically every frame**, not set once at construction. Constructor params provide base values, `render()` computes animated values. + +--- + +## Implementation Plan + +### Phase 1: Uniform Management Helper + +**Goal:** Reduce boilerplate for uniform buffer creation and updates. + +**New file:** `src/gpu/uniform_helper.h` + +```cpp +#pragma once +#include "gpu/gpu.h" +#include <cstring> + +// Generic uniform buffer helper +template <typename T> +class UniformBuffer { + public: + UniformBuffer() = default; + + void init(WGPUDevice device) { + buffer_ = gpu_create_buffer(device, sizeof(T), + WGPUBufferUsage_Uniform | WGPUBufferUsage_CopyDst); + } + + void update(WGPUQueue queue, const T& data) { + wgpuQueueWriteBuffer(queue, buffer_.buffer, 0, &data, sizeof(T)); + } + + GpuBuffer& get() { return buffer_; } + + private: + GpuBuffer buffer_; +}; +``` + +**Benefits:** +- Type-safe uniform updates +- Automatic size calculation +- Reduces manual `wgpuQueueWriteBuffer` calls + +**Size impact:** ~200 bytes (template instantiation minimal) + +--- + +### Phase 2: Effect Parameter Structs + +**Goal:** Define typed parameter structures for configurable effects. + +**Example:** `src/gpu/effects/flash_effect.h` + +```cpp +#pragma once +#include "gpu/effect.h" +#include "gpu/uniform_helper.h" + +struct FlashEffectParams { + float color[3] = {1.0f, 1.0f, 1.0f}; // Default: white + float decay_rate = 0.98f; // Default: fast decay + float trigger_threshold = 0.7f; // Default: trigger on strong beats +}; + +struct FlashUniforms { + float flash_intensity; + float intensity; + float color[3]; // Added: configurable color + float _pad; +}; + +class FlashEffect : public PostProcessEffect { + public: + FlashEffect(const GpuContext& ctx, const FlashEffectParams& params = {}); + void render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) override; + void update_bind_group(WGPUTextureView input_view) override; + + private: + FlashEffectParams params_; + UniformBuffer<FlashUniforms> uniforms_; + float flash_intensity_ = 0.0f; +}; +``` + +**Benefits:** +- Clear separation: constructor params vs runtime state +- Default values for backward compatibility +- Type safety + +**Size impact:** ~50 bytes per effect (struct storage) + +--- + +### Phase 3: Update Shaders to Use Parameters + +**Goal:** Make shaders accept parameterized values. + +**Example:** `assets/final/shaders/flash.wgsl` + +```wgsl +struct Uniforms { + flash_intensity: f32, + intensity: f32, + flash_color: vec3<f32>, // Added: parameterized color + _pad: f32, +}; + +@group(0) @binding(2) var<uniform> uniforms: Uniforms; + +@fragment +fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> { + let color = textureSample(inputTexture, inputSampler, input.uv); + + // Use uniform color instead of hardcoded white + var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity); + + return vec4<f32>(flashed, color.a); +} +``` + +**Benefits:** +- Flexible shader behavior without recompilation +- Single shader supports multiple color variants + +**Size impact:** +12 bytes per uniform struct (3 floats for color) + +--- + +### Phase 4: Update Effect Implementations + +**Goal:** Use new parameter system in effect constructors and render methods. + +**Example:** `src/gpu/effects/flash_effect.cc` + +```cpp +FlashEffect::FlashEffect(const GpuContext& ctx, const FlashEffectParams& params) + : PostProcessEffect(ctx), params_(params) { + + // Shader code (updated to use flash_color) + const char* shader_code = R"( + // ... (shader from Phase 3) + )"; + + pipeline_ = create_post_process_pipeline(ctx_.device, ctx_.format, shader_code); + uniforms_.init(ctx_.device); +} + +void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) { + (void)aspect_ratio; + + // Trigger flash based on configured threshold + if (intensity > params_.trigger_threshold && flash_intensity_ < 0.2f) { + flash_intensity_ = 0.8f; + } + + // Decay based on configured rate + flash_intensity_ *= params_.decay_rate; + + // *** PER-FRAME PARAMETER COMPUTATION *** + // Animate color based on time and beat + float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f)); + float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f)); + float b = params_.color[2] * (1.0f + 0.3f * beat); // Pulse with beat + + // Update uniforms with computed (animated) values + FlashUniforms u = { + .flash_intensity = flash_intensity_, + .intensity = intensity, + .color = {r, g, b}, // Time-dependent, computed every frame + ._pad = 0.0f + }; + uniforms_.update(ctx_.queue, u); + + wgpuRenderPassEncoderSetPipeline(pass, pipeline_); + wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr); + wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0); +} +``` + +**Benefits:** +- Clean separation of parameters and logic +- Easy to test different configurations +- Type-safe uniform updates + +--- + +### Phase 5: Extend .seq File Format + +**Goal:** Allow parameter passing in sequence files. + +**Proposed syntax:** `assets/demo.seq` + +``` +# Old syntax (still supported) +EFFECT + FlashEffect 0.0 1.0 + +# New syntax with parameters +EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 decay=0.95 threshold=0.6 + +# Or JSON-like syntax +EFFECT + FlashEffect 0.0 1.0 {color: [1.0, 0.0, 0.0], decay: 0.95} + +# Multiple instances with different colors +SEQUENCE 0 0 + EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 # Red flash + EFFECT + FlashEffect 2.0 3.0 color=0.0,1.0,0.0 # Green flash + EFFECT + FlashEffect 4.0 5.0 color=0.0,0.0,1.0 # Blue flash +``` + +**Benefits:** +- Artist-friendly configuration +- No code changes for parameter tuning +- Multiple instances with different settings + +**Size impact:** ~100 bytes (parser extension) + +--- + +### Phase 6: Update seq_compiler + +**Goal:** Parse effect parameters and generate constructor calls. + +**File:** `tools/seq_compiler.cc` + +**Generated code:** `src/generated/timeline.cc` + +```cpp +// Before (no parameters) +seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); + +// After (with parameters) +{ + FlashEffectParams p; + p.color[0] = 1.0f; p.color[1] = 0.0f; p.color[2] = 0.0f; // Red + p.decay_rate = 0.95f; + p.trigger_threshold = 0.6f; + seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0); +} +``` + +**Benefits:** +- Compile-time parameter validation +- Type-safe parameter passing +- Zero runtime overhead (parameters baked into binary) + +--- + +## Additional Use Cases + +### Chromatic Aberration Strength + +**Params struct:** +```cpp +struct ChromaAberrationParams { + float strength_multiplier = 0.02f; // Default +}; +``` + +**Shader update:** +```wgsl +struct Uniforms { + time: f32, + intensity: f32, + strength_multiplier: f32, // Added + // ... +}; + +let off = uniforms.strength_multiplier * uniforms.intensity; // Parameterized +``` + +**Usage in .seq:** +``` +EFFECT + ChromaAberrationEffect 0 6 strength=0.05 # Stronger aberration +``` + +--- + +### Gaussian Blur Kernel Size + +**Params struct:** +```cpp +struct GaussianBlurParams { + int kernel_radius = 5; // Default: 5-pixel radius + float sigma = 2.0f; // Default: standard deviation +}; +``` + +**Shader update:** +```wgsl +struct Uniforms { + kernel_radius: i32, + sigma: f32, + // ... +}; + +// Use uniforms.kernel_radius in blur loop +for (var i = -uniforms.kernel_radius; i <= uniforms.kernel_radius; i++) { + // ... +} +``` + +--- + +## Testing Strategy + +### Unit Tests + +**File:** `src/tests/test_effect_params.cc` + +```cpp +void test_flash_effect_default_params() { + // Test default white color + FlashEffectParams params; + assert(params.color[0] == 1.0f && params.color[1] == 1.0f && params.color[2] == 1.0f); +} + +void test_flash_effect_custom_color() { + // Test red flash + FlashEffectParams params; + params.color[0] = 1.0f; + params.color[1] = 0.0f; + params.color[2] = 0.0f; + + GpuContext ctx = /* ... */; + FlashEffect effect(ctx, params); + // Verify effect uses red color +} +``` + +### Integration Tests + +**Verify:** +- .seq file parser handles parameters correctly +- Generated timeline.cc compiles +- Effects render with correct parameters +- Backward compatibility (effects without parameters still work) + +--- + +## Size Budget + +**Estimated size impact:** +- UniformHelper template: ~200 bytes +- Param structs (5 effects × 50 bytes): ~250 bytes +- seq_compiler parser: ~100 bytes +- Generated code overhead: ~50 bytes per parameterized effect +- **Total: ~600-800 bytes** + +**Savings:** +- Reduced boilerplate in effect implementations: ~300 bytes +- Single shader for multiple variants: ~1KB (avoid duplicate shaders) +- **Net impact: ~400 bytes or neutral** + +--- + +## Migration Path + +### Step 1: Add UniformHelper (backward compatible) +- No changes to existing effects +- Test with new helper + +### Step 2: Update FlashEffect (pilot) +- Add FlashEffectParams struct +- Update constructor to accept params (default values = current behavior) +- Update shader to use parameterized color +- Verify backward compatibility (no .seq changes yet) + +### Step 3: Extend .seq syntax +- Update seq_compiler to parse parameters +- Test with FlashEffect only +- Verify generated code compiles + +### Step 4: Migrate other effects +- ChromaAberrationEffect +- GaussianBlurEffect +- DistortEffect +- etc. + +### Step 5: Update demo.seq +- Add parameter specifications for desired effects +- Test visual output + +--- + +## Alternative Approaches Considered + +### Approach A: Virtual parameter interface +**Idea:** Add `virtual void set_param(const char* name, float value)` to Effect base class. + +**Pros:** Very flexible +**Cons:** +- String lookups at runtime (slow, size-costly) +- No type safety +- Poor compile-time validation + +**Verdict:** ❌ Rejected (size and performance concerns) + +--- + +### Approach B: Macro-based parameter system +**Idea:** Use preprocessor macros to generate param structs and parsers. + +**Pros:** Low code duplication +**Cons:** +- Hard to debug +- Poor IDE support +- Opaque error messages + +**Verdict:** ❌ Rejected (maintainability) + +--- + +### Approach C: Runtime parameter animation +**Idea:** Allow parameters to change over time via curves. + +**Pros:** Very flexible (animate flash color over time) +**Cons:** +- Significantly more complex +- Higher runtime cost +- Larger binary size + +**Verdict:** ⚠️ Future work (post-MVP) + +--- + +## Recommended Implementation Order + +1. ✅ **This document** - Design review and approval +2. **Phase 1** - UniformHelper template (~1 hour) +3. **Phase 2** - FlashEffectParams struct (~30 min) +4. **Phase 3** - Update flash.wgsl shader (~30 min) +5. **Phase 4** - Update FlashEffect implementation (~1 hour) +6. **Phase 5** - Extend .seq syntax (~1 hour) +7. **Phase 6** - Update seq_compiler (~2 hours) +8. **Testing** - Unit + integration tests (~1 hour) +9. **Migration** - Apply to 3-5 more effects (~3 hours) + +**Total effort:** ~10-12 hours + +--- + +## Open Questions + +1. **Parameter validation:** Should seq_compiler validate parameter ranges (e.g., color components in [0, 1])? + - Proposal: Yes, with warnings (not errors) for out-of-range values + +2. **Parameter naming:** Use C++ names (`color`) or shader names (`flash_color`)? + - Proposal: C++ names (more natural in .seq files) + +3. **Backward compatibility:** Keep old constructors `Effect(ctx)` or deprecate? + - Proposal: Keep for backward compatibility, use default params + +4. **Parameter persistence:** Store params in SequenceItem for debugging? + - Proposal: No (save size), params only in generated code + +--- + +## Success Criteria + +✅ FlashEffect supports configurable color (red, green, blue, etc.) +✅ ChromaAberrationEffect supports configurable strength +✅ .seq file syntax allows parameter specification +✅ No changes required to effects that don't use parameters +✅ Size impact < 1KB +✅ All existing tests pass +✅ New tests for parameter system pass + +--- + +## Per-Frame Parameter Computation Patterns + +### Pattern 1: Time-based animation +```cpp +void render(..., float time, ...) { + float r = params_.base_color[0] * (0.5f + 0.5f * sinf(time)); + float g = params_.base_color[1] * (0.5f + 0.5f * cosf(time * 1.3f)); + float b = params_.base_color[2]; + // Use r, g, b in uniforms +} +``` + +### Pattern 2: Beat synchronization +```cpp +void render(..., float beat, ...) { + float strength = params_.base_strength * (1.0f + 0.5f * beat); + // Pulses with audio beat +} +``` + +### Pattern 3: Intensity modulation +```cpp +void render(..., float intensity, ...) { + float alpha = params_.base_alpha * intensity; + // Fades with audio intensity +} +``` + +### Pattern 4: Combined animation +```cpp +void render(..., float time, float beat, float intensity, ...) { + float r = params_.color[0] * (0.8f + 0.2f * sinf(time)); // Oscillate + float g = params_.color[1] * (1.0f + 0.3f * beat); // Beat pulse + float b = params_.color[2] * intensity; // Audio reactive + // Complex, multi-factor animation +} +``` + +**Key principle:** `render()` is called every frame - compute parameters dynamically here, not in constructor. + +--- + +## Next Steps + +**Immediate:** +1. Review this document with team/user +2. Get approval on approach +3. Start Phase 1 implementation + +**Future enhancements (post-MVP):** +- Parameter curve system (keyframe animation) +- Parameter validation in seq_compiler +- Visual parameter editor tool +- More effects with parameters (all post-process effects) |
