path: root/doc/archive/SHADER_PARAMETRIZATION_PLAN.md

# Shader Parametrization Plan

## Problem Statement

**Current limitations:**
1. Effect parameters are hardcoded in shader code (e.g., FlashEffect always uses white color)
2. No way to pass parameters from .seq file to effect constructors
3. Each effect reimplements uniform buffer management
4. Uniform structures are manually sized and written
5. Cannot easily vary effect behavior without creating new effect classes

**Use cases:**
- FlashEffect with configurable color (red, blue, green)
- ChromaAberrationEffect with configurable strength multiplier
- GaussianBlurEffect with configurable kernel size
- Any effect with user-defined parameters

---

## Current Architecture Analysis

### Effect Construction (timeline.cc)
```cpp
// Generated by seq_compiler from assets/demo.seq
seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0);
seq->add_effect(std::make_shared<ChromaAberrationEffect>(ctx), 0, 6, 2);
```

**Issue:** Only `GpuContext` passed to constructor, no custom parameters.

### Effect Rendering (flash_effect.cc)
```cpp
void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat,
                         float intensity, float aspect_ratio) {
  // Hardcoded uniform values
  float uniforms[4] = {flash_intensity_, intensity, 0.0f, 0.0f};
  wgpuQueueWriteBuffer(ctx_.queue, uniforms_.buffer, 0, uniforms, sizeof(uniforms));
  // ...
}
```

**Issue:** Parameters hardcoded, uniform writing manual and repetitive.

### Shader Parameters (flash_effect.cc:43-44)
```wgsl
let white = vec3<f32>(1.0, 1.0, 1.0);  // Hardcoded color
let green = vec3<f32>(0.0, 1.0, 0.0);  // Hardcoded alternative
var flashed = mix(color.rgb, green, uniforms.intensity);
```

**Issue:** Flash colors baked into shader code, not parameterizable.

### Chromatic Aberration (chroma_aberration.wgsl:25)
```wgsl
let off = 0.02 * uniforms.intensity;  // Hardcoded strength multiplier
```

**Issue:** Strength multiplier (0.02) is hardcoded, cannot be configured per instance.

---

## Proposed Solution

### Design Principles
1. **Constructor-time parameters:** Base/initial values (base colors, multipliers, modes)
2. **Per-frame computation:** Parameters computed in `render()` based on time/beat/intensity
3. **Uniform management:** Centralized helper to reduce boilerplate
4. **.seq file syntax:** Extend to support base parameter values
5. **Size-conscious:** Minimal overhead (this is a 64k demo)

**IMPORTANT:** Parameters are **computed dynamically every frame**, not set once at construction. Constructor params provide base values, `render()` computes animated values.

---

## Implementation Plan

### Phase 1: Uniform Management Helper

**Goal:** Reduce boilerplate for uniform buffer creation and updates.

**New file:** `src/gpu/uniform_helper.h`

```cpp
#pragma once
#include "gpu/gpu.h"
#include <cstring>

// Generic uniform buffer helper
template <typename T>
class UniformBuffer {
 public:
  UniformBuffer() = default;

  void init(WGPUDevice device) {
    buffer_ = gpu_create_buffer(device, sizeof(T),
                                 WGPUBufferUsage_Uniform | WGPUBufferUsage_CopyDst);
  }

  void update(WGPUQueue queue, const T& data) {
    wgpuQueueWriteBuffer(queue, buffer_.buffer, 0, &data, sizeof(T));
  }

  GpuBuffer& get() { return buffer_; }

 private:
  GpuBuffer buffer_;
};
```

**Benefits:**
- Type-safe uniform updates
- Automatic size calculation
- Reduces manual `wgpuQueueWriteBuffer` calls

**Size impact:** ~200 bytes (template instantiation minimal)

---

### Phase 2: Effect Parameter Structs

**Goal:** Define typed parameter structures for configurable effects.

**Example:** `src/gpu/effects/flash_effect.h`

```cpp
#pragma once
#include "gpu/effect.h"
#include "gpu/uniform_helper.h"

struct FlashEffectParams {
  float color[3] = {1.0f, 1.0f, 1.0f};  // Default: white
  float decay_rate = 0.98f;              // Default: fast decay
  float trigger_threshold = 0.7f;        // Default: trigger on strong beats
};

struct FlashUniforms {
  float flash_intensity;
  float intensity;
  float color[3];  // Added: configurable color
  float _pad;
};

class FlashEffect : public PostProcessEffect {
 public:
  FlashEffect(const GpuContext& ctx, const FlashEffectParams& params = {});
  void render(WGPURenderPassEncoder pass, float time, float beat,
              float intensity, float aspect_ratio) override;
  void update_bind_group(WGPUTextureView input_view) override;

 private:
  FlashEffectParams params_;
  UniformBuffer<FlashUniforms> uniforms_;
  float flash_intensity_ = 0.0f;
};
```

**Benefits:**
- Clear separation: constructor params vs runtime state
- Default values for backward compatibility
- Type safety

**Size impact:** ~50 bytes per effect (struct storage)

---

### Phase 3: Update Shaders to Use Parameters

**Goal:** Make shaders accept parameterized values.

**Example:** `assets/final/shaders/flash.wgsl`

```wgsl
struct Uniforms {
  flash_intensity: f32,
  intensity: f32,
  flash_color: vec3<f32>,  // Added: parameterized color
  _pad: f32,
};

@group(0) @binding(2) var<uniform> uniforms: Uniforms;

@fragment
fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> {
  let color = textureSample(inputTexture, inputSampler, input.uv);

  // Use uniform color instead of hardcoded white
  var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity);

  return vec4<f32>(flashed, color.a);
}
```

**Benefits:**
- Flexible shader behavior without recompilation
- Single shader supports multiple color variants

**Size impact:** +12 bytes per uniform struct (3 floats for color)

---

### Phase 4: Update Effect Implementations

**Goal:** Use new parameter system in effect constructors and render methods.

**Example:** `src/gpu/effects/flash_effect.cc`

```cpp
FlashEffect::FlashEffect(const GpuContext& ctx, const FlashEffectParams& params)
    : PostProcessEffect(ctx), params_(params) {

  // Shader code (updated to use flash_color)
  const char* shader_code = R"(
    // ... (shader from Phase 3)
  )";

  pipeline_ = create_post_process_pipeline(ctx_.device, ctx_.format, shader_code);
  uniforms_.init(ctx_.device);
}

void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat,
                         float intensity, float aspect_ratio) {
  (void)aspect_ratio;

  // Trigger flash based on configured threshold
  if (intensity > params_.trigger_threshold && flash_intensity_ < 0.2f) {
    flash_intensity_ = 0.8f;
  }

  // Decay based on configured rate
  flash_intensity_ *= params_.decay_rate;

  // *** PER-FRAME PARAMETER COMPUTATION ***
  // Animate color based on time and beat
  float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f));
  float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f));
  float b = params_.color[2] * (1.0f + 0.3f * beat);  // Pulse with beat

  // Update uniforms with computed (animated) values
  FlashUniforms u = {
    .flash_intensity = flash_intensity_,
    .intensity = intensity,
    .color = {r, g, b},  // Time-dependent, computed every frame
    ._pad = 0.0f
  };
  uniforms_.update(ctx_.queue, u);

  wgpuRenderPassEncoderSetPipeline(pass, pipeline_);
  wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr);
  wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0);
}
```

**Benefits:**
- Clean separation of parameters and logic
- Easy to test different configurations
- Type-safe uniform updates

---

### Phase 5: Extend .seq File Format

**Goal:** Allow parameter passing in sequence files.

**Proposed syntax:** `assets/demo.seq`

```
# Old syntax (still supported)
EFFECT + FlashEffect 0.0 1.0

# New syntax with parameters
EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 decay=0.95 threshold=0.6

# Or JSON-like syntax
EFFECT + FlashEffect 0.0 1.0 {color: [1.0, 0.0, 0.0], decay: 0.95}

# Multiple instances with different colors
SEQUENCE 0 0
  EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0   # Red flash
  EFFECT + FlashEffect 2.0 3.0 color=0.0,1.0,0.0   # Green flash
  EFFECT + FlashEffect 4.0 5.0 color=0.0,0.0,1.0   # Blue flash
```

**Benefits:**
- Artist-friendly configuration
- No code changes for parameter tuning
- Multiple instances with different settings

**Size impact:** ~100 bytes (parser extension)

---

### Phase 6: Update seq_compiler

**Goal:** Parse effect parameters and generate constructor calls.

**File:** `tools/seq_compiler.cc`

**Generated code:** `src/generated/timeline.cc`

```cpp
// Before (no parameters)
seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0);

// After (with parameters)
{
  FlashEffectParams p;
  p.color[0] = 1.0f; p.color[1] = 0.0f; p.color[2] = 0.0f;  // Red
  p.decay_rate = 0.95f;
  p.trigger_threshold = 0.6f;
  seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0);
}
```

**Benefits:**
- Compile-time parameter validation
- Type-safe parameter passing
- Zero runtime overhead (parameters baked into binary)

---

## Additional Use Cases

### Chromatic Aberration Strength

**Params struct:**
```cpp
struct ChromaAberrationParams {
  float strength_multiplier = 0.02f;  // Default
};
```

**Shader update:**
```wgsl
struct Uniforms {
  time: f32,
  intensity: f32,
  strength_multiplier: f32,  // Added
  // ...
};

let off = uniforms.strength_multiplier * uniforms.intensity;  // Parameterized
```

**Usage in .seq:**
```
EFFECT + ChromaAberrationEffect 0 6 strength=0.05  # Stronger aberration
```

---

### Gaussian Blur Kernel Size

**Params struct:**
```cpp
struct GaussianBlurParams {
  int kernel_radius = 5;     // Default: 5-pixel radius
  float sigma = 2.0f;        // Default: standard deviation
};
```

**Shader update:**
```wgsl
struct Uniforms {
  kernel_radius: i32,
  sigma: f32,
  // ...
};

// Use uniforms.kernel_radius in blur loop
for (var i = -uniforms.kernel_radius; i <= uniforms.kernel_radius; i++) {
  // ...
}
```

---

## Testing Strategy

### Unit Tests

**File:** `src/tests/test_effect_params.cc`

```cpp
void test_flash_effect_default_params() {
  // Test default white color
  FlashEffectParams params;
  assert(params.color[0] == 1.0f && params.color[1] == 1.0f && params.color[2] == 1.0f);
}

void test_flash_effect_custom_color() {
  // Test red flash
  FlashEffectParams params;
  params.color[0] = 1.0f;
  params.color[1] = 0.0f;
  params.color[2] = 0.0f;

  GpuContext ctx = /* ... */;
  FlashEffect effect(ctx, params);
  // Verify effect uses red color
}
```

### Integration Tests

**Verify:**
- .seq file parser handles parameters correctly
- Generated timeline.cc compiles
- Effects render with correct parameters
- Backward compatibility (effects without parameters still work)

---

## Size Budget

**Estimated size impact:**
- UniformHelper template: ~200 bytes
- Param structs (5 effects × 50 bytes): ~250 bytes
- seq_compiler parser: ~100 bytes
- Generated code overhead: ~50 bytes per parameterized effect
- **Total: ~600-800 bytes**

**Savings:**
- Reduced boilerplate in effect implementations: ~300 bytes
- Single shader for multiple variants: ~1KB (avoid duplicate shaders)
- **Net impact: ~400 bytes or neutral**

---

## Migration Path

### Step 1: Add UniformHelper (backward compatible)
- No changes to existing effects
- Test with new helper

### Step 2: Update FlashEffect (pilot)
- Add FlashEffectParams struct
- Update constructor to accept params (default values = current behavior)
- Update shader to use parameterized color
- Verify backward compatibility (no .seq changes yet)

### Step 3: Extend .seq syntax
- Update seq_compiler to parse parameters
- Test with FlashEffect only
- Verify generated code compiles

### Step 4: Migrate other effects
- ChromaAberrationEffect
- GaussianBlurEffect
- DistortEffect
- etc.

### Step 5: Update demo.seq
- Add parameter specifications for desired effects
- Test visual output

---

## Alternative Approaches Considered

### Approach A: Virtual parameter interface
**Idea:** Add `virtual void set_param(const char* name, float value)` to Effect base class.

**Pros:** Very flexible
**Cons:**
- String lookups at runtime (slow, size-costly)
- No type safety
- Poor compile-time validation

**Verdict:** ❌ Rejected (size and performance concerns)

---

### Approach B: Macro-based parameter system
**Idea:** Use preprocessor macros to generate param structs and parsers.

**Pros:** Low code duplication
**Cons:**
- Hard to debug
- Poor IDE support
- Opaque error messages

**Verdict:** ❌ Rejected (maintainability)

---

### Approach C: Runtime parameter animation
**Idea:** Allow parameters to change over time via curves.

**Pros:** Very flexible (animate flash color over time)
**Cons:**
- Significantly more complex
- Higher runtime cost
- Larger binary size

**Verdict:** ⚠️ Future work (post-MVP)

---

## Recommended Implementation Order

1. ✅ **This document** - Design review and approval
2. **Phase 1** - UniformHelper template (~1 hour)
3. **Phase 2** - FlashEffectParams struct (~30 min)
4. **Phase 3** - Update flash.wgsl shader (~30 min)
5. **Phase 4** - Update FlashEffect implementation (~1 hour)
6. **Phase 5** - Extend .seq syntax (~1 hour)
7. **Phase 6** - Update seq_compiler (~2 hours)
8. **Testing** - Unit + integration tests (~1 hour)
9. **Migration** - Apply to 3-5 more effects (~3 hours)

**Total effort:** ~10-12 hours

---

## Open Questions

1. **Parameter validation:** Should seq_compiler validate parameter ranges (e.g., color components in [0, 1])?
   - Proposal: Yes, with warnings (not errors) for out-of-range values

2. **Parameter naming:** Use C++ names (`color`) or shader names (`flash_color`)?
   - Proposal: C++ names (more natural in .seq files)

3. **Backward compatibility:** Keep old constructors `Effect(ctx)` or deprecate?
   - Proposal: Keep for backward compatibility, use default params

4. **Parameter persistence:** Store params in SequenceItem for debugging?
   - Proposal: No (save size), params only in generated code

---

## Success Criteria

✅ FlashEffect supports configurable color (red, green, blue, etc.)
✅ ChromaAberrationEffect supports configurable strength
✅ .seq file syntax allows parameter specification
✅ No changes required to effects that don't use parameters
✅ Size impact < 1KB
✅ All existing tests pass
✅ New tests for parameter system pass

---

## Per-Frame Parameter Computation Patterns

### Pattern 1: Time-based animation
```cpp
void render(..., float time, ...) {
  float r = params_.base_color[0] * (0.5f + 0.5f * sinf(time));
  float g = params_.base_color[1] * (0.5f + 0.5f * cosf(time * 1.3f));
  float b = params_.base_color[2];
  // Use r, g, b in uniforms
}
```

### Pattern 2: Beat synchronization
```cpp
void render(..., float beat, ...) {
  float strength = params_.base_strength * (1.0f + 0.5f * beat);
  // Pulses with audio beat
}
```

### Pattern 3: Intensity modulation
```cpp
void render(..., float intensity, ...) {
  float alpha = params_.base_alpha * intensity;
  // Fades with audio intensity
}
```

### Pattern 4: Combined animation
```cpp
void render(..., float time, float beat, float intensity, ...) {
  float r = params_.color[0] * (0.8f + 0.2f * sinf(time));    // Oscillate
  float g = params_.color[1] * (1.0f + 0.3f * beat);          // Beat pulse
  float b = params_.color[2] * intensity;                     // Audio reactive
  // Complex, multi-factor animation
}
```

**Key principle:** `render()` is called every frame - compute parameters dynamically here, not in constructor.

---

## Next Steps

**Immediate:**
1. Review this document with team/user
2. Get approval on approach
3. Start Phase 1 implementation

**Future enhancements (post-MVP):**
- Parameter curve system (keyframe animation)
- Parameter validation in seq_compiler
- Visual parameter editor tool
- More effects with parameters (all post-process effects)