diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/COMPLETED.md | 19 | ||||
| -rw-r--r-- | doc/HANDOFF_2026-02-08.md | 359 | ||||
| -rw-r--r-- | doc/SHADER_PARAMETRIZATION_PLAN.md | 596 |
3 files changed, 974 insertions, 0 deletions
diff --git a/doc/COMPLETED.md b/doc/COMPLETED.md index bac7206..a4e6a94 100644 --- a/doc/COMPLETED.md +++ b/doc/COMPLETED.md @@ -4,6 +4,25 @@ This file tracks recently completed tasks, organized by completion date. ## Recently Completed (February 8, 2026) +- [x] **Shader Parametrization System (Task #73 Phase 0)** (February 8, 2026) + - **Goal**: Enable per-frame dynamic parameters for shaders and effects via uniform buffers and .seq syntax + - **Implementation**: + - **Phase 1**: Created `UniformHelper` template in `src/gpu/uniform_helper.h` for type-safe uniform buffer management + - **Phase 2**: Added `FlashEffectParams` struct (color[3], decay_rate, trigger_threshold) and `FlashUniforms` struct with proper WGSL alignment + - **Phase 3**: Updated `flash_effect.cc` WGSL shader to use parameterized `flash_color` instead of hardcoded white + - **Phase 4**: Implemented per-frame parameter computation in `render()` method - animates color based on time/beat using sinusoidal modulation + - **Phase 5**: Extended `seq_compiler.cc` to parse key=value parameters from .seq files and generate struct initialization code + - **Critical Bugfix**: Fixed WGSL alignment issue where FlashUniforms was 24 bytes in C++ but shader expected 32 bytes + - Root cause: `vec3<f32>` has 16-byte alignment despite 12-byte size + - Solution: Added explicit padding (`_pad1[2]`, `_pad2`) and static_assert to enforce 32-byte struct size + - Result: Eliminated "Buffer is bound with size 24 where the shader expects 32" validation error + - **Syntax**: `EFFECT + FlashEffect 0 1 color=1.0,0.5,0.5 decay=0.95` in demo.seq + - **Files**: 11 changed (808+ / 40-), 3 new (uniform_helper.h, test_uniform_helper.cc, SHADER_PARAMETRIZATION_PLAN.md) + - **Result**: All 32/32 tests passing, demo runs without validation errors, system ready for extension to other effects + - **Commits**: c7d1dd7 (feature implementation), 775c0ea (alignment fix) + - **Size Impact**: ~400-500 bytes net (UniformHelper overhead + per-effect params) + - **Next**: Task #73 - Extend to ChromaAberrationEffect, GaussianBlurEffect, DistortEffect, SolarizeEffect + - [x] **3D Rendering & Shadow Improvements (Task A)** (February 8, 2026) - [x] **Task A.1 (Shadow Investigation)**: Investigated mesh shadows appearing as bounding boxes. Documented that this is a design limitation of the hybrid renderer (AABB proxy for meshes in SDF pass). Created `doc/DEBUG_SHADOWS.md` with detailed analysis. - [x] **Task A.2 (Plane Scaling Fix)**: Fixed `ObjectType::PLANE` distance calculation for non-uniform scaling. diff --git a/doc/HANDOFF_2026-02-08.md b/doc/HANDOFF_2026-02-08.md new file mode 100644 index 0000000..f796f05 --- /dev/null +++ b/doc/HANDOFF_2026-02-08.md @@ -0,0 +1,359 @@ +# Handoff: Shader Parametrization System (February 8, 2026) + +## Summary +Completed comprehensive shader parametrization system enabling dynamic per-frame parameters for visual effects via uniform buffers and .seq file syntax. + +## Work Completed ✅ + +### Shader Parametrization System (Task #73 Phase 0) +**Goal**: Enable artists to configure visual effects with parameters (color, intensity, decay rates) via .seq files without touching C++ code. + +**Use Cases**: Flash effect with custom colors, chromatic aberration strength control, blur radius adjustment, distortion parameters. + +**Implementation**: + +#### Phase 1: UniformHelper Template +- Created `src/gpu/uniform_helper.h` - Generic type-safe wrapper for WebGPU uniform buffers +- Template class handles buffer creation, updates, and lifetime management +- Zero-overhead abstraction over `gpu_create_buffer()` and `wgpuQueueWriteBuffer()` + +```cpp +template <typename T> +class UniformBuffer { + void init(WGPUDevice device); + void update(WGPUQueue queue, const T& data); + GpuBuffer& get(); +}; +``` + +#### Phase 2: FlashEffect Parameter Structs +- Added `FlashEffectParams` (constructor-time base parameters): + - `color[3]` - RGB flash color (default: white) + - `decay_rate` - Flash fade rate per frame (default: 0.98) + - `trigger_threshold` - Intensity threshold to trigger flash (default: 0.7) + +- Added `FlashUniforms` (GPU buffer layout with WGSL alignment): + - `flash_intensity` (4 bytes, offset 0) + - `intensity` (4 bytes, offset 4) + - `_pad1[2]` (8 bytes, offset 8-15) - **Padding for vec3 alignment** + - `color[3]` (12 bytes, offset 16-27) - **Aligned to 16 bytes** + - `_pad2` (4 bytes, offset 28-31) + - Total: 32 bytes (enforced with `static_assert`) + +#### Phase 3: Parameterized WGSL Shader +- Updated `flash_effect.cc` shader to use `uniforms.flash_color` instead of hardcoded white +- Shader bindings: sampler (0), texture (1), uniforms (2) +- Fragment shader mixes input color with parameterized flash color based on flash intensity + +```wgsl +struct Uniforms { + flash_intensity: f32, + intensity: f32, + flash_color: vec3<f32>, // Parameterized color + _pad: f32, +}; + +fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> { + let color = textureSample(inputTexture, inputSampler, input.uv); + var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity); + return vec4<f32>(flashed, color.a); +} +``` + +#### Phase 4: Per-Frame Parameter Computation +- Implemented dynamic parameter animation in `FlashEffect::render()` method +- Parameters computed each frame based on time, beat, and intensity +- Example: Color modulated by sinusoidal functions of time and beat + +```cpp +// Animate color based on time and beat +const float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f)); +const float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f)); +const float b = params_.color[2] * (1.0f + 0.3f * beat); + +const FlashUniforms u = { + .flash_intensity = flash_intensity_, + .intensity = intensity, + ._pad1 = {0.0f, 0.0f}, + .color = {r, g, b}, + .pad2 = 0.0f +}; +uniforms_.update(ctx_.queue, u); +``` + +#### Phase 5: .seq Syntax Extension +- Extended `seq_compiler.cc` to parse `key=value` parameters after effect class name +- Added `parse_parameters()` function to extract parameter pairs +- Code generation creates `FlashEffectParams` struct initialization for parameterized effects +- Backward compatibility: Effects without parameters use default constructor + +**Syntax Example**: +``` +EFFECT + FlashEffect 0 1 color=1.0,0.5,0.5 decay=0.95 +``` + +**Generated Code**: +```cpp +{ + FlashEffectParams p; + p.color[0] = 1.0f; + p.color[1] = 0.5f; + p.color[2] = 0.5f; + p.decay_rate = 0.95f; + seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0); +} +``` + +--- + +### Critical Bugfix: WGSL Alignment +**Problem**: WebGPU validation error "Buffer is bound with size 24 where the shader expects 32" + +**Root Cause**: WGSL alignment rules require `vec3<f32>` to be 16-byte aligned despite having 12-byte size. Initial `FlashUniforms` struct layout was: +```cpp +struct FlashUniforms { + float flash_intensity; // 0-3 + float intensity; // 4-7 + float color[3]; // 8-19 (WRONG - no padding) + float _pad; // 20-23 +}; // Total: 24 bytes (WRONG) +``` + +**Solution**: Added explicit padding to align `color` array to 16-byte boundary: +```cpp +struct FlashUniforms { + float flash_intensity; // 0-3 + float intensity; // 4-7 + float _pad1[2]; // 8-15 (padding) + float color[3]; // 16-27 (aligned to 16 bytes) + float _pad2; // 28-31 +}; // Total: 32 bytes (CORRECT) +static_assert(sizeof(FlashUniforms) == 32, "FlashUniforms must be 32 bytes"); +``` + +**Impact**: Demo now runs without validation errors, all 32/32 tests pass. + +--- + +## Files Modified + +**New Files (3)**: +- `src/gpu/uniform_helper.h` - UniformBuffer template class +- `src/tests/test_uniform_helper.cc` - Unit test for template compilation +- `doc/SHADER_PARAMETRIZATION_PLAN.md` - Complete design document + +**Modified Files (11)**: +- `src/gpu/effects/flash_effect.h` - Added FlashEffectParams, FlashUniforms structs, parameterized constructor +- `src/gpu/effects/flash_effect.cc` - Implemented per-frame computation, updated shader with parameterized color +- `src/gpu/demo_effects.h` - Removed duplicate FlashEffect definition, added include +- `tools/seq_compiler.cc` - Added parameter parsing and code generation +- `assets/demo.seq` - Added example with parameters +- `src/generated/timeline.cc` - Generated code with struct initialization +- `CMakeLists.txt` - Added test_uniform_helper target +- `TODO.md` - Added Task #73 (extend to other effects), moved completion to summary +- `PROJECT_CONTEXT.md` - Updated "Recently Completed" section +- `doc/COMPLETED.md` - Added detailed completion entry +- `doc/HANDOFF_2026-02-08.md` - This document + +--- + +## Commits Made + +**c7d1dd7** - `feat(gpu): Implement shader parametrization system` +- Phases 1-5 implementation +- 11 files changed (808+ / 40-) +- UniformHelper template + FlashEffect parametrization + .seq syntax + +**775c0ea** - `fix(gpu): Correct FlashUniforms struct alignment for WGSL` +- Critical alignment bugfix (24 → 32 bytes) +- Added static_assert for future safety +- Fixed validation error + +--- + +## Test Results + +**All 32/32 tests passing (100%)** + +**Test Coverage**: +- `test_uniform_helper.cc` - Verifies UniformBuffer template compiles correctly +- `test_demo_effects.cc` - All post-process and scene effects instantiate successfully +- Full shader compilation tests pass with new parameterized shaders + +**Verification**: +- Demo runs without WebGPU validation errors +- Visual effects render with parameterized colors +- .seq file parsing generates correct parameter initialization code +- Per-frame animation produces time-dependent color modulation + +--- + +## Current Status + +**Completed:** +- ✅ Full shader parametrization infrastructure +- ✅ FlashEffect fully parameterized with color/decay control +- ✅ .seq syntax extension with key=value parsing +- ✅ Per-frame dynamic parameter computation +- ✅ Critical WGSL alignment bug fixed +- ✅ Documentation updated (TODO.md, PROJECT_CONTEXT.md, COMPLETED.md) + +**Ready for:** +- Task #73: Extend parametrization to other effects (ChromaAberrationEffect, GaussianBlurEffect, DistortEffect, SolarizeEffect) +- Additional parameter types (vec2, vec4, enums) +- Parameter curve system for keyframe animation (future) + +--- + +## Technical Notes + +### WGSL Alignment Rules +**Critical for future implementations:** +- `vec3<f32>` has **size = 12 bytes** but **alignment = 16 bytes** +- Always add padding before vec3 fields to align to 16-byte boundary +- Use `static_assert(sizeof(YourStruct) == expected_size)` to catch misalignment +- Reference: [WebGPU WGSL Spec - Alignment and Size](https://www.w3.org/TR/WGSL/#alignment-and-size) + +### UniformHelper Pattern +**Reusable template for any effect parameters:** +```cpp +// 1. Define parameter structs +struct YourEffectParams { /* constructor-time params */ }; +struct YourEffectUniforms { /* GPU buffer layout with padding */ }; +static_assert(sizeof(YourEffectUniforms) == expected_size); + +// 2. Add UniformBuffer member +UniformBuffer<YourEffectUniforms> uniforms_; + +// 3. Initialize in constructor +uniforms_.init(ctx_.device); + +// 4. Update in render() +const YourEffectUniforms u = { /* compute values */ }; +uniforms_.update(ctx_.queue, u); + +// 5. Bind in shader +pipeline_ = create_pipeline(...); +bind_group_ = create_bind_group(..., uniforms_.get().buffer, ...); +``` + +### Backward Compatibility +**All existing effects continue to work:** +- Default parameter constructor delegates to parameterized constructor +- Effects without .seq parameters use default values +- No changes required to existing timeline code + +--- + +## Design Document + +**Complete design**: See `doc/SHADER_PARAMETRIZATION_PLAN.md` + +**Key Sections**: +- Motivation and use cases +- 5-phase implementation plan +- WGSL alignment rules and solutions +- Per-frame vs constructor-time parameters +- .seq syntax specification +- Migration path for existing effects +- Size budget analysis (~400-500 bytes) +- Alternative approaches considered + +--- + +## Size Impact + +**Binary Size**: +400-500 bytes net +- UniformHelper template code: ~200 bytes +- FlashEffectParams/Uniforms structs: ~100 bytes +- Per-effect parameter overhead: ~50-100 bytes each +- .seq compiler parameter parsing: ~100 bytes + +**Trade-off**: Acceptable size increase for significant artist workflow improvement (no C++ recompilation for parameter tweaks). + +--- + +## Next Steps (Recommendations) + +### Immediate (Task #73) +Extend parametrization to other effects using FlashEffect as template: + +1. **ChromaAberrationEffect**: + - Parameters: `offset` (vec2), `strength` (float) + - Syntax: `EFFECT + ChromaAberrationEffect 0 5 offset=0.01,0.02 strength=1.5` + +2. **GaussianBlurEffect**: + - Parameters: `radius` (float), `sigma` (float) + - Syntax: `EFFECT + GaussianBlurEffect 2 8 radius=5.0 sigma=2.0` + +3. **DistortEffect** (if exists): + - Parameters: `amount` (float), `speed` (float), `scale` (vec2) + +4. **SolarizeEffect**: + - Parameters: `threshold` (float), `intensity` (float) + +### Future Enhancements +- Parameter curve system (keyframe animation via .seq timeline) +- vec4/mat4 parameter support (color with alpha, transformation matrices) +- Enum parameters (blend modes, filter types) +- Parameter validation and clamping (min/max ranges) + +--- + +## Context for Next Session + +### What Works +- Shader parametrization infrastructure fully functional +- FlashEffect demonstrates complete implementation pattern +- .seq syntax parsing robust with backward compatibility +- All tests passing, demo stable + +### Known Limitations +- Only FlashEffect parametrized (other effects use hardcoded values) +- No parameter validation or range clamping +- No keyframe animation support (future feature) + +### Migration Pattern for Other Effects +1. Copy `FlashEffectParams` / `FlashUniforms` pattern +2. Add `UniformBuffer<YourUniforms>` member +3. Update shader to use uniform values +4. Implement per-frame computation in `render()` +5. Extend `seq_compiler.cc` with effect-specific parameter parsing +6. Test with .seq file, verify alignment with static_assert + +--- + +## User Feedback Summary + +**Session Start**: User requested shader parametrization with use cases (flash colors, aberration strength) + +**Key Clarifications**: +- User confirmed parameters will be "generated in the code (manually, in effect's C++ code)" +- User emphasized "precision: user change parameter values on a per-frame basis. time-dependent" +- Critical feedback: Per-frame computation required, not just static constructor params + +**Critical Bug Report**: User provided log.txt showing GPU validation error (buffer size mismatch) + +**Outcome**: User satisfied, requested commit + documentation update (completed) + +--- + +## Handoff Checklist + +- [x] All tests passing (32/32) +- [x] Working tree clean +- [x] Documentation updated (TODO.md, PROJECT_CONTEXT.md, COMPLETED.md) +- [x] Commits created with detailed messages (c7d1dd7, 775c0ea) +- [x] No known regressions +- [x] Design document complete (SHADER_PARAMETRIZATION_PLAN.md) +- [x] Extension task noted (Task #73) +- [x] Ready for next session + +--- + +**handoff(Claude):** Shader parametrization system complete. FlashEffect fully parameterized with .seq syntax. Critical alignment bug fixed. 32/32 tests passing. Ready for extension to other effects (Task #73). + +--- + +*Generated: February 8, 2026* +*Claude Sonnet 4.5* diff --git a/doc/SHADER_PARAMETRIZATION_PLAN.md b/doc/SHADER_PARAMETRIZATION_PLAN.md new file mode 100644 index 0000000..f5afa27 --- /dev/null +++ b/doc/SHADER_PARAMETRIZATION_PLAN.md @@ -0,0 +1,596 @@ +# Shader Parametrization Plan + +## Problem Statement + +**Current limitations:** +1. Effect parameters are hardcoded in shader code (e.g., FlashEffect always uses white color) +2. No way to pass parameters from .seq file to effect constructors +3. Each effect reimplements uniform buffer management +4. Uniform structures are manually sized and written +5. Cannot easily vary effect behavior without creating new effect classes + +**Use cases:** +- FlashEffect with configurable color (red, blue, green) +- ChromaAberrationEffect with configurable strength multiplier +- GaussianBlurEffect with configurable kernel size +- Any effect with user-defined parameters + +--- + +## Current Architecture Analysis + +### Effect Construction (timeline.cc) +```cpp +// Generated by seq_compiler from assets/demo.seq +seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); +seq->add_effect(std::make_shared<ChromaAberrationEffect>(ctx), 0, 6, 2); +``` + +**Issue:** Only `GpuContext` passed to constructor, no custom parameters. + +### Effect Rendering (flash_effect.cc) +```cpp +void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) { + // Hardcoded uniform values + float uniforms[4] = {flash_intensity_, intensity, 0.0f, 0.0f}; + wgpuQueueWriteBuffer(ctx_.queue, uniforms_.buffer, 0, uniforms, sizeof(uniforms)); + // ... +} +``` + +**Issue:** Parameters hardcoded, uniform writing manual and repetitive. + +### Shader Parameters (flash_effect.cc:43-44) +```wgsl +let white = vec3<f32>(1.0, 1.0, 1.0); // Hardcoded color +let green = vec3<f32>(0.0, 1.0, 0.0); // Hardcoded alternative +var flashed = mix(color.rgb, green, uniforms.intensity); +``` + +**Issue:** Flash colors baked into shader code, not parameterizable. + +### Chromatic Aberration (chroma_aberration.wgsl:25) +```wgsl +let off = 0.02 * uniforms.intensity; // Hardcoded strength multiplier +``` + +**Issue:** Strength multiplier (0.02) is hardcoded, cannot be configured per instance. + +--- + +## Proposed Solution + +### Design Principles +1. **Constructor-time parameters:** Base/initial values (base colors, multipliers, modes) +2. **Per-frame computation:** Parameters computed in `render()` based on time/beat/intensity +3. **Uniform management:** Centralized helper to reduce boilerplate +4. **.seq file syntax:** Extend to support base parameter values +5. **Size-conscious:** Minimal overhead (this is a 64k demo) + +**IMPORTANT:** Parameters are **computed dynamically every frame**, not set once at construction. Constructor params provide base values, `render()` computes animated values. + +--- + +## Implementation Plan + +### Phase 1: Uniform Management Helper + +**Goal:** Reduce boilerplate for uniform buffer creation and updates. + +**New file:** `src/gpu/uniform_helper.h` + +```cpp +#pragma once +#include "gpu/gpu.h" +#include <cstring> + +// Generic uniform buffer helper +template <typename T> +class UniformBuffer { + public: + UniformBuffer() = default; + + void init(WGPUDevice device) { + buffer_ = gpu_create_buffer(device, sizeof(T), + WGPUBufferUsage_Uniform | WGPUBufferUsage_CopyDst); + } + + void update(WGPUQueue queue, const T& data) { + wgpuQueueWriteBuffer(queue, buffer_.buffer, 0, &data, sizeof(T)); + } + + GpuBuffer& get() { return buffer_; } + + private: + GpuBuffer buffer_; +}; +``` + +**Benefits:** +- Type-safe uniform updates +- Automatic size calculation +- Reduces manual `wgpuQueueWriteBuffer` calls + +**Size impact:** ~200 bytes (template instantiation minimal) + +--- + +### Phase 2: Effect Parameter Structs + +**Goal:** Define typed parameter structures for configurable effects. + +**Example:** `src/gpu/effects/flash_effect.h` + +```cpp +#pragma once +#include "gpu/effect.h" +#include "gpu/uniform_helper.h" + +struct FlashEffectParams { + float color[3] = {1.0f, 1.0f, 1.0f}; // Default: white + float decay_rate = 0.98f; // Default: fast decay + float trigger_threshold = 0.7f; // Default: trigger on strong beats +}; + +struct FlashUniforms { + float flash_intensity; + float intensity; + float color[3]; // Added: configurable color + float _pad; +}; + +class FlashEffect : public PostProcessEffect { + public: + FlashEffect(const GpuContext& ctx, const FlashEffectParams& params = {}); + void render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) override; + void update_bind_group(WGPUTextureView input_view) override; + + private: + FlashEffectParams params_; + UniformBuffer<FlashUniforms> uniforms_; + float flash_intensity_ = 0.0f; +}; +``` + +**Benefits:** +- Clear separation: constructor params vs runtime state +- Default values for backward compatibility +- Type safety + +**Size impact:** ~50 bytes per effect (struct storage) + +--- + +### Phase 3: Update Shaders to Use Parameters + +**Goal:** Make shaders accept parameterized values. + +**Example:** `assets/final/shaders/flash.wgsl` + +```wgsl +struct Uniforms { + flash_intensity: f32, + intensity: f32, + flash_color: vec3<f32>, // Added: parameterized color + _pad: f32, +}; + +@group(0) @binding(2) var<uniform> uniforms: Uniforms; + +@fragment +fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> { + let color = textureSample(inputTexture, inputSampler, input.uv); + + // Use uniform color instead of hardcoded white + var flashed = mix(color.rgb, uniforms.flash_color, uniforms.flash_intensity); + + return vec4<f32>(flashed, color.a); +} +``` + +**Benefits:** +- Flexible shader behavior without recompilation +- Single shader supports multiple color variants + +**Size impact:** +12 bytes per uniform struct (3 floats for color) + +--- + +### Phase 4: Update Effect Implementations + +**Goal:** Use new parameter system in effect constructors and render methods. + +**Example:** `src/gpu/effects/flash_effect.cc` + +```cpp +FlashEffect::FlashEffect(const GpuContext& ctx, const FlashEffectParams& params) + : PostProcessEffect(ctx), params_(params) { + + // Shader code (updated to use flash_color) + const char* shader_code = R"( + // ... (shader from Phase 3) + )"; + + pipeline_ = create_post_process_pipeline(ctx_.device, ctx_.format, shader_code); + uniforms_.init(ctx_.device); +} + +void FlashEffect::render(WGPURenderPassEncoder pass, float time, float beat, + float intensity, float aspect_ratio) { + (void)aspect_ratio; + + // Trigger flash based on configured threshold + if (intensity > params_.trigger_threshold && flash_intensity_ < 0.2f) { + flash_intensity_ = 0.8f; + } + + // Decay based on configured rate + flash_intensity_ *= params_.decay_rate; + + // *** PER-FRAME PARAMETER COMPUTATION *** + // Animate color based on time and beat + float r = params_.color[0] * (0.5f + 0.5f * sinf(time * 0.5f)); + float g = params_.color[1] * (0.5f + 0.5f * cosf(time * 0.7f)); + float b = params_.color[2] * (1.0f + 0.3f * beat); // Pulse with beat + + // Update uniforms with computed (animated) values + FlashUniforms u = { + .flash_intensity = flash_intensity_, + .intensity = intensity, + .color = {r, g, b}, // Time-dependent, computed every frame + ._pad = 0.0f + }; + uniforms_.update(ctx_.queue, u); + + wgpuRenderPassEncoderSetPipeline(pass, pipeline_); + wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 0, nullptr); + wgpuRenderPassEncoderDraw(pass, 3, 1, 0, 0); +} +``` + +**Benefits:** +- Clean separation of parameters and logic +- Easy to test different configurations +- Type-safe uniform updates + +--- + +### Phase 5: Extend .seq File Format + +**Goal:** Allow parameter passing in sequence files. + +**Proposed syntax:** `assets/demo.seq` + +``` +# Old syntax (still supported) +EFFECT + FlashEffect 0.0 1.0 + +# New syntax with parameters +EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 decay=0.95 threshold=0.6 + +# Or JSON-like syntax +EFFECT + FlashEffect 0.0 1.0 {color: [1.0, 0.0, 0.0], decay: 0.95} + +# Multiple instances with different colors +SEQUENCE 0 0 + EFFECT + FlashEffect 0.0 1.0 color=1.0,0.0,0.0 # Red flash + EFFECT + FlashEffect 2.0 3.0 color=0.0,1.0,0.0 # Green flash + EFFECT + FlashEffect 4.0 5.0 color=0.0,0.0,1.0 # Blue flash +``` + +**Benefits:** +- Artist-friendly configuration +- No code changes for parameter tuning +- Multiple instances with different settings + +**Size impact:** ~100 bytes (parser extension) + +--- + +### Phase 6: Update seq_compiler + +**Goal:** Parse effect parameters and generate constructor calls. + +**File:** `tools/seq_compiler.cc` + +**Generated code:** `src/generated/timeline.cc` + +```cpp +// Before (no parameters) +seq->add_effect(std::make_shared<FlashEffect>(ctx), 0.0f, 1.f, 0); + +// After (with parameters) +{ + FlashEffectParams p; + p.color[0] = 1.0f; p.color[1] = 0.0f; p.color[2] = 0.0f; // Red + p.decay_rate = 0.95f; + p.trigger_threshold = 0.6f; + seq->add_effect(std::make_shared<FlashEffect>(ctx, p), 0.0f, 1.f, 0); +} +``` + +**Benefits:** +- Compile-time parameter validation +- Type-safe parameter passing +- Zero runtime overhead (parameters baked into binary) + +--- + +## Additional Use Cases + +### Chromatic Aberration Strength + +**Params struct:** +```cpp +struct ChromaAberrationParams { + float strength_multiplier = 0.02f; // Default +}; +``` + +**Shader update:** +```wgsl +struct Uniforms { + time: f32, + intensity: f32, + strength_multiplier: f32, // Added + // ... +}; + +let off = uniforms.strength_multiplier * uniforms.intensity; // Parameterized +``` + +**Usage in .seq:** +``` +EFFECT + ChromaAberrationEffect 0 6 strength=0.05 # Stronger aberration +``` + +--- + +### Gaussian Blur Kernel Size + +**Params struct:** +```cpp +struct GaussianBlurParams { + int kernel_radius = 5; // Default: 5-pixel radius + float sigma = 2.0f; // Default: standard deviation +}; +``` + +**Shader update:** +```wgsl +struct Uniforms { + kernel_radius: i32, + sigma: f32, + // ... +}; + +// Use uniforms.kernel_radius in blur loop +for (var i = -uniforms.kernel_radius; i <= uniforms.kernel_radius; i++) { + // ... +} +``` + +--- + +## Testing Strategy + +### Unit Tests + +**File:** `src/tests/test_effect_params.cc` + +```cpp +void test_flash_effect_default_params() { + // Test default white color + FlashEffectParams params; + assert(params.color[0] == 1.0f && params.color[1] == 1.0f && params.color[2] == 1.0f); +} + +void test_flash_effect_custom_color() { + // Test red flash + FlashEffectParams params; + params.color[0] = 1.0f; + params.color[1] = 0.0f; + params.color[2] = 0.0f; + + GpuContext ctx = /* ... */; + FlashEffect effect(ctx, params); + // Verify effect uses red color +} +``` + +### Integration Tests + +**Verify:** +- .seq file parser handles parameters correctly +- Generated timeline.cc compiles +- Effects render with correct parameters +- Backward compatibility (effects without parameters still work) + +--- + +## Size Budget + +**Estimated size impact:** +- UniformHelper template: ~200 bytes +- Param structs (5 effects × 50 bytes): ~250 bytes +- seq_compiler parser: ~100 bytes +- Generated code overhead: ~50 bytes per parameterized effect +- **Total: ~600-800 bytes** + +**Savings:** +- Reduced boilerplate in effect implementations: ~300 bytes +- Single shader for multiple variants: ~1KB (avoid duplicate shaders) +- **Net impact: ~400 bytes or neutral** + +--- + +## Migration Path + +### Step 1: Add UniformHelper (backward compatible) +- No changes to existing effects +- Test with new helper + +### Step 2: Update FlashEffect (pilot) +- Add FlashEffectParams struct +- Update constructor to accept params (default values = current behavior) +- Update shader to use parameterized color +- Verify backward compatibility (no .seq changes yet) + +### Step 3: Extend .seq syntax +- Update seq_compiler to parse parameters +- Test with FlashEffect only +- Verify generated code compiles + +### Step 4: Migrate other effects +- ChromaAberrationEffect +- GaussianBlurEffect +- DistortEffect +- etc. + +### Step 5: Update demo.seq +- Add parameter specifications for desired effects +- Test visual output + +--- + +## Alternative Approaches Considered + +### Approach A: Virtual parameter interface +**Idea:** Add `virtual void set_param(const char* name, float value)` to Effect base class. + +**Pros:** Very flexible +**Cons:** +- String lookups at runtime (slow, size-costly) +- No type safety +- Poor compile-time validation + +**Verdict:** ❌ Rejected (size and performance concerns) + +--- + +### Approach B: Macro-based parameter system +**Idea:** Use preprocessor macros to generate param structs and parsers. + +**Pros:** Low code duplication +**Cons:** +- Hard to debug +- Poor IDE support +- Opaque error messages + +**Verdict:** ❌ Rejected (maintainability) + +--- + +### Approach C: Runtime parameter animation +**Idea:** Allow parameters to change over time via curves. + +**Pros:** Very flexible (animate flash color over time) +**Cons:** +- Significantly more complex +- Higher runtime cost +- Larger binary size + +**Verdict:** ⚠️ Future work (post-MVP) + +--- + +## Recommended Implementation Order + +1. ✅ **This document** - Design review and approval +2. **Phase 1** - UniformHelper template (~1 hour) +3. **Phase 2** - FlashEffectParams struct (~30 min) +4. **Phase 3** - Update flash.wgsl shader (~30 min) +5. **Phase 4** - Update FlashEffect implementation (~1 hour) +6. **Phase 5** - Extend .seq syntax (~1 hour) +7. **Phase 6** - Update seq_compiler (~2 hours) +8. **Testing** - Unit + integration tests (~1 hour) +9. **Migration** - Apply to 3-5 more effects (~3 hours) + +**Total effort:** ~10-12 hours + +--- + +## Open Questions + +1. **Parameter validation:** Should seq_compiler validate parameter ranges (e.g., color components in [0, 1])? + - Proposal: Yes, with warnings (not errors) for out-of-range values + +2. **Parameter naming:** Use C++ names (`color`) or shader names (`flash_color`)? + - Proposal: C++ names (more natural in .seq files) + +3. **Backward compatibility:** Keep old constructors `Effect(ctx)` or deprecate? + - Proposal: Keep for backward compatibility, use default params + +4. **Parameter persistence:** Store params in SequenceItem for debugging? + - Proposal: No (save size), params only in generated code + +--- + +## Success Criteria + +✅ FlashEffect supports configurable color (red, green, blue, etc.) +✅ ChromaAberrationEffect supports configurable strength +✅ .seq file syntax allows parameter specification +✅ No changes required to effects that don't use parameters +✅ Size impact < 1KB +✅ All existing tests pass +✅ New tests for parameter system pass + +--- + +## Per-Frame Parameter Computation Patterns + +### Pattern 1: Time-based animation +```cpp +void render(..., float time, ...) { + float r = params_.base_color[0] * (0.5f + 0.5f * sinf(time)); + float g = params_.base_color[1] * (0.5f + 0.5f * cosf(time * 1.3f)); + float b = params_.base_color[2]; + // Use r, g, b in uniforms +} +``` + +### Pattern 2: Beat synchronization +```cpp +void render(..., float beat, ...) { + float strength = params_.base_strength * (1.0f + 0.5f * beat); + // Pulses with audio beat +} +``` + +### Pattern 3: Intensity modulation +```cpp +void render(..., float intensity, ...) { + float alpha = params_.base_alpha * intensity; + // Fades with audio intensity +} +``` + +### Pattern 4: Combined animation +```cpp +void render(..., float time, float beat, float intensity, ...) { + float r = params_.color[0] * (0.8f + 0.2f * sinf(time)); // Oscillate + float g = params_.color[1] * (1.0f + 0.3f * beat); // Beat pulse + float b = params_.color[2] * intensity; // Audio reactive + // Complex, multi-factor animation +} +``` + +**Key principle:** `render()` is called every frame - compute parameters dynamically here, not in constructor. + +--- + +## Next Steps + +**Immediate:** +1. Review this document with team/user +2. Get approval on approach +3. Start Phase 1 implementation + +**Future enhancements (post-MVP):** +- Parameter curve system (keyframe animation) +- Parameter validation in seq_compiler +- Visual parameter editor tool +- More effects with parameters (all post-process effects) |