summaryrefslogtreecommitdiff
path: root/doc/GPU_EFFECTS_TEST_ANALYSIS.md
diff options
context:
space:
mode:
Diffstat (limited to 'doc/GPU_EFFECTS_TEST_ANALYSIS.md')
-rw-r--r--doc/GPU_EFFECTS_TEST_ANALYSIS.md648
1 files changed, 648 insertions, 0 deletions
diff --git a/doc/GPU_EFFECTS_TEST_ANALYSIS.md b/doc/GPU_EFFECTS_TEST_ANALYSIS.md
new file mode 100644
index 0000000..f5b74ce
--- /dev/null
+++ b/doc/GPU_EFFECTS_TEST_ANALYSIS.md
@@ -0,0 +1,648 @@
+# GPU Effects Test Coverage Analysis
+
+## Current State
+
+### Coverage Status
+**GPU Directory Coverage: Very Low** (~20% estimated)
+
+### File Breakdown (32 files total)
+
+#### Core Infrastructure (5 files)
+- `gpu.{h,cc}` - WebGPU initialization and management
+- `effect.{h,cc}` - Base Effect/Sequence/MainSequence classes
+- `texture_manager.{h,cc}` - Texture loading and procedural generation
+
+#### Effects System (27 files)
+**Base Effects:**
+- `demo_effects.{h,cc}` - Effect registration and factory
+
+**Individual Effects (19 implementations):**
+1. `chroma_aberration_effect.cc` - Chromatic aberration post-process
+2. `distort_effect.cc` - Distortion effect
+3. `fade_effect.{h,cc}` - Fade in/out
+4. `flash_cube_effect.{h,cc}` - Flashing cube
+5. `flash_effect.{h,cc}` - Screen flash on beat
+6. `gaussian_blur_effect.cc` - Blur post-process
+7. `heptagon_effect.cc` - Heptagon shape
+8. `hybrid_3d_effect.{h,cc}` - 3D + SDF hybrid rendering
+9. `moving_ellipse_effect.cc` - Animated ellipse
+10. `particle_spray_effect.cc` - Particle spray
+11. `particles_effect.cc` - Particle system
+12. `passthrough_effect.cc` - Identity post-process
+13. `solarize_effect.cc` - Solarize color effect
+14. `theme_modulation_effect.{h,cc}` - Theme color modulation
+
+**Utilities:**
+- `post_process_helper.{h,cc}` - Post-process pipeline utilities
+- `shader_composer.{h,cc}` - Shader composition system
+- `shaders.{h,cc}` - Shader registration
+
+### Existing Tests (4 files)
+1. ✅ `test_shader_assets.cc` - Basic shader keyword validation
+2. ✅ `test_shader_compilation.cc` - Real WebGPU shader compilation
+3. ✅ `test_shader_composer.cc` - Shader composition logic
+4. ⚠️ `test_texture_manager.cc` - Minimal (just compilation check)
+
+### Coverage Gaps
+**Completely Untested:**
+- ❌ `effect.cc` - Sequence/Effect lifecycle (0% coverage)
+- ❌ All 19 effect implementations (0% coverage)
+- ❌ `demo_effects.cc` - Effect factory (0% coverage)
+- ❌ `post_process_helper.cc` - Pipeline utilities (0% coverage)
+- ❌ `gpu.cc` - Initialization logic (partially tested via integration tests)
+
+---
+
+## Testing Challenges
+
+### Challenge 1: WebGPU Initialization Overhead
+**Problem**: Every test needs a full WebGPU device/queue/surface setup.
+
+**Current Approach** (test_shader_compilation.cc):
+```cpp
+// 100+ lines of initialization boilerplate
+WGPUInstance instance = wgpuCreateInstance(nullptr);
+WGPUAdapter adapter = ...; // Complex callback-based async request
+WGPUDevice device = ...; // Complex callback-based async request
+```
+
+**Issues**:
+- Heavy: ~150ms per test for GPU initialization
+- Unreliable: May fail in headless CI environments
+- Verbose: 100+ lines of boilerplate per test file
+
+### Challenge 2: Rendering Requires a Surface
+**Problem**: Most tests need a real window for `render_frame()`.
+
+**Current Approach** (test_3d_render.cc, test_mesh.cc):
+```cpp
+PlatformState state = platform_init(false, 640, 480); // Creates GLFW window
+WGPUSurface surface = platform_create_wgpu_surface(instance, &state);
+main_seq.render_frame(time, beat, peak, aspect, surface);
+```
+
+**Issues**:
+- GUI dependency: Can't run in headless CI
+- Manual testing: Requires human to verify visual output
+- No assertions: Can't validate correctness automatically
+
+### Challenge 3: No Frame Validation
+**Problem**: How do you assert that a visual effect is correct?
+
+**Current Gap**:
+- No pixel readback infrastructure
+- No reference image comparison
+- No "golden master" test images
+
+---
+
+## Proposed Solution: Headless Rendering with Frame Sink
+
+### Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────┐
+│ Test Framework │
+├─────────────────────────────────────────────────────────┤
+│ │
+│ ┌──────────────┐ ┌──────────────────────────┐ │
+│ │ MockSurface │─────▶│ OffscreenFramebuffer │ │
+│ └──────────────┘ │ (WGPUTexture) │ │
+│ │ └──────────────────────────┘ │
+│ │ │ │
+│ │ ▼ │
+│ │ ┌──────────────────────────┐ │
+│ └──────────────▶│ PixelReadback │ │
+│ │ (wgpuBufferMapAsync) │ │
+│ └──────────────────────────┘ │
+│ │ │
+│ ▼ │
+│ ┌──────────────────────────┐ │
+│ │ Assertions │ │
+│ │ (pixel checks, hashes) │ │
+│ └──────────────────────────┘ │
+└─────────────────────────────────────────────────────────┘
+```
+
+### Key Components
+
+#### 1. WebGPU Test Fixture (Reusable)
+```cpp
+// src/tests/webgpu_test_fixture.h
+class WebGPUTestFixture {
+ public:
+ bool init(); // One-time setup
+ void shutdown();
+
+ WGPUDevice device() const { return device_; }
+ WGPUQueue queue() const { return queue_; }
+ WGPUTextureFormat format() const { return WGPUTextureFormat_BGRA8Unorm; }
+
+ private:
+ WGPUInstance instance_ = nullptr;
+ WGPUAdapter adapter_ = nullptr;
+ WGPUDevice device_ = nullptr;
+ WGPUQueue queue_ = nullptr;
+};
+```
+
+**Benefits**:
+- Shared initialization across all GPU tests
+- ~100 lines of boilerplate → 3 lines per test
+- Can gracefully skip tests if GPU unavailable
+
+#### 2. Offscreen Render Target
+```cpp
+// src/tests/offscreen_render_target.h
+class OffscreenRenderTarget {
+ public:
+ OffscreenRenderTarget(WGPUDevice device, int width, int height);
+ ~OffscreenRenderTarget();
+
+ WGPUTexture texture() const { return texture_; }
+ WGPUTextureView view() const { return view_; }
+
+ // Simulate surface behavior without actual window
+ WGPUSurfaceTexture get_current_texture();
+
+ // Readback pixels for validation
+ std::vector<uint8_t> read_pixels();
+
+ private:
+ WGPUDevice device_;
+ WGPUTexture texture_;
+ WGPUTextureView view_;
+ int width_, height_;
+};
+```
+
+**Key Feature**: No window needed! Pure GPU texture operations.
+
+#### 3. Effect Test Helpers
+```cpp
+// src/tests/effect_test_helpers.h
+
+// Helper: Test effect lifecycle
+void test_effect_lifecycle(Effect* effect, MainSequence* demo);
+
+// Helper: Test effect render (smoke test - no crash)
+void test_effect_render(Effect* effect, WGPURenderPassEncoder pass);
+
+// Helper: Test post-process bind group update
+void test_post_process_bind_group(PostProcessEffect* effect,
+ WGPUTextureView input);
+
+// Helper: Validate rendered output (pixel checks)
+bool validate_pixels(const std::vector<uint8_t>& pixels,
+ int width, int height,
+ std::function<bool(uint8_t r, uint8_t g, uint8_t b, uint8_t a)> predicate);
+
+// Helper: Hash-based validation (deterministic output check)
+uint64_t hash_pixels(const std::vector<uint8_t>& pixels);
+```
+
+---
+
+## Proposed Test Structure
+
+### Test 1: Effect Base Classes (`test_effect_base.cc`)
+
+**Goal**: Test Effect/Sequence/MainSequence lifecycle
+
+```cpp
+void test_effect_construction() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return; // Skip if no GPU
+
+ FlashEffect effect(fixture.device(), fixture.queue(), fixture.format());
+ assert(!effect.is_initialized);
+
+ MainSequence main_seq;
+ main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());
+
+ effect.init(&main_seq);
+ assert(effect.is_initialized);
+}
+
+void test_sequence_add_effect() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return;
+
+ auto effect = std::make_shared<FlashEffect>(/*...*/);
+
+ Sequence seq;
+ seq.add_effect(effect, 0.0f, 10.0f, 0);
+
+ // Should not activate before start time
+ seq.update_active_list(-1.0f);
+ // Verify not active
+
+ // Should activate at start time
+ seq.update_active_list(0.0f);
+ // Verify active
+
+ // Should deactivate after end time
+ seq.update_active_list(11.0f);
+ // Verify not active
+}
+
+void test_sequence_priority_sorting() {
+ // Add effects with different priorities
+ // Verify render order via collect_active_effects()
+}
+```
+
+**Coverage Impact**: effect.cc 0% → ~60%
+
+---
+
+### Test 2: Post-Process Effects (`test_post_process_effects.cc`)
+
+**Goal**: Test all post-process effects (flash, blur, distort, etc.)
+
+```cpp
+void test_flash_effect_basic() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return;
+
+ FlashEffect effect(fixture.device(), fixture.queue(), fixture.format());
+
+ MainSequence main_seq;
+ main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());
+ effect.init(&main_seq);
+
+ OffscreenRenderTarget target(fixture.device(), 256, 256);
+ effect.update_bind_group(target.view());
+
+ // Create render pass
+ WGPUCommandEncoder encoder = wgpuDeviceCreateCommandEncoder(fixture.device(), nullptr);
+ WGPURenderPassDescriptor pass_desc = create_simple_pass(target.view());
+ WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc);
+
+ // Render with low intensity (no flash)
+ effect.render(pass, 0.0f, 0.0f, 0.1f, 1.0f);
+ wgpuRenderPassEncoderEnd(pass);
+
+ // Submit and readback
+ WGPUCommandBuffer cmd = wgpuCommandEncoderFinish(encoder, nullptr);
+ wgpuQueueSubmit(fixture.queue(), 1, &cmd);
+
+ // Validate: should not be pure white (no flash triggered)
+ auto pixels = target.read_pixels();
+ bool has_non_white = validate_pixels(pixels, 256, 256,
+ [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
+ return r < 255 || g < 255 || b < 255; // At least one pixel not white
+ });
+ assert(has_non_white);
+}
+
+void test_flash_effect_trigger() {
+ // Test with high intensity (>0.7) - should trigger flash
+ // Validate some pixels are white/bright
+}
+
+// Similar tests for:
+// - test_gaussian_blur_effect()
+// - test_chroma_aberration_effect()
+// - test_distort_effect()
+// - test_solarize_effect()
+// - test_passthrough_effect()
+```
+
+**Coverage Impact**: 6 post-process effects 0% → ~50%
+
+---
+
+### Test 3: Scene Effects (`test_scene_effects.cc`)
+
+**Goal**: Test scene-rendering effects (geometry, 3D, particles)
+
+```cpp
+void test_heptagon_effect() {
+ // Test basic render without crash
+ // Optionally: readback and check for non-black pixels (shape rendered)
+}
+
+void test_moving_ellipse_effect() {
+ // Test animation over time (different time values)
+}
+
+void test_particle_effects() {
+ // Test ParticleSprayEffect
+ // Test ParticlesEffect
+}
+
+void test_hybrid_3d_effect() {
+ // Test 3D rendering with SDF raymarching
+ // This is complex - may just be smoke test
+}
+```
+
+**Coverage Impact**: 8 scene effects 0% → ~40%
+
+---
+
+### Test 4: Effect Factory (`test_demo_effects.cc`)
+
+**Goal**: Test effect registration and instantiation
+
+```cpp
+void test_effect_factory_registration() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return;
+
+ // All effects should be auto-registered
+ const auto& effects = GetEffectRegistry();
+ assert(effects.size() >= 15); // We have ~19 effects
+
+ // Check specific effects exist
+ assert(effects.find("FlashEffect") != effects.end());
+ assert(effects.find("FlashCubeEffect") != effects.end());
+ // etc.
+}
+
+void test_effect_factory_instantiation() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return;
+
+ MainSequence main_seq;
+ main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());
+
+ // Create effect via factory
+ auto effect = CreateEffect("FlashEffect", fixture.device(),
+ fixture.queue(), fixture.format(), nullptr);
+ assert(effect != nullptr);
+ assert(dynamic_cast<FlashEffect*>(effect.get()) != nullptr);
+
+ // Init should not crash
+ effect->init(&main_seq);
+}
+```
+
+**Coverage Impact**: demo_effects.cc 0% → ~70%
+
+---
+
+### Test 5: Integration Test (`test_mainsequence_render.cc`)
+
+**Goal**: Test full render pipeline with multiple effects
+
+```cpp
+void test_mainsequence_full_render() {
+ WebGPUTestFixture fixture;
+ if (!fixture.init()) return;
+
+ MainSequence main_seq;
+ main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());
+
+ // Add a sequence with multiple effects
+ auto seq = std::make_shared<Sequence>();
+ seq->add_effect(std::make_shared<FlashEffect>(/*...*/), 0.0f, 10.0f, 0);
+ seq->add_effect(std::make_shared<PassthroughEffect>(/*...*/), 0.0f, 10.0f, 1);
+
+ main_seq.add_sequence(seq, 0.0f, 0);
+ seq->init(&main_seq);
+
+ // Create offscreen target
+ OffscreenRenderTarget target(fixture.device(), 256, 256);
+
+ // Render a frame
+ // Note: Need to adapt render_frame() to accept texture instead of surface
+ // OR create a mock surface that wraps the offscreen texture
+
+ main_seq.render_frame_offscreen(0.5f, 0.0f, 0.5f, 1.0f, target.texture());
+
+ // Validate: frame rendered without crash, some pixels non-black
+ auto pixels = target.read_pixels();
+ bool rendered = validate_pixels(pixels, 256, 256,
+ [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
+ return r > 0 || g > 0 || b > 0; // At least one pixel not black
+ });
+ assert(rendered);
+}
+```
+
+**Coverage Impact**: effect.cc MainSequence 0% → ~50%
+
+---
+
+## Implementation Plan
+
+### Phase 1: Foundation (Week 1) - 8 hours
+1. **Create WebGPUTestFixture** (2h)
+ - Extract init code from test_shader_compilation.cc
+ - Add graceful skip if GPU unavailable
+ - Test on macOS, Linux
+
+2. **Create OffscreenRenderTarget** (3h)
+ - WGPUTexture creation without surface
+ - Pixel readback via wgpuBufferMapAsync
+ - Validation helpers (hash, predicate checks)
+
+3. **Create Effect Test Helpers** (2h)
+ - Lifecycle test helper
+ - Render smoke test helper
+ - Pixel validation utilities
+
+4. **Add render_frame_offscreen() to MainSequence** (1h)
+ - Variant of render_frame() that targets offscreen texture
+ - Or: Mock surface that wraps offscreen texture
+
+---
+
+### Phase 2: Base Tests (Week 2) - 10 hours
+1. **test_effect_base.cc** (4h)
+ - Effect construction/init
+ - Sequence lifecycle (add, activate, deactivate)
+ - Priority sorting
+ - Collect active effects
+
+2. **test_demo_effects.cc** (2h)
+ - Effect registry validation
+ - Effect factory instantiation
+
+3. **test_post_process_helper.cc** (2h)
+ - Pipeline creation
+ - Bind group management
+
+4. **CI Integration** (2h)
+ - Update CMakeLists.txt
+ - Test graceful skip in headless CI
+ - Document GPU test requirements
+
+---
+
+### Phase 3: Individual Effects (Week 3-4) - 20 hours
+1. **test_post_process_effects.cc** (8h)
+ - FlashEffect (with trigger logic)
+ - GaussianBlurEffect
+ - ChromaAberrationEffect
+ - DistortEffect
+ - SolarizeEffect
+ - PassthroughEffect
+
+2. **test_scene_effects.cc** (8h)
+ - HeptagonEffect
+ - MovingEllipseEffect
+ - FlashCubeEffect
+ - ParticleSprayEffect
+ - ParticlesEffect
+ - FadeEffect
+ - ThemeModulationEffect
+
+3. **test_hybrid_3d.cc** (4h)
+ - Hybrid3DEffect (complex, may be smoke test only)
+
+---
+
+### Phase 4: Integration (Week 5) - 6 hours
+1. **test_mainsequence_render.cc** (4h)
+ - Full render pipeline
+ - Multiple sequences
+ - Effect priority ordering
+ - Frame-by-frame simulation
+
+2. **Coverage Report** (2h)
+ - Generate coverage with new tests
+ - Update documentation
+ - Identify remaining gaps
+
+---
+
+## Expected Coverage Impact
+
+### Before (Current State)
+```
+gpu/
+├── gpu.cc ~20% (partially via integration tests)
+├── effect.cc 0% ❌
+├── texture_manager.cc 10% ⚠️
+├── demo_effects.cc 0% ❌
+├── effects/
+│ ├── (19 effect files) 0% ❌
+│ ├── post_process_helper.cc 0% ❌
+│ ├── shader_composer.cc 80% ✅ (already tested)
+│ └── shaders.cc 5% ⚠️
+
+Overall GPU Coverage: ~20%
+```
+
+### After (With Proposed Tests)
+```
+gpu/
+├── gpu.cc ~30% (some init paths tested)
+├── effect.cc ~60% ✅ (lifecycle, sequence logic)
+├── texture_manager.cc ~40% (basic ops tested)
+├── demo_effects.cc ~70% ✅ (factory tested)
+├── effects/
+│ ├── (19 effect files) ~45% ✅ (basic render tested)
+│ ├── post_process_helper.cc ~65% ✅
+│ ├── shader_composer.cc 80% (unchanged)
+│ └── shaders.cc ~30% (registration tested)
+
+Overall GPU Coverage: ~50% ✅ (20% → 50% = +150% increase)
+```
+
+---
+
+## Alternative: Minimal Approach (If Time-Constrained)
+
+### Quick Wins (2-3 hours total)
+
+1. **test_effect_lifecycle.cc** (1.5h)
+ - Just test Effect/Sequence construction, add_effect, activation
+ - No rendering, just state checks
+ - Gets effect.cc from 0% → 40%
+
+2. **test_effect_factory.cc** (1h)
+ - Test effect registry and instantiation
+ - Gets demo_effects.cc from 0% → 60%
+
+3. **Smoke Test for Each Effect** (30min)
+ - Loop through all effects, construct + init
+ - Verify no crashes
+ - Gets effects/*.cc from 0% → 20%
+
+**Result**: GPU coverage 20% → 35% with minimal effort
+
+---
+
+## Recommendations
+
+### Priority 1: Foundation (Do First)
+- ✅ Create WebGPUTestFixture (shared across all GPU tests)
+- ✅ Create OffscreenRenderTarget (enables headless testing)
+- ✅ test_effect_base.cc (high impact, tests core logic)
+
+### Priority 2: Smoke Tests (Quick Wins)
+- ✅ test_demo_effects.cc (factory validation)
+- ✅ Loop through all effects, test construction
+
+### Priority 3: Detailed Effect Tests (Long-Term)
+- ⚠️ Individual effect render validation (time-consuming)
+- ⚠️ Pixel-level assertions (requires golden masters)
+
+### Priority 4: Integration Tests (Optional)
+- ⚠️ Full MainSequence render pipeline (complex setup)
+
+---
+
+## Open Questions
+
+1. **Pixel Readback Performance**: Is `wgpuBufferMapAsync` fast enough for CI?
+ - Typical latency: ~5-10ms per frame
+ - For 50 tests × 1 frame each = ~500ms (acceptable)
+
+2. **Headless CI Support**: Will WebGPU work without display server?
+ - macOS: ✅ Works (Metal backend)
+ - Linux: ⚠️ Requires Vulkan/X11/Wayland
+ - Win32: ✅ Works (D3D12 backend)
+ - Solution: Gracefully skip if init fails
+
+3. **Golden Master Images**: Do we need reference images?
+ - For now: ❌ No (too fragile, GPU differences)
+ - Instead: Use coarse checks (non-black, brightness, hash)
+
+4. **Mock vs Real GPU**: Should we mock WebGPU API?
+ - Verdict: ❌ No mocking - use real GPU but offscreen
+ - Mocking WebGPU is too complex and low-value
+
+---
+
+## Files to Create
+
+### Test Infrastructure (3 files)
+```
+src/tests/webgpu_test_fixture.{h,cc} # Shared WebGPU init
+src/tests/offscreen_render_target.{h,cc} # Headless rendering
+src/tests/effect_test_helpers.h # Common test utilities
+```
+
+### Test Suites (6 files)
+```
+src/tests/test_effect_base.cc # Effect/Sequence lifecycle
+src/tests/test_demo_effects.cc # Effect factory
+src/tests/test_post_process_helper.cc # Pipeline utilities
+src/tests/test_post_process_effects.cc # All post-process effects
+src/tests/test_scene_effects.cc # All scene effects
+src/tests/test_mainsequence_render.cc # Integration test
+```
+
+**Total**: 9 new files (~2000 lines)
+
+---
+
+## Conclusion
+
+**Recommended Approach**: Start with foundation + base tests (Phase 1-2)
+- Achieves GPU coverage: 20% → 40% with moderate effort
+- Provides infrastructure for future detailed tests
+- Low risk: graceful skip if GPU unavailable
+- No GUI dependency: fully headless
+
+**Stretch Goal**: Add individual effect tests (Phase 3)
+- Achieves GPU coverage: 40% → 50%
+- Higher effort, diminishing returns
+- Nice-to-have, not critical
+
+**Key Innovation**: Offscreen rendering eliminates need for windows/displays while still testing real GPU code paths.