# GPU Effects Test Coverage Analysis ## Current State ### Coverage Status **GPU Directory Coverage: Very Low** (~20% estimated) ### File Breakdown (32 files total) #### Core Infrastructure (5 files) - `gpu.{h,cc}` - WebGPU initialization and management - `effect.{h,cc}` - Base Effect/Sequence/MainSequence classes - `texture_manager.{h,cc}` - Texture loading and procedural generation #### Effects System (27 files) **Base Effects:** - `demo_effects.{h,cc}` - Effect registration and factory **Individual Effects (19 implementations):** 1. `chroma_aberration_effect.cc` - Chromatic aberration post-process 2. `distort_effect.cc` - Distortion effect 3. `fade_effect.{h,cc}` - Fade in/out 4. `flash_cube_effect.{h,cc}` - Flashing cube 5. `flash_effect.{h,cc}` - Screen flash on beat 6. `gaussian_blur_effect.cc` - Blur post-process 7. `heptagon_effect.cc` - Heptagon shape 8. `hybrid_3d_effect.{h,cc}` - 3D + SDF hybrid rendering 9. `moving_ellipse_effect.cc` - Animated ellipse 10. `particle_spray_effect.cc` - Particle spray 11. `particles_effect.cc` - Particle system 12. `passthrough_effect.cc` - Identity post-process 13. `solarize_effect.cc` - Solarize color effect 14. `theme_modulation_effect.{h,cc}` - Theme color modulation **Utilities:** - `post_process_helper.{h,cc}` - Post-process pipeline utilities - `shader_composer.{h,cc}` - Shader composition system - `shaders.{h,cc}` - Shader registration ### Existing Tests (4 files) 1. ✅ `test_shader_assets.cc` - Basic shader keyword validation 2. ✅ `test_shader_compilation.cc` - Real WebGPU shader compilation 3. ✅ `test_shader_composer.cc` - Shader composition logic 4. ⚠️ `test_texture_manager.cc` - Minimal (just compilation check) ### Coverage Gaps **Completely Untested:** - ❌ `effect.cc` - Sequence/Effect lifecycle (0% coverage) - ❌ All 19 effect implementations (0% coverage) - ❌ `demo_effects.cc` - Effect factory (0% coverage) - ❌ `post_process_helper.cc` - Pipeline utilities (0% coverage) - ❌ `gpu.cc` - Initialization logic (partially tested via integration tests) --- ## Testing Challenges ### Challenge 1: WebGPU Initialization Overhead **Problem**: Every test needs a full WebGPU device/queue/surface setup. **Current Approach** (test_shader_compilation.cc): ```cpp // 100+ lines of initialization boilerplate WGPUInstance instance = wgpuCreateInstance(nullptr); WGPUAdapter adapter = ...; // Complex callback-based async request WGPUDevice device = ...; // Complex callback-based async request ``` **Issues**: - Heavy: ~150ms per test for GPU initialization - Unreliable: May fail in headless CI environments - Verbose: 100+ lines of boilerplate per test file ### Challenge 2: Rendering Requires a Surface **Problem**: Most tests need a real window for `render_frame()`. **Current Approach** (test_3d_render.cc, test_mesh.cc): ```cpp PlatformState state = platform_init(false, 640, 480); // Creates GLFW window WGPUSurface surface = platform_create_wgpu_surface(instance, &state); main_seq.render_frame(time, beat, peak, aspect, surface); ``` **Issues**: - GUI dependency: Can't run in headless CI - Manual testing: Requires human to verify visual output - No assertions: Can't validate correctness automatically ### Challenge 3: No Frame Validation **Problem**: How do you assert that a visual effect is correct? **Current Gap**: - No pixel readback infrastructure - No reference image comparison - No "golden master" test images --- ## Proposed Solution: Headless Rendering with Frame Sink ### Architecture Overview ``` ┌─────────────────────────────────────────────────────────┐ │ Test Framework │ ├─────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────┐ ┌──────────────────────────┐ │ │ │ MockSurface │─────▶│ OffscreenFramebuffer │ │ │ └──────────────┘ │ (WGPUTexture) │ │ │ │ └──────────────────────────┘ │ │ │ │ │ │ │ ▼ │ │ │ ┌──────────────────────────┐ │ │ └──────────────▶│ PixelReadback │ │ │ │ (wgpuBufferMapAsync) │ │ │ └──────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌──────────────────────────┐ │ │ │ Assertions │ │ │ │ (pixel checks, hashes) │ │ │ └──────────────────────────┘ │ └─────────────────────────────────────────────────────────┘ ``` ### Key Components #### 1. WebGPU Test Fixture (Reusable) ```cpp // src/tests/webgpu_test_fixture.h class WebGPUTestFixture { public: bool init(); // One-time setup void shutdown(); WGPUDevice device() const { return device_; } WGPUQueue queue() const { return queue_; } WGPUTextureFormat format() const { return WGPUTextureFormat_BGRA8Unorm; } private: WGPUInstance instance_ = nullptr; WGPUAdapter adapter_ = nullptr; WGPUDevice device_ = nullptr; WGPUQueue queue_ = nullptr; }; ``` **Benefits**: - Shared initialization across all GPU tests - ~100 lines of boilerplate → 3 lines per test - Can gracefully skip tests if GPU unavailable #### 2. Offscreen Render Target ```cpp // src/tests/offscreen_render_target.h class OffscreenRenderTarget { public: OffscreenRenderTarget(WGPUDevice device, int width, int height); ~OffscreenRenderTarget(); WGPUTexture texture() const { return texture_; } WGPUTextureView view() const { return view_; } // Simulate surface behavior without actual window WGPUSurfaceTexture get_current_texture(); // Readback pixels for validation std::vector read_pixels(); private: WGPUDevice device_; WGPUTexture texture_; WGPUTextureView view_; int width_, height_; }; ``` **Key Feature**: No window needed! Pure GPU texture operations. #### 3. Effect Test Helpers ```cpp // src/tests/effect_test_helpers.h // Helper: Test effect lifecycle void test_effect_lifecycle(Effect* effect, MainSequence* demo); // Helper: Test effect render (smoke test - no crash) void test_effect_render(Effect* effect, WGPURenderPassEncoder pass); // Helper: Test post-process bind group update void test_post_process_bind_group(PostProcessEffect* effect, WGPUTextureView input); // Helper: Validate rendered output (pixel checks) bool validate_pixels(const std::vector& pixels, int width, int height, std::function predicate); // Helper: Hash-based validation (deterministic output check) uint64_t hash_pixels(const std::vector& pixels); ``` --- ## Proposed Test Structure ### Test 1: Effect Base Classes (`test_effect_base.cc`) **Goal**: Test Effect/Sequence/MainSequence lifecycle ```cpp void test_effect_construction() { WebGPUTestFixture fixture; if (!fixture.init()) return; // Skip if no GPU FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); assert(!effect.is_initialized); MainSequence main_seq; main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); effect.init(&main_seq); assert(effect.is_initialized); } void test_sequence_add_effect() { WebGPUTestFixture fixture; if (!fixture.init()) return; auto effect = std::make_shared(/*...*/); Sequence seq; seq.add_effect(effect, 0.0f, 10.0f, 0); // Should not activate before start time seq.update_active_list(-1.0f); // Verify not active // Should activate at start time seq.update_active_list(0.0f); // Verify active // Should deactivate after end time seq.update_active_list(11.0f); // Verify not active } void test_sequence_priority_sorting() { // Add effects with different priorities // Verify render order via collect_active_effects() } ``` **Coverage Impact**: effect.cc 0% → ~60% --- ### Test 2: Post-Process Effects (`test_post_process_effects.cc`) **Goal**: Test all post-process effects (flash, blur, distort, etc.) ```cpp void test_flash_effect_basic() { WebGPUTestFixture fixture; if (!fixture.init()) return; FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); MainSequence main_seq; main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); effect.init(&main_seq); OffscreenRenderTarget target(fixture.device(), 256, 256); effect.update_bind_group(target.view()); // Create render pass WGPUCommandEncoder encoder = wgpuDeviceCreateCommandEncoder(fixture.device(), nullptr); WGPURenderPassDescriptor pass_desc = create_simple_pass(target.view()); WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc); // Render with low intensity (no flash) effect.render(pass, 0.0f, 0.0f, 0.1f, 1.0f); wgpuRenderPassEncoderEnd(pass); // Submit and readback WGPUCommandBuffer cmd = wgpuCommandEncoderFinish(encoder, nullptr); wgpuQueueSubmit(fixture.queue(), 1, &cmd); // Validate: should not be pure white (no flash triggered) auto pixels = target.read_pixels(); bool has_non_white = validate_pixels(pixels, 256, 256, [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { return r < 255 || g < 255 || b < 255; // At least one pixel not white }); assert(has_non_white); } void test_flash_effect_trigger() { // Test with high intensity (>0.7) - should trigger flash // Validate some pixels are white/bright } // Similar tests for: // - test_gaussian_blur_effect() // - test_chroma_aberration_effect() // - test_distort_effect() // - test_solarize_effect() // - test_passthrough_effect() ``` **Coverage Impact**: 6 post-process effects 0% → ~50% --- ### Test 3: Scene Effects (`test_scene_effects.cc`) **Goal**: Test scene-rendering effects (geometry, 3D, particles) ```cpp void test_heptagon_effect() { // Test basic render without crash // Optionally: readback and check for non-black pixels (shape rendered) } void test_moving_ellipse_effect() { // Test animation over time (different time values) } void test_particle_effects() { // Test ParticleSprayEffect // Test ParticlesEffect } void test_hybrid_3d_effect() { // Test 3D rendering with SDF raymarching // This is complex - may just be smoke test } ``` **Coverage Impact**: 8 scene effects 0% → ~40% --- ### Test 4: Effect Factory (`test_demo_effects.cc`) **Goal**: Test effect registration and instantiation ```cpp void test_effect_factory_registration() { WebGPUTestFixture fixture; if (!fixture.init()) return; // All effects should be auto-registered const auto& effects = GetEffectRegistry(); assert(effects.size() >= 15); // We have ~19 effects // Check specific effects exist assert(effects.find("FlashEffect") != effects.end()); assert(effects.find("FlashCubeEffect") != effects.end()); // etc. } void test_effect_factory_instantiation() { WebGPUTestFixture fixture; if (!fixture.init()) return; MainSequence main_seq; main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); // Create effect via factory auto effect = CreateEffect("FlashEffect", fixture.device(), fixture.queue(), fixture.format(), nullptr); assert(effect != nullptr); assert(dynamic_cast(effect.get()) != nullptr); // Init should not crash effect->init(&main_seq); } ``` **Coverage Impact**: demo_effects.cc 0% → ~70% --- ### Test 5: Integration Test (`test_mainsequence_render.cc`) **Goal**: Test full render pipeline with multiple effects ```cpp void test_mainsequence_full_render() { WebGPUTestFixture fixture; if (!fixture.init()) return; MainSequence main_seq; main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); // Add a sequence with multiple effects auto seq = std::make_shared(); seq->add_effect(std::make_shared(/*...*/), 0.0f, 10.0f, 0); seq->add_effect(std::make_shared(/*...*/), 0.0f, 10.0f, 1); main_seq.add_sequence(seq, 0.0f, 0); seq->init(&main_seq); // Create offscreen target OffscreenRenderTarget target(fixture.device(), 256, 256); // Render a frame // Note: Need to adapt render_frame() to accept texture instead of surface // OR create a mock surface that wraps the offscreen texture main_seq.render_frame_offscreen(0.5f, 0.0f, 0.5f, 1.0f, target.texture()); // Validate: frame rendered without crash, some pixels non-black auto pixels = target.read_pixels(); bool rendered = validate_pixels(pixels, 256, 256, [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { return r > 0 || g > 0 || b > 0; // At least one pixel not black }); assert(rendered); } ``` **Coverage Impact**: effect.cc MainSequence 0% → ~50% --- ## Implementation Plan ### Phase 1: Foundation (Week 1) - 8 hours 1. **Create WebGPUTestFixture** (2h) - Extract init code from test_shader_compilation.cc - Add graceful skip if GPU unavailable - Test on macOS, Linux 2. **Create OffscreenRenderTarget** (3h) - WGPUTexture creation without surface - Pixel readback via wgpuBufferMapAsync - Validation helpers (hash, predicate checks) 3. **Create Effect Test Helpers** (2h) - Lifecycle test helper - Render smoke test helper - Pixel validation utilities 4. **Add render_frame_offscreen() to MainSequence** (1h) - Variant of render_frame() that targets offscreen texture - Or: Mock surface that wraps offscreen texture --- ### Phase 2: Base Tests (Week 2) - 10 hours 1. **test_effect_base.cc** (4h) - Effect construction/init - Sequence lifecycle (add, activate, deactivate) - Priority sorting - Collect active effects 2. **test_demo_effects.cc** (2h) - Effect registry validation - Effect factory instantiation 3. **test_post_process_helper.cc** (2h) - Pipeline creation - Bind group management 4. **CI Integration** (2h) - Update CMakeLists.txt - Test graceful skip in headless CI - Document GPU test requirements --- ### Phase 3: Individual Effects (Week 3-4) - 20 hours 1. **test_post_process_effects.cc** (8h) - FlashEffect (with trigger logic) - GaussianBlurEffect - ChromaAberrationEffect - DistortEffect - SolarizeEffect - PassthroughEffect 2. **test_scene_effects.cc** (8h) - HeptagonEffect - MovingEllipseEffect - FlashCubeEffect - ParticleSprayEffect - ParticlesEffect - FadeEffect - ThemeModulationEffect 3. **test_hybrid_3d.cc** (4h) - Hybrid3DEffect (complex, may be smoke test only) --- ### Phase 4: Integration (Week 5) - 6 hours 1. **test_mainsequence_render.cc** (4h) - Full render pipeline - Multiple sequences - Effect priority ordering - Frame-by-frame simulation 2. **Coverage Report** (2h) - Generate coverage with new tests - Update documentation - Identify remaining gaps --- ## Expected Coverage Impact ### Before (Current State) ``` gpu/ ├── gpu.cc ~20% (partially via integration tests) ├── effect.cc 0% ❌ ├── texture_manager.cc 10% ⚠️ ├── demo_effects.cc 0% ❌ ├── effects/ │ ├── (19 effect files) 0% ❌ │ ├── post_process_helper.cc 0% ❌ │ ├── shader_composer.cc 80% ✅ (already tested) │ └── shaders.cc 5% ⚠️ Overall GPU Coverage: ~20% ``` ### After (With Proposed Tests) ``` gpu/ ├── gpu.cc ~30% (some init paths tested) ├── effect.cc ~60% ✅ (lifecycle, sequence logic) ├── texture_manager.cc ~40% (basic ops tested) ├── demo_effects.cc ~70% ✅ (factory tested) ├── effects/ │ ├── (19 effect files) ~45% ✅ (basic render tested) │ ├── post_process_helper.cc ~65% ✅ │ ├── shader_composer.cc 80% (unchanged) │ └── shaders.cc ~30% (registration tested) Overall GPU Coverage: ~50% ✅ (20% → 50% = +150% increase) ``` --- ## Alternative: Minimal Approach (If Time-Constrained) ### Quick Wins (2-3 hours total) 1. **test_effect_lifecycle.cc** (1.5h) - Just test Effect/Sequence construction, add_effect, activation - No rendering, just state checks - Gets effect.cc from 0% → 40% 2. **test_effect_factory.cc** (1h) - Test effect registry and instantiation - Gets demo_effects.cc from 0% → 60% 3. **Smoke Test for Each Effect** (30min) - Loop through all effects, construct + init - Verify no crashes - Gets effects/*.cc from 0% → 20% **Result**: GPU coverage 20% → 35% with minimal effort --- ## Recommendations ### Priority 1: Foundation (Do First) - ✅ Create WebGPUTestFixture (shared across all GPU tests) - ✅ Create OffscreenRenderTarget (enables headless testing) - ✅ test_effect_base.cc (high impact, tests core logic) ### Priority 2: Smoke Tests (Quick Wins) - ✅ test_demo_effects.cc (factory validation) - ✅ Loop through all effects, test construction ### Priority 3: Detailed Effect Tests (Long-Term) - ⚠️ Individual effect render validation (time-consuming) - ⚠️ Pixel-level assertions (requires golden masters) ### Priority 4: Integration Tests (Optional) - ⚠️ Full MainSequence render pipeline (complex setup) --- ## Open Questions 1. **Pixel Readback Performance**: Is `wgpuBufferMapAsync` fast enough for CI? - Typical latency: ~5-10ms per frame - For 50 tests × 1 frame each = ~500ms (acceptable) 2. **Headless CI Support**: Will WebGPU work without display server? - macOS: ✅ Works (Metal backend) - Linux: ⚠️ Requires Vulkan/X11/Wayland - Win32: ✅ Works (D3D12 backend) - Solution: Gracefully skip if init fails 3. **Golden Master Images**: Do we need reference images? - For now: ❌ No (too fragile, GPU differences) - Instead: Use coarse checks (non-black, brightness, hash) 4. **Mock vs Real GPU**: Should we mock WebGPU API? - Verdict: ❌ No mocking - use real GPU but offscreen - Mocking WebGPU is too complex and low-value --- ## Files to Create ### Test Infrastructure (3 files) ``` src/tests/webgpu_test_fixture.{h,cc} # Shared WebGPU init src/tests/offscreen_render_target.{h,cc} # Headless rendering src/tests/effect_test_helpers.h # Common test utilities ``` ### Test Suites (6 files) ``` src/tests/test_effect_base.cc # Effect/Sequence lifecycle src/tests/test_demo_effects.cc # Effect factory src/tests/test_post_process_helper.cc # Pipeline utilities src/tests/test_post_process_effects.cc # All post-process effects src/tests/test_scene_effects.cc # All scene effects src/tests/test_mainsequence_render.cc # Integration test ``` **Total**: 9 new files (~2000 lines) --- ## Conclusion **Recommended Approach**: Start with foundation + base tests (Phase 1-2) - Achieves GPU coverage: 20% → 40% with moderate effort - Provides infrastructure for future detailed tests - Low risk: graceful skip if GPU unavailable - No GUI dependency: fully headless **Stretch Goal**: Add individual effect tests (Phase 3) - Achieves GPU coverage: 40% → 50% - Higher effort, diminishing returns - Nice-to-have, not critical **Key Innovation**: Offscreen rendering eliminates need for windows/displays while still testing real GPU code paths.