diff options
Diffstat (limited to 'GPU_EFFECTS_TEST_ANALYSIS.md')
| -rw-r--r-- | GPU_EFFECTS_TEST_ANALYSIS.md | 648 |
1 files changed, 648 insertions, 0 deletions
diff --git a/GPU_EFFECTS_TEST_ANALYSIS.md b/GPU_EFFECTS_TEST_ANALYSIS.md new file mode 100644 index 0000000..f5b74ce --- /dev/null +++ b/GPU_EFFECTS_TEST_ANALYSIS.md @@ -0,0 +1,648 @@ +# GPU Effects Test Coverage Analysis + +## Current State + +### Coverage Status +**GPU Directory Coverage: Very Low** (~20% estimated) + +### File Breakdown (32 files total) + +#### Core Infrastructure (5 files) +- `gpu.{h,cc}` - WebGPU initialization and management +- `effect.{h,cc}` - Base Effect/Sequence/MainSequence classes +- `texture_manager.{h,cc}` - Texture loading and procedural generation + +#### Effects System (27 files) +**Base Effects:** +- `demo_effects.{h,cc}` - Effect registration and factory + +**Individual Effects (19 implementations):** +1. `chroma_aberration_effect.cc` - Chromatic aberration post-process +2. `distort_effect.cc` - Distortion effect +3. `fade_effect.{h,cc}` - Fade in/out +4. `flash_cube_effect.{h,cc}` - Flashing cube +5. `flash_effect.{h,cc}` - Screen flash on beat +6. `gaussian_blur_effect.cc` - Blur post-process +7. `heptagon_effect.cc` - Heptagon shape +8. `hybrid_3d_effect.{h,cc}` - 3D + SDF hybrid rendering +9. `moving_ellipse_effect.cc` - Animated ellipse +10. `particle_spray_effect.cc` - Particle spray +11. `particles_effect.cc` - Particle system +12. `passthrough_effect.cc` - Identity post-process +13. `solarize_effect.cc` - Solarize color effect +14. `theme_modulation_effect.{h,cc}` - Theme color modulation + +**Utilities:** +- `post_process_helper.{h,cc}` - Post-process pipeline utilities +- `shader_composer.{h,cc}` - Shader composition system +- `shaders.{h,cc}` - Shader registration + +### Existing Tests (4 files) +1. ✅ `test_shader_assets.cc` - Basic shader keyword validation +2. ✅ `test_shader_compilation.cc` - Real WebGPU shader compilation +3. ✅ `test_shader_composer.cc` - Shader composition logic +4. ⚠️ `test_texture_manager.cc` - Minimal (just compilation check) + +### Coverage Gaps +**Completely Untested:** +- ❌ `effect.cc` - Sequence/Effect lifecycle (0% coverage) +- ❌ All 19 effect implementations (0% coverage) +- ❌ `demo_effects.cc` - Effect factory (0% coverage) +- ❌ `post_process_helper.cc` - Pipeline utilities (0% coverage) +- ❌ `gpu.cc` - Initialization logic (partially tested via integration tests) + +--- + +## Testing Challenges + +### Challenge 1: WebGPU Initialization Overhead +**Problem**: Every test needs a full WebGPU device/queue/surface setup. + +**Current Approach** (test_shader_compilation.cc): +```cpp +// 100+ lines of initialization boilerplate +WGPUInstance instance = wgpuCreateInstance(nullptr); +WGPUAdapter adapter = ...; // Complex callback-based async request +WGPUDevice device = ...; // Complex callback-based async request +``` + +**Issues**: +- Heavy: ~150ms per test for GPU initialization +- Unreliable: May fail in headless CI environments +- Verbose: 100+ lines of boilerplate per test file + +### Challenge 2: Rendering Requires a Surface +**Problem**: Most tests need a real window for `render_frame()`. + +**Current Approach** (test_3d_render.cc, test_mesh.cc): +```cpp +PlatformState state = platform_init(false, 640, 480); // Creates GLFW window +WGPUSurface surface = platform_create_wgpu_surface(instance, &state); +main_seq.render_frame(time, beat, peak, aspect, surface); +``` + +**Issues**: +- GUI dependency: Can't run in headless CI +- Manual testing: Requires human to verify visual output +- No assertions: Can't validate correctness automatically + +### Challenge 3: No Frame Validation +**Problem**: How do you assert that a visual effect is correct? + +**Current Gap**: +- No pixel readback infrastructure +- No reference image comparison +- No "golden master" test images + +--- + +## Proposed Solution: Headless Rendering with Frame Sink + +### Architecture Overview + +``` +┌─────────────────────────────────────────────────────────┐ +│ Test Framework │ +├─────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────┐ ┌──────────────────────────┐ │ +│ │ MockSurface │─────▶│ OffscreenFramebuffer │ │ +│ └──────────────┘ │ (WGPUTexture) │ │ +│ │ └──────────────────────────┘ │ +│ │ │ │ +│ │ ▼ │ +│ │ ┌──────────────────────────┐ │ +│ └──────────────▶│ PixelReadback │ │ +│ │ (wgpuBufferMapAsync) │ │ +│ └──────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌──────────────────────────┐ │ +│ │ Assertions │ │ +│ │ (pixel checks, hashes) │ │ +│ └──────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +### Key Components + +#### 1. WebGPU Test Fixture (Reusable) +```cpp +// src/tests/webgpu_test_fixture.h +class WebGPUTestFixture { + public: + bool init(); // One-time setup + void shutdown(); + + WGPUDevice device() const { return device_; } + WGPUQueue queue() const { return queue_; } + WGPUTextureFormat format() const { return WGPUTextureFormat_BGRA8Unorm; } + + private: + WGPUInstance instance_ = nullptr; + WGPUAdapter adapter_ = nullptr; + WGPUDevice device_ = nullptr; + WGPUQueue queue_ = nullptr; +}; +``` + +**Benefits**: +- Shared initialization across all GPU tests +- ~100 lines of boilerplate → 3 lines per test +- Can gracefully skip tests if GPU unavailable + +#### 2. Offscreen Render Target +```cpp +// src/tests/offscreen_render_target.h +class OffscreenRenderTarget { + public: + OffscreenRenderTarget(WGPUDevice device, int width, int height); + ~OffscreenRenderTarget(); + + WGPUTexture texture() const { return texture_; } + WGPUTextureView view() const { return view_; } + + // Simulate surface behavior without actual window + WGPUSurfaceTexture get_current_texture(); + + // Readback pixels for validation + std::vector<uint8_t> read_pixels(); + + private: + WGPUDevice device_; + WGPUTexture texture_; + WGPUTextureView view_; + int width_, height_; +}; +``` + +**Key Feature**: No window needed! Pure GPU texture operations. + +#### 3. Effect Test Helpers +```cpp +// src/tests/effect_test_helpers.h + +// Helper: Test effect lifecycle +void test_effect_lifecycle(Effect* effect, MainSequence* demo); + +// Helper: Test effect render (smoke test - no crash) +void test_effect_render(Effect* effect, WGPURenderPassEncoder pass); + +// Helper: Test post-process bind group update +void test_post_process_bind_group(PostProcessEffect* effect, + WGPUTextureView input); + +// Helper: Validate rendered output (pixel checks) +bool validate_pixels(const std::vector<uint8_t>& pixels, + int width, int height, + std::function<bool(uint8_t r, uint8_t g, uint8_t b, uint8_t a)> predicate); + +// Helper: Hash-based validation (deterministic output check) +uint64_t hash_pixels(const std::vector<uint8_t>& pixels); +``` + +--- + +## Proposed Test Structure + +### Test 1: Effect Base Classes (`test_effect_base.cc`) + +**Goal**: Test Effect/Sequence/MainSequence lifecycle + +```cpp +void test_effect_construction() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; // Skip if no GPU + + FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); + assert(!effect.is_initialized); + + MainSequence main_seq; + main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); + + effect.init(&main_seq); + assert(effect.is_initialized); +} + +void test_sequence_add_effect() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; + + auto effect = std::make_shared<FlashEffect>(/*...*/); + + Sequence seq; + seq.add_effect(effect, 0.0f, 10.0f, 0); + + // Should not activate before start time + seq.update_active_list(-1.0f); + // Verify not active + + // Should activate at start time + seq.update_active_list(0.0f); + // Verify active + + // Should deactivate after end time + seq.update_active_list(11.0f); + // Verify not active +} + +void test_sequence_priority_sorting() { + // Add effects with different priorities + // Verify render order via collect_active_effects() +} +``` + +**Coverage Impact**: effect.cc 0% → ~60% + +--- + +### Test 2: Post-Process Effects (`test_post_process_effects.cc`) + +**Goal**: Test all post-process effects (flash, blur, distort, etc.) + +```cpp +void test_flash_effect_basic() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; + + FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); + + MainSequence main_seq; + main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); + effect.init(&main_seq); + + OffscreenRenderTarget target(fixture.device(), 256, 256); + effect.update_bind_group(target.view()); + + // Create render pass + WGPUCommandEncoder encoder = wgpuDeviceCreateCommandEncoder(fixture.device(), nullptr); + WGPURenderPassDescriptor pass_desc = create_simple_pass(target.view()); + WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc); + + // Render with low intensity (no flash) + effect.render(pass, 0.0f, 0.0f, 0.1f, 1.0f); + wgpuRenderPassEncoderEnd(pass); + + // Submit and readback + WGPUCommandBuffer cmd = wgpuCommandEncoderFinish(encoder, nullptr); + wgpuQueueSubmit(fixture.queue(), 1, &cmd); + + // Validate: should not be pure white (no flash triggered) + auto pixels = target.read_pixels(); + bool has_non_white = validate_pixels(pixels, 256, 256, + [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { + return r < 255 || g < 255 || b < 255; // At least one pixel not white + }); + assert(has_non_white); +} + +void test_flash_effect_trigger() { + // Test with high intensity (>0.7) - should trigger flash + // Validate some pixels are white/bright +} + +// Similar tests for: +// - test_gaussian_blur_effect() +// - test_chroma_aberration_effect() +// - test_distort_effect() +// - test_solarize_effect() +// - test_passthrough_effect() +``` + +**Coverage Impact**: 6 post-process effects 0% → ~50% + +--- + +### Test 3: Scene Effects (`test_scene_effects.cc`) + +**Goal**: Test scene-rendering effects (geometry, 3D, particles) + +```cpp +void test_heptagon_effect() { + // Test basic render without crash + // Optionally: readback and check for non-black pixels (shape rendered) +} + +void test_moving_ellipse_effect() { + // Test animation over time (different time values) +} + +void test_particle_effects() { + // Test ParticleSprayEffect + // Test ParticlesEffect +} + +void test_hybrid_3d_effect() { + // Test 3D rendering with SDF raymarching + // This is complex - may just be smoke test +} +``` + +**Coverage Impact**: 8 scene effects 0% → ~40% + +--- + +### Test 4: Effect Factory (`test_demo_effects.cc`) + +**Goal**: Test effect registration and instantiation + +```cpp +void test_effect_factory_registration() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; + + // All effects should be auto-registered + const auto& effects = GetEffectRegistry(); + assert(effects.size() >= 15); // We have ~19 effects + + // Check specific effects exist + assert(effects.find("FlashEffect") != effects.end()); + assert(effects.find("FlashCubeEffect") != effects.end()); + // etc. +} + +void test_effect_factory_instantiation() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; + + MainSequence main_seq; + main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); + + // Create effect via factory + auto effect = CreateEffect("FlashEffect", fixture.device(), + fixture.queue(), fixture.format(), nullptr); + assert(effect != nullptr); + assert(dynamic_cast<FlashEffect*>(effect.get()) != nullptr); + + // Init should not crash + effect->init(&main_seq); +} +``` + +**Coverage Impact**: demo_effects.cc 0% → ~70% + +--- + +### Test 5: Integration Test (`test_mainsequence_render.cc`) + +**Goal**: Test full render pipeline with multiple effects + +```cpp +void test_mainsequence_full_render() { + WebGPUTestFixture fixture; + if (!fixture.init()) return; + + MainSequence main_seq; + main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); + + // Add a sequence with multiple effects + auto seq = std::make_shared<Sequence>(); + seq->add_effect(std::make_shared<FlashEffect>(/*...*/), 0.0f, 10.0f, 0); + seq->add_effect(std::make_shared<PassthroughEffect>(/*...*/), 0.0f, 10.0f, 1); + + main_seq.add_sequence(seq, 0.0f, 0); + seq->init(&main_seq); + + // Create offscreen target + OffscreenRenderTarget target(fixture.device(), 256, 256); + + // Render a frame + // Note: Need to adapt render_frame() to accept texture instead of surface + // OR create a mock surface that wraps the offscreen texture + + main_seq.render_frame_offscreen(0.5f, 0.0f, 0.5f, 1.0f, target.texture()); + + // Validate: frame rendered without crash, some pixels non-black + auto pixels = target.read_pixels(); + bool rendered = validate_pixels(pixels, 256, 256, + [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { + return r > 0 || g > 0 || b > 0; // At least one pixel not black + }); + assert(rendered); +} +``` + +**Coverage Impact**: effect.cc MainSequence 0% → ~50% + +--- + +## Implementation Plan + +### Phase 1: Foundation (Week 1) - 8 hours +1. **Create WebGPUTestFixture** (2h) + - Extract init code from test_shader_compilation.cc + - Add graceful skip if GPU unavailable + - Test on macOS, Linux + +2. **Create OffscreenRenderTarget** (3h) + - WGPUTexture creation without surface + - Pixel readback via wgpuBufferMapAsync + - Validation helpers (hash, predicate checks) + +3. **Create Effect Test Helpers** (2h) + - Lifecycle test helper + - Render smoke test helper + - Pixel validation utilities + +4. **Add render_frame_offscreen() to MainSequence** (1h) + - Variant of render_frame() that targets offscreen texture + - Or: Mock surface that wraps offscreen texture + +--- + +### Phase 2: Base Tests (Week 2) - 10 hours +1. **test_effect_base.cc** (4h) + - Effect construction/init + - Sequence lifecycle (add, activate, deactivate) + - Priority sorting + - Collect active effects + +2. **test_demo_effects.cc** (2h) + - Effect registry validation + - Effect factory instantiation + +3. **test_post_process_helper.cc** (2h) + - Pipeline creation + - Bind group management + +4. **CI Integration** (2h) + - Update CMakeLists.txt + - Test graceful skip in headless CI + - Document GPU test requirements + +--- + +### Phase 3: Individual Effects (Week 3-4) - 20 hours +1. **test_post_process_effects.cc** (8h) + - FlashEffect (with trigger logic) + - GaussianBlurEffect + - ChromaAberrationEffect + - DistortEffect + - SolarizeEffect + - PassthroughEffect + +2. **test_scene_effects.cc** (8h) + - HeptagonEffect + - MovingEllipseEffect + - FlashCubeEffect + - ParticleSprayEffect + - ParticlesEffect + - FadeEffect + - ThemeModulationEffect + +3. **test_hybrid_3d.cc** (4h) + - Hybrid3DEffect (complex, may be smoke test only) + +--- + +### Phase 4: Integration (Week 5) - 6 hours +1. **test_mainsequence_render.cc** (4h) + - Full render pipeline + - Multiple sequences + - Effect priority ordering + - Frame-by-frame simulation + +2. **Coverage Report** (2h) + - Generate coverage with new tests + - Update documentation + - Identify remaining gaps + +--- + +## Expected Coverage Impact + +### Before (Current State) +``` +gpu/ +├── gpu.cc ~20% (partially via integration tests) +├── effect.cc 0% ❌ +├── texture_manager.cc 10% ⚠️ +├── demo_effects.cc 0% ❌ +├── effects/ +│ ├── (19 effect files) 0% ❌ +│ ├── post_process_helper.cc 0% ❌ +│ ├── shader_composer.cc 80% ✅ (already tested) +│ └── shaders.cc 5% ⚠️ + +Overall GPU Coverage: ~20% +``` + +### After (With Proposed Tests) +``` +gpu/ +├── gpu.cc ~30% (some init paths tested) +├── effect.cc ~60% ✅ (lifecycle, sequence logic) +├── texture_manager.cc ~40% (basic ops tested) +├── demo_effects.cc ~70% ✅ (factory tested) +├── effects/ +│ ├── (19 effect files) ~45% ✅ (basic render tested) +│ ├── post_process_helper.cc ~65% ✅ +│ ├── shader_composer.cc 80% (unchanged) +│ └── shaders.cc ~30% (registration tested) + +Overall GPU Coverage: ~50% ✅ (20% → 50% = +150% increase) +``` + +--- + +## Alternative: Minimal Approach (If Time-Constrained) + +### Quick Wins (2-3 hours total) + +1. **test_effect_lifecycle.cc** (1.5h) + - Just test Effect/Sequence construction, add_effect, activation + - No rendering, just state checks + - Gets effect.cc from 0% → 40% + +2. **test_effect_factory.cc** (1h) + - Test effect registry and instantiation + - Gets demo_effects.cc from 0% → 60% + +3. **Smoke Test for Each Effect** (30min) + - Loop through all effects, construct + init + - Verify no crashes + - Gets effects/*.cc from 0% → 20% + +**Result**: GPU coverage 20% → 35% with minimal effort + +--- + +## Recommendations + +### Priority 1: Foundation (Do First) +- ✅ Create WebGPUTestFixture (shared across all GPU tests) +- ✅ Create OffscreenRenderTarget (enables headless testing) +- ✅ test_effect_base.cc (high impact, tests core logic) + +### Priority 2: Smoke Tests (Quick Wins) +- ✅ test_demo_effects.cc (factory validation) +- ✅ Loop through all effects, test construction + +### Priority 3: Detailed Effect Tests (Long-Term) +- ⚠️ Individual effect render validation (time-consuming) +- ⚠️ Pixel-level assertions (requires golden masters) + +### Priority 4: Integration Tests (Optional) +- ⚠️ Full MainSequence render pipeline (complex setup) + +--- + +## Open Questions + +1. **Pixel Readback Performance**: Is `wgpuBufferMapAsync` fast enough for CI? + - Typical latency: ~5-10ms per frame + - For 50 tests × 1 frame each = ~500ms (acceptable) + +2. **Headless CI Support**: Will WebGPU work without display server? + - macOS: ✅ Works (Metal backend) + - Linux: ⚠️ Requires Vulkan/X11/Wayland + - Win32: ✅ Works (D3D12 backend) + - Solution: Gracefully skip if init fails + +3. **Golden Master Images**: Do we need reference images? + - For now: ❌ No (too fragile, GPU differences) + - Instead: Use coarse checks (non-black, brightness, hash) + +4. **Mock vs Real GPU**: Should we mock WebGPU API? + - Verdict: ❌ No mocking - use real GPU but offscreen + - Mocking WebGPU is too complex and low-value + +--- + +## Files to Create + +### Test Infrastructure (3 files) +``` +src/tests/webgpu_test_fixture.{h,cc} # Shared WebGPU init +src/tests/offscreen_render_target.{h,cc} # Headless rendering +src/tests/effect_test_helpers.h # Common test utilities +``` + +### Test Suites (6 files) +``` +src/tests/test_effect_base.cc # Effect/Sequence lifecycle +src/tests/test_demo_effects.cc # Effect factory +src/tests/test_post_process_helper.cc # Pipeline utilities +src/tests/test_post_process_effects.cc # All post-process effects +src/tests/test_scene_effects.cc # All scene effects +src/tests/test_mainsequence_render.cc # Integration test +``` + +**Total**: 9 new files (~2000 lines) + +--- + +## Conclusion + +**Recommended Approach**: Start with foundation + base tests (Phase 1-2) +- Achieves GPU coverage: 20% → 40% with moderate effort +- Provides infrastructure for future detailed tests +- Low risk: graceful skip if GPU unavailable +- No GUI dependency: fully headless + +**Stretch Goal**: Add individual effect tests (Phase 3) +- Achieves GPU coverage: 40% → 50% +- Higher effort, diminishing returns +- Nice-to-have, not critical + +**Key Innovation**: Offscreen rendering eliminates need for windows/displays while still testing real GPU code paths. |