diff options
Diffstat (limited to 'GPU_EFFECTS_TEST_ANALYSIS.md')
| -rw-r--r-- | GPU_EFFECTS_TEST_ANALYSIS.md | 648 |
1 files changed, 0 insertions, 648 deletions
diff --git a/GPU_EFFECTS_TEST_ANALYSIS.md b/GPU_EFFECTS_TEST_ANALYSIS.md deleted file mode 100644 index f5b74ce..0000000 --- a/GPU_EFFECTS_TEST_ANALYSIS.md +++ /dev/null @@ -1,648 +0,0 @@ -# GPU Effects Test Coverage Analysis - -## Current State - -### Coverage Status -**GPU Directory Coverage: Very Low** (~20% estimated) - -### File Breakdown (32 files total) - -#### Core Infrastructure (5 files) -- `gpu.{h,cc}` - WebGPU initialization and management -- `effect.{h,cc}` - Base Effect/Sequence/MainSequence classes -- `texture_manager.{h,cc}` - Texture loading and procedural generation - -#### Effects System (27 files) -**Base Effects:** -- `demo_effects.{h,cc}` - Effect registration and factory - -**Individual Effects (19 implementations):** -1. `chroma_aberration_effect.cc` - Chromatic aberration post-process -2. `distort_effect.cc` - Distortion effect -3. `fade_effect.{h,cc}` - Fade in/out -4. `flash_cube_effect.{h,cc}` - Flashing cube -5. `flash_effect.{h,cc}` - Screen flash on beat -6. `gaussian_blur_effect.cc` - Blur post-process -7. `heptagon_effect.cc` - Heptagon shape -8. `hybrid_3d_effect.{h,cc}` - 3D + SDF hybrid rendering -9. `moving_ellipse_effect.cc` - Animated ellipse -10. `particle_spray_effect.cc` - Particle spray -11. `particles_effect.cc` - Particle system -12. `passthrough_effect.cc` - Identity post-process -13. `solarize_effect.cc` - Solarize color effect -14. `theme_modulation_effect.{h,cc}` - Theme color modulation - -**Utilities:** -- `post_process_helper.{h,cc}` - Post-process pipeline utilities -- `shader_composer.{h,cc}` - Shader composition system -- `shaders.{h,cc}` - Shader registration - -### Existing Tests (4 files) -1. ✅ `test_shader_assets.cc` - Basic shader keyword validation -2. ✅ `test_shader_compilation.cc` - Real WebGPU shader compilation -3. ✅ `test_shader_composer.cc` - Shader composition logic -4. ⚠️ `test_texture_manager.cc` - Minimal (just compilation check) - -### Coverage Gaps -**Completely Untested:** -- ❌ `effect.cc` - Sequence/Effect lifecycle (0% coverage) -- ❌ All 19 effect implementations (0% coverage) -- ❌ `demo_effects.cc` - Effect factory (0% coverage) -- ❌ `post_process_helper.cc` - Pipeline utilities (0% coverage) -- ❌ `gpu.cc` - Initialization logic (partially tested via integration tests) - ---- - -## Testing Challenges - -### Challenge 1: WebGPU Initialization Overhead -**Problem**: Every test needs a full WebGPU device/queue/surface setup. - -**Current Approach** (test_shader_compilation.cc): -```cpp -// 100+ lines of initialization boilerplate -WGPUInstance instance = wgpuCreateInstance(nullptr); -WGPUAdapter adapter = ...; // Complex callback-based async request -WGPUDevice device = ...; // Complex callback-based async request -``` - -**Issues**: -- Heavy: ~150ms per test for GPU initialization -- Unreliable: May fail in headless CI environments -- Verbose: 100+ lines of boilerplate per test file - -### Challenge 2: Rendering Requires a Surface -**Problem**: Most tests need a real window for `render_frame()`. - -**Current Approach** (test_3d_render.cc, test_mesh.cc): -```cpp -PlatformState state = platform_init(false, 640, 480); // Creates GLFW window -WGPUSurface surface = platform_create_wgpu_surface(instance, &state); -main_seq.render_frame(time, beat, peak, aspect, surface); -``` - -**Issues**: -- GUI dependency: Can't run in headless CI -- Manual testing: Requires human to verify visual output -- No assertions: Can't validate correctness automatically - -### Challenge 3: No Frame Validation -**Problem**: How do you assert that a visual effect is correct? - -**Current Gap**: -- No pixel readback infrastructure -- No reference image comparison -- No "golden master" test images - ---- - -## Proposed Solution: Headless Rendering with Frame Sink - -### Architecture Overview - -``` -┌─────────────────────────────────────────────────────────┐ -│ Test Framework │ -├─────────────────────────────────────────────────────────┤ -│ │ -│ ┌──────────────┐ ┌──────────────────────────┐ │ -│ │ MockSurface │─────▶│ OffscreenFramebuffer │ │ -│ └──────────────┘ │ (WGPUTexture) │ │ -│ │ └──────────────────────────┘ │ -│ │ │ │ -│ │ ▼ │ -│ │ ┌──────────────────────────┐ │ -│ └──────────────▶│ PixelReadback │ │ -│ │ (wgpuBufferMapAsync) │ │ -│ └──────────────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌──────────────────────────┐ │ -│ │ Assertions │ │ -│ │ (pixel checks, hashes) │ │ -│ └──────────────────────────┘ │ -└─────────────────────────────────────────────────────────┘ -``` - -### Key Components - -#### 1. WebGPU Test Fixture (Reusable) -```cpp -// src/tests/webgpu_test_fixture.h -class WebGPUTestFixture { - public: - bool init(); // One-time setup - void shutdown(); - - WGPUDevice device() const { return device_; } - WGPUQueue queue() const { return queue_; } - WGPUTextureFormat format() const { return WGPUTextureFormat_BGRA8Unorm; } - - private: - WGPUInstance instance_ = nullptr; - WGPUAdapter adapter_ = nullptr; - WGPUDevice device_ = nullptr; - WGPUQueue queue_ = nullptr; -}; -``` - -**Benefits**: -- Shared initialization across all GPU tests -- ~100 lines of boilerplate → 3 lines per test -- Can gracefully skip tests if GPU unavailable - -#### 2. Offscreen Render Target -```cpp -// src/tests/offscreen_render_target.h -class OffscreenRenderTarget { - public: - OffscreenRenderTarget(WGPUDevice device, int width, int height); - ~OffscreenRenderTarget(); - - WGPUTexture texture() const { return texture_; } - WGPUTextureView view() const { return view_; } - - // Simulate surface behavior without actual window - WGPUSurfaceTexture get_current_texture(); - - // Readback pixels for validation - std::vector<uint8_t> read_pixels(); - - private: - WGPUDevice device_; - WGPUTexture texture_; - WGPUTextureView view_; - int width_, height_; -}; -``` - -**Key Feature**: No window needed! Pure GPU texture operations. - -#### 3. Effect Test Helpers -```cpp -// src/tests/effect_test_helpers.h - -// Helper: Test effect lifecycle -void test_effect_lifecycle(Effect* effect, MainSequence* demo); - -// Helper: Test effect render (smoke test - no crash) -void test_effect_render(Effect* effect, WGPURenderPassEncoder pass); - -// Helper: Test post-process bind group update -void test_post_process_bind_group(PostProcessEffect* effect, - WGPUTextureView input); - -// Helper: Validate rendered output (pixel checks) -bool validate_pixels(const std::vector<uint8_t>& pixels, - int width, int height, - std::function<bool(uint8_t r, uint8_t g, uint8_t b, uint8_t a)> predicate); - -// Helper: Hash-based validation (deterministic output check) -uint64_t hash_pixels(const std::vector<uint8_t>& pixels); -``` - ---- - -## Proposed Test Structure - -### Test 1: Effect Base Classes (`test_effect_base.cc`) - -**Goal**: Test Effect/Sequence/MainSequence lifecycle - -```cpp -void test_effect_construction() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; // Skip if no GPU - - FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); - assert(!effect.is_initialized); - - MainSequence main_seq; - main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); - - effect.init(&main_seq); - assert(effect.is_initialized); -} - -void test_sequence_add_effect() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; - - auto effect = std::make_shared<FlashEffect>(/*...*/); - - Sequence seq; - seq.add_effect(effect, 0.0f, 10.0f, 0); - - // Should not activate before start time - seq.update_active_list(-1.0f); - // Verify not active - - // Should activate at start time - seq.update_active_list(0.0f); - // Verify active - - // Should deactivate after end time - seq.update_active_list(11.0f); - // Verify not active -} - -void test_sequence_priority_sorting() { - // Add effects with different priorities - // Verify render order via collect_active_effects() -} -``` - -**Coverage Impact**: effect.cc 0% → ~60% - ---- - -### Test 2: Post-Process Effects (`test_post_process_effects.cc`) - -**Goal**: Test all post-process effects (flash, blur, distort, etc.) - -```cpp -void test_flash_effect_basic() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; - - FlashEffect effect(fixture.device(), fixture.queue(), fixture.format()); - - MainSequence main_seq; - main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); - effect.init(&main_seq); - - OffscreenRenderTarget target(fixture.device(), 256, 256); - effect.update_bind_group(target.view()); - - // Create render pass - WGPUCommandEncoder encoder = wgpuDeviceCreateCommandEncoder(fixture.device(), nullptr); - WGPURenderPassDescriptor pass_desc = create_simple_pass(target.view()); - WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc); - - // Render with low intensity (no flash) - effect.render(pass, 0.0f, 0.0f, 0.1f, 1.0f); - wgpuRenderPassEncoderEnd(pass); - - // Submit and readback - WGPUCommandBuffer cmd = wgpuCommandEncoderFinish(encoder, nullptr); - wgpuQueueSubmit(fixture.queue(), 1, &cmd); - - // Validate: should not be pure white (no flash triggered) - auto pixels = target.read_pixels(); - bool has_non_white = validate_pixels(pixels, 256, 256, - [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { - return r < 255 || g < 255 || b < 255; // At least one pixel not white - }); - assert(has_non_white); -} - -void test_flash_effect_trigger() { - // Test with high intensity (>0.7) - should trigger flash - // Validate some pixels are white/bright -} - -// Similar tests for: -// - test_gaussian_blur_effect() -// - test_chroma_aberration_effect() -// - test_distort_effect() -// - test_solarize_effect() -// - test_passthrough_effect() -``` - -**Coverage Impact**: 6 post-process effects 0% → ~50% - ---- - -### Test 3: Scene Effects (`test_scene_effects.cc`) - -**Goal**: Test scene-rendering effects (geometry, 3D, particles) - -```cpp -void test_heptagon_effect() { - // Test basic render without crash - // Optionally: readback and check for non-black pixels (shape rendered) -} - -void test_moving_ellipse_effect() { - // Test animation over time (different time values) -} - -void test_particle_effects() { - // Test ParticleSprayEffect - // Test ParticlesEffect -} - -void test_hybrid_3d_effect() { - // Test 3D rendering with SDF raymarching - // This is complex - may just be smoke test -} -``` - -**Coverage Impact**: 8 scene effects 0% → ~40% - ---- - -### Test 4: Effect Factory (`test_demo_effects.cc`) - -**Goal**: Test effect registration and instantiation - -```cpp -void test_effect_factory_registration() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; - - // All effects should be auto-registered - const auto& effects = GetEffectRegistry(); - assert(effects.size() >= 15); // We have ~19 effects - - // Check specific effects exist - assert(effects.find("FlashEffect") != effects.end()); - assert(effects.find("FlashCubeEffect") != effects.end()); - // etc. -} - -void test_effect_factory_instantiation() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; - - MainSequence main_seq; - main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); - - // Create effect via factory - auto effect = CreateEffect("FlashEffect", fixture.device(), - fixture.queue(), fixture.format(), nullptr); - assert(effect != nullptr); - assert(dynamic_cast<FlashEffect*>(effect.get()) != nullptr); - - // Init should not crash - effect->init(&main_seq); -} -``` - -**Coverage Impact**: demo_effects.cc 0% → ~70% - ---- - -### Test 5: Integration Test (`test_mainsequence_render.cc`) - -**Goal**: Test full render pipeline with multiple effects - -```cpp -void test_mainsequence_full_render() { - WebGPUTestFixture fixture; - if (!fixture.init()) return; - - MainSequence main_seq; - main_seq.init_test(fixture.device(), fixture.queue(), fixture.format()); - - // Add a sequence with multiple effects - auto seq = std::make_shared<Sequence>(); - seq->add_effect(std::make_shared<FlashEffect>(/*...*/), 0.0f, 10.0f, 0); - seq->add_effect(std::make_shared<PassthroughEffect>(/*...*/), 0.0f, 10.0f, 1); - - main_seq.add_sequence(seq, 0.0f, 0); - seq->init(&main_seq); - - // Create offscreen target - OffscreenRenderTarget target(fixture.device(), 256, 256); - - // Render a frame - // Note: Need to adapt render_frame() to accept texture instead of surface - // OR create a mock surface that wraps the offscreen texture - - main_seq.render_frame_offscreen(0.5f, 0.0f, 0.5f, 1.0f, target.texture()); - - // Validate: frame rendered without crash, some pixels non-black - auto pixels = target.read_pixels(); - bool rendered = validate_pixels(pixels, 256, 256, - [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) { - return r > 0 || g > 0 || b > 0; // At least one pixel not black - }); - assert(rendered); -} -``` - -**Coverage Impact**: effect.cc MainSequence 0% → ~50% - ---- - -## Implementation Plan - -### Phase 1: Foundation (Week 1) - 8 hours -1. **Create WebGPUTestFixture** (2h) - - Extract init code from test_shader_compilation.cc - - Add graceful skip if GPU unavailable - - Test on macOS, Linux - -2. **Create OffscreenRenderTarget** (3h) - - WGPUTexture creation without surface - - Pixel readback via wgpuBufferMapAsync - - Validation helpers (hash, predicate checks) - -3. **Create Effect Test Helpers** (2h) - - Lifecycle test helper - - Render smoke test helper - - Pixel validation utilities - -4. **Add render_frame_offscreen() to MainSequence** (1h) - - Variant of render_frame() that targets offscreen texture - - Or: Mock surface that wraps offscreen texture - ---- - -### Phase 2: Base Tests (Week 2) - 10 hours -1. **test_effect_base.cc** (4h) - - Effect construction/init - - Sequence lifecycle (add, activate, deactivate) - - Priority sorting - - Collect active effects - -2. **test_demo_effects.cc** (2h) - - Effect registry validation - - Effect factory instantiation - -3. **test_post_process_helper.cc** (2h) - - Pipeline creation - - Bind group management - -4. **CI Integration** (2h) - - Update CMakeLists.txt - - Test graceful skip in headless CI - - Document GPU test requirements - ---- - -### Phase 3: Individual Effects (Week 3-4) - 20 hours -1. **test_post_process_effects.cc** (8h) - - FlashEffect (with trigger logic) - - GaussianBlurEffect - - ChromaAberrationEffect - - DistortEffect - - SolarizeEffect - - PassthroughEffect - -2. **test_scene_effects.cc** (8h) - - HeptagonEffect - - MovingEllipseEffect - - FlashCubeEffect - - ParticleSprayEffect - - ParticlesEffect - - FadeEffect - - ThemeModulationEffect - -3. **test_hybrid_3d.cc** (4h) - - Hybrid3DEffect (complex, may be smoke test only) - ---- - -### Phase 4: Integration (Week 5) - 6 hours -1. **test_mainsequence_render.cc** (4h) - - Full render pipeline - - Multiple sequences - - Effect priority ordering - - Frame-by-frame simulation - -2. **Coverage Report** (2h) - - Generate coverage with new tests - - Update documentation - - Identify remaining gaps - ---- - -## Expected Coverage Impact - -### Before (Current State) -``` -gpu/ -├── gpu.cc ~20% (partially via integration tests) -├── effect.cc 0% ❌ -├── texture_manager.cc 10% ⚠️ -├── demo_effects.cc 0% ❌ -├── effects/ -│ ├── (19 effect files) 0% ❌ -│ ├── post_process_helper.cc 0% ❌ -│ ├── shader_composer.cc 80% ✅ (already tested) -│ └── shaders.cc 5% ⚠️ - -Overall GPU Coverage: ~20% -``` - -### After (With Proposed Tests) -``` -gpu/ -├── gpu.cc ~30% (some init paths tested) -├── effect.cc ~60% ✅ (lifecycle, sequence logic) -├── texture_manager.cc ~40% (basic ops tested) -├── demo_effects.cc ~70% ✅ (factory tested) -├── effects/ -│ ├── (19 effect files) ~45% ✅ (basic render tested) -│ ├── post_process_helper.cc ~65% ✅ -│ ├── shader_composer.cc 80% (unchanged) -│ └── shaders.cc ~30% (registration tested) - -Overall GPU Coverage: ~50% ✅ (20% → 50% = +150% increase) -``` - ---- - -## Alternative: Minimal Approach (If Time-Constrained) - -### Quick Wins (2-3 hours total) - -1. **test_effect_lifecycle.cc** (1.5h) - - Just test Effect/Sequence construction, add_effect, activation - - No rendering, just state checks - - Gets effect.cc from 0% → 40% - -2. **test_effect_factory.cc** (1h) - - Test effect registry and instantiation - - Gets demo_effects.cc from 0% → 60% - -3. **Smoke Test for Each Effect** (30min) - - Loop through all effects, construct + init - - Verify no crashes - - Gets effects/*.cc from 0% → 20% - -**Result**: GPU coverage 20% → 35% with minimal effort - ---- - -## Recommendations - -### Priority 1: Foundation (Do First) -- ✅ Create WebGPUTestFixture (shared across all GPU tests) -- ✅ Create OffscreenRenderTarget (enables headless testing) -- ✅ test_effect_base.cc (high impact, tests core logic) - -### Priority 2: Smoke Tests (Quick Wins) -- ✅ test_demo_effects.cc (factory validation) -- ✅ Loop through all effects, test construction - -### Priority 3: Detailed Effect Tests (Long-Term) -- ⚠️ Individual effect render validation (time-consuming) -- ⚠️ Pixel-level assertions (requires golden masters) - -### Priority 4: Integration Tests (Optional) -- ⚠️ Full MainSequence render pipeline (complex setup) - ---- - -## Open Questions - -1. **Pixel Readback Performance**: Is `wgpuBufferMapAsync` fast enough for CI? - - Typical latency: ~5-10ms per frame - - For 50 tests × 1 frame each = ~500ms (acceptable) - -2. **Headless CI Support**: Will WebGPU work without display server? - - macOS: ✅ Works (Metal backend) - - Linux: ⚠️ Requires Vulkan/X11/Wayland - - Win32: ✅ Works (D3D12 backend) - - Solution: Gracefully skip if init fails - -3. **Golden Master Images**: Do we need reference images? - - For now: ❌ No (too fragile, GPU differences) - - Instead: Use coarse checks (non-black, brightness, hash) - -4. **Mock vs Real GPU**: Should we mock WebGPU API? - - Verdict: ❌ No mocking - use real GPU but offscreen - - Mocking WebGPU is too complex and low-value - ---- - -## Files to Create - -### Test Infrastructure (3 files) -``` -src/tests/webgpu_test_fixture.{h,cc} # Shared WebGPU init -src/tests/offscreen_render_target.{h,cc} # Headless rendering -src/tests/effect_test_helpers.h # Common test utilities -``` - -### Test Suites (6 files) -``` -src/tests/test_effect_base.cc # Effect/Sequence lifecycle -src/tests/test_demo_effects.cc # Effect factory -src/tests/test_post_process_helper.cc # Pipeline utilities -src/tests/test_post_process_effects.cc # All post-process effects -src/tests/test_scene_effects.cc # All scene effects -src/tests/test_mainsequence_render.cc # Integration test -``` - -**Total**: 9 new files (~2000 lines) - ---- - -## Conclusion - -**Recommended Approach**: Start with foundation + base tests (Phase 1-2) -- Achieves GPU coverage: 20% → 40% with moderate effort -- Provides infrastructure for future detailed tests -- Low risk: graceful skip if GPU unavailable -- No GUI dependency: fully headless - -**Stretch Goal**: Add individual effect tests (Phase 3) -- Achieves GPU coverage: 40% → 50% -- Higher effort, diminishing returns -- Nice-to-have, not critical - -**Key Innovation**: Offscreen rendering eliminates need for windows/displays while still testing real GPU code paths. |