path: root/GPU_EFFECTS_TEST_ANALYSIS.md

# GPU Effects Test Coverage Analysis

## Current State

### Coverage Status
**GPU Directory Coverage: Very Low** (~20% estimated)

### File Breakdown (32 files total)

#### Core Infrastructure (5 files)
- `gpu.{h,cc}` - WebGPU initialization and management
- `effect.{h,cc}` - Base Effect/Sequence/MainSequence classes
- `texture_manager.{h,cc}` - Texture loading and procedural generation

#### Effects System (27 files)
**Base Effects:**
- `demo_effects.{h,cc}` - Effect registration and factory

**Individual Effects (19 implementations):**
1. `chroma_aberration_effect.cc` - Chromatic aberration post-process
2. `distort_effect.cc` - Distortion effect
3. `fade_effect.{h,cc}` - Fade in/out
4. `flash_cube_effect.{h,cc}` - Flashing cube
5. `flash_effect.{h,cc}` - Screen flash on beat
6. `gaussian_blur_effect.cc` - Blur post-process
7. `heptagon_effect.cc` - Heptagon shape
8. `hybrid_3d_effect.{h,cc}` - 3D + SDF hybrid rendering
9. `moving_ellipse_effect.cc` - Animated ellipse
10. `particle_spray_effect.cc` - Particle spray
11. `particles_effect.cc` - Particle system
12. `passthrough_effect.cc` - Identity post-process
13. `solarize_effect.cc` - Solarize color effect
14. `theme_modulation_effect.{h,cc}` - Theme color modulation

**Utilities:**
- `post_process_helper.{h,cc}` - Post-process pipeline utilities
- `shader_composer.{h,cc}` - Shader composition system
- `shaders.{h,cc}` - Shader registration

### Existing Tests (4 files)
1. ✅ `test_shader_assets.cc` - Basic shader keyword validation
2. ✅ `test_shader_compilation.cc` - Real WebGPU shader compilation
3. ✅ `test_shader_composer.cc` - Shader composition logic
4. ⚠️ `test_texture_manager.cc` - Minimal (just compilation check)

### Coverage Gaps
**Completely Untested:**
- ❌ `effect.cc` - Sequence/Effect lifecycle (0% coverage)
- ❌ All 19 effect implementations (0% coverage)
- ❌ `demo_effects.cc` - Effect factory (0% coverage)
- ❌ `post_process_helper.cc` - Pipeline utilities (0% coverage)
- ❌ `gpu.cc` - Initialization logic (partially tested via integration tests)

---

## Testing Challenges

### Challenge 1: WebGPU Initialization Overhead
**Problem**: Every test needs a full WebGPU device/queue/surface setup.

**Current Approach** (test_shader_compilation.cc):
```cpp
// 100+ lines of initialization boilerplate
WGPUInstance instance = wgpuCreateInstance(nullptr);
WGPUAdapter adapter = ...; // Complex callback-based async request
WGPUDevice device = ...;   // Complex callback-based async request
```

**Issues**:
- Heavy: ~150ms per test for GPU initialization
- Unreliable: May fail in headless CI environments
- Verbose: 100+ lines of boilerplate per test file

### Challenge 2: Rendering Requires a Surface
**Problem**: Most tests need a real window for `render_frame()`.

**Current Approach** (test_3d_render.cc, test_mesh.cc):
```cpp
PlatformState state = platform_init(false, 640, 480); // Creates GLFW window
WGPUSurface surface = platform_create_wgpu_surface(instance, &state);
main_seq.render_frame(time, beat, peak, aspect, surface);
```

**Issues**:
- GUI dependency: Can't run in headless CI
- Manual testing: Requires human to verify visual output
- No assertions: Can't validate correctness automatically

### Challenge 3: No Frame Validation
**Problem**: How do you assert that a visual effect is correct?

**Current Gap**:
- No pixel readback infrastructure
- No reference image comparison
- No "golden master" test images

---

## Proposed Solution: Headless Rendering with Frame Sink

### Architecture Overview

```
┌─────────────────────────────────────────────────────────┐
│                  Test Framework                          │
├─────────────────────────────────────────────────────────┤
│                                                           │
│  ┌──────────────┐      ┌──────────────────────────┐    │
│  │ MockSurface  │─────▶│ OffscreenFramebuffer     │    │
│  └──────────────┘      │ (WGPUTexture)            │    │
│        │               └──────────────────────────┘    │
│        │                         │                       │
│        │                         ▼                       │
│        │               ┌──────────────────────────┐    │
│        └──────────────▶│ PixelReadback            │    │
│                        │ (wgpuBufferMapAsync)     │    │
│                        └──────────────────────────┘    │
│                                  │                       │
│                                  ▼                       │
│                        ┌──────────────────────────┐    │
│                        │ Assertions                │    │
│                        │ (pixel checks, hashes)   │    │
│                        └──────────────────────────┘    │
└─────────────────────────────────────────────────────────┘
```

### Key Components

#### 1. WebGPU Test Fixture (Reusable)
```cpp
// src/tests/webgpu_test_fixture.h
class WebGPUTestFixture {
 public:
  bool init();  // One-time setup
  void shutdown();

  WGPUDevice device() const { return device_; }
  WGPUQueue queue() const { return queue_; }
  WGPUTextureFormat format() const { return WGPUTextureFormat_BGRA8Unorm; }

 private:
  WGPUInstance instance_ = nullptr;
  WGPUAdapter adapter_ = nullptr;
  WGPUDevice device_ = nullptr;
  WGPUQueue queue_ = nullptr;
};
```

**Benefits**:
- Shared initialization across all GPU tests
- ~100 lines of boilerplate → 3 lines per test
- Can gracefully skip tests if GPU unavailable

#### 2. Offscreen Render Target
```cpp
// src/tests/offscreen_render_target.h
class OffscreenRenderTarget {
 public:
  OffscreenRenderTarget(WGPUDevice device, int width, int height);
  ~OffscreenRenderTarget();

  WGPUTexture texture() const { return texture_; }
  WGPUTextureView view() const { return view_; }

  // Simulate surface behavior without actual window
  WGPUSurfaceTexture get_current_texture();

  // Readback pixels for validation
  std::vector<uint8_t> read_pixels();

 private:
  WGPUDevice device_;
  WGPUTexture texture_;
  WGPUTextureView view_;
  int width_, height_;
};
```

**Key Feature**: No window needed! Pure GPU texture operations.

#### 3. Effect Test Helpers
```cpp
// src/tests/effect_test_helpers.h

// Helper: Test effect lifecycle
void test_effect_lifecycle(Effect* effect, MainSequence* demo);

// Helper: Test effect render (smoke test - no crash)
void test_effect_render(Effect* effect, WGPURenderPassEncoder pass);

// Helper: Test post-process bind group update
void test_post_process_bind_group(PostProcessEffect* effect,
                                    WGPUTextureView input);

// Helper: Validate rendered output (pixel checks)
bool validate_pixels(const std::vector<uint8_t>& pixels,
                     int width, int height,
                     std::function<bool(uint8_t r, uint8_t g, uint8_t b, uint8_t a)> predicate);

// Helper: Hash-based validation (deterministic output check)
uint64_t hash_pixels(const std::vector<uint8_t>& pixels);
```

---

## Proposed Test Structure

### Test 1: Effect Base Classes (`test_effect_base.cc`)

**Goal**: Test Effect/Sequence/MainSequence lifecycle

```cpp
void test_effect_construction() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return; // Skip if no GPU

  FlashEffect effect(fixture.device(), fixture.queue(), fixture.format());
  assert(!effect.is_initialized);

  MainSequence main_seq;
  main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());

  effect.init(&main_seq);
  assert(effect.is_initialized);
}

void test_sequence_add_effect() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return;

  auto effect = std::make_shared<FlashEffect>(/*...*/);

  Sequence seq;
  seq.add_effect(effect, 0.0f, 10.0f, 0);

  // Should not activate before start time
  seq.update_active_list(-1.0f);
  // Verify not active

  // Should activate at start time
  seq.update_active_list(0.0f);
  // Verify active

  // Should deactivate after end time
  seq.update_active_list(11.0f);
  // Verify not active
}

void test_sequence_priority_sorting() {
  // Add effects with different priorities
  // Verify render order via collect_active_effects()
}
```

**Coverage Impact**: effect.cc 0% → ~60%

---

### Test 2: Post-Process Effects (`test_post_process_effects.cc`)

**Goal**: Test all post-process effects (flash, blur, distort, etc.)

```cpp
void test_flash_effect_basic() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return;

  FlashEffect effect(fixture.device(), fixture.queue(), fixture.format());

  MainSequence main_seq;
  main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());
  effect.init(&main_seq);

  OffscreenRenderTarget target(fixture.device(), 256, 256);
  effect.update_bind_group(target.view());

  // Create render pass
  WGPUCommandEncoder encoder = wgpuDeviceCreateCommandEncoder(fixture.device(), nullptr);
  WGPURenderPassDescriptor pass_desc = create_simple_pass(target.view());
  WGPURenderPassEncoder pass = wgpuCommandEncoderBeginRenderPass(encoder, &pass_desc);

  // Render with low intensity (no flash)
  effect.render(pass, 0.0f, 0.0f, 0.1f, 1.0f);
  wgpuRenderPassEncoderEnd(pass);

  // Submit and readback
  WGPUCommandBuffer cmd = wgpuCommandEncoderFinish(encoder, nullptr);
  wgpuQueueSubmit(fixture.queue(), 1, &cmd);

  // Validate: should not be pure white (no flash triggered)
  auto pixels = target.read_pixels();
  bool has_non_white = validate_pixels(pixels, 256, 256,
    [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
      return r < 255 || g < 255 || b < 255; // At least one pixel not white
    });
  assert(has_non_white);
}

void test_flash_effect_trigger() {
  // Test with high intensity (>0.7) - should trigger flash
  // Validate some pixels are white/bright
}

// Similar tests for:
// - test_gaussian_blur_effect()
// - test_chroma_aberration_effect()
// - test_distort_effect()
// - test_solarize_effect()
// - test_passthrough_effect()
```

**Coverage Impact**: 6 post-process effects 0% → ~50%

---

### Test 3: Scene Effects (`test_scene_effects.cc`)

**Goal**: Test scene-rendering effects (geometry, 3D, particles)

```cpp
void test_heptagon_effect() {
  // Test basic render without crash
  // Optionally: readback and check for non-black pixels (shape rendered)
}

void test_moving_ellipse_effect() {
  // Test animation over time (different time values)
}

void test_particle_effects() {
  // Test ParticleSprayEffect
  // Test ParticlesEffect
}

void test_hybrid_3d_effect() {
  // Test 3D rendering with SDF raymarching
  // This is complex - may just be smoke test
}
```

**Coverage Impact**: 8 scene effects 0% → ~40%

---

### Test 4: Effect Factory (`test_demo_effects.cc`)

**Goal**: Test effect registration and instantiation

```cpp
void test_effect_factory_registration() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return;

  // All effects should be auto-registered
  const auto& effects = GetEffectRegistry();
  assert(effects.size() >= 15); // We have ~19 effects

  // Check specific effects exist
  assert(effects.find("FlashEffect") != effects.end());
  assert(effects.find("FlashCubeEffect") != effects.end());
  // etc.
}

void test_effect_factory_instantiation() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return;

  MainSequence main_seq;
  main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());

  // Create effect via factory
  auto effect = CreateEffect("FlashEffect", fixture.device(),
                             fixture.queue(), fixture.format(), nullptr);
  assert(effect != nullptr);
  assert(dynamic_cast<FlashEffect*>(effect.get()) != nullptr);

  // Init should not crash
  effect->init(&main_seq);
}
```

**Coverage Impact**: demo_effects.cc 0% → ~70%

---

### Test 5: Integration Test (`test_mainsequence_render.cc`)

**Goal**: Test full render pipeline with multiple effects

```cpp
void test_mainsequence_full_render() {
  WebGPUTestFixture fixture;
  if (!fixture.init()) return;

  MainSequence main_seq;
  main_seq.init_test(fixture.device(), fixture.queue(), fixture.format());

  // Add a sequence with multiple effects
  auto seq = std::make_shared<Sequence>();
  seq->add_effect(std::make_shared<FlashEffect>(/*...*/), 0.0f, 10.0f, 0);
  seq->add_effect(std::make_shared<PassthroughEffect>(/*...*/), 0.0f, 10.0f, 1);

  main_seq.add_sequence(seq, 0.0f, 0);
  seq->init(&main_seq);

  // Create offscreen target
  OffscreenRenderTarget target(fixture.device(), 256, 256);

  // Render a frame
  // Note: Need to adapt render_frame() to accept texture instead of surface
  // OR create a mock surface that wraps the offscreen texture

  main_seq.render_frame_offscreen(0.5f, 0.0f, 0.5f, 1.0f, target.texture());

  // Validate: frame rendered without crash, some pixels non-black
  auto pixels = target.read_pixels();
  bool rendered = validate_pixels(pixels, 256, 256,
    [](uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
      return r > 0 || g > 0 || b > 0; // At least one pixel not black
    });
  assert(rendered);
}
```

**Coverage Impact**: effect.cc MainSequence 0% → ~50%

---

## Implementation Plan

### Phase 1: Foundation (Week 1) - 8 hours
1. **Create WebGPUTestFixture** (2h)
   - Extract init code from test_shader_compilation.cc
   - Add graceful skip if GPU unavailable
   - Test on macOS, Linux

2. **Create OffscreenRenderTarget** (3h)
   - WGPUTexture creation without surface
   - Pixel readback via wgpuBufferMapAsync
   - Validation helpers (hash, predicate checks)

3. **Create Effect Test Helpers** (2h)
   - Lifecycle test helper
   - Render smoke test helper
   - Pixel validation utilities

4. **Add render_frame_offscreen() to MainSequence** (1h)
   - Variant of render_frame() that targets offscreen texture
   - Or: Mock surface that wraps offscreen texture

---

### Phase 2: Base Tests (Week 2) - 10 hours
1. **test_effect_base.cc** (4h)
   - Effect construction/init
   - Sequence lifecycle (add, activate, deactivate)
   - Priority sorting
   - Collect active effects

2. **test_demo_effects.cc** (2h)
   - Effect registry validation
   - Effect factory instantiation

3. **test_post_process_helper.cc** (2h)
   - Pipeline creation
   - Bind group management

4. **CI Integration** (2h)
   - Update CMakeLists.txt
   - Test graceful skip in headless CI
   - Document GPU test requirements

---

### Phase 3: Individual Effects (Week 3-4) - 20 hours
1. **test_post_process_effects.cc** (8h)
   - FlashEffect (with trigger logic)
   - GaussianBlurEffect
   - ChromaAberrationEffect
   - DistortEffect
   - SolarizeEffect
   - PassthroughEffect

2. **test_scene_effects.cc** (8h)
   - HeptagonEffect
   - MovingEllipseEffect
   - FlashCubeEffect
   - ParticleSprayEffect
   - ParticlesEffect
   - FadeEffect
   - ThemeModulationEffect

3. **test_hybrid_3d.cc** (4h)
   - Hybrid3DEffect (complex, may be smoke test only)

---

### Phase 4: Integration (Week 5) - 6 hours
1. **test_mainsequence_render.cc** (4h)
   - Full render pipeline
   - Multiple sequences
   - Effect priority ordering
   - Frame-by-frame simulation

2. **Coverage Report** (2h)
   - Generate coverage with new tests
   - Update documentation
   - Identify remaining gaps

---

## Expected Coverage Impact

### Before (Current State)
```
gpu/
├── gpu.cc                   ~20% (partially via integration tests)
├── effect.cc                  0% ❌
├── texture_manager.cc        10% ⚠️
├── demo_effects.cc             0% ❌
├── effects/
│   ├── (19 effect files)       0% ❌
│   ├── post_process_helper.cc  0% ❌
│   ├── shader_composer.cc     80% ✅ (already tested)
│   └── shaders.cc              5% ⚠️

Overall GPU Coverage: ~20%
```

### After (With Proposed Tests)
```
gpu/
├── gpu.cc                   ~30% (some init paths tested)
├── effect.cc                 ~60% ✅ (lifecycle, sequence logic)
├── texture_manager.cc        ~40% (basic ops tested)
├── demo_effects.cc           ~70% ✅ (factory tested)
├── effects/
│   ├── (19 effect files)     ~45% ✅ (basic render tested)
│   ├── post_process_helper.cc ~65% ✅
│   ├── shader_composer.cc     80% (unchanged)
│   └── shaders.cc            ~30% (registration tested)

Overall GPU Coverage: ~50% ✅ (20% → 50% = +150% increase)
```

---

## Alternative: Minimal Approach (If Time-Constrained)

### Quick Wins (2-3 hours total)

1. **test_effect_lifecycle.cc** (1.5h)
   - Just test Effect/Sequence construction, add_effect, activation
   - No rendering, just state checks
   - Gets effect.cc from 0% → 40%

2. **test_effect_factory.cc** (1h)
   - Test effect registry and instantiation
   - Gets demo_effects.cc from 0% → 60%

3. **Smoke Test for Each Effect** (30min)
   - Loop through all effects, construct + init
   - Verify no crashes
   - Gets effects/*.cc from 0% → 20%

**Result**: GPU coverage 20% → 35% with minimal effort

---

## Recommendations

### Priority 1: Foundation (Do First)
- ✅ Create WebGPUTestFixture (shared across all GPU tests)
- ✅ Create OffscreenRenderTarget (enables headless testing)
- ✅ test_effect_base.cc (high impact, tests core logic)

### Priority 2: Smoke Tests (Quick Wins)
- ✅ test_demo_effects.cc (factory validation)
- ✅ Loop through all effects, test construction

### Priority 3: Detailed Effect Tests (Long-Term)
- ⚠️ Individual effect render validation (time-consuming)
- ⚠️ Pixel-level assertions (requires golden masters)

### Priority 4: Integration Tests (Optional)
- ⚠️ Full MainSequence render pipeline (complex setup)

---

## Open Questions

1. **Pixel Readback Performance**: Is `wgpuBufferMapAsync` fast enough for CI?
   - Typical latency: ~5-10ms per frame
   - For 50 tests × 1 frame each = ~500ms (acceptable)

2. **Headless CI Support**: Will WebGPU work without display server?
   - macOS: ✅ Works (Metal backend)
   - Linux: ⚠️ Requires Vulkan/X11/Wayland
   - Win32: ✅ Works (D3D12 backend)
   - Solution: Gracefully skip if init fails

3. **Golden Master Images**: Do we need reference images?
   - For now: ❌ No (too fragile, GPU differences)
   - Instead: Use coarse checks (non-black, brightness, hash)

4. **Mock vs Real GPU**: Should we mock WebGPU API?
   - Verdict: ❌ No mocking - use real GPU but offscreen
   - Mocking WebGPU is too complex and low-value

---

## Files to Create

### Test Infrastructure (3 files)
```
src/tests/webgpu_test_fixture.{h,cc}       # Shared WebGPU init
src/tests/offscreen_render_target.{h,cc}   # Headless rendering
src/tests/effect_test_helpers.h            # Common test utilities
```

### Test Suites (6 files)
```
src/tests/test_effect_base.cc              # Effect/Sequence lifecycle
src/tests/test_demo_effects.cc             # Effect factory
src/tests/test_post_process_helper.cc      # Pipeline utilities
src/tests/test_post_process_effects.cc     # All post-process effects
src/tests/test_scene_effects.cc            # All scene effects
src/tests/test_mainsequence_render.cc      # Integration test
```

**Total**: 9 new files (~2000 lines)

---

## Conclusion

**Recommended Approach**: Start with foundation + base tests (Phase 1-2)
- Achieves GPU coverage: 20% → 40% with moderate effort
- Provides infrastructure for future detailed tests
- Low risk: graceful skip if GPU unavailable
- No GUI dependency: fully headless

**Stretch Goal**: Add individual effect tests (Phase 3)
- Achieves GPU coverage: 40% → 50%
- Higher effort, diminishing returns
- Nice-to-have, not critical

**Key Innovation**: Offscreen rendering eliminates need for windows/displays while still testing real GPU code paths.