# To-Do List

**High-level task tracker.** See design docs for details. Completed: `doc/COMPLETED.md`

---

## Priority 1: Spectral Brush Editor (Task #5) [IN PROGRESS]

Procedural spectrogram tool: 50-100× compression (5 KB .spec → ~100 bytes C++).

**Design:** `doc/SPECTRAL_BRUSH_2.md` (MQ-based v2)

---

## Priority 2: Test Infrastructure Maintenance [ONGOING]

**Status:** 35/35 tests passing

**Outstanding TODOs:**

1. **test_fft.cc:87** - Investigate FFT-DCT algorithm discrepancy

## Priority 4: Audio System Enhancements [LOW PRIORITY]

1. **`synth.cc`: use `ola_decode()` from `src/audio/ola.h`** — the OLA decode logic in
   `synth_render()` is currently inlined for frame-by-frame lazy decoding. Refactor to
   call `ola_decode()` for consistency with `spectool` and the test (requires decoupling
   the per-frame lazy path, e.g. decode a full block on demand then serve samples).

2. **GPU-Accelerated PCM Synthesis:**
   - Compute shader for direct PCM generation (bypass spectrogram)
   - Write to compute buffer, readback to synth

---

## Priority 4: 3D System Enhancements (Task #18)

Pipeline for importing complex 3D scenes to replace hardcoded geometry.

**Status:** C++ object data loading complete. Shader SDF integration pending.

---

## Priority 4: WGSL Modularization (Task #50) [RECURRENT]

Ongoing shader code hygiene for granular, reusable snippets.

---

## Priority 4: Wine/Windows Black Screen

`demo64k.exe` runs under Wine (wgpu-native v27, Vulkan/MoltenVK) but shows a black window — no visuals rendered. Audio and timeline progress correctly. GPU device/adapter init succeeds.

**Likely causes to investigate:**
- Swapchain format mismatch (Wine Vulkan may prefer BGRA8 over RGBA8)
- Surface present failing silently (check `WGPUSurfaceGetCurrentTexture` status)
- Render pass output not reaching the surface (missing present call or wrong texture view)

**To reproduce:** `./scripts/run_win.sh` — window opens, stays black.

---

## CNN v3 — U-Net + FiLM [IN PROGRESS]

U-Net architecture with FiLM conditioning. Runtime style control via beat/audio.
Richer G-buffer input (normals, depth, material IDs). Per-pixel testability across
PyTorch / HTML WebGPU / C++ WebGPU.

**Design:** `cnn_v3/docs/CNN_V3.md`

**Phases:**
1. ✅ G-buffer: `GBufferEffect` integrated. SDF/shadow placeholder (shadow=1, transp=0).
2. ✅ Training infrastructure: `blender_export.py`, `pack_blender_sample.py`, `pack_photo_sample.py`
3. ✅ WGSL shaders: cnn_v3_common (snippet), enc0, enc1, bottleneck, dec1, dec0
4. ✅ C++ `CNNv3Effect`: 5 compute passes, FiLM uniform upload, `set_film_params()` API
   - Params alignment fix: WGSL `vec3u` align=16 → C++ structs 64/96 bytes
   - Weight offsets as explicit formulas (e.g. `20*4*9+4`)
   - FiLM γ/β: identity defaults; real values require trained MLP (see below)
5. Parity validation (test vectors, ≤1/255 per pixel)

**FiLM MLP training** (blocks meaningful Phase 4 output):
- Needs `cnn_v3/training/train_cnn_v3.py` — not yet written
- MLP: `Linear(5→16) → ReLU → Linear(16→48)` trained jointly with U-Net
- Input: `[beat_phase, beat_time/8, audio_intensity, style_p0, style_p1]`
- Output: γ/β for enc0(4ch) + enc1(8ch) + dec1(4ch) + dec0(4ch) = 40 floats
- Trained weights (~3 KB f16) stored in `.bin` after conv weights; loaded at runtime
- See `cnn_v3/docs/CNN_V3.md` §5 for full MLP spec and §11 for training pipeline plan

## Future: CNN v2 8-bit Quantization

Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB).

**Requirements:** Quantization-aware training (QAT)
**Design:** `cnn_v2/docs/CNN_V2.md`

---

## Future: Size Optimization (64k Target)

- Task #22: Windows Native Platform (Win32)
- Task #28: Spectrogram Quantization
- Task #34: Full STL Removal
- Task #35: CRT Replacement

**Measure:** `./scripts/measure_size.sh`

---

**Backlog:** `doc/BACKLOG.md` for untriaged ideas