path: root/doc/BACKLOG.md

# Future Goals & Ideas (Untriaged)

This file contains low-priority tasks and ideas that have not yet been triaged for active development.

---

## Audio Tools

### Task #64: specplay Enhancements
Extend audio analysis tool with new features:
- **Priority 1**: Spectral visualization (ASCII art), waveform display, frequency analysis, dynamic range
- **Priority 2**: Diff mode (compare .wav vs .spec), batch mode (CSV report, find clipping)
- **Priority 3**: WAV export (.spec → .wav), normalization
- **Priority 4**: Spectral envelope, harmonic analysis, onset detection
- **Priority 5**: Interactive mode (seek, loop, volume control)

See `tools/specplay_README.md` for detailed feature list.

### Task #65: Data-Driven Tempo Control
Move tempo variation from code to data files.

**Current**: `g_tempo_scale` is hardcoded in `main.cc` with manual animation curves

**Goal**: Define tempo curves in `.seq` or `.track` files

**Approach A**: Add TEMPO directive to `.seq` format
- Example: `TEMPO 0.0 1.0`, `TEMPO 10.0 2.0`, `TEMPO 20.0 1.0`
- seq_compiler generates tempo curve array in timeline.cc

**Approach B**: Add tempo column to music.track
- Each pattern trigger can specify tempo_scale override
- tracker_compiler generates tempo events in music_data.cc

**Benefits**: Non-programmers can edit tempo, easier iteration, version control friendly

**Priority**: Low (current approach works)

### Task #67: DCT/FFT Performance Benchmarking
Add timing measurements to audio tests.

**Goal**: Compare performance of different DCT/IDCT implementations

**Location**: Add timing code to `test_dct.cc` or `test_fft.cc`

**Measurements**:
- Reference IDCT/FDCT (naive O(N²))
- FFT-based DCT/IDCT (current O(N log N))
- Future x86_64 SIMD-optimized versions

**Output Format**:
- Average time per transform (microseconds)
- Throughput (transforms per second)
- Speedup factor vs reference

**Test Sizes**: DCT_SIZE=512 (production), plus 128, 256, 1024 for scaling

**Implementation**:
- Use `std::chrono::high_resolution_clock`
- Run 1000+ iterations to reduce noise
- Report min/avg/max times
- Guard with `#if !defined(STRIP_ALL)`

**Priority**: Very Low (nice-to-have)

### Task #69: Convert Audio Pipeline to Clipped Int16
Use clipped int16 for all audio processing.

**Current**: Float32 throughout (generation, mixing, synthesis, output)

**Goal**: Convert to int16 for faster processing and reduced memory

**Rationale**:
- Simpler arithmetic (no float operations)
- Smaller memory footprint (2 bytes vs 4 bytes)
- Hardware-native format (most audio devices use int16)
- Eliminates float→int16 conversion at output
- Natural clipping behavior

**Scope**:
- Output path: Definitely convert (backends, WAV dump)
- Synthesis: Consider keeping float32 for quality
- Mixing: Could use int16 with overflow handling
- Asset storage: Already int16 in .spec files

**Implementation Phases**:
1. **Phase 1: Output Only** (~50 lines) - Convert `synth_render()` output to int16
2. **Phase 2: Mixing Stage** (~200 lines) - Convert voice mixing to int16 arithmetic
3. **Phase 3: Full Pipeline** (~500+ lines) - Convert spectrograms to int16 storage

**Trade-offs**:
- Quality loss: 16-bit vs 32-bit float precision
- Dynamic range: Limited to [-32768, 32767]
- Clipping: Must handle overflow carefully
- Code complexity: Saturation arithmetic

**Testing Requirements**:
- Verify no audible quality degradation
- Ensure clipping behavior matches float version
- Check mixing overflow doesn't cause artifacts
- Validate WAV dumps bit-identical

**Size Impact**:
- Phase 1: Negligible (~50 bytes)
- Phase 2: ~100-200 bytes
- Phase 3: 50% memory, ~1-2KB code savings

**Priority**: Low (final optimization only if 64k budget requires it)

**Notes**: Quality must be validated - may not be worth trade-off

---

## Developer Tools

### Task #66: External Asset Loading for Debugging
mmap() asset files instead of embedded data.

**Current**: All assets embedded in `assets_data.cc` (regenerate on every change)

**Goal**: Load assets from external files in debug builds for faster iteration

**Scope**: macOS only, non-STRIP_ALL builds only

**Implementation**:
- Add `DEMO_ENABLE_EXTERNAL_ASSETS` CMake option
- Modify `GetAsset()` to check for external file first (e.g., `assets/final/<name>`)
- Use `mmap()` to map file into memory
- Fallback to embedded data if file not found

**Benefits**: Edit shaders/assets without regenerating assets_data.cc (~10s rebuild)

**Trade-offs**: Adds runtime file I/O, only useful during development

**Priority**: Low (current workflow acceptable)

---

## Visual Effects

### Task #73: Extend Shader Parametrization [IN PROGRESS - 2/4 complete]
Extend uniform parameter system to remaining effects.

**Goal**: Add parametrization to DistortEffect, SolarizeEffect

**Pattern**: Follow FlashEffect implementation (UniformHelper, params struct, .seq syntax)

**Completed**: ChromaAberrationEffect (offset_scale, angle), GaussianBlurEffect (strength)

**Priority**: Medium (quality-of-life for artists)

**Estimated Impact**: ~200-300 bytes per effect

### Task #52: Procedural SDF Font
Minimal bezier/spline set for [A-Z, 0-9] and SDF rendering.

### Task #55: SDF Random Planes Intersection
Implement `sdPolyhedron` (crystal/gem shapes) via plane intersection.

### Task #54: Tracy Integration
Integrate Tracy debugger for performance profiling.

### Task #58: Advanced Shader Factorization
Further factorize WGSL code into smaller, reusable snippets.

### Task #59: Comprehensive RNG Library
Add WGSL snippets for float/vec2/vec3 noise (Perlin, Gyroid, etc.) and random number generators.

### Task #60: OOP Refactoring
Investigate if more C++ code can be made object-oriented without size penalty (vs functional style).

### Task #61: GPU Procedural Generation
Implement system to generate procedural data (textures, geometry) on GPU and read back to CPU.

### Task #62: Physics Engine Enhancements (PBD & Rotation)
- **Task #62.1**: Quaternion rotation for `Object3D` with angular momentum
- **Task #62.2**: Position Based Dynamics (PBD) - Re-evaluate velocity after resolving collisions/constraints

### Task #63: Refactor Large Files
Split `src/gpu/gpu.cc`, `src/3d/visual_debug.cc` and `src/gpu/effect.cc` into sub-functionalities.

---

## Performance Optimization

### Task #70: SIMD x86_64 Implementation
Implement critical functions using intrinsics for x86_64 platforms.

**Goal**: Optimize hot paths for audio and procedural generation

**Scope**:
- IDCT/FDCT transforms
- Audio mixing and voice synthesis
- CPU-side procedural texture/geometry generation

**Constraint**: Non-critical; fallback to generic C++ must be maintained

**Priority**: Very Low