diff options
| -rw-r--r-- | CLAUDE.md | 4 | ||||
| -rw-r--r-- | PROJECT_CONTEXT.md | 8 | ||||
| -rw-r--r-- | TODO.md | 53 | ||||
| -rw-r--r-- | doc/COMPLETED.md | 5 | ||||
| -rw-r--r-- | doc/HOWTO.md | 133 | ||||
| -rw-r--r-- | doc/archive/AUDIO_WAV_DRIFT_BUG.md (renamed from doc/AUDIO_WAV_DRIFT_BUG.md) | 0 | ||||
| -rw-r--r-- | doc/archive/FILE_HIERARCHY_CLEANUP_2026-02-13.md (renamed from doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md) | 0 |
7 files changed, 30 insertions, 173 deletions
@@ -21,8 +21,8 @@ # Arch: ARCHITECTURE.md, CODING_STYLE.md, BACKLOG.md, TOOLS_REFERENCE.md, CONTEXT_MAINTENANCE.md # TIER 4: ARCHIVE (Historical/Debugging Only) -# Active: COMPLETED.md, CNN_DEBUG.md, FILE_HIERARCHY_CLEANUP_2026-02-13.md -# Archived: doc/archive/* (handoffs, analyses, build proposals, timing fixes) +# Active: COMPLETED.md, CNN_DEBUG.md +# Archived: doc/archive/* (handoffs, analyses, build proposals, cleanups) # ============================================ # PROJECT RULES (IMPORTANT) diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index e8071c3..dee9c77 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -39,14 +39,18 @@ - **Effects:** CNN post-processing: CNNEffect (v1) and CNNv2Effect operational. CNN v2: sigmoid activation, storage buffer weights (~3.2 KB), 7D static features, dynamic layers. Training stable, convergence validated. - **Tools:** CNN test tool operational. Texture readback utility functional. Timeline editor (web-based, beat-aligned, audio playback). - **Build:** Asset dependency tracking. Size measurement. Hot-reload (debug-only). -- **Sequence:** DAG-based effect routing with explicit node system. Python compiler with topological sort and ping-pong optimization. 7 effects operational (Passthrough, Placeholder, GaussianBlur, Heptagon, Particles, RotatingCube, Hybrid3D). See `doc/SEQUENCE.md`. +- **Sequence:** DAG-based effect routing with explicit node system. Python compiler with topological sort and ping-pong optimization. 9 effects operational (Passthrough, Placeholder, GaussianBlur, Heptagon, Particles, RotatingCube, Hybrid3D, Flash, PeakMeter). See `doc/SEQUENCE.md`. - **Testing:** **35/35 passing** --- ## Next Up -See `TODO.md` for current priorities and active tasks. +**Active:** Spectral Brush Editor (procedural compression), CNN v2 quantization +**Ongoing:** Test infrastructure maintenance (35/35 passing) +**Future:** Size optimization (64k target), 3D enhancements + +See `TODO.md` for details. --- @@ -1,8 +1,6 @@ # To-Do List -**High-level task tracker.** See individual design docs for implementation details. - -**Completed tasks:** `doc/COMPLETED.md` +**High-level task tracker.** See design docs for details. Completed: `doc/COMPLETED.md` --- @@ -14,36 +12,13 @@ Procedural spectrogram tool: 50-100× compression (5 KB .spec → ~100 bytes C++ --- -## ~~Priority 2: Workspace System (Task #77)~~ [COMPLETED] - -Self-contained workspaces for parallel demo development. - -**Design:** `doc/WORKSPACE_SYSTEM.md` - -**Usage:** `cmake -B build -DDEMO_WORKSPACE=main` - ---- - -## Priority 2: CNN v2 - Parametric Static Features (Task #85) [COMPLETE] +## Priority 2: CNN v2 8-bit Quantization -Enhanced CNN post-processing with multi-dimensional feature inputs. +Reduce weights from f16 (~3.2 KB) to i8 (~1.6 KB). +**Requirements:** Quantization-aware training (QAT) **Design:** `cnn_v2/docs/CNN_V2.md` -**Status:** -- ✅ Full implementation complete and validated -- ✅ Sigmoid activation (smooth gradients, fixes training collapse) -- ✅ Training pipeline: patch-based, stable convergence -- ✅ All tests passing (34/36, 2 unrelated script failures) - -**Specs:** -- 7D static features (RGBD + UV + sin + bias) -- Storage buffer weights (~3.2 KB, 8→4→4 channels) -- Sigmoid for layer 0 & final, ReLU for middle layers -- <10 KB target achieved - -**TODO:** 8-bit quantization (2× reduction, needs QAT). - --- ## Priority 3: Test Infrastructure Maintenance [ONGOING] @@ -65,26 +40,6 @@ Enhanced CNN post-processing with multi-dimensional feature inputs. 4. **test_fft.cc:87** - Investigate FFT-DCT algorithm discrepancy - May need different algorithm or fix existing one ---- - -## ~~Priority 3: Tracker Humanization & Sample Offset~~ [IMPLEMENTED] - -Enhance tracker with sample offset and humanization for realistic playback. - -**Status:** ✅ Both features implemented and tested (commit e9dde3c) - -**Features:** -1. **Sample Offset (compile-time):** Intrinsic offset per sample, zero runtime cost - - `.track` syntax: `SAMPLE <name> OFFSET <sec>` - - Applied during compilation, preserves beat sync - -2. **Humanization (runtime, deterministic):** Per-note timing/volume variation - - `.track` syntax: `HUMANIZE SEED <int> TIMING <pct> VOLUME <pct>` - - Deterministic RNG ensures identical playback and WAV export - -**Test:** `data/test_humanize.track` - ---- ## Priority 4: Audio System Enhancements [LOW PRIORITY] diff --git a/doc/COMPLETED.md b/doc/COMPLETED.md index debfc3d..2a22845 100644 --- a/doc/COMPLETED.md +++ b/doc/COMPLETED.md @@ -48,10 +48,9 @@ Use `read @doc/archive/FILENAME.md` to access archived documents. - Compile-time ping-pong optimization (aliased nodes) - Unified preprocess/postprocess per sequence - Python compiler (seq_compiler_v2.py) generates optimized C++ SequenceV2 subclasses - - **Testing**: 34/36 passing (2 v1-dependent tests disabled: test_demo_effects, test_sequence) + - **Testing**: 35/35 passing (all v2 tests ported) - **Status**: demo64k and test_demo run successfully, all seek positions work - - **TODO**: Port CNN effects to v2, flatten mode implementation, remaining effects - - **Design**: `doc/SEQUENCE_v2.md`, Plan: `doc/archive/SEQUENCE_V2_MIGRATION_PLAN.md` + - **Design**: `doc/SEQUENCE.md`, Archive: `doc/archive/SEQUENCE_V2_MIGRATION_PLAN.md` ## Recently Completed (February 14, 2026) diff --git a/doc/HOWTO.md b/doc/HOWTO.md index 4cafaa2..f1401df 100644 --- a/doc/HOWTO.md +++ b/doc/HOWTO.md @@ -96,147 +96,46 @@ make run_util_tests # Utility tests ## Training -### Patch-Based (Recommended) -Extracts patches at salient points, trains on center pixels only (matches WGSL sliding window): +### CNN v1 (Legacy) ```bash -# Train with 32×32 patches at detected corners/edges +# Patch-based (recommended) ./cnn_v1/training/train_cnn.py \ --input training/input/ --target training/output/ \ --patch-size 32 --patches-per-image 64 --detector harris \ - --layers 3 --kernel_sizes 3,5,3 --epochs 5000 --batch_size 16 \ - --checkpoint-every 1000 -``` - -**Training behavior:** -- Loss computed only on center pixels (excludes conv padding borders) -- For 3-layer network: excludes 3px border on each side -- Matches GPU shader sliding-window paradigm - -**Detectors:** `harris` (default), `fast`, `shi-tomasi`, `gradient` + --layers 3 --kernel_sizes 3,5,3 --epochs 5000 -### Full-Image -Processes entire image with sliding window (matches WGSL): -```bash -./cnn_v1/training/train_cnn.py \ - --input training/input/ --target training/output/ \ - --layers 3 --kernel_sizes 3,5,3 --epochs 10000 --batch_size 8 \ - --checkpoint-every 1000 +# Export shaders +./cnn_v1/training/train_cnn.py --export-only checkpoints/checkpoint.pth ``` -### Export & Validation -```bash -# Generate shaders from checkpoint -./cnn_v1/training/train_cnn.py --export-only checkpoints/checkpoint_epoch_5000.pth - -# Generate ground truth (sliding window, no tiling) -./cnn_v1/training/train_cnn.py --infer input.png \ - --export-only checkpoints/checkpoint_epoch_5000.pth \ - --output ground_truth.png -``` - -**Inference:** Processes full image with sliding window (each pixel from NxN neighborhood). No tiling artifacts. - -**Kernel sizes:** 3×3 (36 weights), 5×5 (100 weights), 7×7 (196 weights) - ### CNN v2 Training -Enhanced CNN with parametric static features (7D input: RGBD + UV + sin encoding + bias). - -**Complete Pipeline** (recommended): ```bash -# Train → Export → Build → Validate (default config) +# Default pipeline (train → export → validate) ./cnn_v2/scripts/train_cnn_v2_full.sh -# Rapid debug (1 layer, 3×3, 5 epochs) -./cnn_v2/scripts/train_cnn_v2_full.sh --num-layers 1 --kernel-sizes 3 --epochs 5 --output-weights test.bin - -# Custom training parameters -./cnn_v2/scripts/train_cnn_v2_full.sh --epochs 500 --batch-size 32 --checkpoint-every 100 +# Quick debug (1 layer, 5 epochs) +./cnn_v2/scripts/train_cnn_v2_full.sh --num-layers 1 --epochs 5 # Custom architecture -./cnn_v2/scripts/train_cnn_v2_full.sh --kernel-sizes 3,5,3 --num-layers 3 --mip-level 1 - -# Custom output path -./cnn_v2/scripts/train_cnn_v2_full.sh --output-weights workspaces/test/cnn_weights.bin - -# Grayscale loss (compute loss on luminance instead of RGBA) -./cnn_v2/scripts/train_cnn_v2_full.sh --grayscale-loss - -# Custom directories -./cnn_v2/scripts/train_cnn_v2_full.sh --input training/input --target training/target_2 +./cnn_v2/scripts/train_cnn_v2_full.sh --kernel-sizes 3,5,3 --epochs 500 -# Full-image mode (instead of patch-based) -./cnn_v2/scripts/train_cnn_v2_full.sh --full-image --image-size 256 +# Validation only +./cnn_v2/scripts/train_cnn_v2_full.sh --validate -# See all options +# All options ./cnn_v2/scripts/train_cnn_v2_full.sh --help ``` -**Defaults:** 200 epochs, 3×3 kernels, 8→4→4 channels, batch-size 16, patch-based (8×8, harris detector). -- Live progress with single-line update -- Always saves final checkpoint (regardless of --checkpoint-every interval) -- When multiple kernel sizes provided (e.g., 3,5,3), num_layers derived from list length -- Validates all input images on final epoch -- Exports binary weights (storage buffer architecture) -- Streamlined output: single-line export summary, compact validation -- All parameters configurable via command-line +**Defaults:** 200 epochs, 3×3 kernels, 8→4→4 channels, patch-based (8×8). Outputs ~3.2 KB f16 weights. -**Validation Only** (skip training): +**Manual export:** ```bash -# Use latest checkpoint -./cnn_v2/scripts/train_cnn_v2_full.sh --validate - -# Use specific checkpoint -./cnn_v2/scripts/train_cnn_v2_full.sh --validate checkpoints/checkpoint_epoch_50.pth -``` - -**Manual Training:** -```bash -# Default config -./cnn_v2/training/train_cnn_v2.py \ - --input training/input/ --target training/target_2/ \ - --epochs 100 --batch-size 16 --checkpoint-every 5 - -# Custom architecture (per-layer kernel sizes) -./cnn_v2/training/train_cnn_v2.py \ - --input training/input/ --target training/target_2/ \ - --kernel-sizes 1,3,5 \ - --epochs 5000 --batch-size 16 - -# Mip-level for p0-p3 features (0=original, 1=half, 2=quarter, 3=eighth) -./cnn_v2/training/train_cnn_v2.py \ - --input training/input/ --target training/target_2/ \ - --mip-level 1 \ - --epochs 100 --batch-size 16 - -# Grayscale loss (compute loss on luminance Y = 0.299*R + 0.587*G + 0.114*B) -./cnn_v2/training/train_cnn_v2.py \ - --input training/input/ --target training/target_2/ \ - --grayscale-loss \ - --epochs 100 --batch-size 16 -``` - -**Export Binary Weights:** -```bash -# Verbose output (shows all layer details) -./training/export_cnn_v2_weights.py checkpoints/checkpoint_epoch_100.pth \ +./training/export_cnn_v2_weights.py checkpoints/checkpoint.pth \ --output-weights workspaces/main/cnn_v2_weights.bin - -# Quiet mode (single-line summary) -./training/export_cnn_v2_weights.py checkpoints/checkpoint_epoch_100.pth \ - --output-weights workspaces/main/cnn_v2_weights.bin \ - --quiet -``` - -Generates binary format: header + layer info + f16 weights (~3.2 KB for 3-layer model). -Storage buffer architecture allows dynamic layer count. -Use `--quiet` for streamlined output in scripts (used automatically by train_cnn_v2_full.sh). - -**TODO:** 8-bit quantization for 2× size reduction (~1.6 KB). Requires quantization-aware training (QAT). - ``` -**Validation:** Use HTML tool (`cnn_v2/tools/cnn_v2_test/index.html`) for CNN v2 validation. See `cnn_v2/docs/CNN_V2_WEB_TOOL.md`. +See `cnn_v2/docs/CNN_V2.md` for architecture details and web validation tool. --- diff --git a/doc/AUDIO_WAV_DRIFT_BUG.md b/doc/archive/AUDIO_WAV_DRIFT_BUG.md index 050dd49..050dd49 100644 --- a/doc/AUDIO_WAV_DRIFT_BUG.md +++ b/doc/archive/AUDIO_WAV_DRIFT_BUG.md diff --git a/doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md b/doc/archive/FILE_HIERARCHY_CLEANUP_2026-02-13.md index 8af5efd..8af5efd 100644 --- a/doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md +++ b/doc/archive/FILE_HIERARCHY_CLEANUP_2026-02-13.md |