From 65f7d74cbd4b34640dda73100c45f25ad468349d Mon Sep 17 00:00:00 2001 From: skal Date: Fri, 13 Feb 2026 08:36:09 +0100 Subject: Documentation: Update for file hierarchy reorganization Updated docs to reflect February 13, 2026 changes: - doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md: Complete summary - doc/WORKSPACE_SYSTEM.md: Current structure, workspace.cfg format - doc/SHADER_REUSE_INVESTIGATION.md: Implementation status - PROJECT_CONTEXT.md: Workspace and shader system updates Key changes documented: - src/app/ application structure - workspaces/{music,weights,obj,shaders}/ layout - common/shaders/ shared shader system - Eliminated 36 duplicate shaders - Asset packer path normalization handoff(Claude): Documentation updated for hierarchy cleanup --- PROJECT_CONTEXT.md | 4 +- doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md | 202 +++++++++++++++++++++++++++++++ doc/SHADER_REUSE_INVESTIGATION.md | 21 +++- doc/WORKSPACE_SYSTEM.md | 60 ++++++--- 4 files changed, 267 insertions(+), 20 deletions(-) create mode 100644 doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 83bfef6..d0e04e2 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -32,9 +32,9 @@ ## Current Status - **Timing System:** **Beat-based timelines** for musical synchronization. Sequences defined in beats, converted to seconds at runtime. Effects receive both physical time (constant) and beat time (musical). Variable tempo affects audio only. See `doc/BEAT_TIMING.md`. -- **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=`. Shared common assets. +- **Workspace system:** Multi-workspace support. Easy switching with `-DDEMO_WORKSPACE=`. Organized structure: `music/`, `weights/`, `obj/`, `shaders/`. Shared common shaders in `common/shaders/`. See `doc/WORKSPACE_SYSTEM.md`. - **Audio:** Sample-accurate sync. Zero heap allocations per frame. Variable tempo. Comprehensive tests. -- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition. +- **Shaders:** Parameterized effects (UniformHelper, .seq syntax). Beat-synchronized animation support (`beat_time`, `beat_phase`). Modular WGSL composition with ShaderComposer. 20 shared common shaders (math, render, compute). - **3D:** Hybrid SDF/rasterization with BVH. Binary scene loader. Blender pipeline. - **Effects:** CNN post-processing: CNNEffect (v1) and CNNv2Effect operational. CNN v2: storage buffer weights (~3.2 KB), 7D static features, dynamic layers. Validated and loading correctly. TODO: 8-bit quantization. - **Tools:** CNN test tool (readback works, output incorrect - under investigation). Texture readback utility functional. Timeline editor (web-based, beat-aligned, audio playback). diff --git a/doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md b/doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md new file mode 100644 index 0000000..8af5efd --- /dev/null +++ b/doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md @@ -0,0 +1,202 @@ +# File Hierarchy Cleanup - February 13, 2026 + +## Summary + +Comprehensive reorganization of project file structure for improved maintainability and code reuse. + +--- + +## Changes Implemented + +### 1. Application Entry Points → `src/app/` + +**Before:** +``` +src/ + main.cc + stub_main.cc + test_demo.cc +``` + +**After:** +``` +src/app/ + main.cc + stub_main.cc + test_demo.cc +``` + +**Impact:** Cleaner src/ directory, separates application code from libraries. + +--- + +### 2. Workspace Reorganization + +**Before:** +``` +workspaces/main/ + assets/music/*.spec + shaders/*.wgsl + obj/*.obj + +assets/ + demo.seq + music.track + originals/ + common/ + final/ +``` + +**After:** +``` +workspaces/{main,test}/ + music/ # Audio samples (.spec) + weights/ # CNN binary weights (.bin) + obj/ # 3D models (.obj) + shaders/ # Workspace-specific WGSL only + +common/ + shaders/ # Shared WGSL utilities + math/ + render/ + compute/ + +tools/ + originals/ # Source audio files + test_demo.seq + test_demo.track +``` + +**Removed:** +- `assets/` directory (legacy structure) +- `assets/common/` (replaced by `common/`) +- `assets/final/` (superseded by workspaces) +- `assets/originals/` → `tools/originals/` + +--- + +### 3. Shared Shader System + +**Problem:** 36 duplicate shader files across workspaces (byte-identical). + +**Solution:** Implemented Option 1 from `SHADER_REUSE_INVESTIGATION.md`. + +**Structure:** +``` +common/shaders/ + common_uniforms.wgsl + lighting.wgsl + passthrough.wgsl + ray_box.wgsl + ray_triangle.wgsl + sdf_primitives.wgsl + skybox.wgsl + math/ + common_utils.wgsl + noise.wgsl + sdf_shapes.wgsl + sdf_utils.wgsl + render/ + lighting_utils.wgsl + scene_query_bvh.wgsl + scene_query_linear.wgsl + shadows.wgsl + compute/ + gen_blend.wgsl + gen_grid.wgsl + gen_mask.wgsl + gen_noise.wgsl + gen_perlin.wgsl +``` + +**Reference in assets.txt:** +``` +SHADER_COMMON_UNIFORMS, NONE, ../../common/shaders/common_uniforms.wgsl +SHADER_MATH_NOISE, NONE, ../../common/shaders/math/noise.wgsl +``` + +**Asset Packer Enhancement:** +- Added `#include ` for path normalization +- Implemented `lexically_normal()` to resolve `../../common/` references +- Cross-platform path handling for workspace-relative includes + +--- + +## Updated Files + +### Configuration +- `workspaces/main/workspace.cfg` - Updated asset_dirs and shader_dirs +- `workspaces/test/workspace.cfg` - Updated asset_dirs and shader_dirs +- `workspaces/main/assets.txt` - Common shader references +- `workspaces/test/assets.txt` - Common shader references + +### Build System +- `cmake/DemoExecutables.cmake` - src/app/ paths, test_demo paths +- `cmake/DemoCodegen.cmake` - Removed legacy fallback paths +- `cmake/Validation.cmake` - Workspace shader paths + +### Tools +- `tools/asset_packer.cc` - Filesystem path normalization +- `scripts/gen_spectrograms.sh` - tools/originals/ paths +- `scripts/train_cnn_v2_full.sh` - workspaces/main/weights/ paths +- `training/export_cnn_v2_weights.py` - workspaces/main/weights/ paths + +### Application +- `src/app/main.cc` - Hot-reload workspace paths + +--- + +## Metrics + +**File Reduction:** +- Removed 36 duplicate shader files +- Deleted legacy assets/ structure (~70 files) +- Net: ~100 files eliminated + +**Disk Space:** +- Common shaders: 20 files +- Per-workspace shaders: ~30-35 files +- Saved: ~36 shader duplicates + +**Workspace Structure:** +``` +workspaces/main/: 31 shaders (workspace-specific) +workspaces/test/: 19 shaders (workspace-specific) +common/: 20 shaders (shared) +Total unique: 70 shaders (vs 106 before) +``` + +--- + +## Benefits + +1. **Single Source of Truth:** Common shaders in one location +2. **No Duplication:** Bug fixes apply everywhere automatically +3. **Clear Separation:** Common vs workspace-specific code +4. **Size Optimization:** Important for 64k target +5. **Maintainability:** Easier to understand and modify +6. **Workspace Isolation:** Each workspace still self-contained for specific content + +--- + +## Migration Notes + +**For New Workspaces:** +1. Create `workspaces/new_workspace/` with subdirs: `music/`, `weights/`, `obj/`, `shaders/` +2. Reference common shaders: `../../common/shaders/...` +3. Add workspace-specific shaders to local `shaders/` +4. Update `workspace.cfg`: `asset_dirs = ["music/", "weights/", "obj/"]` + +**For Common Shader Changes:** +- Edit files in `common/shaders/` +- Changes apply to all workspaces immediately +- Run full build to verify all workspaces + +--- + +## Documentation Updated + +- `doc/WORKSPACE_SYSTEM.md` - New structure reflected +- `doc/SHADER_REUSE_INVESTIGATION.md` - Implementation status +- `doc/PROJECT_CONTEXT.md` - Current project state +- `doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md` - This document diff --git a/doc/SHADER_REUSE_INVESTIGATION.md b/doc/SHADER_REUSE_INVESTIGATION.md index 4f83f1d..0eb36f9 100644 --- a/doc/SHADER_REUSE_INVESTIGATION.md +++ b/doc/SHADER_REUSE_INVESTIGATION.md @@ -1,6 +1,25 @@ # Shader Code Reuse Investigation -## Current State +## ✅ Implementation Status + +**Date:** February 13, 2026 +**Solution:** Option 1 - Shared Common Directory +**Status:** IMPLEMENTED + +**Results:** +- Created `common/shaders/` with 20 shared shader files +- Eliminated 36 duplicate files across workspaces +- Asset references use `../../common/shaders/...` +- Enhanced asset_packer with filesystem path normalization +- Build passes, all tests pass + +See `doc/FILE_HIERARCHY_CLEANUP_2026-02-13.md` for full details. + +--- + +## Investigation (Historical) + +### Current State ### Duplication Analysis - **36 duplicate shader files** between main and test workspaces diff --git a/doc/WORKSPACE_SYSTEM.md b/doc/WORKSPACE_SYSTEM.md index e8e2615..9362da6 100644 --- a/doc/WORKSPACE_SYSTEM.md +++ b/doc/WORKSPACE_SYSTEM.md @@ -46,7 +46,7 @@ This makes it hard to: 4. Shaders hard to categorize (shared vs demo-specific) 5. Adding new demos requires scattered changes -## Proposed Structure +## Current Structure (Implemented) ### Workspace Directory Layout @@ -57,10 +57,10 @@ This makes it hard to: timeline.seq # Visual effects music.track # Audio patterns assets.txt # Asset list - /assets/ - /music/*.spec # Demo-specific audio - /meshes/*.obj # Demo-specific meshes - /shaders/ # Demo-specific shaders + /music/*.spec # Audio samples + /weights/*.bin # CNN binary weights + /obj/*.obj # 3D models + /shaders/ # Workspace-specific shaders only custom_effect.wgsl /test/ # Test/validation demo @@ -68,23 +68,41 @@ This makes it hard to: timeline.seq music.track assets.txt - /assets/ + /music/ + /weights/ + /obj/ /shaders/ - /experiments/ # Experimental demos - /demo2024_revision/ - workspace.cfg - ... - -/assets/common/ # Shared resources +/common/ # Shared resources /shaders/ /math/ # Shared math utilities common_utils.wgsl - sdf.wgsl - /common_uniforms/ # Shared uniforms - common.wgsl - /audio/ - standard_drums.spec # Shared samples + noise.wgsl + sdf_shapes.wgsl + sdf_utils.wgsl + /render/ # Shared rendering helpers + lighting_utils.wgsl + scene_query_bvh.wgsl + scene_query_linear.wgsl + shadows.wgsl + /compute/ # Shared compute shaders + gen_blend.wgsl + gen_grid.wgsl + gen_mask.wgsl + gen_noise.wgsl + gen_perlin.wgsl + common_uniforms.wgsl + lighting.wgsl + passthrough.wgsl + ray_box.wgsl + ray_triangle.wgsl + sdf_primitives.wgsl + skybox.wgsl + +/tools/ + originals/ # Source audio files (.wav, .aif) + test_demo.seq # Test demo timeline + test_demo.track # Test demo music ``` ### Workspace Configuration @@ -96,6 +114,14 @@ name = "Main Demo" description = "Production 64k demo" version = "1.0" +[build] +target = "demo64k" +timeline = "timeline.seq" +music = "music.track" +assets = "assets.txt" +asset_dirs = ["music/", "weights/", "obj/"] +shader_dirs = ["shaders/"] + [build] # Output binary name target = "demo64k" -- cgit v1.2.3