summaryrefslogtreecommitdiff
path: root/doc/CONTEXT_MAINTENANCE.md
diff options
context:
space:
mode:
authorskal <pascal.massimino@gmail.com>2026-02-07 15:34:46 +0100
committerskal <pascal.massimino@gmail.com>2026-02-07 15:34:46 +0100
commita9a151a4fdcd46f4737abe98c654c1ec619ef425 (patch)
tree33d0bf801552cd21a9dd008d32c79cbe6c4c3e72 /doc/CONTEXT_MAINTENANCE.md
parenta0dd0a27c4d6831fb2fb5ad81283f36512ef16ef (diff)
docs: Reorganize documentation with tiered hierarchy for context optimization
Major documentation reorganization to reduce AI agent context size by ~58% and establish sustainable maintenance practices. ## File Moves (Root → doc/) - Move COMPLETED.md (new), HANDOFF*.md, *_ANALYSIS.md, *_SUMMARY.md to doc/ - Keep only 5 essential files in root: CLAUDE.md, GEMINI.md, PROJECT_CONTEXT.md, TODO.md, README.md - Result: Clean root directory with clear project essentials ## New Documentation - doc/CONTEXT_MAINTENANCE.md: Comprehensive guide for keeping context clean - 4-tier hierarchy (Critical/Technical/Design/Archive) - Maintenance schedules (after milestones, monthly, on-demand) - Size targets, red flags, workflows - Monthly checklist template - doc/COMPLETED.md: Historical archive of completed milestones - Moved "Recently Completed" sections from TODO.md and PROJECT_CONTEXT.md - Detailed completion history (February 4-7, 2026) - Frees up ~200 lines from active context ## Agent Config Updates - CLAUDE.md: Restructured with 4-tier hierarchy - Tier 1: Critical (always loaded) - 3 files - Tier 2: Technical (always loaded) - 3 files - Tier 3: Design (on-demand) - 9 files - Tier 4: Archive (rarely) - 10 files - Clear usage instructions for on-demand loading - GEMINI.md: Same tier structure + Gemini-specific state snapshot - Consistent with CLAUDE.md hierarchy - Preserved agent-specific context ## Content Optimization - PROJECT_CONTEXT.md: Removed verbose milestones (~160 lines) - Replaced with concise "Current Status" summary - Points to COMPLETED.md for history - TODO.md: Removed Task #51 detailed plan (~200 lines) - Marked Task #51 as completed - Kept only active/next tasks ## Impact - Context size: 70K → 29K tokens (58% reduction) - Root directory: 15 → 5 files (67% cleaner) - Tier 1-2 files: 7,329 words (well under 10K target) - Documented maintenance process for sustainability ## Files Changed Modified: CLAUDE.md, GEMINI.md, PROJECT_CONTEXT.md, TODO.md New: doc/COMPLETED.md, doc/CONTEXT_MAINTENANCE.md Moved: 10 technical docs from root to doc/
Diffstat (limited to 'doc/CONTEXT_MAINTENANCE.md')
-rw-r--r--doc/CONTEXT_MAINTENANCE.md332
1 files changed, 332 insertions, 0 deletions
diff --git a/doc/CONTEXT_MAINTENANCE.md b/doc/CONTEXT_MAINTENANCE.md
new file mode 100644
index 0000000..c228cd7
--- /dev/null
+++ b/doc/CONTEXT_MAINTENANCE.md
@@ -0,0 +1,332 @@
+# Context Maintenance Guide
+
+This document describes how to maintain clean, efficient context for AI agents (Claude, Gemini) working on this project.
+
+## Philosophy
+
+**Goal**: Keep AI context focused on **actionable, current information** while archiving **historical details** for reference.
+
+**Principle**: The context should answer "What am I working on **now**?" not "What did we do 6 months ago?"
+
+---
+
+## File Organization Hierarchy
+
+### Tier 1: Critical (Always Loaded)
+These files are essential for every session and should remain lean:
+
+- **CLAUDE.md** / **GEMINI.md** - Agent configuration (keep to ~30 lines)
+- **PROJECT_CONTEXT.md** - High-level project overview (keep to ~200 lines)
+ - Current status summary (not detailed milestones)
+ - Architectural overview (brief)
+ - Next priorities (not detailed plans)
+- **TODO.md** - Active tasks only (keep to ~300 lines)
+ - Current and next tasks
+ - Brief attack plans (not 200-line implementation details)
+ - Mark completed tasks and reference COMPLETED.md
+- **README.md** - Quick start guide
+
+### Tier 2: Technical Reference (Always Loaded)
+Essential for daily work:
+
+- **doc/HOWTO.md** - Usage instructions, common commands
+- **doc/CONTRIBUTING.md** - Coding guidelines, commit policies
+- **doc/AI_RULES.md** - Agent-specific rules
+
+### Tier 3: Design Docs (Load On-Demand)
+Load these only when working on specific subsystems:
+
+- **doc/ASSET_SYSTEM.md** - Asset pipeline details
+- **doc/BUILD.md** - Build system details
+- **doc/3D.md** - 3D rendering architecture
+- **doc/SPEC_EDITOR.md** - Spectral editor design
+- **doc/TRACKER.md** - Audio tracker system
+- **doc/PROCEDURAL.md** - Procedural generation
+- **doc/ANALYSIS_VARIABLE_TEMPO_V2.md** - Tempo system analysis
+
+### Tier 4: Historical Archive (Load Rarely)
+Reference material for understanding past decisions:
+
+- **doc/COMPLETED.md** - Detailed completion history
+- **doc/HANDOFF*.md** - Agent handoff documents
+- **doc/*_ANALYSIS.md** - Technical investigations
+- **doc/*_SUMMARY.md** - Task/feature summaries
+
+---
+
+## Maintenance Schedule
+
+### After Major Milestone (Immediately)
+**Trigger**: Completed a multi-week task or significant feature
+
+**Actions**:
+1. ✅ Move detailed completion notes from TODO.md to COMPLETED.md
+2. ✅ Update PROJECT_CONTEXT.md "Current Status" (remove old "Recently Completed")
+3. ✅ Archive detailed implementation plans from TODO.md
+4. ✅ Update "Next Up" section in PROJECT_CONTEXT.md
+
+**Time**: 10-15 minutes
+
+### Monthly Review (1st of Month)
+**Trigger**: Calendar-based
+
+**Actions**:
+1. ✅ Archive tasks completed >30 days ago from TODO.md → COMPLETED.md
+2. ✅ Review PROJECT_CONTEXT.md for outdated "Recently Completed" items
+3. ✅ Check for temporary analysis docs in root directory → move to doc/
+4. ✅ Verify CLAUDE.md/GEMINI.md only load Tier 1-2 files by default
+
+**Time**: 15-20 minutes
+
+### On-Demand (When Context Feels Slow)
+**Trigger**: AI responses taking longer, context bloat suspected
+
+**Actions**:
+1. ✅ Run token analysis: `wc -w CLAUDE.md PROJECT_CONTEXT.md TODO.md doc/*.md | sort -rn`
+2. ✅ Check for oversized files (>500 lines in Tier 1-2)
+3. ✅ Aggressively archive verbose sections
+4. ✅ Consider splitting large design docs
+
+**Time**: 30-45 minutes
+
+---
+
+## What to Archive
+
+### ✅ ALWAYS Archive:
+- Detailed implementation plans for completed tasks (>100 lines)
+- Verbose milestone descriptions with step-by-step breakdowns
+- Analysis documents for resolved issues
+- Deprecated design docs (keep note in main doc pointing to archive)
+- Old handoff documents (>1 month old)
+
+### ⚠️ SUMMARIZE Before Archiving:
+- Major architectural decisions (keep 2-3 line summary)
+- Critical bug resolutions (keep root cause + solution)
+- Performance optimizations (keep before/after metrics)
+
+### ❌ NEVER Archive:
+- Current task priorities
+- Essential coding guidelines (CONTRIBUTING.md, AI_RULES.md)
+- Quick-start instructions (README.md, HOWTO.md)
+- Active architectural constraints
+
+---
+
+## Red Flags (Time to Clean Up)
+
+### File Size Red Flags:
+- PROJECT_CONTEXT.md >500 lines
+- TODO.md >800 lines
+- Any Tier 1-2 file >1000 lines
+
+### Content Red Flags:
+- "Recently Completed" section >100 lines in PROJECT_CONTEXT.md
+- TODO.md contains tasks marked `[DONE]` for >30 days
+- Detailed implementation plans (>50 lines) for completed tasks in TODO.md
+- Analysis documents in root directory (should be in doc/)
+
+### Context Red Flags:
+- AI responses noticeably slower
+- Token usage >50K on initial load
+- Repeated "let me read..." for basic project info
+
+---
+
+## Archiving Workflow
+
+### 1. Identify Content to Archive
+```bash
+# Find large files
+find . -name "*.md" -type f -exec wc -l {} + | sort -rn | head -10
+
+# Find old completed tasks in TODO.md
+grep -n "\[x\].*202[0-9]-[0-9][0-9]-[0-9][0-9]" TODO.md
+```
+
+### 2. Move to COMPLETED.md
+```markdown
+## Recently Completed (YYYY-MM-DD)
+
+- [x] **Task Name**: Brief summary
+ - Key points (2-3 bullets max)
+ - Result: One-line outcome
+```
+
+### 3. Update Source Files
+- Replace verbose sections with: `**Note**: Detailed history in doc/COMPLETED.md`
+- Update cross-references
+- Keep high-level summaries
+
+### 4. Update Agent Configs
+Update CLAUDE.md and GEMINI.md if files moved:
+```markdown
+# Design docs (load on-demand when needed)
+# Use: "read @doc/COMPLETED.md" for detailed history
+```
+
+### 5. Verify
+```bash
+# Check context size
+cat CLAUDE.md PROJECT_CONTEXT.md TODO.md | wc -w
+# Target: <10,000 words for Tier 1 files
+
+# Verify no broken references
+grep -r "@doc/" CLAUDE.md GEMINI.md PROJECT_CONTEXT.md TODO.md
+```
+
+---
+
+## Common Mistakes to Avoid
+
+### ❌ Don't:
+- Archive current priorities or active tasks
+- Remove essential architectural context
+- Break cross-references without updating
+- Move files without updating CLAUDE.md/GEMINI.md
+- Delete content (archive instead)
+- Keep outdated "Recently Completed" sections >3 months
+
+### ✅ Do:
+- Preserve key technical insights (in summaries)
+- Update file references after moving docs
+- Keep audit trail (note where content moved)
+- Maintain searchable keywords (for finding archives)
+- Regular small cleanups (better than big purges)
+
+---
+
+## Quick Reference: File Moves
+
+When moving files from root to doc/:
+
+```bash
+# 1. Move file (use git mv if tracked)
+git mv OLDFILE.md doc/OLDFILE.md
+
+# 2. Update CLAUDE.md
+# Change: @OLDFILE.md
+# To: # Use: "read @doc/OLDFILE.md" when needed
+
+# 3. Update GEMINI.md (same as above)
+
+# 4. Update cross-references in other files
+grep -r "OLDFILE.md" *.md doc/*.md
+
+# 5. Commit with clear message
+git commit -m "docs: Move OLDFILE.md to doc/ for better organization"
+```
+
+---
+
+## Context Size Targets
+
+### Optimal Targets:
+- **CLAUDE.md**: <30 lines
+- **PROJECT_CONTEXT.md**: <300 lines
+- **TODO.md**: <500 lines
+- **Total Tier 1-2**: <10,000 words (~40K tokens)
+
+### Warning Thresholds:
+- **PROJECT_CONTEXT.md**: >500 lines
+- **TODO.md**: >800 lines
+- **Total Tier 1-2**: >15,000 words (~60K tokens)
+
+### Critical (Clean Up Immediately):
+- **PROJECT_CONTEXT.md**: >800 lines
+- **TODO.md**: >1200 lines
+- **Total Tier 1-2**: >20,000 words (~80K tokens)
+
+---
+
+## Examples
+
+### Good: Lean TODO.md Entry
+```markdown
+- [ ] **Task #72: Implement Audio Compression**
+ - Goal: Reduce .spec file sizes by 50%
+ - Approach: Quantize to uint16_t, use RLE encoding
+ - See: doc/AUDIO_COMPRESSION_PLAN.md for details
+```
+
+### Bad: Verbose TODO.md Entry
+```markdown
+- [ ] **Task #72: Implement Audio Compression**
+ - Goal: Reduce .spec file sizes by 50%
+ - Background: Currently using float32 which is wasteful...
+ - Research: Investigated 5 compression algorithms...
+ - Step 1: Create compression interface
+ - Define CompressorBase class
+ - Add compress() and decompress() methods
+ - Implement factory pattern for algorithm selection
+ - Step 2: Implement RLE compressor
+ - [100+ more lines of detailed implementation]
+```
+
+### Good: Lean PROJECT_CONTEXT.md Status
+```markdown
+### Current Status
+- Audio: Stable, 93% test coverage, variable tempo support
+- 3D: Hybrid SDF/raster, BVH acceleration, physics system
+- Shaders: Modular WGSL, comprehensive compilation tests
+```
+
+### Bad: Verbose PROJECT_CONTEXT.md Status
+```markdown
+### Recently Completed
+
+#### Audio Peak Measurement (February 7, 2026)
+- **Real-Time Peak Fix**: Fixed critical bug where... [20 lines]
+- **Peak Decay Optimization**: Changed decay from 0.95 to 0.7... [15 lines]
+- **SilentBackend**: Created test backend with... [25 lines]
+- [200+ more lines of detailed milestones]
+```
+
+---
+
+## Questions?
+
+If unsure whether to archive something, ask:
+
+1. **Is this about work completed >30 days ago?** → Archive
+2. **Is this a detailed implementation plan?** → Archive (keep 2-line summary)
+3. **Is this needed for next week's work?** → Keep
+4. **Is this >50 lines of historical detail?** → Archive
+5. **Would a new contributor need this immediately?** → Keep if yes
+
+When in doubt, **archive with a pointer**. Better to have clean context and occasionally load an archive than to bloat every session.
+
+---
+
+## Maintenance Checklist
+
+Copy this for monthly reviews:
+
+```markdown
+## Context Maintenance - [Month YYYY]
+
+### File Sizes (before)
+- [ ] PROJECT_CONTEXT.md: ___ lines
+- [ ] TODO.md: ___ lines
+- [ ] Tier 1-2 total: ___ words
+
+### Actions Taken
+- [ ] Archived tasks >30 days old to COMPLETED.md
+- [ ] Moved temp analysis docs to doc/
+- [ ] Updated PROJECT_CONTEXT.md current status
+- [ ] Simplified TODO.md task descriptions
+- [ ] Verified CLAUDE.md/GEMINI.md config
+- [ ] Updated cross-references
+
+### File Sizes (after)
+- [ ] PROJECT_CONTEXT.md: ___ lines (target: <300)
+- [ ] TODO.md: ___ lines (target: <500)
+- [ ] Tier 1-2 total: ___ words (target: <10K)
+
+### Notes
+[Any issues encountered, files moved, etc.]
+```
+
+---
+
+*Last updated: February 7, 2026*