diff options
Diffstat (limited to 'tools/timeline_editor/ROADMAP.md')
| -rw-r--r-- | tools/timeline_editor/ROADMAP.md | 734 |
1 files changed, 43 insertions, 691 deletions
diff --git a/tools/timeline_editor/ROADMAP.md b/tools/timeline_editor/ROADMAP.md index 836c6f6..a3ef268 100644 --- a/tools/timeline_editor/ROADMAP.md +++ b/tools/timeline_editor/ROADMAP.md @@ -1,708 +1,60 @@ -# Timeline Editor - Feature Roadmap +# Timeline Editor - Roadmap -This document outlines planned enhancements for the interactive timeline editor. +## Completed ---- +- ✅ **V2 Format Support** - NODE declarations, arrow syntax, buffer chain visualization (2026-02-17) +- ✅ **Core editing** - Drag/drop, properties panel, add/delete +- ✅ **Beat timing** - Quantize grid, snap-to-beat, BPM slider +- ✅ **Audio playback** - WAV visualization, seek, auto-expand +- ✅ **CPU load** - Effect density visualization -## Known Bugs (High Priority) +## Future Work -### Audio Playback Integration Issues +### Phase 1: Productivity (Medium Priority) -1. ~~**Audio waveform doesn't scale with zoom nor follow timeline**~~ ✅ FIXED - - Waveform now correctly syncs with timeline at all zoom levels +**Undo/Redo** (10-12h) +- Command pattern for mutations +- Ctrl+Z/Ctrl+Shift+Z +- History stack (50 commands) -2. ~~**Playback indicator doesn't follow zoom and height issues**~~ ✅ FIXED - - Red bar now dynamically spans full timeline height - - Position correctly accounts for pixelsPerSecond +**Copy/Paste** (6-8h) +- Ctrl+C/V/D for duplication +- Paste with timing adjustment -3. ~~**Sequences overlap timeline at scroll origin**~~ ✅ FIXED - - Proper padding prevents overlap with timeline border +**Multi-Select** (12-15h) +- Ctrl/Shift/box select +- Batch operations (move, delete, priority) -4. ~~**Timeline and waveform should be fixed, not floating**~~ ✅ FIXED - - Sticky header stays at top during scroll +**Extended Shortcuts** (4-6h) +- Arrow keys for fine movement +- Shift+Arrow for resize +- E/N for create effect/sequence -5. ~~**Status indicator causes reflow**~~ ✅ FIXED - - Messages now fixed positioned at top-right - - No layout shift when appearing/disappearing +### Phase 2: Advanced (Low Priority) ---- +**Tracker Overlay** (12-15h) +- Visualize music.track patterns +- Show sample triggers (kick, snare) +- Align visual effects with audio events -## Phase 1: Core Editing Features (High Priority) +**Templates** (8-10h) +- Save/load effect patterns +- "Intro", "Beat Drop" presets -### 1.0 Sequence V2 Format Support ✅ COMPLETED -**Goal:** Support explicit node routing and DAG-based effect composition. +**Search/Filter** (6-8h) +- Filter by class/priority/time +- "Go to" dialog -**Features:** -- ✅ Parse NODE declarations with typed buffers (u8x4_norm, f32x4, f16x8, depth24, compute_f32) -- ✅ Parse arrow syntax: `EFFECT + ClassName input1 input2 -> output1 output2 start end` -- ✅ Serialize v2 format with NODE declarations -- ✅ Validate node references (only when explicit NODEs declared) -- ✅ Buffer chain visualization in properties panel and tooltips -- ✅ Node editor modal for adding/deleting node declarations -- ✅ Backward compatible with auto-inferred nodes (no NODE declarations) +**Auto-Save** (4-6h) +- LocalStorage every 30s +- Session restore -**Implementation:** -- New module: `timeline-format.js` (parse, serialize, validate, render) -- Updated data model: effects have `inputs[]` and `outputs[]` arrays -- Property panel: separate fields for inputs/outputs, buffer chain display -- Validation: blocks save if undeclared nodes referenced (when NODEs explicit) - -**Status:** ✅ COMPLETED (2026-02-17) -**Effort:** ~12 hours actual -**Impact:** Essential for v2 DAG architecture support - ---- - -### 1.1 Snap-to-Beat ⭐ Priority: HIGH -**Goal:** Enable precise musical timing by snapping items to beat boundaries. - -**Features:** -- Parse BPM from demo.seq file (`# BPM 120`) -- Calculate beat grid based on BPM (e.g., 120 BPM = 0.5s per beat) -- Visual beat markers on timeline (lighter gray lines between second markers) -- Toggle snap mode (keyboard shortcut: `S` key) -- When dragging, snap position to nearest beat -- Status indicator showing current snap mode and BPM - -**Implementation Details:** -```javascript -// Calculate beat positions -const beatsPerSecond = bpm / 60.0; -const secondsPerBeat = 60.0 / bpm; - -// Snap function -function snapToBeat(timeInSeconds) { - if (!snapEnabled) return timeInSeconds; - const beatIndex = Math.round(timeInSeconds * beatsPerSecond); - return beatIndex * secondsPerBeat; -} - -// Visual beat markers -for (let beat = 0; beat < maxBeats; beat++) { - const beatTime = beat * secondsPerBeat; - // Render subtle vertical line at beatTime -} -``` - -**UI Additions:** -- Checkbox: "☑️ Snap to Beat (S)" in zoom controls -- Display: "BPM: 120 | Beat: 0.5s" in status bar -- Visual: Dashed vertical lines at beat boundaries - -**Effort:** 4-6 hours -**Dependencies:** None -**Impact:** Makes musical timing much easier - ---- - -### 1.2 Create New Effects ⭐ Priority: HIGH -**Goal:** Add effects to sequences without manual property editing. - -**Features:** -- "➕ Add Effect" button (enabled when sequence is selected) -- Modal dialog or inline form for effect creation: - - Effect class name (dropdown or autocomplete) - - Start time (relative to sequence) - - Duration - - Priority (default: 0) - - Constructor arguments (optional text field) -- Double-click on sequence to add effect at that position -- Right-click context menu option - -**Implementation Details:** -```javascript -function showEffectCreationDialog(sequenceIndex, timeHint = 0) { - const dialog = document.createElement('div'); - dialog.className = 'modal-dialog'; - dialog.innerHTML = ` - <h3>New Effect</h3> - <label>Class Name: <input id="newEffectClass" list="effectClasses"></label> - <datalist id="effectClasses"> - <option>FlashEffect</option> - <option>FadeEffect</option> - <option>HeptagonEffect</option> - <!-- Add more from known effects --> - </datalist> - <label>Start Time: <input type="number" id="newEffectStart" value="${timeHint}"></label> - <label>Duration: <input type="number" id="newEffectDuration" value="1.0"></label> - <label>Priority: <input type="number" id="newEffectPriority" value="0"></label> - <label>Arguments: <input type="text" id="newEffectArgs"></label> - <button onclick="createEffect()">Create</button> - <button onclick="closeDialog()">Cancel</button> - `; - document.body.appendChild(dialog); -} - -function createEffect() { - const effect = { - type: 'effect', - className: document.getElementById('newEffectClass').value, - startTime: parseFloat(document.getElementById('newEffectStart').value), - endTime: parseFloat(document.getElementById('newEffectStart').value) + - parseFloat(document.getElementById('newEffectDuration').value), - priority: parseInt(document.getElementById('newEffectPriority').value), - args: document.getElementById('newEffectArgs').value - }; - - sequences[selectedSequenceIndex].effects.push(effect); - closeDialog(); - renderTimeline(); -} -``` - -**UI Additions:** -- Button: "➕ Add Effect to Sequence" (appears when sequence selected) -- Modal dialog with form -- Context menu on right-click -- Keyboard shortcut: `E` key when sequence selected - -**Effort:** 6-8 hours -**Dependencies:** Need list of available effect classes -**Impact:** Essential for productive editing - ---- - -### 1.2b Priority Editing UI ⭐ Priority: HIGH -**Goal:** Enable visual editing of sequence and effect priorities. - -**Features:** -- **Sequence Priority**: Edit absolute priority (0-9 for scene, 10+ for post-processing) -- **Effect Priority Modifier**: Toggle between `+`, `=`, `-` modifiers -- **Visual Priority Indicators**: Color-code or label items by priority level -- **Z-order visualization**: Show stacking order visually - -**UI Design:** - -**For Sequences:** -``` -Properties Panel: -┌─────────────────────────────┐ -│ Sequence Priority │ -│ ┌─────────────────────────┐ │ -│ │ Priority: [0] ▲▼ │ │ -│ └─────────────────────────┘ │ -│ (0-9: Scene, 10+: Post-FX) │ -└─────────────────────────────┘ - -Visual indicator on timeline: -[0] Sequence 1 (dark blue) -[1] Sequence 2 (medium blue) -[5] Sequence 3 (light blue) -``` - -**For Effects:** -``` -Properties Panel: -┌─────────────────────────────┐ -│ Effect Priority Modifier │ -│ ┌─────────────────────────┐ │ -│ │ ○ + Increment (normal) │ │ -│ │ ○ = Same as previous │ │ -│ │ ● - Decrement (bg) │ │ -│ └─────────────────────────┘ │ -│ Computed priority: -1 │ -└─────────────────────────────┘ - -Visual indicator on timeline: -[-1] FlashCubeEffect (darker gray, background) -[0] FlashEffect (normal gray) -[1] FadeEffect (lighter gray) -``` - -**Implementation:** -```javascript -// Sequence priority editor -function renderSequencePriorityEditor(seq) { - return ` - <div class="property-group"> - <label>Sequence Priority</label> - <div class="priority-control"> - <input type="number" id="seqPriority" value="${seq.priority}" - min="0" max="20" step="1"> - <button onclick="adjustPriority(-1)">▼</button> - <button onclick="adjustPriority(+1)">▲</button> - </div> - <small>0-9: Scene effects | 10+: Post-processing</small> - </div> - `; -} - -// Effect priority modifier editor -function renderEffectPriorityEditor(effect) { - return ` - <div class="property-group"> - <label>Priority Modifier</label> - <div class="priority-modifier-group"> - <label> - <input type="radio" name="priorityMod" value="+" - ${effect.priorityModifier === '+' ? 'checked' : ''}> - + Increment (normal) - </label> - <label> - <input type="radio" name="priorityMod" value="=" - ${effect.priorityModifier === '=' ? 'checked' : ''}> - = Same as previous - </label> - <label> - <input type="radio" name="priorityMod" value="-" - ${effect.priorityModifier === '-' ? 'checked' : ''}> - - Decrement (background) - </label> - </div> - <small>Computed priority: ${effect.priority}</small> - </div> - `; -} - -// Recalculate effect priorities after modifier change -function recalculatePriorities(sequence) { - let currentPriority = -1; - for (const effect of sequence.effects) { - if (effect.priorityModifier === '+') currentPriority++; - else if (effect.priorityModifier === '-') currentPriority--; - // '=' keeps current priority - effect.priority = currentPriority; - } -} -``` - -**Visual Priority Indicators (CSS):** -```css -/* Sequence priority colors */ -.sequence[data-priority="0"] { border-color: #264f78; } /* Dark blue */ -.sequence[data-priority="1"] { border-color: #0e639c; } /* Medium blue */ -.sequence[data-priority="2"] { border-color: #1177bb; } /* Light blue */ - -/* Effect priority badge */ -.effect::before { - content: "[" attr(data-priority) "]"; - position: absolute; - top: 2px; - left: 2px; - font-size: 10px; - background: rgba(0,0,0,0.5); - padding: 2px 4px; - border-radius: 2px; -} - -/* Background effects (negative priority) */ -.effect[data-priority^="-"] { - opacity: 0.7; - background: #2d2d30; -} -``` - -**Validation:** -- Warn if sequence priorities conflict (same time + same priority) -- Show warning if effect priority order doesn't match visual stacking -- Suggest reordering effects if priority sequence is non-contiguous - -**Effort:** 6-8 hours -**Priority:** HIGH (essential for proper z-order control) -**Dependencies:** None -**Impact:** Enables precise control over render order - ---- - -### 1.3 Overlap Detection & Warnings ⭐ Priority: MEDIUM -**Goal:** Visualize and warn about timeline conflicts. - -**Features:** -- Detect sequence overlaps (same priority, overlapping time ranges) -- Detect effect overlaps within same sequence -- Visual indicators: - - Red border around overlapping items - - Warning icon (⚠️) on overlapping items - - Tooltip explaining conflict -- Warnings panel listing all conflicts -- Filter timeline to show only conflicts - -**Detection Logic:** -```javascript -function detectOverlaps() { - const conflicts = []; - - // Check sequence overlaps (same priority) - for (let i = 0; i < sequences.length; i++) { - for (let j = i + 1; j < sequences.length; j++) { - const seq1 = sequences[i]; - const seq2 = sequences[j]; - - if (seq1.priority !== seq2.priority) continue; - - const seq1End = seq1.startTime + getSequenceDuration(seq1); - const seq2End = seq2.startTime + getSequenceDuration(seq2); - - if (seq1.startTime < seq2End && seq2.startTime < seq1End) { - conflicts.push({ - type: 'sequence', - items: [i, j], - severity: 'warning', - message: `Sequences ${i+1} and ${j+1} overlap with same priority ${seq1.priority}` - }); - } - } - } - - // Check effect overlaps within sequences - sequences.forEach((seq, seqIdx) => { - for (let i = 0; i < seq.effects.length; i++) { - for (let j = i + 1; j < seq.effects.length; j++) { - const eff1 = seq.effects[i]; - const eff2 = seq.effects[j]; - - if (eff1.priority !== eff2.priority) continue; - - if (eff1.startTime < eff2.endTime && eff2.startTime < eff1.endTime) { - conflicts.push({ - type: 'effect', - sequenceIndex: seqIdx, - items: [i, j], - severity: 'info', - message: `Effects overlap in sequence ${seqIdx+1}` - }); - } - } - } - }); - - return conflicts; -} -``` - -**Visual Indicators:** -```css -.sequence.conflict { - border-color: #f48771; - box-shadow: 0 0 10px rgba(244, 135, 113, 0.5); -} - -.effect.conflict { - border-color: #f48771; -} - -.conflict-warning { - position: absolute; - top: 2px; - right: 2px; - font-size: 14px; -} -``` - -**UI Additions:** -- Panel: "⚠️ Conflicts" (collapsible, shows list of warnings) -- Button: "🔍 Show Only Conflicts" (filters view) -- Visual: Red borders and warning icons on conflicting items -- Tooltip: Hover over warning icon for details - -**Effort:** 8-10 hours -**Dependencies:** None -**Impact:** Prevents rendering issues and z-order bugs - ---- - -## Phase 2: Productivity Enhancements (Medium Priority) - -### 2.1 Undo/Redo System -**Goal:** Allow users to experiment without fear of breaking things. - -**Implementation:** -- Command pattern for all mutations -- History stack (max 50 commands) -- Keyboard shortcuts: `Ctrl+Z` (undo), `Ctrl+Shift+Z` (redo) -- Buttons in toolbar - -**Effort:** 10-12 hours -**Priority:** MEDIUM - ---- - -### 2.2 Multi-Select & Batch Operations -**Goal:** Edit multiple items at once. - -**Features:** -- Click + Ctrl/Cmd to add to selection -- Click + Shift for range selection -- Drag box to select multiple -- Batch operations: - - Move all selected items together - - Delete all selected - - Adjust priority of all selected - - Duplicate selection - -**Effort:** 12-15 hours -**Priority:** MEDIUM - ---- - -### 2.3 Copy/Paste & Duplication -**Goal:** Reuse sequences and effects quickly. - -**Features:** -- `Ctrl+C` to copy selected item(s) -- `Ctrl+V` to paste at current time -- `Ctrl+D` to duplicate in place -- Paste adjusts timing to avoid overlaps (optional) - -**Effort:** 6-8 hours -**Priority:** MEDIUM - ---- - -### 2.4 Keyboard Shortcuts & Navigation -**Goal:** Power users can work without mouse. - -**Shortcuts:** -- `Space`: Play/pause preview indicator (if implemented) -- `Arrow Keys`: Move selected item (fine adjustment) -- `Shift+Arrow`: Resize selected effect duration -- `Delete` / `Backspace`: Delete selected -- `S`: Toggle snap-to-beat -- `E`: Create effect in selected sequence -- `N`: New sequence -- `1-9`: Jump to sequence 1-9 -- `/`: Focus search/filter box - -**Effort:** 4-6 hours -**Priority:** MEDIUM - ---- - -### 2.5 Music.track Visualization & Overlay -**Goal:** Display tracker patterns and samples alongside timeline for precise alignment. - -**Features:** -- Load `music.track` file (parse tracker score format) -- Render tracker patterns as horizontal bars above/below sequences -- Display sample triggers (kick, snare, bass notes) as vertical markers -- Color-code different sample types (drums, bass, melody) -- Show pattern names and beat divisions -- Toggle visibility (checkbox: "Show Tracker Score") -- Sync horizontal scale with timeline (pixels per second) -- Help users align visual sequences with audio events - -**Implementation Details:** -```javascript -// Parse music.track file format (similar to demo.seq parser) -function parseTrackerScore(content) { - // Extract patterns, events, sample triggers - // Return structured data: { patterns: [...], events: [...] } -} - -// Render tracker overlay on canvas or DOM elements -function renderTrackerOverlay() { - // Draw pattern bars - // Draw sample trigger markers (vertical lines) - // Label with sample names (KICK_1, BASS_E2, etc.) -} -``` - -**Visual Design:** -- Tracker patterns: Semi-transparent colored bars (green/yellow/purple) -- Sample triggers: Thin vertical lines with labels -- Pattern names: Small text above bars -- Positioned above waveform, below time markers - -**Use Case:** -- Load `music.track` to see exactly where drums/bass/melody trigger -- Align visual effect sequences with musical events -- Verify timing matches tracker score before testing in demo - -**Effort:** 12-15 hours -**Priority:** MEDIUM -**Dependencies:** Requires `music.track` file format specification - ---- - -## Phase 3: Advanced Features (Low Priority) - -### 3.1 Timeline Playback Indicator -**Goal:** Visualize current playback position. - -**Features:** -- Vertical red line showing current time -- Play/pause button -- Time advances at real-time speed (or adjustable) -- Click timeline to seek -- Does NOT render effects (just shows position) - -**Effort:** 4-6 hours -**Priority:** LOW -**Note:** This is ONLY a position indicator, not a preview renderer - ---- - -### 3.2 Templates & Presets -**Goal:** Quick-start with common patterns. - -**Features:** -- Template library (e.g., "Intro Sequence", "Beat Drop", "Outro") -- Save custom templates -- Drag template onto timeline to instantiate -- Templates include sequences + effects with relative timing - -**Effort:** 8-10 hours -**Priority:** LOW - ---- - -### 3.3 Search & Filter -**Goal:** Navigate large timelines quickly. - -**Features:** -- Search bar: Filter by effect class name -- Filter by priority -- Filter by time range -- Highlight matching items -- "Go to" dialog (jump to specific time) - -**Effort:** 6-8 hours -**Priority:** LOW - ---- - -### 3.4 Timeline Comparison View -**Goal:** Compare two versions of the timeline. - -**Features:** -- Load two .seq files side by side -- Diff view showing additions/deletions/changes -- Color-coded: green = added, red = removed, yellow = modified -- Merge changes from one timeline to another - -**Effort:** 15-20 hours -**Priority:** LOW - ---- - -### 3.5 Export Formats -**Goal:** Support alternative output formats. - -**Features:** -- Export to JSON (for programmatic processing) -- Export to CSV (for spreadsheet analysis) -- Export to Markdown table (for documentation) -- Import from JSON/CSV - -**Effort:** 6-8 hours -**Priority:** LOW - ---- - -### 3.6 Auto-Save & Local Storage -**Goal:** Never lose work. - -**Features:** -- Auto-save to browser localStorage every 30 seconds -- Restore last session on load -- Warning if leaving page with unsaved changes -- Session history (last 5 sessions) - -**Effort:** 4-6 hours -**Priority:** LOW - ---- - -## Implementation Priority - -**Immediate (Next 1-2 weeks):** -1. ✅ Snap-to-Beat (4-6h) - Makes editing much easier -2. ✅ Create New Effects (6-8h) - Essential for productivity -3. ✅ Priority Editing UI (6-8h) - Essential for z-order control - -**Short-term (Next 1-2 months):** -4. ✅ Overlap Detection (8-10h) - Prevents bugs -5. ✅ Undo/Redo (10-12h) - Safety net for experimentation -6. ✅ Copy/Paste (6-8h) - Common workflow - -**Medium-term (Next 3-6 months):** -6. Multi-Select (12-15h) -7. Keyboard Shortcuts (4-6h) -8. Timeline Playback Indicator (4-6h) - -**Long-term (Nice to have):** -9. Templates (8-10h) -10. Search/Filter (6-8h) -11. Timeline Comparison (15-20h) -12. Export Formats (6-8h) -13. Auto-Save (4-6h) - ---- - -## Total Effort Estimates - -**Phase 1 (Core):** 28-36 hours (was 22-28, added priority editing) -**Phase 2 (Productivity):** 38-51 hours -**Phase 3 (Advanced):** 51-74 hours - -**Total:** ~117-161 hours for full feature set - ---- - -## Development Approach - -**Incremental Rollout:** -- Each feature should be a separate commit -- Keep the single-file HTML structure for simplicity -- Add features in priority order -- Test with real demo.seq files after each feature - -**Code Organization (when file gets too large):** -- Consider splitting into: - - `index.html` (UI structure + styles) - - `editor.js` (main logic) - - `parser.js` (seq file parsing) - - `timeline.js` (rendering) - - `commands.js` (undo/redo system) - -**Testing Strategy:** -- Manual testing with assets/demo.seq -- Test edge cases: empty files, single sequence, many overlaps -- Browser compatibility: Chrome, Firefox, Safari -- No automated tests (keep it simple for now) - ---- +**Export** (6-8h) +- JSON/CSV/Markdown formats ## Design Principles -1. **Keep it simple:** Avoid over-engineering -2. **No dependencies:** Stay pure HTML/JS/CSS -3. **Instant feedback:** All operations should feel immediate -4. **Visual clarity:** Color-coding, icons, clear labels -5. **Forgiving UX:** Undo/redo, non-destructive operations -6. **Offline-first:** No network required - ---- - -## Questions for Future Consideration - -1. **Effect Class Validation:** - - Should we validate effect class names against known effects? - - Source of truth: Parse C++ code? Hardcoded list? User-maintained? - -2. **Priority System:** - - Current system uses integers (-1, 0, 1, 2...) - - Should we add named priority levels? (Background, Normal, Foreground, Top) - -3. **BPM Changes:** - - What if BPM changes mid-timeline? - - Support for tempo changes? - -4. **Collaboration:** - - Should multiple people be able to edit the same timeline? - - Real-time collaboration? (probably overkill) - -5. **Preview Integration:** - - Should we ever integrate visual preview? - - Current answer: NO (out of scope, would require WebAssembly + full engine) - ---- - -## Success Metrics - -**v1.0 (Current):** ✅ Basic editing works -**v1.1 (Phase 1):** Snap-to-beat + effect creation = productive editing -**v2.0 (Phase 2):** Undo/redo + multi-select = professional tool -**v3.0 (Phase 3):** Advanced features = power user delight - -**Goal:** Make timeline editing 10x faster than manual text editing. +- Keep it simple (no over-engineering) +- Pure ES6 (no dependencies) +- Instant feedback +- Offline-first |