# Float32Array Subarray Optimization Analysis ## Background `Float32Array.subarray(start, end)` creates a **view** on the same underlying buffer without copying data: - **Memory**: No allocation, shares underlying ArrayBuffer - **Speed**: O(1) operation vs O(N) copy - **Lifetime**: View is valid as long as parent array exists ## Current State ### ✅ Already Optimized (Good Examples) **Location 1: Mini Spectrum Viewer (line 1423)** ```javascript draw_spectrum(state.referenceSpectrogram.subarray(pos, pos + size), true); ``` ✅ Correct usage - extracting single frame for display **Location 2: Procedural Spectrum Viewer (line 1438)** ```javascript draw_spectrum(fullProcSpec.subarray(pos, pos + size), false); ``` ✅ Correct usage - extracting single frame for display ### ❌ Optimization Opportunities ## Optimization 1: IDCT Frame Extraction (HIGH IMPACT) **Location**: `spectrogramToAudio()` function (line 1477-1480) **Current Code:** ```javascript // Extract frame (no windowing - window is only for analysis, not synthesis) const frame = new Float32Array(dctSize); for (let b = 0; b < dctSize; b++) { frame[b] = spectrogram[frameIdx * dctSize + b]; } // IDCT const timeFrame = javascript_idct_512(frame); ``` **Analysis:** - Creates new Float32Array for each frame - Copies 512 floats per frame - For typical audio (16s @ 32kHz): ~500 frames - **Total**: 500 allocations + 256K float copies **Why Safe to Optimize:** - `javascript_idct_fft()` only **reads** input (verified in dct.js:166-206) - Input array is not modified - Parent spectrogram remains valid throughout loop **Optimized Code:** ```javascript // Extract frame directly (no copy needed - IDCT doesn't modify input) const pos = frameIdx * dctSize; const frame = spectrogram.subarray(pos, pos + dctSize); // IDCT const timeFrame = javascript_idct_512(frame); ``` **Impact:** - Eliminates 500 allocations - Eliminates 256K float copies - ~30-50% faster audio synthesis - Reduced GC pressure ## Optimization 2: DCT Frame Windowing (MEDIUM COMPLEXITY) **Location**: `audioToSpectrogram()` function (line 364-371) **Current Code:** ```javascript const frame = new Float32Array(DCT_SIZE); // Extract windowed frame for (let i = 0; i < DCT_SIZE; i++) { if (frameStart + i < audioData.length) { frame[i] = audioData[frameStart + i] * window[i]; } } // Compute DCT (forward transform) const dctCoeffs = javascript_dct_512(frame); ``` **Analysis:** - Creates new Float32Array for each frame - Must apply window function (element-wise multiplication) - For typical audio (16s @ 32kHz): ~1000 frames - **Total**: 1000 allocations + windowing operation **Why NOT Straightforward:** - Cannot use direct subarray because we need to apply window - Window function modifies values: `audioData[i] * window[i]` - DCT reads input (verified in dct.js:122-160), doesn't modify **Optimization Options:** ### Option A: Reuse Single Buffer (RECOMMENDED) ```javascript // Allocate once outside loop const frameBuffer = new Float32Array(DCT_SIZE); for (let frameIdx = 0; frameIdx < numFrames; frameIdx++) { const frameStart = frameIdx * hopSize; // Reuse buffer (windowing operation required) for (let i = 0; i < DCT_SIZE; i++) { if (frameStart + i < audioData.length) { frameBuffer[i] = audioData[frameStart + i] * window[i]; } else { frameBuffer[i] = 0; } } // Compute DCT const dctCoeffs = javascript_dct_512(frameBuffer); // Store in spectrogram for (let b = 0; b < DCT_SIZE; b++) { spectrogram[frameIdx * DCT_SIZE + b] = dctCoeffs[b]; } } ``` **Impact:** - Eliminates 999 of 1000 allocations (reuses 1 buffer) - Same windowing cost (unavoidable) - ~10-15% faster analysis - Reduced GC pressure ### Option B: Modify DCT to Accept Windowing Function ```javascript // More complex - would require DCT function signature change const dctCoeffs = javascript_dct_512_windowed( audioData.subarray(frameStart, frameStart + DCT_SIZE), window ); ``` **Not recommended**: More complex, breaks API compatibility ## Optimization 3: Curve Spectrogram Access (ALREADY OPTIMAL) **Location**: Curve class `getSpectrogram()` (curve.js) **Current Code:** ```javascript getSpectrogram() { if (!this.dirty && this.cachedSpectrogram) { return this.cachedSpectrogram; // Returns reference } this.cachedSpectrogram = this.computeSpectrogram(); this.dirty = false; return this.cachedSpectrogram; } ``` **Analysis:** ✅ Already optimal - returns direct reference to Float32Array ✅ No copying needed - consumers use subarray() or direct access ## Optimizations NOT Applicable ### DCT/IDCT Internal Arrays **Locations**: dct.js lines 126-127, 169-170 ```javascript const real = new Float32Array(N); const imag = new Float32Array(N); ``` **Why Not Optimized:** - FFT needs writable buffers (in-place algorithm) - Cannot use subarray() - would modify parent - Allocation is necessary ## Implementation Plan ### Phase 1: IDCT Frame Extraction (10 minutes) 1. Update `spectrogramToAudio()` (line 1477-1480) 2. Replace copy loop with `subarray()` 3. Test audio playback 4. Verify no regressions ### Phase 2: DCT Frame Buffer Reuse (15 minutes) 1. Update `audioToSpectrogram()` (line 362-379) 2. Allocate single buffer outside loop 3. Reuse buffer for windowing 4. Test .wav loading 5. Verify spectrogram quality ## Testing Checklist - [ ] Load .wav file - should work - [ ] Play procedural audio - should work - [ ] Play original audio - should work - [ ] Visual spectrogram rendering - should match - [ ] No JavaScript errors in console - [ ] Memory usage doesn't increase over time (no leaks) ## Expected Performance Gains **Audio Playback (16s @ 32kHz):** - Before: ~500 allocations, 256K float copies - After: 0 extra allocations, 0 copies - **Speedup**: 30-50% faster synthesis **WAV Analysis (16s @ 32kHz):** - Before: ~1000 allocations - After: 1 allocation (reused buffer) - **Speedup**: 10-15% faster analysis **Overall:** - Reduced GC pressure - Lower memory footprint - Smoother playback on slower machines ## Safety Verification **IDCT Optimization:** ✅ `javascript_idct_fft()` verified read-only (dct.js:175-186) ✅ Only reads `input[k]`, writes to separate `real`/`imag` buffers ✅ Safe to pass subarray **DCT Optimization:** ✅ `javascript_dct_fft()` verified read-only (dct.js:131-133) ✅ Only reads `input[2*i]` and `input[2*i+1]`, writes to separate buffers ✅ Safe to reuse buffer (not subarray due to windowing) ## References - MDN: TypedArray.subarray() - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/subarray - Performance: Subarray is O(1), copying is O(N) - Memory: Subarray shares ArrayBuffer, no allocation