diff options
| author | skal <pascal.massimino@gmail.com> | 2026-05-14 19:09:39 +0200 |
|---|---|---|
| committer | skal <pascal.massimino@gmail.com> | 2026-05-14 19:11:28 +0200 |
| commit | 6ef8f578817ee0134fd5867ca3b80590e3eb2368 (patch) | |
| tree | 5550607e5c4a16ca237bfa4430ac1ef1f5d80c5d /GEMINI.md | |
| parent | 4bcbe13dab5ffb64d93cc61956f07ee5168a84c9 (diff) | |
ans: order-0 rANS coder + WGSL asset compression
Adds src/util/ans.{h,cc}, a per-chunk-adaptive order-0 rANS entropy
coder. Decoder is always built; encoder is gated on ANS_ENABLE_ENCODER
(tools only). Both sides take an optional 256-entry initial_counts
table to seed the adaptive model.
The per-chunk initial state is (1 << kBits). Higher initial states
(e.g. with a signature packed into the upper bits) force a renorm-emit
at iter 0 that the decoder never consumes, corrupting multi-chunk
streams once stats become skewed.
Asset pipeline:
- AssetRecord gains 'compression' and 'uncompressed_size' fields.
- asset_packer scans every WGSL file to build a corpus-wide byte
histogram, then ANS-encodes each shader using that histogram as the
seed. Histogram and accessor are emitted alongside the asset table.
Round-trip verification runs at pack time for every compressed
asset; failures fall back to uncompressed storage.
- asset_manager decompresses on first GetAsset(), caches the
heap-allocated buffer, and DropAsset / ReloadAssetsFromFile free it
along with the procedural cache.
- Disk-load (dev) builds are unchanged: WGSL paths stay as filenames.
Tests:
- src/tests/util/test_ans.cc: roundtrip variants (empty, single byte,
single-symbol run, all-zeros, random uniform/skewed, repeated ASCII),
seeded-vs-uniform compression, rejection of mismatched counts /
corruption / truncation, PeekUncompressedSize.
- 37/37 dev, 36/36 STRIP_ALL.
Compression observed: WGSL shaders shrink to ~0.62-0.71x in the main
workspace (81 of 105 assets qualify).
Docs:
- doc/ANS.md (new): algorithm, bitstream, API, asset pipeline
integration, compression numbers, limitations, tests.
- doc/ASSET_SYSTEM.md: new Compression section + updated technical
guarantees for compressed assets.
- doc/COMPLETED.md: May 2026 entry.
- PROJECT_CONTEXT.md: Build status line mentions WGSL ANS compression.
- CLAUDE.md, GEMINI.md: tier-3 build doc list includes ANS.md.
Diffstat (limited to 'GEMINI.md')
| -rw-r--r-- | GEMINI.md | 2 |
1 files changed, 1 insertions, 1 deletions
@@ -21,7 +21,7 @@ # CNN: @cnn_v1/docs/CNN_V1_EFFECT.md, @cnn_v2/docs/CNN_V2.md, @cnn_v2/docs/CNN_V2_BINARY_FORMAT.md # 3D/Graphics: @doc/3D.md, @doc/GPU_PROCEDURAL_PHASE4.md, @doc/MASKING_SYSTEM.md, @doc/SDF_EFFECT_GUIDE.md # Scene: @doc/SCENE_FORMAT.md, @doc/SEQUENCE.md, @doc/WORKSPACE_SYSTEM.md -# Build: @doc/ASSET_SYSTEM.md, @doc/BUILD.md, @doc/CMAKE_MODULES.md, @doc/SIZE_MEASUREMENT.md +# Build: @doc/ASSET_SYSTEM.md, @doc/ANS.md, @doc/BUILD.md, @doc/CMAKE_MODULES.md, @doc/SIZE_MEASUREMENT.md # Rendering: @doc/GEOM_BUFFER.md, @doc/SHADER_REUSE_INVESTIGATION.md, @doc/UNIFORM_BUFFER_GUIDELINES.md, @doc/WGPU_HELPERS.md, @doc/AUXILIARY_TEXTURE_INIT.md # Tools: @doc/test_demo_README.md, @doc/HOT_RELOAD.md, @doc/HEADLESS_MODE.md, @doc/RECIPE.md, @doc/TOOLS_REFERENCE.md # Arch: @doc/ARCHITECTURE.md, @doc/CODING_STYLE.md, @doc/BACKLOG.md, @doc/CONTEXT_MAINTENANCE.md |