1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
|
# Sequence v2 System Documentation
**Status:** Phase 1 & 2 complete (Foundation + Compiler operational)
## Quick Start
```bash
# Compile v2 timeline to C++
python3 tools/seq_compiler_v2.py input.seq --output generated/timeline_v2.cc
# Final mode (flattened, no vtables)
python3 tools/seq_compiler_v2.py input.seq --output timeline_v2.cc --flatten
```
## v2 Timeline Syntax
```
# BPM 120
SEQUENCE 0.0 0 "sequence_name"
# Node declarations (optional, auto-inferred as u8x4_norm)
NODE gbuf_normal f16x8
NODE depth depth24
# Asset dependencies
ASSET shader_blur
# Effect routing: inputs... -> outputs...
EFFECT + DeferredRender source depth -> gbuf_normal gbuf_pos 0.0 10.0
EFFECT + Compose gbuf_normal gbuf_pos -> composed 0.0 10.0
EFFECT + Blur composed -> sink 0.0 10.0
```
**Node types:** `u8x4_norm`, `f32x4`, `f16x8`, `depth24`, `compute_f32`
**Priority modifiers:** `+` (increment), `=` (same), `-` (decrement)
---
This document describes the high-level ideas for an improved Sequence + Effects system "version 2".
## Goal
* more flexibility and less boilerplate
* fully configurable by text (.seq)
* no compatibility with previous version of sequences
* sequence can be stacked (with priority) and arranged in a timeline to create a demo
## Structure
### Sequence
A 'Sequence' unit consist of:
* A predeclared set of named assets and Effects that are used during the sequence
* a start-time, end-time
* a globally-visible set of internal time-dependent params (UniformsSequenceParams) derived from GlobalParams
* a globally-visible set of fixed params values (UniformsSequenceStaticParams) used to configure effects
* an input in RGBAu8 format ("Source"). Black by default (RGB=0,0,0,255). Buffer format: u8x4_norm (see below)
* an output in RGBAu8 format ("Sink"). This output goes to the next sequence in stack or to physical display
* the sink of a Sequence is the source of the next Sequence in the stack, or goes to screen if it's the last sequence in stack
* three programatic sections: 'Preprocess', 'EffectFlow' (time-ordered set of effects), 'Postprocess'
* a set of internal buffers (Nodes = compute buffers or textures, used as input or render targets by effects stack)
* this Nodes are visible to the final post-process effect
* Node persists during the whole sequence
* at compile time, one can detect which Node can actually be ping-pong buffers
* each Sequence's preprocess and postprocess is unique and attached to the sequence
### Effects
The effect unit consists in:
* An 'Effect' within the Sequence uses the UniformsSequenceParams and GlobalParams to update its internal state
* Receives updated InputNode from the previous effects (InputNode = buffer, textures, g-buffer, ...)
* processes them with a shader or c++ code (3d engine, etc.)
* fills its OutputNode, passed to the next effect or the final postprocessing step
* uses the sequence's assets and params only
* if needed an effect predefines an alias of one of its internal buffer as its declared 'postprocess_output' OutputNode
* by default, postprocess_output = sequence RGBA Source
* Effect's are not attached to a particular Sequence, and can be used in any Sequence
### Preprocess:
* the preprocessing step sets up the local UniformsSequenceParams (prepare timers, objects' transforms, camera position, fills Uniform buffers, etc.)
* it's a single function attached to the Sequence unit (virtual method?)
### Postprocess:
* the post-process unique (virtual) function is responsible for the final assembly into OuputNode
* it can take any internal Node as input (most likely the last effects' output) and produces its final Sink content
## How it works
### initialization
at initialization time, the sequence knows from compile-time:
* the list of assets it needs to be ready at start time
* the list of effects needed, their input nodes, their output node
* the list of internal textures, buffer, render target, depth-buffer it will need for its stack effect to work
* which intern Node buffers can actually be ping-pong buffers, to optimize resource
* generally speaking, the lifetime of a buffer during a frame is known at compile time and can be optimized
### Flow
* preprocess: prepare internal state at time t: UniformsSequenceParams
* for each effects:
** bind the InputNode nodes (previous effect buffers, compute buffers, etc.)
** runs the compute, vertex and fragment passes
** fills its output Nodes
* process: turns its pre-declared input Nodes into its unique Sink node, ready for next sequence, or display
## Requirement
* unified code to flow the engine
* textual description of the sequence/effects arrangement (similar to the .seq)
* update the HTML editor tool
* unified description of buffer, texture and compute buffers at compilation time
** example, of a sequence's internal Node declaration:
"NODE Name1 u8x16_norm"
"NODE Name2 f32x4"
"NODE Name3 f16x8"
"Node Name4 u8x4_norm" <- Source/Sink of a sequence, four bytes
* each effect
** declares its requirement as input Node.
** declares its output Node
** can access the C++ version and WebGPU versions of GlobalParams, UniformsSequenceParams and UniformsSequenceStaticParams
** are not attached to a particular sequence, but can be used in many
* validation and optimization at compile time (generating c++ code) by seq_compiler
* compilation can generate boilerplate preprocess() / postprocess() functions
* no backward compatibility needed. Sequence v1 can go.
* the HTML Editor overall code, look and functionalities need to be preserved though. Just adapted to v2.
** basically the sequence box will need to show input Node and output Node and have an editable property panel for these
** same for effects: ability to edit their input Node and output Node names
* a lot of v1 effects can be deleted (solarized, chroma aberration, etc.): they will be implemented ad-hoc in postprocess() with a single macro call within the final shader
* need a minimal migration plan for code.
|
