1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
|
# Sequence v2 System Documentation
This document describes the high-level ideas for an improved Sequence + Effects system "version 2".
## Goal
* more flexibility and less boilerplate
* fully configurable by text (.seq)
* no compatibility with previous version of sequences
* sequence can be stacked (with priority) and arranged in a timeline to create a demo
## Structure
### Sequence
A 'Sequence' unit consist of:
* A predeclared set of named assets and Effects that are used during the sequence
* a start-time, end-time
* a globally-visible set of internal time-dependent params (UniformsSequenceParams) derived from GlobalParams
* a globally-visible set of fixed params values (UniformsSequenceStaticParams) used to configure effects
* an input in RGBAu8 format ("Source"). Black by default (RGB=0,0,0,255). Buffer format: u8x4_norm (see below)
* an output in RGBAu8 format ("Sink"). This output goes to the next sequence in stack or to physical display
* the sink of a Sequence is the source of the next Sequence in the stack, or goes to screen if it's the last sequence in stack
* three programatic sections: 'Preprocess', 'EffectFlow' (time-ordered set of effects), 'Postprocess'
* a set of internal buffers (Nodes = compute buffers or textures, used as input or render targets by effects stack)
* this Nodes are visible to the final post-process effect
* Node persists during the whole sequence
* at compile time, one can detect which Node can actually be ping-pong buffers
* each Sequence's preprocess and postprocess is unique and attached to the sequence
### Effects
The effect unit consists in:
* An 'Effect' within the Sequence uses the UniformsSequenceParams and GlobalParams to update its internal state
* Receives updated InputNode from the previous effects (InputNode = buffer, textures, g-buffer, ...)
* processes them with a shader or c++ code (3d engine, etc.)
* fills its OutputNode, passed to the next effect or the final postprocessing step
* uses the sequence's assets and params only
* if needed an effect predefines an alias of one of its internal buffer as its declared 'postprocess_output' OutputNode
* by default, postprocess_output = sequence RGBA Source
* Effect's are not attached to a particular Sequence, and can be used in any Sequence
### Preprocess:
* the preprocessing step sets up the local UniformsSequenceParams (prepare timers, objects' transforms, camera position, fills Uniform buffers, etc.)
* it's a single function attached to the Sequence unit (virtual method?)
### Postprocess:
* the post-process unique (virtual) function is responsible for the final assembly into OuputNode
* it can take any internal Node as input (most likely the last effects' output) and produces its final Sink content
## How it works
### initialization
at initialization time, the sequence knows from compile-time:
* the list of assets it needs to be ready at start time
* the list of effects needed, their input nodes, their output node
* the list of internal textures, buffer, render target, depth-buffer it will need for its stack effect to work
* which intern Node buffers can actually be ping-pong buffers, to optimize resource
* generally speaking, the lifetime of a buffer during a frame is known at compile time and can be optimized
### Flow
* preprocess: prepare internal state at time t: UniformsSequenceParams
* for each effects:
** bind the InputNode nodes (previous effect buffers, compute buffers, etc.)
** runs the compute, vertex and fragment passes
** fills its output Nodes
* process: turns its pre-declared input Nodes into its unique Sink node, ready for next sequence, or display
## Requirement
* unified code to flow the engine
* textual description of the sequence/effects arrangement (similar to the .seq)
* update the HTML editor tool
* unified description of buffer, texture and compute buffers at compilation time
** example, of a sequence's internal Node declaration:
"NODE Name1 u8x16_norm"
"NODE Name2 f32x4"
"NODE Name3 f16x8"
"Node Name4 u8x4_norm" <- Source/Sink of a sequence, four bytes
* each effect
** declares its requirement as input Node.
** declares its output Node
** can access the C++ version and WebGPU versions of GlobalParams, UniformsSequenceParams and UniformsSequenceStaticParams
** are not attached to a particular sequence, but can be used in many
* validation and optimization at compile time (generating c++ code) by seq_compiler
* compilation can generate boilerplate preprocess() / postprocess() functions
* no backward compatibility needed. Sequence v1 can go.
* the HTML Editor overall code, look and functionalities need to be preserved though. Just adapted to v2.
** basically the sequence box will need to show input Node and output Node and have an editable property panel for these
** same for effects: ability to edit their input Node and output Node names
* a lot of v1 effects can be deleted (solarized, chroma aberration, etc.): they will be implemented ad-hoc in postprocess() with a single macro call within the final shader
* need a minimal migration plan for code.
|
