# Sequence v2 System Documentation

This document describes the high-level ideas for an improved Sequence + Effects system "version 2".

## Goal

* more flexibility and less boilerplate
* fully configurable by text (.seq)
* no compatibility with previous version of sequences
* sequence can be stacked (with priority) and arranged in a timeline to create a demo

## Structure

### Sequence

A 'Sequence' unit consist of:
  * A predeclared set of named assets and Effects that are used during the sequence
  * a start-time, end-time
  * a globally-visible set of internal time-dependent params (UniformsSequenceParams) derived from GlobalParams
  * a globally-visible set of fixed params values (UniformsSequenceStaticParams) used to configure effects
  * an input in RGBAu8 format ("Source"). Black by default (RGB=0,0,0,255). Buffer format: u8x4_norm (see below)
  * an output in RGBAu8 format ("Sink"). This output goes to the next sequence in stack or to physical display
  * the sink of a Sequence is the source of the next Sequence in the stack, or goes to screen if it's the last sequence in stack
  * three programatic sections: 'Preprocess', 'EffectFlow' (time-ordered set of effects), 'Postprocess'
  * a set of internal buffers (Nodes = compute buffers or textures, used as input or render targets by effects stack)
  * this Nodes are visible to the final post-process effect
  * Node persists during the whole sequence
  * at compile time, one can detect which Node can actually be ping-pong buffers
  * each Sequence's preprocess and postprocess is unique and attached to the sequence

### Effects

The effect unit consists in:
  * An 'Effect' within the Sequence uses the UniformsSequenceParams and GlobalParams to update its internal state
  * Receives updated InputNode from the previous effects (InputNode = buffer, textures, g-buffer, ...)
  * processes them with a shader or c++ code (3d engine, etc.)
  * fills its OutputNode, passed to the next effect or the final postprocessing step
  * uses the sequence's assets and params only
  * if needed an effect predefines an alias of one of its internal buffer as its declared 'postprocess_output' OutputNode
  * by default, postprocess_output = sequence RGBA Source
  * Effect's are not attached to a particular Sequence, and can be used in any Sequence

### Preprocess:
  * the preprocessing step sets up the local UniformsSequenceParams (prepare timers, objects' transforms, camera position, fills Uniform buffers, etc.) 
  * it's a single function attached to the Sequence unit (virtual method?)

### Postprocess:
  * the post-process unique (virtual) function is responsible for the final assembly into OuputNode
  * it can take any internal Node as input (most likely the last effects' output) and produces its final Sink content

## How it works

### initialization

at initialization time, the sequence knows from compile-time:
  * the list of assets it needs to be ready at start time
  * the list of effects needed, their input nodes, their output node
  * the list of internal textures, buffer, render target, depth-buffer it will need for its stack effect to work
  * which intern Node buffers can actually be ping-pong buffers, to optimize resource
  * generally speaking, the lifetime of a buffer during a frame is known at compile time and can be optimized

### Flow
  * preprocess: prepare internal state at time t: UniformsSequenceParams
  * for each effects:
     ** bind the InputNode nodes (previous effect buffers, compute buffers, etc.)
     ** runs the compute, vertex and fragment passes
     ** fills its output Nodes
  * process: turns its pre-declared input Nodes into its unique Sink node, ready for next sequence, or display

## Requirement

  * unified code to flow the engine
  * textual description of the sequence/effects arrangement (similar to the .seq)
  * update the HTML editor tool
  * unified description of buffer, texture and compute buffers at compilation time
    ** example, of a sequence's internal Node declaration:
     "NODE Name1 u8x16_norm"
     "NODE Name2 f32x4"
     "NODE Name3 f16x8"
     "Node Name4 u8x4_norm" <- Source/Sink of a sequence, four bytes
  * each effect
    ** declares its requirement as input Node.
    ** declares its output Node
    ** can access the C++ version and WebGPU versions of GlobalParams, UniformsSequenceParams and UniformsSequenceStaticParams
    ** are not attached to a particular sequence, but can be used in many
  * validation and optimization at compile time (generating c++ code) by seq_compiler
  * compilation can generate boilerplate preprocess() / postprocess() functions  
  * no backward compatibility needed. Sequence v1 can go.
  * the HTML Editor overall code, look and functionalities need to be preserved though. Just adapted to v2.
    ** basically the sequence box will need to show input Node and output Node and have an editable property panel for these
    ** same for effects: ability to edit their input Node and output Node names
  * a lot of v1 effects can be deleted (solarized, chroma aberration, etc.): they will be implemented ad-hoc in postprocess() with a single macro call within the final shader
  * need a minimal migration plan for code.