toonz2/library/shaders

Toonz Shader Fxs Manual

==========================================================

1. Introduction


Toonz 7.1 allows users to write new Fxs using GLSL (the
OpenGL Shading Language).

Shader Fx interfaces are read once at Toonz's startup,
but the underlying fx algorithm can be modified in
real time to ease the fx creation process.


Users reading these notes for the first time may want to
refer to the official GLSL guide at:

http://www.opengl.org/documentation/glsl/

Up-and-running examples of GLSL (fragment) shader programs
can be found at the GLSL sanbox gallery, from which some of
the provided examples are adapted from (requires a
WebGL-compatible web browser, such as Firefox or Google
Chrome):

http://glsl.heroku.com/

Further examples can be found at the beautiful gallery at:

https://www.shadertoy.com/

==========================================================

2. Requirements


The most recent version of your graphics drivers, as well
as a fairly recent graphics card.

Specifically, graphics drivers must support OpenGL 2.1,
Transform Feedback and Pixel Buffers (either as a built-in
feature or through extensions).

==========================================================

3. Limitations


Shader fxs are rendered on the GPU, meaning that they are
typically executed in a massively parallel fashion - ie fast.

However, since most systems only adopt one GPU, only one
Shader fx is allowed to be rendered at the same time.
This means that Shader Fxs do not take advantage of multiple
rendering threads in a Toonz rendering process like common
CPU-based fxs do.


Shader Fx are intended to apply a fragment shader on the
output surface for the fx. In other words, each output pixel
is processed separately using the supplied fragment shader
program.

This prevents the implementation of more complex output
patterns that span multiple pixels at the same time.

Furthermore, there is no way to specify intermediate buffer
objects to read or write data to - which is often a common
need when writing fxs.

==========================================================

3. Implementing a Shader Fx


In order to implement a shader fx it's currently necessary
to either create or edit the following files:

a. /config/current.txt

This file hosts the associations between fxs and
their parameters and the names displayed in the GUI
(which are not locale-dependent).

b. /profiles/layouts/fxs/fxs.lst

The list of fxs as displayed in the right-click
contextual menus like "Add Fx" or "Insert Fx"

c. /profiles/layouts/fxs/.xml

Parameters tabbing in the Fx Parameters Editor

d. /shaders/.xml

The Shader Fx interface.

e. The actual shader program files


Please, observe that the paths and names outside brackets
are mandatory.

Apart from point (d) and (e) discussed separately, it is best
to locate existing entries and emulate their behavior.
You can typically find related entries by searching "Shader"
in each file.

==========================================================

4. The Shader Interface File


The Shader Fx Interface file at (3.d) is an xml document that
defines the main properties of the fx.

Specifically:

a. Shader program files to be compiled at run-time

b. Input ports for the fx

c. Parameters

d. Restrictions to the class of world/output coordinates
transforms handled by the fx

The file is read once when Toonz starts, so any modification
will not be recognized until Toonz is restarted.


The complete recognized file structure is as follows:


// (4.a) The applied fragment shader

SHADER_myShaderName // Internal name of the fx (mandatory, a simple app-unique literal id)


"programs/myShader.frag" // The shader program file (3.e), relative to the
// path of the interface file.


// (4.b) - Only a *fixed* number of ports allowed
// A first port
"Source" // The displayed port name


// Second port
"Control"


// (4.a) Vertex shader used to acquire the geometry of
// input images. See (5.b).
SHADER_myShader_ports // The unique id for the vertex shader program (mandatory)


"programs/myShader_ports.vert"




// (4.a) Vertex shader used to calculate the fx's bbox.
// See (5.c).
SHADER_myShader_bbox


"programs/myShader_bbox.vert"



// (4.d) Optional, see (5.a)
isotropic // May be either 'any' (default) or 'isotropic'.
// Isotropic transforms exclude shears and non-uniform scales.

// (4.c)


float radius // Parameter declaration

// Additional Paramater attributes (can be omitted)
10 // The parameter default


0 20 // The parameter range


length // The parameter concept type - or, how it is represented
// by the Toonz GUI




float angle


angle_ui // Concepts of type _ui are editable in
// camera stand
"My Angle"





// Composite parameter concepts can be formed by 2 or
// more parameters
polar_ui


"My Polar Coordinates"

// List of involved parameters
radius


angle



----------------------------------------------------------

4.1. Parameter Declarations


Parameters are introduced by a declaration typically matching
the corresponding GLSL variable declaration.

The complete recognized list of supported parameter types is:

bool, float, vec2, int, ivec2, rgb, rgba


The 'rgb' and 'rgba' types map to GLSL 'vec3' and 'vec4'
variables respectively, but are displayed with the appropriate
color editors by Toonz - plus, the range of their components
automatically maps from [0, 255] in Toonz and the Shader
Interface file to [0.0, 1.0] in the corresponding shader program
files.

----------------------------------------------------------

4.2. Parameter Concepts


Parameter 'concepts' are additional parameter properties that
regard the way Toonz represents a certain parameter type.

For example, a 'float' variable type may either indicate
an angle, the length of a segment, a percentage value,
and more.

Fx writers may want to explicitly specify a parameter concept
for the following reasons:

a. Impose a measure to the parameter (e.g. degress, inches, %)

b. Make the parameter editable in camera-stand


The complete list of supported parameter concepts is the following:

percent - Displayed with the percentage '%' unit

length - Displayed in length units (inches, mm, cm, etc..)

angle - Displayed in angular units '�'

point - A vec2 displayed in length units

radius_ui - Like length, displaying a radius in camstand. May compose with a point (the center)

width_ui - Like length, displaying a vertical line width. May compose with the line's angle.

angle_ui - Like angle, displaying it in camstand

point_ui - Like point, in camstand

xy_ui - Composes two float types in a point

vector_ui - Composes two float types in an 'arrow'-like vector

polar_ui - Like vector_ui, from a length and an angle

size_ui - Displays a square indicating a size. May compose width and height in a rect.

quad_ui - Composes 4 points in a quadrilateral

rect_ui - Composes width, height, and the optional center point in a rect

==========================================================

5. Shader program files


A shader program file is a simple text file containing the
actual algorithms of a shader fx.

In the current implementation of Toonz Shader Fxs, there are
3 possible shader program files that need to be specified:

a. The main fragment shader program, responsible of
executing the code that actually renders the fx

b. An optional vertex shader program to calculate the
geometries of contents required from input ports

c. An optional vertex shader program to calculate the
bounding box of the fx output

----------------------------------------------------------

5.a. The 'MainProgram' Fragment Shader


The main program is in practice a standard GLSL fragment
shader - however, Toonz will provide it a set of additional
uniform input variables that must be addressed to correctly
compute the desired output.


The complete list of additional variables always supplied
by Toonz is:

uniform mat3 worldToOutput;
uniform mat3 outputToWorld;

These matrix variables describe the affine transforms mapping
output coordinates to Toonz's world coordinates, and vice-versa.

They include an additional coordinate as an OpenGL version-portable
way to perform translations by natural multiplication - transforming
a point is then done like:

vec2 worldPoint = (outputToWorld * vec3(outPoint, 1.0)).xy

Fx parameters are typically intended in world coordinates,
and should be adjusted through these transforms - for example,
a camstand-displayed radius value must be multiplied by the
'worldToOutput' scale factors in order to get the corresponding
value in output coordinates.

World/Output transforms may be restricted to a specific sub-class
of affine transforms by specifying so in the Shader Interface File.

Restricting to isotropic transforms may be useful to simplify
cases where angular values are taken into account, since this
transforms class preserves angles by allowing only uniform scales,
rotations and translations. Non-uniform scales and shears are
later applied by Toonz on the produced fx output if necessary.


In case input ports have been specified, we also have:

uniform sampler2D inputImage[n];
uniform mat3 outputToInput[n];
uniform mat3 inputToOutput[n];

The sampler variables correspond to the input content to the
fx. The matrix variables are the reference transforms from
output to input variables, and vice-versa.


Additional uniform variables corresponding to fx parameters
will also be supplied by Toonz. For example, if a "float radius"
parameter was specified, a corresponding

uniform float radius;

input variable will be provided to the program.


WARNING: Toonz requires that *output* colors must be
'premultiplied' - that is, common RGB components
(in the range [0, 1]) must be stored multiplied
by their alpha component.

----------------------------------------------------------

5.b. The optional 'PortsProgram' Vertex Shader


The shader program (b) is required in case an fx specifies
input ports, AND it needs to calculate some input content
in a different region than the required output.
It can be neglected otherwise.

For example, a blur fx requires that input contents outside
the required output rectangle are 'blurred in' it.


The 'PortsProgram' vertex shader is a one-shot shader
run by Toonz on a single dummy vertex - which uses
OpenGL 3.0's "Transform Feedback" extension to return a
set of predefined 'varying' output variables


The complete set of variables supplied by Toonz and required
in output by the program is:

uniform mat3 worldToOutput;
uniform mat3 outputToWorld;
uniform vec4 outputRect;

varying vec4 inputRect[portsCount];
varying vec4 worldToInput[portsCount];

The transforms are intended in the same way as (5.a).

The outputRect and inputRect[] variables store the
(left, bottom, right, top) rect components in output
and input coordinates respectively.

Parameter input variables are obviously also supplied.


WARNING: *All* the required output variables must be
declared AND filled with values.

There is no recognized default for them, and the
fx will (silently) fail to render if some are not
assigned.

----------------------------------------------------------

5.c. The optional 'BBoxProgram' Vertex Shader


Some fx may be able to restrict their opaque renderable
area inside a rect.

For example, blurring an image will 'blur out' the image
content by the specified blur radius. Beyond that, the fx
will render full transparent pixels. Thus, the bounding
box of the fx in this case will be calculated as the
input bounding box, enlarged by the blur radius.

The default output bounding box is assumed to be infinite;
if that is the case, the BBoxProgram can be omitted.


Fx writers may want to supply an explicit program to
calculate the bounding box of the fx, given its input
bounding boxes. This is be useful in Toonz's rendering
pipeline because the software is then allowed to
restrict memory allocation (and fxs calculations)
for the output image to said output bounding box, resulting
in less memory consumption and increased speed.


The complete set of variables supplied by Toonz and required
in output by the program is:

uniform vec4 infiniteRect;
uniform vec4 inputBBox[portsCount];

varying vec4 outputBBox;

The infiniteRect variable should be used to identify both
input and output infinite bboxes.