SciTech-AV-Video-DVP(Digital Video Processing)-CV/CG-ffmpeg-libavfilter:数字过滤器-

This document describes filters, sources, and sinks provided by the libavfilter library.

Filtergraph Syntax

• Filters in the same linear chain are separated by commas,
• distinct linear chains of filters are separated by semicolons.
• The points where the linear chains join are labelled by names enclosed in square brackets.
• Some filters take in input a list of parameters:
  • they are specified after the filter name and an equal sign,
  • they are separated from each other by a colon.
• There exist so-called:
  • source filters that do not have an audio/video input,
  • sink filters that will not have audio/video output.

Filtering Introduction

Filtering in FFmpeg is enabled through the libavfilter library.

In libavfilter, a filter can have multiple inputs and multiple outputs.

To illustrate the sorts of things that are possible, we consider the following filtergraph.

Example of filtergraph:

                [main]
input --> split ---------------------> overlay --> output
            |                             ^
            |[tmp]                  [flip]|
            +-----> crop --> vflip -------+

This filtergraph splits the input stream in two streams, then,

sends one stream through the crop filter and the vflip filter,

before merging it back with the other stream by overlaying it on top.

You can use the following command to achieve this:

ffmpeg -i INPUT -vf "split [main][tmp]; [tmp] crop=iw:ih/2:0:0, vflip [flip]; [main][flip] overlay=0:H/2" OUTPUT

The result will be that the top half of the video is mirrored onto the bottom half of the output video.

In our example,

• crop,vflip are in one linear chain,
• split and overlay are separately in another.
• the split filter generates two outputs that are associated to the labels [main] and [tmp].
• the stream sent to the second output of split, labelled as [tmp], is processed through the crop filter.
• the crop filter crops away the lower half part of the video, and then vertically flipped.
• The overlay filter takes in input the first unchanged output of the split filter (which was labelled as [main]), and overlay on its lower half the output generated by the crop, vflip filterchain.

# 4.1 Filtergraph syntax
A filtergraph has a textual representation, which:
* is recognized by:
- the `-filter/-vf/-af` and `-filter_complex` **options in ffmpeg**
- `-vf/-af` in ffplay
* and by **the avfilter_graph_parse_ptr() function** `defined in` `libavfilter/avfilter.h`.

• A filterchain consists of a sequence of connected filters, each one connected to the previous one in the sequence.
• A filterchain is represented by a list of ","-separated filter descriptions.
• A filtergraph consists of a sequence of filterchains.
• A sequence of filterchains is represented by a list of ";"-separated filterchain descriptions.
• A filter is represented by a string of the form:
  • [in_link_1]...
  • [in_link_N]filter_name@id=arguments[out_link_1]...
  • [out_link_M]
• filter_name:
  • is the name of the filter class of which the described filter is an instance of,
  • and has to be the name of one of the filter classes registered in the program optionally followed by "@id".
  • The name of the filter class is optionally followed by** a string** "=arguments

arguments is a string which contains the parameters used to initialize the filter instance.

It may have one of two forms:

• A ':'-separated list of key=value pairs.
• A ':'-separated list of value.
  
  In this case, the keys are assumed to be the option names in the order they are declared. E.g.
  
  the fade filter declares three options in this order – type, start_frame and nb_frames.
  
  Then the parameter list in:0:30 means that:
  • the value in is assigned to the option type,
  • 0 to start_frame
  • 30 to nb_frames.

A ':'-separated list of mixed direct value and long key=value pairs.

• The direct value must precede the key=value pairs,
• and follow the same constraints order of the previous point.

The following key=value pairs can be set in any preferred order.

If the option value itself is a list of items (e.g. the format filter takes a list of pixel formats),

the items in the list are usually separated by '|'.

The list of arguments can be quoted using the character "'" as initial and ending mark,

and the character '\' for escaping the characters within the quoted text;

otherwise the argument string is considered terminated when the next special character (belonging to the set '[]=;,' ) is encountered.

A special syntax implemented in the ffmpeg CLI tool allows loading option values from files.

This is done be prepending a slash '/' to the option name, then

the supplied value is interpreted as a path from which the actual value is loaded. E.g.

ffmpeg -i <INPUT> -vf drawtext=/text=/tmp/some_text <OUTPUT>

will load the text to be drawn from /tmp/some_text.

API users wishing to implement a similar feature should use the avfilter_graph_segment_*() functions together with custom IO code.

The name and arguments of the filter are optionally preceded and followed by a list of link labels.

A link label allows one to name a link and associate it to a filter output or input pad.

the preceding labels in_link_1 ... in_link_N, are associated to the filter input pads,

the following labels out_link_1 ... out_link_M, are associated to the filter output pads.

When two link labels with the same name are found in the filtergraph,

a link between the corresponding input and output pad is created.

If an output pad is not labelled, it is linked by default to the first unlabelled input pad of the next filter in the filterchain.

For example in the filterchain

nullsrc, split[L1], [L2]overlay, nullsink

the split filter instance has two output pads, and the overlay filter instance two input pads.

The first output pad of split is labelled "L1",

the first input pad of overlay is labelled "L2",

and the second output pad of split is linked to the second input pad of overlay, which are both unlabelled.

In a filter description,

if the input label of the first filter is not specified, "in" is assumed;

if the output label of the last filter is not specified, "out" is assumed.

In a complete filterchain, all the unlabelled filter input and output pads must be connected.

A filtergraph is considered valid if all the filter input and output pads of all the filterchains are connected.

Leading and trailing whitespaces (space, tabs, or line feeds) separating tokens in the filtergraph specification are ignored.

This means that the filtergraph can be expressed using empty lines and spaces to improve redability.

For example, the filtergraph:

testsrc,split[L1],hflip[L2];[L1][L2] hstack

can be represented as:

testsrc,
split [L1], hflip [L2];

[L1][L2] hstack

Libavfilter will automatically insert scale filters where format conversion is required.

It is possible to specify swscale flags for those automatically inserted scalers by prepending sws_flags=flags; to the filtergraph description.

Here is a BNF description of the filtergraph syntax:

NAME             ::= sequence of alphanumeric characters and '_'
FILTER_NAME      ::= NAME["@"NAME]
LINKLABEL        ::= "[" NAME "]"
LINKLABELS       ::= LINKLABEL [LINKLABELS]
FILTER_ARGUMENTS ::= sequence of chars (possibly quoted)
FILTER           ::= [LINKLABELS] FILTER_NAME ["=" FILTER_ARGUMENTS] [LINKLABELS]
FILTERCHAIN      ::= FILTER [,FILTERCHAIN]
FILTERGRAPH      ::= [sws_flags=flags;] FILTERCHAIN [;FILTERGRAPH]

4.2 Notes on filtergraph escaping

Filtergraph description composition entails several levels of escaping. See (ffmpeg-utils)the "Quoting and escaping" section in the ffmpeg-utils(1) manual for more information about the employed escaping procedure.

A first level escaping affects the content of each filter option value, which may contain the special character : used to separate values, or one of the escaping characters '.

A second level escaping affects the whole filter description, which may contain the escaping characters ' or the special characters [],; used by the filtergraph description.

Finally, when you specify a filtergraph on a shell commandline, you need to perform a third level escaping for the shell special characters contained within it.

For example, consider the following string to be embedded in the drawtext filter description text value:

this is a 'string': may contain one, or more, special characters

This string contains the ' special escaping character, and the : special character, so it needs to be escaped in this way:

text=this is a 'string': may contain one, or more, special characters

A second level of escaping is required when embedding the filter description in a filtergraph description, in order to escape all the filtergraph special characters. Thus the example above becomes:

drawtext=text=this is a \'string\'\: may contain one, or more, special characters

(note that in addition to the ' escaping special characters, also , needs to be escaped).

Finally an additional level of escaping is needed when writing the filtergraph description in a shell command, which depends on the escaping rules of the adopted shell. For example, assuming that \ is special and needs to be escaped with another , the previous string will finally result in:

-vf "drawtext=text=this is a \\\'string\\\'\\: may contain one\, or more\, special characters"

In order to avoid cumbersome escaping when using a commandline tool accepting a filter specification as input, it is advisable to avoid direct inclusion of the filter or options specification in the shell.

For example, in case of the drawtext filter, you might prefer to use the textfile option in place of text to specify the text to render.