Fluentd Forward Output

Defines a Fluentd/Bit forward Unix socket output to write TenXObject instance and template field values.

For an example configuration, see the FluentBit output.

Configuration

To configure the Fluentd Forward output module, Edit these settings.

Below is the default configuration from: forward/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/output/event/forward/config.yaml
• Within cloned repo
  ./pipelines/run/output/event/forward/config.yaml

Download

# 🔟❎ 'run' fluent forward output configuration

# Configure fluent forward outputs to write TenXObject instance and template field values.
# To learn more see https://doc.log10x.com/run/output/event/forward

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/forward

# ============================== File Options =================================

# Multiple forward outputs can be defined below
outputForward:

  # --------------------------- 10x Encoding Output ---------------------------

    # Encode TenXObjects: https://doc.log10x.com/run/transform/#template-encoding

    # 'hostAddress' specifies the host address of target fluent instance
  - hostAddress: $=TenXEnv.get("TENX_REMOTE_FORWARD_HOST", "127.0.0.1")

    # 'hostPort' specifies the host port of target fluent instance
    hostPort: $=parseInt(TenXEnv.get("TENX_REMOTE_FORWARD_PORT", 24224))

    # 'filter' sets a JavaScript expression that TenXObjects must evaluate as truthy to write to the output.
    #  To learn more see https://doc.log10x.com/run/output/regulate/#filter-expressions
    filter: null

    # 'fields' defines the fields to write for each TenXObject sent to this output.
    #  To learn more see https://doc.log10x.com/run/output/stream/#outputfields
    fields:
      - fullText # https://doc.log10x.com/api/js/#fulltext-string

    # 'encodeType' specifies how 'fields' are encoded to output. Possible values: [json,delimited].
    #  To learn more see https://doc.log10x.com/run/output/event/forward/#outputforwardencodetype
    encodeType: delimited

    # 'writeTemplates' controls whether to write new templates for TenXObjects written to this output.
    writeTemplates: true

    # 'tagValue' specifies a default tag for forwarded events without a source tag
    tagValue: ""

Options

Specify the options below to configure multiple Fluentd Forward output:

Name	Description	Category
outputForwardAddress	Location of target Unix domain socket for encoded object field values	Host
outputForwardHostAddress	Host address of target fluent instance	Host
outputForwardHostPort	Host port of target fluent instance	Host
outputForwardFields	List of TenXObject field names to encode	Encoding
outputForwardWriteObjects	If true, encode TenXObjects [outputForwardFields](https://doc.log10x.com/run/output/event/forward/#outputforwardfields) to this output	Encoding
outputForwardWriteTemplates	If true, write new TenXTemplates to this output	Encoding
outputForwardTagField	Field name to extract from an TenXObjects as its Fluentd 'tag' value	Encoding
outputForwardTagValue	Default tag value for TenXObjects	Encoding
outputForwardFilter	A JavaScript expression an TenXObject must evaluate as 'truthy' against to be written to this output	Encoding
outputForwardEncodeType	The output format in which to encode TenXObject values. Possible values:[json, delimited]	Encoding
outputForwardEncodeDelimiter	The delimiter placed between key and values pairs if 'outputForwardEncodeType' is 'delimited'	Encoding

Host

outputForwardAddress

Location of target Unix domain socket for encoded object field values.

Type	Default	Category
File		Host

Specifies a Unix domain socket address to write TenXObject instance and template field values. if specified, outputForwardHostAddress and outputForwardHostPort are ignored.

outputForwardHostAddress

Host address of target fluent instance.

Type	Default	Category
String	127.0.0.1	Host

Specifies a host address used together with outputForwardHostPort are ignored to encode object field values to. Ignored if outputForwardAddress is specified.

outputForwardHostPort

Host port of target fluent instance.

Type	Default	Category
Number	24224	Host

Specifies a host por used together with outputForwardHostAddress are ignored to encode object field values to. Ignored if outputForwardAddress is specified.

Encoding

outputForwardFields

List of TenXObject field names to encode.

Type	Default	Category
List	[]	Encoding

Specifies a list of TenXObject intrinsic/calculated/extracted field names to extract and write to the output.

outputForwardWriteObjects

If true, encode TenXObjects [outputForwardFields](https://doc.log10x.com/run/output/event/forward/#outputforwardfields) to this output.

Type	Default	Category
String	true	Encoding

Determine whether to write outputForwardFields values of an TenXObject passed to this output. This configuration is typically defined using a JavaScript expression.

To enable/disable the output depending on whether a launch argument, environment variable, or JVM -D option evaluates as truthy, use:

  outputFileWriteObjects: $=TenXEnv.get("myForwardAddress")

To learn more see TenXEnv.get.

outputForwardWriteTemplates

If true, write new TenXTemplates to this output.

Type	Default	Category
String	false	Encoding

Controls whether to write template values of TenXObjects passed to this output. The output will only emit TenXTemplates that the pipeline did not load at startup via the templateFile argument, and will only emit an TenXTemplate once based on its templateHash value.

outputForwardTagField

Field name to extract from an TenXObjects as its Fluentd 'tag' value.

Type	Default	Category
String	the TenXObject's 'source' intrinsic field value, if present	Encoding

Specifies an intrinsic/extracted/calculated TenXObject field name to use as the TenXObject's forward tag value. To learn more see Fluentd tagging.

outputForwardTagValue

Default tag value for TenXObjects.

Type	Default	Category
String	""	Encoding

Specifies an default value for TenXObject forward tag value, if the internal 'tag' and outputForwardTagField returned empty.

outputForwardFilter

A JavaScript expression an TenXObject must evaluate as 'truthy' against to be written to this output.

Type	Default	Category
String	""	Encoding

Specifies a JavaScript expression that an TenXObject must evaluate as truthy to write its instance/template field values to this output. For example, outputForwardFilter: this.timestamp will only emit timestamped TenXObjects.

outputForwardEncodeType

The output format in which to encode TenXObject values. Possible values:[json, delimited].

Type	Default	Category
String	delimited	Encoding

Specifies how outputForwardFields are encoded to a stream/event to this output. Possible values:

• json: formats names and values for the current TenXObject as: {"field":"str","field2":1, "field3": true}
• delimited: formats values for the current TenXObject as: "str",1,true The outputForwardEncodeDelimiter argument determines the separator to use.

outputForwardEncodeDelimiter

The delimiter placed between key and values pairs if 'outputForwardEncodeType' is 'delimited'.

Type	Default	Category
String		Encoding

Sets the delimiter to separate TenXObject field values when outputForwardEncodeType is delimited.

This module is defined in forward/module.yaml.