Fluent Bit Event Output

Defines a Fluent Bit process that spawns and writes TenXObject instances and template field values.

This output integrates a lightweight, customizable Fluent Bit pipeline for shipping processed events to destinations such as Datadog, Splunk, Elasticsearch, and CloudWatch.

Use cases:

Lambda launcher: Ships optimized CloudWatch Logs events to analytics platforms. The Python handler pipes expanded events to the 10x Engine via stdin; Fluent Bit handles output shipping to the configured destination.
Storage Streamer: Ships query results from cloud storage (e.g., S3) to log analyzers and time-series databases.

Configuration

To configure the Fluent Bit event output module, Edit these settings.

CloudwatchDatadogElasticsearchSplunkStdout

Below is the default configuration from: fluentbit/cloudwatch.yaml.

Edit Online

# 🔟❎ 'run' Fluent Bit output configuration for events stream output to AWS Cloudwatch

# Launch a Fluent Bit process into whose stdin pipe to write TenXObjects field values and TenXTemplates
# To learn more see https://doc.log10x.com/run/output/event/process/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/fluentbit


# ============================ Fluent Bit Options =============================

# Multiple Fluent Bit outputs can be defined below

fluentbitOut:

    # ------------------------------ General ----------------------------------

    # 'writeObjects' controls whether to write TenXObject field values to output.
  - writeObjects: true

    # 'writeTemplates' controls whether to write new template for TenXObjects written to
    #  this output. To learn more about TenXTemplates, see: https://doc.log10x.com/run/template
    writeTemplates: false

    # ------------------------------ Process ----------------------------------

    # 'command' specifies the command to execute to launch the Fluent Bit sub-process.
    command: /opt/fluent-bit/bin/fluent-bit

    # 'inputArgs' specifies list of values to pass as startup arguments to the spawned Fluent Bit sub-process
    inputArgs:
      # Sets the Fluent Bit input to read from stdin and route TenXTemplates
      # See https://github.com/log-10x/config/blob/main/pipelines/run/output/event/fluentbit/fluentbit-10x-input.conf
      - -c
      - $=path("fluentbit-10x-input.conf")

    # Setting the Fluent Bit output to aws cloudwatch.
    # for full config options of the Fluent Bit cloudwatch plugin, see -
    # https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch
    #
    # NOTE - authenticating with AWS is handled by Fluent Bit using credentials providers
    # as defined in - https://docs.fluentbit.io/manual/administration/aws-credentials
    #
    outputArgs:
      ############### Events output ###############

      # Set event output to Cloudwatch
      - "-o"
      - "cloudwatch_logs"
      - "-m"
      - "input"

      # Setting destination within AWS Cloudwatch
      - "-p"
      - "log_group_name=my-tenx-logs-stream"
      - "-p"
      - "log_stream_prefix=events-"
      - "-p"
      - "auto_create_group=true"
      - "-p"
      - "$=\"region=\" + TenXEnv.get(\"AWS_REGION\", \"us-east-1\")"

      # Flush every second
      - "-f"
      - "1"

      ############### Templates output ###############

      # Set templates output to Cloudwatch
      - "-o"
      - "cloudwatch_logs"
      - "-m"
      - "templates"

      # Setting destination within AWS Cloudwatch
      - "-p"
      - "log_group_name=my-tenx-logs-stream"
      - "-p"
      - "log_stream_prefix=templates-"
      - "-p"
      - "auto_create_group=true"
      - "-p"
      - "$=\"region=\" + TenXEnv.get(\"AWS_REGION\", \"us-east-1\")"

      # Flush every second
      - "-f"
      - "1"

    # 'destroyMode' is set to 'cached' to allow a fluent-bit sub-process to
    #  to be reused across multiple 10x pipeline executions within a host JVM.
    destroyMode: cached

    # 'maxCacheSize' sets the size of the cache to used for caching fluent-bit instances
    maxCacheSize: 1

    # 'lazyLaunch' determines whether the fluent-bit instance will launch immediately
    # during the pipeline initialization, or only on-demand.
    lazyLaunch: true

    # ------------------------------ Encoding ---------------------------------

    # 'filter' specifies an expression that any TenXObject passed to this output must
    #  evaluate as truthy against to write its instance/template values to stdout.
    #  For example, 'filter: this.timestamp' will only emit timestamped objects.
    #  To learn more see: https://doc.log10x.com/api/js/#TenXBaseObject+timestamped
    filter: null

    #  This corresponds to the stdin format defined by fluent-bit in:
    #  https://docs.fluentbit.io/manual/pipeline/inputs/standard-input
    #  The 'text' argument below could be replaced by any list of intrinsic/calculated/extracted fields.
    #  The "" delimiter instructs 'joinFields' to produce JSON output.
    fields:  #
      - open="["
      - timestamp=dateFormat("+%s.%N")
      - separator=","
      - fields=joinFields("", "text")
      - close="]"

Below is the default configuration from: fluentbit/datadog.yaml (* Required Fields).

Edit Online

# 🔟❎ 'run' Fluent Bit output configuration for events stream output to Datadog

# Launch a Fluent Bit process into whose stdin pipe to write TenXObjects field values and TenXTemplates
# To learn more see http://doc.log10x.com/run/output/event/process/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/fluentbit


# ============================ Fluent Bit Options =============================

# Multiple Fluent Bit outputs can be defined below

fluentbitOut:

    # ------------------------------ General ----------------------------------

    # 'writeObjects' controls whether to write TenXObject field values to output.
  - writeObjects: true

    # 'writeTemplates' controls whether to write new template for TenXObjects written to
    #  this output. To learn more about TenXTemplates, see: http://doc.log10x.com/run/template
    writeTemplates: false

    # ------------------------------ Process ----------------------------------

    # 'command' specifies the command to execute to launch the Fluent Bit sub-process.
    command: /opt/fluent-bit/bin/fluent-bit

    # 'inputArgs' specifies list of values to pass as startup arguments to the spawned Fluent Bit sub-process
    inputArgs:
      # Sets the Fluent Bit input to read from stdin and route TenXTemplates
      # See https://github.com/log-10x/config/blob/main/pipelines/run/output/event/fluentbit/fluentbit-10x-input.conf
      - -c
      - $=path("fluentbit-10x-input.conf")

    # Setting the Fluent Bit output to Datadog
    # for full config options of the Fluent Bit datadog plugin, see -
    # https://docs.fluentbit.io/manual/pipeline/outputs/datadog
    #
    # Authentication requires the DD_API_KEY environment variable.
    # Set DD_SITE to your Datadog region (default: datadoghq.com).
    #
    outputArgs:
      ############### Events output ###############

      # Set event output to Datadog
      - "-o"
      - "datadog"
      - "-m"
      - "input"

      # Setting API key
      - "-p"
      - "$=\"apikey=\" + TenXEnv.get(\"DD_API_KEY\")" # (❗ REQUIRED)

      # Setting intake host (derived from DD_SITE, e.g. datadoghq.com, datadoghq.eu, us5.datadoghq.com)
      - "-p"
      - "$=\"Host=http-intake.logs.\" + TenXEnv.get(\"DD_SITE\", \"datadoghq.com\")"

      # TLS and compression
      - "-p"
      - "TLS=on"
      - "-p"
      - "compress=gzip"

      # Source and tag metadata
      - "-p"
      - "dd_source=log10x"
      - "-p"
      - "$=\"dd_tags=\" + TenXEnv.get(\"DD_TAGS\", \"\")"

      # Use 'text' field as the message body
      - "-p"
      - "dd_message_key=text"

      # Flush every second
      - "-f"
      - "1"

      ############### Templates output ###############

      # Set templates output to Datadog
      - "-o"
      - "datadog"
      - "-m"
      - "templates"

      # Setting API key
      - "-p"
      - "$=\"apikey=\" + TenXEnv.get(\"DD_API_KEY\")" # (❗ REQUIRED)

      # Setting intake host
      - "-p"
      - "$=\"Host=http-intake.logs.\" + TenXEnv.get(\"DD_SITE\", \"datadoghq.com\")"

      # TLS and compression
      - "-p"
      - "TLS=on"
      - "-p"
      - "compress=gzip"

      # Source and tag metadata for templates
      - "-p"
      - "dd_source=log10x-templates"
      - "-p"
      - "$=\"dd_tags=\" + TenXEnv.get(\"DD_TAGS\", \"\")"

      # Flush every second
      - "-f"
      - "1"

    # 'destroyMode' is set to 'cached' to allow a fluent-bit sub-process to
    #  to be reused across multiple 10x pipeline executions within a host JVM.
    destroyMode: cached

    # 'maxCacheSize' sets the size of the cache to used for caching fluent-bit instances
    maxCacheSize: 1

    # 'lazyLaunch' determines whether the fluent-bit instance will launch immediately
    # during the pipeline initialization, or only on-demand.
    lazyLaunch: true

    # ------------------------------ Encoding ---------------------------------

    # 'filter' specifies an expression that any TenXObject passed to this output must
    #  evaluate as truthy against to write its instance/template values to stdout.
    #  For example, 'filter: this.timestamp' will only emit timestamped objects.
    #  To learn more see: http://doc.log10x.com/api/js/#TenXBaseObject+timestamped
    filter: null

    #  This corresponds to the stdin format defined by fluent-bit in:
    #  https://docs.fluentbit.io/manual/pipeline/inputs/standard-input
    #  The 'text' argument below could be replaced by any list of intrinsic/calculated/extracted fields.
    #  The "" delimiter instructs 'joinFields' to produce JSON output.
    fields:  #
      - open="["
      - timestamp=dateFormat("+%s.%N")
      - separator=","
      - fields=joinFields("", "text")
      - close="]"

Below is the default configuration from: fluentbit/elasticsearch.yaml (* Required Fields).

Edit Online

# 🔟❎ 'run' Fluent Bit output configuration for events stream output to Elasticsearch

# Launch a Fluent Bit process into whose stdin pipe to write TenXObjects field values and TenXTemplates
# To learn more see https://doc.log10x.com/run/output/event/process/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/fluentbit


# ============================ Fluent Bit Options =============================

# Multiple Fluent Bit outputs can be defined below

fluentbitOut:

    # ------------------------------ General ----------------------------------

    # 'writeObjects' controls whether to write TenXObject field values to output.
  - writeObjects: true

    # 'writeTemplates' controls whether to write new template for TenXObjects written to
    #  this output. To learn more about TenXTemplates, see: https://doc.log10x.com/run/template
    writeTemplates: false

    # ------------------------------ Process ----------------------------------

    # 'command' specifies the command to execute to launch the Fluent Bit sub-process.
    command: /opt/fluent-bit/bin/fluent-bit

    # 'inputArgs' specifies list of values to pass as startup arguments to the spawned Fluent Bit sub-process
    inputArgs:
      # Sets the Fluent Bit input to read from stdin and route TenXTemplates
      # See https://github.com/log-10x/config/blob/main/pipelines/run/output/event/fluentbit/fluentbit-10x-input.conf
      - -c
      - $=path("fluentbit-10x-input.conf")

    # Setting the Fluent Bit output to Elasticsearch
    # for full config options of the Fluent Bit elasticsearch plugin, see -
    # https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch
    #
    outputArgs:

      ############### Events output ###############

      # Set event output to Elasticsearch
      - "-o"
      - "es"
      - "-m"
      - "input"

      # Setting host
      - "-p"
      - "$=\"host=\" + TenXEnv.get(\"ELASTICSEARCH_HOST\", \"elasticsearch-master\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"port=\" + TenXEnv.get(\"ELASTICSEARCH_PORT\", \"9200\")" # (❗ REQUIRED)

      # Setting up credentials
      - "-p"
      - "$=\"HTTP_User=\" + TenXEnv.get(\"ELASTICSEARCH_USERNAME\", \"elastic\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"HTTP_Passwd=\" + TenXEnv.get(\"ELASTICSEARCH_PASSWORD\")" # (❗ REQUIRED)

      # Setting the index destination
      - "-p"
      - "Logstash_Format=On"

      # Config retry
      - "-p"
      - "retry_limit=False"

      # Support for removal of types from elasticsearch
      - "-p"
      - "Suppress_Type_Name=On"

      ############### Templates output ###############

      # Set templates output to Elasticsearch
      - "-o"
      - "es"
      - "-m"
      - "templates"

      # Setting host
      - "-p"
      - "$=\"host=\" + TenXEnv.get(\"ELASTICSEARCH_HOST\", \"elasticsearch-master\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"port=\" + TenXEnv.get(\"ELASTICSEARCH_PORT\", \"9200\")" # (❗ REQUIRED)

      # Setting up credentials
      - "-p"
      - "$=\"HTTP_User=\" + TenXEnv.get(\"ELASTICSEARCH_USERNAME\", \"elastic\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"HTTP_Passwd=\" + TenXEnv.get(\"ELASTICSEARCH_PASSWORD\")" # (❗ REQUIRED)

      # Setting the templates index destination
      - "-p"
      - "index=l1es_dml"
      - "-p"
      - "id_key=templateHash"

      # Config retry
      - "-p"
      - "retry_limit=False"

      # Support for removal of types from elasticsearch
      - "-p"
      - "Suppress_Type_Name=On"

    # 'destroyMode' is set to 'cached' to allow a fluent-bit sub-process to
    #  to be reused across multiple 10x pipeline executions within a host JVM.
    destroyMode: cached

    # 'maxCacheSize' sets the size of the cache to used for caching fluent-bit instances
    maxCacheSize: 1

    # 'lazyLaunch' determines whether the fluent-bit instance will launch immediately
    # during the pipeline initialization, or only on-demand.
    lazyLaunch: true

    # ------------------------------ Encoding ---------------------------------

    # 'filter' specifies an expression that any TenXObject passed to this output must
    #  evaluate as truthy against to write its instance/template values to stdout.
    #  For example, 'filter: this.timestamp' will only emit timestamped objects.
    #  To learn more see: https://doc.log10x.com/api/js/#TenXBaseObject+timestamped
    filter: null

    #  This corresponds to the stdin format defined by fluent-bit in:
    #  https://docs.fluentbit.io/manual/pipeline/inputs/standard-input
    #  The 'text' argument below could be replaced by any list of intrinsic/calculated/extracted fields.
    #  The "" delimiter instructs 'joinFields' to produce JSON output.
    fields:  #
      - open="["
      - timestamp=dateFormat("+%s.%N")
      - separator=","
      - fields=joinFields("", "text")
      - close="]"

Below is the default configuration from: fluentbit/splunk.yaml (* Required Fields).

Edit Online

# 🔟❎ 'run' Fluent Bit output configuration for events stream output to Splunk

# Launch a Fluent Bit process into whose stdin pipe to write TenXObjects field values and TenXTemplates
# To learn more see https://doc.log10x.com/run/output/event/process/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/fluentbit


# ============================ Fluent Bit Options =============================

# Multiple Fluent Bit outputs can be defined below

fluentbitOut:

    # ------------------------------ General ----------------------------------

    # 'writeObjects' controls whether to write TenXObject field values to output.
  - writeObjects: true

    # 'writeTemplates' controls whether to write new template for TenXObjects written to
    #  this output. To learn more about TenXTemplates, see: https://doc.log10x.com/run/template
    writeTemplates: false

    # ------------------------------ Process ----------------------------------

    # 'command' specifies the command to execute to launch the Fluent Bit sub-process.
    command: /opt/fluent-bit/bin/fluent-bit

    # 'inputArgs' specifies list of values to pass as startup arguments to the spawned Fluent Bit sub-process
    inputArgs:
      # Sets the Fluent Bit input to read from stdin and route TenXTemplates
      # See https://github.com/log-10x/config/blob/main/pipelines/run/output/event/fluentbit/fluentbit-10x-input.conf
      - -c
      - $=path("fluentbit-10x-input.conf")

    # Setting the Fluent Bit output to Splunk
    # for full config options of the Fluent Bit splunk plugin, see -
    # https://docs.fluentbit.io/manual/pipeline/outputs/splunk
    #
    # Fluent Bit is using Splunk's HTTP event collector (HEC) for emitting events
    # for instructions on setting up HEC on splunk, see -
    # https://dev.splunk.com/enterprise/docs/devtools/httpeventcollector
    #
    outputArgs:

      ############### Events output ###############

      # Set event output to Splunk
      - "-o"
      - "splunk"
      - "-m"
      - "input"

      # Setting host
      - "-p"
      - "$=\"host=\" + TenXEnv.get(\"SPLUNK_HOST\", \"127.0.0.1\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"port=\" + TenXEnv.get(\"SPLUNK_PORT\", \"8088\")" # (❗ REQUIRED)

      # Setting up credentials
      - "-p"
      - "$=\"splunk_token=\" + TenXEnv.get(\"SPLUNK_HEC_TOKEN\")" # (❗ REQUIRED)

      # Setting the index destination
      - "-p"
      - "event_index=main"
      - "-p"
      - "event_source=fluent-bit-tenx-stream"
      - "-p"
      - "event_sourcetype=tenx_encoded"

      # Setting TLS
      - "-p"
      - "tls=on"
      - "-p"
      - "tls.verify=on"

      # Don't send raw
      - "-p"
      - "splunk_send_raw=off"

      ############### Templates output ###############

      # Set templates output to Elasticsearch
      - "-o"
      - "es"
      - "-m"
      - "templates"

      # Setting host
      - "-p"
      - "$=\"host=\" + TenXEnv.get(\"SPLUNK_HOST\", \"127.0.0.1\")" # (❗ REQUIRED)
      - "-p"
      - "$=\"port=\" + TenXEnv.get(\"SPLUNK_PORT\", \"8088\")" # (❗ REQUIRED)

      # Setting up credentials
      - "-p"
      - "$=\"splunk_token=\" + TenXEnv.get(\"SPLUNK_HEC_TOKEN\")" # (❗ REQUIRED)

      # Setting the templates index destination
      - "-p"
      - "event_index=main"
      - "-p"
      - "event_source=fluent-bit-tenx-stream"
      - "-p"
      - "event_sourcetype=tenx_dml_raw_json"

      # Setting TLS
      - "-p"
      - "tls=on"
      - "-p"
      - "tls.verify=on"

      # Don't send raw
      - "-p"
      - "splunk_send_raw=off"

    # 'destroyMode' is set to 'cached' to allow a fluent-bit sub-process to
    #  to be reused across multiple 10x pipeline executions within a host JVM.
    destroyMode: cached

    # 'maxCacheSize' sets the size of the cache to used for caching fluent-bit instances
    maxCacheSize: 1

    # 'lazyLaunch' determines whether the fluent-bit instance will launch immediately
    # during the pipeline initialization, or only on-demand.
    lazyLaunch: true

    # ------------------------------ Encoding ---------------------------------

    # 'filter' specifies an expression that any TenXObject passed to this output must
    #  evaluate as truthy against to write its instance/template values to stdout.
    #  For example, 'filter: this.timestamp' will only emit timestamped objects.
    #  To learn more see: https://doc.log10x.com/api/js/#TenXBaseObject+timestamped
    filter: null

    #  This corresponds to the stdin format defined by fluent-bit in:
    #  https://docs.fluentbit.io/manual/pipeline/inputs/standard-input
    #  The 'text' argument below could be replaced by any list of intrinsic/calculated/extracted fields.
    #  The "" delimiter instructs 'joinFields' to produce JSON output.
    fields:  #
      - open="["
      - timestamp=dateFormat("+%s.%N")
      - separator=","
      - fields=joinFields("", "text")
      - close="]"

Below is the default configuration from: fluentbit/stdout.yaml.

Edit Online

# 🔟❎ 'run' Fluent Bit output configuration for events stream output to Standard output

# Launch a Fluent Bit process into whose stdin pipe to write TenXObjects field values and TenXTemplates
# To learn more see https://doc.log10x.com/run/output/event/process/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:
  - run/modules/output/event/fluentbit


# ============================ Fluent Bit Options =============================

# Multiple Fluent Bit outputs can be defined below

fluentbitOut:

    # ------------------------------ General ----------------------------------

    # 'writeObjects' controls whether to write TenXObject field values to output.
  - writeObjects: true

    # 'writeTemplates' controls whether to write new template for TenXObjects written to
    #  this output. To learn more about TenXTemplates, see: https://doc.log10x.com/run/template
    writeTemplates: false

    # ------------------------------ Process ----------------------------------

    # 'command' specifies the command to execute to launch the Fluent Bit sub-process.
    command: /opt/fluent-bit/bin/fluent-bit

    # 'inputArgs' specifies list of values to pass as startup arguments to the spawned Fluent Bit sub-process
    inputArgs:
      # Sets the Fluent Bit input to read from stdin and route TenXTemplates
      # See https://github.com/log-10x/config/blob/main/pipelines/run/output/event/fluentbit/fluentbit-10x-input.conf
      - -c
      - $=path("fluentbit-10x-input.conf")

    # Setting the Fluent Bit output to stdout
    outputArgs:

      ############### Events output ###############

      # Set event output to stdout
      - "-o"
      - "stdout"
      - "-m"
      - "input"

      ############### Templates output ###############

      # Set templates output to stdout
      - "-o"
      - "stdout"
      - "-m"
      - "templates"

    # 'destroyMode' is set to 'cached' to allow a fluent-bit sub-process to
    #  to be reused across multiple 10x pipeline executions within a host JVM.
    destroyMode: cached

    # 'maxCacheSize' sets the size of the cache to used for caching fluent-bit instances
    maxCacheSize: 1

    # 'lazyLaunch' determines whether the fluent-bit instance will launch immediately
    # during the pipeline initialization, or only on-demand.
    lazyLaunch: true

    # ------------------------------ Encoding ---------------------------------

    # 'filter' specifies an expression that any TenXObject passed to this output must
    #  evaluate as truthy against to write its instance/template values to stdout.
    #  For example, 'filter: this.timestamp' will only emit timestamped objects.
    #  To learn more see: https://doc.log10x.com/api/js/#TenXBaseObject+timestamped
    filter: null

    #  This corresponds to the stdin format defined by fluent-bit in:
    #  https://docs.fluentbit.io/manual/pipeline/inputs/standard-input
    #  The 'text' argument below could be replaced by any list of intrinsic/calculated/extracted fields.
    #  The "" delimiter instructs 'joinFields' to produce JSON output.
    fields:  #
      - open="["
      - timestamp=dateFormat("+%s.%N")
      - separator=","
      - fields=joinFields("", "text")
      - close="]"

Options

Specify the options below to configure multiple Fluent Bit event output:

Name	Description	Category
fluentbitOutCommand	Command to launch Fluent Bit	Process
fluentbitOutOutputArgs	Arguments to pass to the spawned Fluent Bit process setting up output	Process
fluentbitOutLazyLaunch	Sets whether to launch the process on-demand	Process
fluentbitOutInputArgs	Arguments to pass to the spawned Fluent Bit process setting up input	Advanced
fluentbitOutStartupWaitMs	Wait time for process to startup	Advanced
fluentbitOutStartupWaitPattern	Wait for an output from the process at startup	Advanced
fluentbitOutDestroyMode	Control how to terminate the sub-process on output close	Lifecycle
fluentbitOutDestroyWait	Duration to wait for spawned sub-process to terminate	Lifecycle
fluentbitOutMaxCacheSize	Max number of sub-processes to keep alive across subsequent 10x pipeline executions	Lifecycle
fluentbitOutFields	List of TenXObject field names to encode	Encoding
fluentbitOutFilter	A JavaScript expression an TenXObject must evaluate as 'truthy' against to be written to this output	Encoding
fluentbitOutWriteObjects	If true, TenXObject field values specified by [fluentbitOutFields](https://doc.log10x.com/run/output/event/fluentbit/#fluentbitoutfields) are encoded to this output	Encoding
fluentbitOutWriteTemplates	If true, write new TenXTemplates to this output	Encoding
fluentbitOutEncodeType	The output format in which to encode TenXObject values. Possible values:[json, delimited]	Encoding
fluentbitOutEncodedLinePrefix	The line prefix prepended to the encoded event if 'fluentbitOutEncodeType' is 'delimited'	Encoding
fluentbitOutEncodeDelimiter	The delimiter placed between key and values pairs if 'fluentbitOutEncodeType' is 'delimited'	Encoding

Process

`fluentbitOutCommand`

Command to launch Fluent Bit.

Type	Required	Category
String	✔	Process

Specifies the command to execute to launch the Fluent Bit sub-process.

`fluentbitOutOutputArgs`

Arguments to pass to the spawned Fluent Bit process setting up output.

Type	Default	Category
List	[]	Process

List of values to pass as startup arguments to the spawned Fluent Bit sub-process intended to set up it's output plugin(s).

`fluentbitOutLazyLaunch`

Sets whether to launch the process on-demand.

Type	Default	Category
Boolean	false	Process

Controls whether to spawn the target sub-process upon the creation of the output or on-demand. If true, launch the sub-process when the first TenXObject instance is written to its stdin.

Advanced

`fluentbitOutInputArgs`

Arguments to pass to the spawned Fluent Bit process setting up input.

Type	Default	Category
List	[-c, fluentbit-10x-input.conf]	Advanced

List of values to pass as startup arguments to the spawned Fluent Bit sub-process intended to set up it's stdin input.

`fluentbitOutStartupWaitMs`

Wait time for process to startup.

Type	Default	Category
Number	5000	Advanced

Sets a startup delay wait time in milliseconds before the process is considered available for events. if provided, will add a wait as part of starting the process before emitting events into it. max value for this argument is 10,000 (ie 10 seconds). if fluentbitOutStartupWaitPattern is also provided, the process will be considered a failure if the wait time has passed before a line matching fluentbitOutStartupWaitPattern was emitted.

`fluentbitOutStartupWaitPattern`

Wait for an output from the process at startup.

Type	Default	Category
String	\[\ssp\s]\s+stream\s+processor\s+started\b	Advanced

Sets a regex pattern to wait for the process to emit for it to be considered available for events. if provided, will wait until the process has emitted a line to it's output that matches the regex before emitting events into it. if fluentbitOutStartupWaitMs is also provided, will only wait for the specified wait time, after which if no line was emitted to match the provided regex the process will be considered a failure.

Lifecycle

`fluentbitOutDestroyMode`

Control how to terminate the sub-process on output close.

Type	Default	Category
String	graceful	Lifecycle

Controls how to treat spawned sub-process when this output closes. Possible values:

Value	Description
graceful (default)	The sub-process is destroyed gracefully with a timeout period of fluentbitOutDestroyWait.
forcible	Terminates the sub-process immediately.
cached	The sub-process remains alive for future pipelines that launch the same command and argument combination within the host JVM. A maximum of fluentbitOutMaxCacheSize can be kept alive. Useful for launching shippers (e.g., Filebeat, Fluent Bit) across multiple 10x pipeline runs in a long-living server environment.

`fluentbitOutDestroyWait`

Duration to wait for spawned sub-process to terminate.

Type	Default	Category
String	""	Lifecycle

Controls the time duration that the output stream will wait for the sub-process to terminate gracefully (e.g., '5s'). If the process timeouts, the output will forcibly terminate it. If set to 0, the output stream will wait indefinitely for the sub-process to terminate. This value only applies when fluentbitOutDestroyMode is set to graceful.

`fluentbitOutMaxCacheSize`

Max number of sub-processes to keep alive across subsequent 10x pipeline executions.

Type	Default	Category
Number	0	Lifecycle

Sets the global maximum number of processes that can retained globally across 10x pipeline executions within the host JVM. This value is useful when launching shippers (e.g., Filebeat, Fluent Bit) for use across multiple 10x pipeline executions within a server environment. This value only applies when fluentbitOutDestroyMode is set to cached. If fluentbitOutMaxCacheSize is set to 0, no caching is performed, and sub-processes are destroyed gracefully.

Encoding

`fluentbitOutFields`

List of TenXObject field names to encode.

Type	Default	Category
List	[]	Encoding

Specifies a list of TenXObject intrinsic/calculated/extracted field names to extract and write to the output.

`fluentbitOutFilter`

A JavaScript expression an TenXObject must evaluate as 'truthy' against to be written to this output.

Type	Default	Category
String	""	Encoding

Specifies a JavaScript expression that an TenXObject must evaluate as truthy to write its instance/template field values to this output.

For example, to only emit timestamped TenXObjects, specify:

fluentbitOutFilter: this.timestamped

`fluentbitOutWriteObjects`

If true, TenXObject field values specified by [fluentbitOutFields](https://doc.log10x.com/run/output/event/fluentbit/#fluentbitoutfields) are encoded to this output.

Type	Default	Category
String	true	Encoding

Determine whether to write fluentbitOutFields values of an TenXObject passed to this output. This configuration is typically defined using a JavaScript expression.

To enable/disable the output depending on whether a launch argument, environment variable, or JVM -D option evaluates as truthy, use:

  fluentbitOutWriteObjects: $=TenXEnv.get("myProcCmd")

To learn more see TenXEnv.get.

`fluentbitOutWriteTemplates`

If true, write new TenXTemplates to this output.

Type	Default	Category
String	false	Encoding

Controls whether to write template values of TenXObjects passed to this output. The output will only emit TenXTemplates that the pipeline did not load at startup via thetemplateFile argument, and will only emit an TenXTemplate once based on its templateHash value.

`fluentbitOutEncodeType`

The output format in which to encode TenXObject values. Possible values:[json, delimited].

Type	Default	Category
String	delimited	Encoding

Specifies how to encode fluentbitOutFields values to this output. Possible values:

json: formats names and values for the current TenXObject as: {"field":"str","field2":1, "field3": true}
delimited: formats values for the current TenXObject as: "str",1,true The fluentbitOutEncodeDelimiter argument determines the separator to use.

`fluentbitOutEncodedLinePrefix`

The line prefix prepended to the encoded event if 'fluentbitOutEncodeType' is 'delimited'.

Type	Default	Category
String		Encoding

Sets the char which is prepended to the emitted TenXObject field values when fluentbitOutEncodeType is delimited.

`fluentbitOutEncodeDelimiter`

The delimiter placed between key and values pairs if 'fluentbitOutEncodeType' is 'delimited'.

Type	Default	Category
String		Encoding

Sets the char delimiter to separate TenXObject field values when fluentbitOutEncodeType is delimited.

This module is defined in fluentbit/module.yaml.