Fluent Bit

Runs 10x Engine as a sidecar to Fluent Bit for reporting, receiving, and optimizing events before they ship to their destination (Elasticsearch, Splunk, S3, Kafka, …). Fluent Bit and Log10x run as peer processes and exchange events over the Fluent Forward protocol in both directions — works against any stock Fluent Bit build and the official Fluent Bit Helm chart with a values overlay.

Architecture

graph LR
    A["<div style='font-size: 14px;'>📂 Sources</div><div style='font-size: 10px;'>tail, http, k8s</div>"] --> F["<div style='font-size: 14px;'>🧪 Filters</div><div style='font-size: 10px;'>Match app.*</div>"]
    F --> B["<div style='font-size: 14px;'>📤 out_forward</div><div style='font-size: 10px;'>:24224</div>"]
    B --> E["<div style='font-size: 14px;'>⚡ 10x Engine</div><div style='font-size: 10px;'>Receive/Optimize</div>"]
    E --> C["<div style='font-size: 14px;'>📥 in_forward</div><div style='font-size: 10px;'>:24225 Tag_Prefix tenx.</div>"]
    C --> D["<div style='font-size: 14px;'>📤 Destinations</div><div style='font-size: 10px;'>ES, Splunk, S3, Kafka</div>"]

    classDef input fill:#2563eb,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef filter fill:#ea580c,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef engine fill:#7c3aed,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef output fill:#16a34a,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8

    class A input
    class B filter
    class C filter
    class D output
    class E engine
    class F filter

Data Flow

• 📂 Sources — Your existing Fluent Bit inputs (tail, http, forward, systemd, …) tag events with your normal scheme (e.g. app.*, k8s.*).
• 🧪 Filters — Your enrichment filters (kubernetes, modify, parser, lua, …) Match your source tags only (e.g. Match app.*). They run exactly once before the event is handed off to Log10x.
• 📤 out_forward → Log10x — A forward output with Match app.* (or whatever your source tag pattern is) ships the enriched event to the Log10x sidecar over TCP :24224.
• ⚡ 10x Engine — The Receiver app applies rate/policy-based filtering and optionally compacts events for volume reduction.
• 📥 in_forward → destinations — Processed events come back to Fluent Bit on :24225 via a forward input with Tag_Prefix tenx., which prepends tenx. to every returning tag. Your destination outputs Match tenx.*; the tenx. namespace is what keeps filters from re-firing and the out_forward to Log10x from looping events back. Fluent Bit has no label/scope concept, so tag-prefix namespacing is the bypass mechanism.
• 📤 Destinations — Your destination outputs (es, splunk, kafka, s3, …) Match tenx.* and ship to the real destinations. The original tag is preserved after the prefix (tenx.app.foo), so destinations that route on the suffix still work.

What an event looks like on the way back

The record structure of the original Fluent Bit event is preserved end-to-end — every field comes back to your destination outputs with the same name and same position. The wire tag has a tenx. prefix prepended on egress (the bypass mechanism); the original tag is the suffix. What changes in the record body depends on the Receiver app mode:

Mode	Difference vs the event Fluent Bit sent in
Receive (default)	None. Same record. Tag is tenx.<original>.
Receive + symbolMessageHashField <name>	Adds one new field with the symbol-pattern hash (a stable identifier for the message pattern — usable as a dedup key, metric dimension, or correlation ID).
receiverOptimize true	The value of the message field (log by default, or whatever fluentbitInputMessageField is set to) is replaced with a compact encoded form. A separate tenx-template event is emitted with the template needed to decode it. All other fields stay verbatim.
receiverOptimize true + symbolMessageHashField <name>	Both of the above.

The original Fluent Bit tag is carried by the Forward protocol itself and surfaces on the event as its source inside Log10x — used for rate-based grouping and re-emitted as the wire tag on the return path (with tenx. prepended by the egress forward input). Internally, Log10x's Fluent Bit input module reads the message text from the field named by fluentbitInputMessageField (default log); when the Receiver app is configured with k8sExtractorName: fluentK8s, the kubernetes.* sub-object is also materialized as enrichment fields for use by message-pattern and rate filtering.

Key Files

File	Purpose
stream.yaml	Fluent Bit Forward input + output stream definitions
conf/tenx-sidecar.conf	Reference Fluent Bit config showing tag-prefix bypass via Tag_Prefix tenx.

Quickstart

1. Run Log10x:

tenx @run/input/forwarder/fluentbit @apps/receiver

2. Wire up your Fluent Bit config — include the sidecar recipe and Match your sources with the recipe's tag conventions:

fluent-bit.conf

@INCLUDE ${TENX_MODULES}/pipelines/run/modules/input/forwarder/fluentbit/conf/tenx-sidecar.conf

[INPUT]
    Name         tail
    Path         /var/log/app.log
    Tag          app.logs           # any tag NOT starting with `tenx.`
    Parser       json

3. Point destination outputs at the tenx.* namespace (the recipe defaults to stdout for testing):

[OUTPUT]
    Name         your_output_plugin
    Match        tenx.*             # only matches events that came back from Log10x
    # ... destination config

For Splunk integration see the 10x for Splunk documentation. For Kubernetes deployment via the official Fluent Bit Helm chart see the Helm sidecar overlay.

Config Files

To configure the Fluent Bit module, Edit these files.

Below is the default configuration from: fluentbit/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/input/forwarder/fluentbit/config.yaml
• Within cloned repo
  ./pipelines/run/input/forwarder/fluentbit/config.yaml

Download

# 🔟❎ 'run' Fluent Bit receiver configuration
#
# To learn more see https://doc.log10x.com/run/input/forwarder/fluentbit/

tenx: run

# =============================== Dependencies ================================

include:
  - run/input/forwarder/config.yaml
  - run/modules/input/forwarder/fluentbit

# ============================= Fluent Bit Options ============================

fluentbit:

  # ----------------------------- Input Options -----------------------------

  input:

    # 'port' specifies the TCP port to listen on for events from Fluent Bit's
    #  forward output plugin.
    port: 24224

    # 'path' switches the input to a Unix domain socket instead of TCP.
    #  When set, takes precedence over 'port'.
    # path: /tmp/tenx-fluentbit-in.sock

    # 'messageField' is the name of the JSON field in each Fluent Bit record
    #  carrying the log line text.
    messageField: log

  # ----------------------------- Output Options ----------------------------

  output:

    # 'host' specifies the TCP host of Fluent Bit's forward input receiving
    #  processed events from Log10x.
    host: 127.0.0.1

    # 'port' specifies the TCP port of Fluent Bit's forward input. MUST differ
    #  from input.port or the two would collide on the same listener.
    port: 24225

    # 'path' switches the output to a Unix domain socket of TCP.
    #  When set, takes precedence over 'host'/'port'.
    # path: /tmp/tenx-fluentbit-out.sock

    # 'encodeType' is the on-wire format for the Forward record.
    #  - 'delimited': each top-level field of the rendered record becomes its
    #    own Forward record field — preserves the original record's structure.
    #  - 'json': the whole record is wrapped as one stringified field.
    encodeType: delimited

Options

Specify the options below to configure the Fluent Bit:

Name	Description	Category
fluentbitInputPort	TCP port to listen on for events from Fluent Bit's forward output	Input
fluentbitInputPath	Unix domain socket path to listen on (Linux/macOS)	Input
fluentbitInputMessageField	Name of the JSON field carrying the original log line	Input
fluentbitOutputHost	TCP host of the Fluent Bit forward input receiving processed events	Output
fluentbitOutputPort	TCP port of the Fluent Bit forward input receiving processed events	Output
fluentbitOutputPath	Unix domain socket path of the Fluent Bit forward input (Linux/macOS)	Output
fluentbitOutputEncodeType	Output format when outputFields are set. Possible values: [json, delimited]	Output

Input

fluentbitInputPort

TCP port to listen on for events from Fluent Bit's forward output.

Type	Default	Category
String	24224	Input

TCP port where Log10x listens for events sent by Fluent Bit's forward output. Match this against the Port in your Fluent Bit [OUTPUT] block (Name forward).

fluentbitInputPath

Unix domain socket path to listen on (Linux/macOS).

Type	Default	Category
String	""	Input

When set, Log10x listens on a Unix domain socket instead of TCP — useful on Linux/macOS to avoid TCP overhead. Configure Fluent Bit's forward output with Unix_Path set to a matching path. Ignored on Windows.

fluentbitInputMessageField

Name of the JSON field carrying the original log line.

Type	Default	Category
String	log	Input

Name of the field in each Fluent Bit record that contains the actual log message text. The default (log) matches Fluent Bit's tail input convention for container and file logs. Set to e.g. message if your sources emit the log line under a different key.

Output

fluentbitOutputHost

TCP host of the Fluent Bit forward input receiving processed events.

Type	Default	Category
String	127.0.0.1	Output

Hostname or IP where Fluent Bit's forward input is listening for processed events from Log10x. Pairs with fluentbitOutputPort.

fluentbitOutputPort

TCP port of the Fluent Bit forward input receiving processed events.

Type	Default	Category
String	24225	Output

TCP port where Fluent Bit's forward input is listening for processed events from Log10x. MUST differ from the port Log10x's own Forward input listens on (default 24224) or the two would collide on the same socket.

fluentbitOutputPath

Unix domain socket path of the Fluent Bit forward input (Linux/macOS).

Type	Default	Category
String	""	Output

Optional Unix domain socket path for Fluent Bit's forward input. When set, takes precedence over fluentbitOutputHost / fluentbitOutputPort. Linux/macOS only — ignored on Windows.

fluentbitOutputEncodeType

Output format when outputFields are set. Possible values: [json, delimited].

Type	Default	Category
String	delimited	Output

Specifies how the combined output (main event field plus outputFields) is encoded when writing back to Fluent Bit. Possible values:

• json: formats all fields as a JSON object
• delimited: formats field values separated by the output delimiter Only takes effect when fluentbitOutputFields is set.

This module is defined in fluentbit/module.yaml.