Receive

Read events from a Fluent Bit forwarder to transform into typed TenXObjects and filter using local/centralized receiver policies. This module is a component of the Receiver app.

Architecture

graph LR
    A["<div style='font-size: 14px;'>📂 Fluent Bit</div><div style='font-size: 10px;'>inputs</div>"] --> B["<div style='font-size: 14px;'>🔧 Lua Filter</div><div style='font-size: 10px;'>tenx.lua</div>"]
    B --> C["<div style='font-size: 14px;'>⚡ 10x Receiver</div><div style='font-size: 10px;'>filter policy</div>"]
    C --> D["<div style='font-size: 14px;'>🔌 Unix/TCP</div><div style='font-size: 10px;'>input</div>"]
    D --> E["<div style='font-size: 14px;'>📤 Fluent Bit</div><div style='font-size: 10px;'>outputs</div>"]

    classDef input fill:#2563eb,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef filter fill:#ea580c,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef engine fill:#7c3aed,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef socket fill:#0891b2,stroke:#0e7490,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef output fill:#16a34a,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8

    class A input
    class B filter
    class C engine
    class D socket
    class E output

Data Flow

• 📂 Fluent Bit Inputs - Collect logs from files, containers, or other sources
• 🔧 Lua Filter - Intercepts ALL events and pipes them to 10x sidecar
• ⚡ 10x Receiver - Applies rate/policy-based filtering, drops noisy events
• 🔌 Unix/TCP Input - Receives FILTERED events back from the sidecar
• 📤 Fluent Bit Outputs - Only filtered events ship to final destinations

Key Characteristics

Feature	Description
🚦 Rate Limiting	Filter events based on per-template rate limits
📋 Policy-Based	Apply local or centralized filtering policies
💰 Cost Control	Reduce log volume and costs by dropping noisy events
🔧 Lua Filter	Uses Fluent Bit's native Lua scripting for sidecar launch

Sidecar Relay

This module configures a Fluent Bit Lua filter and Unix/TCP input. The Lua filter launches a 10x sidecar process and passes it collected events to receive. The sidecar relays received events back to Fluent Bit via the configured Unix/TCP input to ship to output(s) (e.g., Splunk, S3).

Install

Nix/Win/OSX k8s

See the Log10x Receiver Fluent Bit run instructions

Deploy to k8s via Helm

See the Log10x Receiver Fluent Bit deployment instructions

Config Files

To configure the Fluent Bit Receiver module, Edit these files.

Below is the default configuration from: receive/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/input/forwarder/fluentbit/receive/config.yaml
• Within cloned repo
  ./pipelines/run/input/forwarder/fluentbit/receive/config.yaml

Download

# 🔟❎ 'run' Fluent Bit receiver configuration

# Configure an input that reads events from a Fluent Bit forwarder and
# an output that writes received TenXObjects back to Fluent Bit.
# When 'receiverOptimize' is enabled, events are encoded for volume reduction.

# This config file operates in conjunction with the instructions provided in:
# https://doc.log10x.com/run/input/forwarder/fluentbit/receive/#install

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include:

  # Parent module declaring shared forwarder options (outputEncodeField, outputWriteTemplates, encodeObjects)
  - run/modules/input/forwarder/module.yaml

  - run/modules/input/forwarder/fluentbit/receive/module.yaml

  # Include the stdin input to read events from the forwarder
  - run/modules/input/forwarder/fluentbit/input

  # Include forward output to write events into the forwarder
  # Windows: Forward protocol over TCP (port 24224) - Unix: Forward protocol over Unix domain socket
  - '$="run/modules/input/forwarder/fluentbit/output/" + (TenXString.includes(TenXEnv.get("os.name"), "Windows") ? "forward" : "unix")'

# =============================== Mode Options ================================

# Mode is selected here by two user flags:
# - `receiverOptimize=true`        → compact-all: encoded=encode() for every event
# - `compactReceiverLookupFile=<p>` → compact-lookup: encoded=shouldEncode()?encode():fullText
# - neither                          → receive: plain fullText
#
# Substitution happens at config-load (`$=yield` defers until options are bound).
# The transport stream.yaml then parses the resulting literal — shouldEncode() is
# only in the parsed expression when compactReceiver is actually configured, so
# CompactObject's body is never compiled in modes that don't need it.

outputEncodeField: '$=yield TenXEnv.get("compactReceiverLookupFile") ? "encoded=shouldEncode() ? encode() : fullText" : (TenXEnv.get("receiverOptimize") ? "encoded=encode()" : "fullText")'

outputWriteTemplates: $=yield TenXEnv.get("receiverOptimize", false) || TenXEnv.get("compactReceiverLookupFile", false)

# Legacy flag retained for fluentd/stdio and logstash input extractor name.
# For fluentbit/unix + fluentbit/forward the mode is driven by outputEncodeField above.
encodeObjects: $=yield TenXEnv.get("receiverOptimize", false) || TenXEnv.get("compactReceiverLookupFile", false)

# ============================= Fluent Bit Options ============================

fluentbit:
  # 'outputFields' specifies TenXObject field names to emit alongside the main event.
  #  When set, enrichment fields are included in the sidecar output for forwarder post-processing.
  #  Example: outputFields: [level, group, symbolMessage]
  outputFields: []

  # 'outputEncodeType' controls the output format when outputFields is set.
  #  Possible values: 'json' or 'delimited'
  outputEncodeType: delimited

Options

Specify the options below to configure the Fluent Bit Receiver:

Name	Description
fluentbitOutputFields	List of TenXObject field names to include alongside the main event in output
fluentbitOutputEncodeType	Output format when outputFields are set. Possible values: [json, delimited]

fluentbitOutputFields

List of TenXObject field names to include alongside the main event in output.

Type	Default
List	[]

Specifies TenXObject field names to emit alongside the main event text when writing back to the Fluent Bit forwarder. When empty (default), only the main event field is written. Example fields: level, group, symbolMessage.

fluentbitOutputEncodeType

Output format when outputFields are set. Possible values: [json, delimited].

Type	Default
String	delimited

Specifies how the combined output (main event field plus outputFields) is encoded when writing back to the Fluent Bit forwarder. Possible values:

• json: formats all fields as a JSON object
• delimited: formats field values separated by the output delimiter Only takes effect when fluentbitOutputFields is set.

This module is defined in receive/module.yaml.