Log Forwarder Inputs

Runs the 10x Engine in-path with your log forwarder to process collected log events before they ship to output destinations (e.g., Splunk, Elasticsearch, S3). Depending on the forwarder, 10x runs as a separate log10x/edge-10x sidecar container added via a values or kustomize overlay (Fluentd, Fluent Bit, Logstash, OTel Collector, Vector), as an image swap to the embedded 10x variant (Filebeat, log10x/filebeat-10x), or as a file relay (Splunk UF, Datadog Agent); 5 of the 8 supported forwarders run the sidecar.

The design enables 10x apps, the Reporter (read-only DaemonSet alongside the forwarder) and the Receiver (embedded or sidecar per forwarder, with pass, sample, compact, tier_down, offload, and drop actions plus read-only observation), to process events at the source while integrating with existing log forwarders (e.g., Fluentd/Bit).

Extensibility

All forwarder input modules utilize core IPC I/O modules (e.g., stdin, Unix) as building blocks for integrating with bundled forwarders (e.g., Fluentd/Bit) and to serve as a reference for supporting additional forwarder types.

Modules

• Fluent Bit

  ---

  Receive and optimize events collected by Fluent Bit via the Fluent Forward protocol.

  More info

• Fluentd

  ---

  Receive and optimize events collected by Fluentd via the Fluent Forward protocol.

  More info

• OpenTelemetry Collector

  ---

  Receive and optimize events collected by the OpenTelemetry Collector via OTLP/gRPC (both directions).

  More info

• Vector

  ---

  Receive and optimize events collected by Vector via the socket sink and fluent source.

  More info

• Filebeat

  ---

  Report, receive, and optimize events collected by Filebeat forwarders.

  More info

• Logstash

  ---

  Receive and optimize events collected by Logstash via newline-delimited JSON.

  More info

• Splunk UF Input

  ---

  Receive and optimize events before shipping to Splunk via Universal Forwarder.

  More info

• Datadog Agent Input

  ---

  Receive and optimize events before shipping to Datadog via Datadog Agent.

  More info

Config Files

To configure the Log Forwarder inputs module, Edit these files.

Below is the default configuration from: forwarder/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/input/forwarder/config.yaml
• Within cloned repo
  ./pipelines/run/input/forwarder/config.yaml

Download

# 🔟❎ 'run' shared forwarder configuration
#
tenx: run

# =============================== Dependencies ================================

include:

  # Shared forwarder options (mode dispatch, output encoding)
  - run/modules/input/forwarder/module.yaml

# =============================== Mode Options ================================
#
# The Receiver app picks the encoding mode based on user flags:
#   default                      → emit events verbatim (`fullText`)
#   receiverOptimize=true        → compact-all (lossless compaction of every event)
#   compactReceiverLookupFile=…  → compact-lookup (per-container decision)
#
# `symbolMessageHashField` (optional, unset by default) adds a stable
# pattern-hash field alongside the encoded/fullText output: when set, encode()
# and fullText are called as `encode("<field>")` / `fullText("<field>")`.
#
# Unset is the receive-path default and it means VERBATIM: the record the
# forwarder gets back is the record it sent, byte for byte, with no field added.
# The hash is still computed and still rides the TenXObject internally as
# `tenx_hash` for metrics and aggregation, it just does not reach the wire.
# `symbolMessageHashField my_custom_hash` (CLI arg or env var) opts in and names
# the field.

output:

  # outputEncodeField is the per-event output field expression, resolved once at
  # config-load by the `$=yield` ternary below. Written as a YAML `|-` block
  # scalar for readability: the engine treats the newlines/indentation between
  # tokens as ordinary whitespace, so this compiles identically to the original
  # one-liner. (Block scalars do NOT process backslash escapes, so the `\"`
  # below reach the expression literally, exactly what encode("field") needs.)
  #
  # The three resolved modes (the regulator marks routeState via route(action),
  # where action is the per-service disposition: drop/offload/tier_down/compact/
  # sample/pass; this picks the output FORMAT, while `outputSoftDrop` also flips
  # the stream filter):
  #   receiverOptimize=true   → encoded=encode()                                                     (compact every event)
  #   outputOffload truthy    → fullText("routeState")                                               (FULL text + marker; for S3 offload, never compacted)
  #   outputSoftDrop truthy   → encoded=isRoute("compact") ? encode("routeState") : fullText("routeState")  (compact the compact-marked slice; every other action stays fullText so the forwarder routes it by routeState)
  #   neither                 → fullText                                                             (receive / hard-drop)
  # The `symbolMessageHashField` variant (when set) passes the hash field name
  # into encode()/fullText() so a stable pattern-hash ships alongside the output.
  #
  # ROUTE MARKER: outputOffload and outputSoftDrop both splice the `routeState`
  # field onto every forwarded event so the downstream forwarder can route by the
  # per-service action name (offload → the customer's own S3, tier_down → the SIEM
  # cheap tier, compact → encoded bytes to the SIEM, pass/sample → the SIEM, drop →
  # suppressed). The splice writes the route NAME as a JSON STRING (e.g.
  # `"routeState":"offload"` / `"routeState":"pass"`), so a forwarder match must be
  # string-equality against the action name, NOT boolean truthiness. hard-drop
  # omits dropped events entirely, so it emits no marker (nothing to route).
  encodeField: |-
    $=yield TenXEnv.get("receiverOptimize")
      ? ("encoded=" + (TenXEnv.get("symbolMessageHashField") ? ("encode(\"" + TenXEnv.get("symbolMessageHashField") + "\")") : "encode()"))
      : (TenXEnv.get("outputOffload")
          ? (TenXEnv.get("symbolMessageHashField") ? ("fullText(\"" + TenXEnv.get("symbolMessageHashField") + "\",\"routeState\")") : "fullText(\"routeState\")")
          : (TenXEnv.get("outputSoftDrop")
              ? ("encoded=isRoute(\"compact\") ? "
                  + (TenXEnv.get("symbolMessageHashField") ? ("encode(\"" + TenXEnv.get("symbolMessageHashField") + "\",\"routeState\")") : "encode(\"routeState\")")
                  + " : "
                  + (TenXEnv.get("symbolMessageHashField") ? ("fullText(\"" + TenXEnv.get("symbolMessageHashField") + "\",\"routeState\")") : "fullText(\"routeState\")"))
              : (TenXEnv.get("symbolMessageHashField") ? ("fullText(\"" + TenXEnv.get("symbolMessageHashField") + "\")") : "fullText")))

  # outputWriteTemplates = true whenever events are encoded (compact-all or soft-drop)
  writeTemplates: $=yield TenXEnv.get("receiverOptimize") || TenXEnv.get("outputSoftDrop")

  # outputDropFilter decides whether a (possibly regulator-marked) object is written.
  # Resolved ONCE here so the soft/hard logic lives next to encodeField instead of
  # being duplicated in every output stream; streams just reference $?outputDropFilter.
  # (Named outputDropFilter, not outputFilter -- the latter is a built-in per-stream
  # engine option and would collide.)
  #   outputSoftDrop or outputOffload truthy → "isObject"                    (write all; dropped events flow for routing/compaction)
  #   unset (hard, default)                  → "isObject && !this.isRoute("drop")" (omit dropped objects)
  dropFilter: '$=yield (TenXEnv.get("outputSoftDrop") || TenXEnv.get("outputOffload")) ? "isObject" : "isObject && !this.isRoute(\"drop\")"'

Options

Specify the options below to configure the Log Forwarder inputs:

Name	Description
outputEncodeField	The single field expression each forwarder's output stream writes per event
outputWriteTemplates	Whether the forwarder emits new TenXTemplates alongside encoded events
emit_dropped	Whether regulator-marked (`routeState="drop"`) events survive the output filter
compact_emitted	Whether all emitted events are written as `encode()` bytes or as `fullText`
outputDropFilter	The filter expression each output stream uses to decide whether a marked object is written

outputEncodeField

The single field expression each forwarder's output stream writes per event.

Type	Default
String	fullText

Derived by the customer-facing config.yaml from compact_emitted (and from the legacy per-pattern compactReceiverLookupFile lookup when set):

• compact_emitted=false (default): fullText, every event passes through full text
• compact_emitted=true: encoded=encode(), every emitted event compacted losslessly
• compact-lookup (legacy): encoded=shouldEncode() ? encode() : fullText, per-pattern decision via the compactReceiver module (requires compactReceiverLookupFile)

Advanced users can override directly to customize the output field expression.

outputWriteTemplates

Whether the forwarder emits new TenXTemplates alongside encoded events.

Type	Default
String	false

True for encoding modes (compact-all, compact-lookup) so decoders can reconstruct events from templateHash + vars. False for receive mode (no encoding → no templates needed).

emit_dropped

Whether regulator-marked (`routeState="drop"`) events survive the output filter.

Type	Default
String	""

Single boolean controlling whether the regulator-dropped slice reaches the forwarder. Orthogonal to compact_emitted.

• unset (default) = hard-drop: outputDropFilter resolves to isObject && !isRoute("drop"), dropped events never leave the engine.
• truthy = emit: outputDropFilter resolves to isObject, all events flow. The downstream forwarder routes by the wire-level routeState record field.

Use truthiness (engine string == is identity compare): set a non-empty value (e.g. true) to emit dropped events, leave unset to hard-drop.

compact_emitted

Whether all emitted events are written as `encode()` bytes or as `fullText`.

Type	Default
String	""

Single boolean controlling whether ALL emitted events are compacted on the wire. Orthogonal to emit_dropped. Applies uniformly: when set, both kept and dropped events (if emit_dropped) emit as encode(); when unset, both emit as fullText.

• unset (default) = fullText: outputEncodeField resolves to fullText
• truthy = compact: outputEncodeField resolves to encoded=encode()

Use truthiness (engine string == is identity compare): set a non-empty value (e.g. true) for compact, leave unset for fullText.

Per-pattern compact-lookup mode (via compactReceiverLookupFile) still overrides this when the lookup file is configured.

outputDropFilter

The filter expression each output stream uses to decide whether a marked object is written.

Type	Default
String	isObject \&\& !isRoute("drop")

Resolved once by the customer-facing config.yaml from emit_dropped, so the emit/hard-drop logic lives in one place instead of being duplicated per stream:

• hard-drop (default): isObject && !isRoute("drop"), dropped objects not written
• emit (emit_dropped truthy): isObject, everything written; routing decision moves downstream to the forwarder based on the wire-level routeState field.

Each output stream references it via $?outputDropFilter. Named outputDropFilter rather than outputFilter, which is a built-in per-stream engine option.

This module is defined in forwarder/module.yaml.