OTel Collector

Runs 10x Engine as a sidecar to the OpenTelemetry Collector for reporting, receiving, and optimizing events before they ship to their destination (Elasticsearch, Splunk, S3, Kafka, …). The Collector and Log10x run as peer processes — the Collector sends events to Log10x via its native OTLP/gRPC exporter and receives processed events back via its OTLP/gRPC receiver. The OTLP wire preserves resource attributes, scope info, log-record attributes, severity, timestamp, and body end-to-end, so k8s metadata (k8s.pod.name, k8s.namespace.name, k8s.container.name, labels, …) round-trips back to your destinations.

Distribution

Both the OTLP receiver and OTLP exporter ship in the core otelcol distribution — no otelcol-contrib build is required. Tested against otelcol v0.151.0+.

Architecture

graph LR
    A["<div style='font-size: 14px;'>📂 Receivers</div><div style='font-size: 10px;'>filelog, otlp, journald</div>"] --> F["<div style='font-size: 14px;'>🧪 logs/to-tenx</div><div style='font-size: 10px;'>k8sattributes, resource, transform</div>"]
    F --> B["<div style='font-size: 14px;'>📤 otlp exporter</div><div style='font-size: 10px;'>OTLP/gRPC → :4317</div>"]
    B --> E["<div style='font-size: 14px;'>⚡ 10x Engine</div><div style='font-size: 10px;'>Receive/Optimize</div>"]
    E --> C["<div style='font-size: 14px;'>📥 otlp receiver</div><div style='font-size: 10px;'>:24225 (no processors)</div>"]
    C --> D["<div style='font-size: 14px;'>📤 Destinations</div><div style='font-size: 10px;'>ES, Splunk, S3, Kafka</div>"]

    classDef input fill:#2563eb,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef filter fill:#ea580c,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef engine fill:#7c3aed,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef output fill:#16a34a,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8

    class A input
    class B filter
    class C filter
    class D output
    class E engine
    class F filter

Data Flow

• 📂 Receivers — Your existing OTel receivers (filelog, otlp, journald, kafka, …) feed events into the logs/to-tenx pipeline.
• 🧪 logs/to-tenx — Your enrichment processors (k8sattributes, resource, attributes, transform, filter, …) run here exactly once before the event is handed off to Log10x. The processors live on this pipeline only; the return-path pipeline never sees them.
• 📤 OTLP exporter → Log10x — The Collector forwards the enriched event to the Log10x sidecar over OTLP/gRPC on TCP :4317. Resource attributes, scope info, log-record attributes, and the body all travel on the wire.
• ⚡ 10x Engine — The Receiver app applies rate/policy-based filtering and optionally compacts events for volume reduction.
• 📥 logs/from-tenx — Processed events come back to the Collector on :24225 via its OTLP/gRPC receiver. Keep this pipeline processor-free; the destination exporters consume events directly so enrichment never re-fires on the return path.
• 📤 Destinations — Your OTel destination exporters (elasticsearch, splunkhec, kafka, awss3, …) consume the return pipeline and ship to the real destinations.

What an event looks like on the way back

Every attribute that came in over OTLP — resource attributes, scope info, log-record attributes, body — round-trips back to the Collector. The LogRecord.body carries the message verbatim in its original AnyValue shape (so a body.stringValue is byte-for-byte unchanged); the resource-vs-log-attribute distinction is collapsed (everything comes back as a log-record attribute), but no data is lost. What changes between in and out depends on the Receiver app mode:

Mode	Difference vs the event the Collector sent in
Receive (default)	None. Same record.
Receive + symbolMessageHashField <name>	Same record + one new field named <name> carrying the symbol-pattern hash (a stable identifier for the message pattern — usable as a dedup key, metric dimension, or correlation ID).
receiverOptimize true	The value of the field captured by otelCollectorInputMessageField (default body) is replaced with a compact encoded form. A separate tenx-template event is emitted with the template needed to decode it. All other fields stay verbatim.
receiverOptimize true + symbolMessageHashField <name>	Both of the above.

Log10x reads the log line text via the JSON field configured by otelCollectorInputMessageField (default body); the tag field stamped by the input (from service.name, falling back to k8s.pod.name, then to the literal "otel") becomes the event's source. All other resource and log-record attributes (k8s.pod.name, k8s.namespace.name, service.name, …) come through as flat top-level fields on the record, available for message-pattern and rate filtering.

Key Files

File	Purpose
stream.yaml	OTel Collector OTLP/gRPC input + output stream definitions
conf/tenx-sidecar.yaml	Reference Collector config showing the two-pipeline split with no return-path processors

Quickstart

1. Run Log10x:

tenx @run/input/forwarder/otel-collector @apps/receiver

2. Wire up your Collector config — start from the sidecar recipe and add your real receivers + destination exporters. The logs/to-tenx pipeline carries everything from sources through enrichment to the otlp/tenx exporter; the logs/from-tenx pipeline carries the returning events directly to your destination exporters:

otelcol.yaml

receivers:
  filelog:
    include: [/var/log/app.log]

  # Receive processed events back from Log10x over OTLP/gRPC
  otlp/tenx:
    protocols:
      grpc:
        endpoint: 0.0.0.0:24225

processors:
  batch: {}

exporters:
  # Hand off to the Log10x sidecar over OTLP/gRPC
  otlp/tenx:
    endpoint: 127.0.0.1:4317
    tls:
      insecure: true

  debug:
    verbosity: detailed

service:
  pipelines:
    logs/to-tenx:
      receivers: [filelog]
      processors: [batch]
      exporters: [otlp/tenx]
    logs/from-tenx:
      receivers: [otlp/tenx]
      exporters: [debug]

For Splunk integration see the 10x for Splunk documentation. For Kubernetes deployment via the official OpenTelemetry Collector Helm chart see the Helm sidecar overlay.

Config Files

To configure the OpenTelemetry Collector module, Edit these files.

Below is the default configuration from: otel-collector/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/input/forwarder/otel-collector/config.yaml
• Within cloned repo
  ./pipelines/run/input/forwarder/otel-collector/config.yaml

Download

# 🔟❎ 'run' OpenTelemetry Collector receiver configuration
#
# To learn more see https://doc.log10x.com/run/input/forwarder/otel-collector/

tenx: run

# =============================== Dependencies ================================

include:
  - run/input/forwarder/config.yaml
  - run/modules/input/forwarder/otel-collector

# ====================== OpenTelemetry Collector Options ======================

otelCollector:

  # ----------------------------- Input Options -----------------------------

  input:

    # 'port' specifies the TCP port to listen on for OTLP/gRPC events from the
    #  OpenTelemetry Collector's `otlp` exporter.
    port: 4317

    # 'messageField' is the name of the JSON field on the rendered record
    #  carrying the log line text.
    messageField: body

  # ----------------------------- Output Options ----------------------------

  output:

    # 'host' specifies the TCP host of the OpenTelemetry Collector's `otlp`
    #  receiver receiving processed events from Log10x.
    host: 127.0.0.1

    # 'port' specifies the TCP port of the Collector's `otlp` receiver. MUST
    #  differ from input.port or the two would collide on the same listener.
    port: 24225

    # 'encodeType' is the on-wire encoding for the rendered record.
    #  - 'delimited': each top-level field of the rendered record becomes an
    #    attribute on the returning OTLP LogRecord; the `body` field is lifted
    #    into LogRecord.body so the message round-trips byte-for-byte.
    #  - 'json': the whole record is wrapped as one stringified field.
    encodeType: delimited

Options

Specify the options below to configure the OpenTelemetry Collector:

Name	Description	Category
otelCollectorInputPort	TCP port to listen on for OTLP/gRPC events from the OpenTelemetry Collector	Input
otelCollectorInputMessageField	Dotted JSON path to the field carrying the log line text	Input
otelCollectorOutputHost	TCP host of the OpenTelemetry Collector's OTLP/gRPC receiver	Output
otelCollectorOutputPort	TCP port of the OpenTelemetry Collector's OTLP/gRPC receiver	Output
otelCollectorOutputEncodeType	Output format when outputFields are set. Possible values: [json, delimited]	Output

Input

otelCollectorInputPort

TCP port to listen on for OTLP/gRPC events from the OpenTelemetry Collector.

Type	Default	Category
String	4317	Input

TCP port where Log10x listens for OTLP/gRPC log records from the OpenTelemetry Collector's otlp exporter. Match this against the endpoint of your Collector exporter.

otelCollectorInputMessageField

Dotted JSON path to the field carrying the log line text.

Type	Default	Category
String	body	Input

Top-level JSON field name on the input record that carries the actual log line text. The default (body) matches the flattened shape the input emits for plain-text OTLP bodies. Set to a different field name if your Collector pipeline copies the log line into a different attribute before exporting (e.g. an <attr_key> set via a transform).

Output

otelCollectorOutputHost

TCP host of the OpenTelemetry Collector's OTLP/gRPC receiver.

Type	Default	Category
String	127.0.0.1	Output

Hostname or IP where the OpenTelemetry Collector's otlp receiver is listening for processed events from Log10x. Pairs with otelCollectorOutputPort.

otelCollectorOutputPort

TCP port of the OpenTelemetry Collector's OTLP/gRPC receiver.

Type	Default	Category
String	24225	Output

TCP port where the OpenTelemetry Collector's otlp receiver is listening for processed events from Log10x. MUST differ from the port Log10x's own OTLP/gRPC input listens on (default 4317) or the two would collide on the same listener.

otelCollectorOutputEncodeType

Output format when outputFields are set. Possible values: [json, delimited].

Type	Default	Category
String	delimited	Output

Specifies how the combined output (main event field plus outputFields) is encoded when writing back to the OpenTelemetry Collector. Possible values:

• json: formats all fields as a JSON object
• delimited: formats field values separated by the output delimiter Only takes effect when otelCollectorOutputFields is set.

This module is defined in otel-collector/module.yaml.