Vector

Runs 10x Engine as a sidecar to Vector for reporting, receiving, and optimizing events before they ship to their destination (Elasticsearch, Splunk, S3, Kafka, …). Vector and Log10x run as peer processes, Vector sends events to Log10x via its native socket sink (newline-delimited JSON over TCP or Unix socket) and receives processed events back via its native fluent source (Fluent Forward protocol). Works against any stock Vector build (Linux/macOS/Windows) and the official vector/vector Helm chart with a values overlay.

Architecture

graph LR
    A["<div style='font-size: 14px;'>📂 Sources</div><div style='font-size: 10px;'>file, kubernetes_logs, journald</div>"] --> F["<div style='font-size: 14px;'>🧪 transforms</div><div style='font-size: 10px;'>enrichment</div>"]
    F --> B["<div style='font-size: 14px;'>📤 socket sink</div><div style='font-size: 10px;'>tcp/unix → :9000</div>"]
    B --> E["<div style='font-size: 14px;'>⚡ 10x Engine</div><div style='font-size: 10px;'>Receive/Optimize</div>"]
    E --> C["<div style='font-size: 14px;'>📥 fluent source</div><div style='font-size: 10px;'>:9001 (no transforms)</div>"]
    C --> D["<div style='font-size: 14px;'>📤 Destinations</div><div style='font-size: 10px;'>ES, Splunk, S3, Kafka</div>"]

    classDef input fill:#2563eb,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef filter fill:#ea580c,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef engine fill:#7c3aed,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef output fill:#16a34a,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8

    class A input
    class B filter
    class C filter
    class D output
    class E engine
    class F filter

Data Flow

📂 Sources, Your existing Vector sources (file, kubernetes_logs, journald, socket, …) feed events into the enrichment transforms.
🧪 Transforms, Your enrichment transforms (remap, filter, route, …) run here exactly once before the event is handed off to Log10x. The recipe places them between your sources and the tenx_in sink, so the bypass is structural, not configured.
📤 socket sink → Log10x, Vector forwards the enriched event to the Log10x sidecar over TCP :9000 (or a Unix socket on Linux/macOS) as newline-delimited JSON.
⚡ 10x Engine, The Receiver app applies rate/policy-based filtering and optionally compacts events for volume reduction.
📥 fluent source, Processed events come back to Vector on :9001 over the Fluent Forward protocol. Only your destination sinks consume tenx_out; no transforms sit between them, so enrichment never re-fires.
📤 Destinations, Vector's destination sinks (elasticsearch, splunk_hec, kafka, aws_s3, …) consume tenx_out and ship to the real destinations.

What an event looks like on the way back

The record structure of the original Vector event is preserved end-to-end, every field comes back to your destination sinks with the same name and same position. What changes depends on the Receiver app mode:

Mode	Difference vs the event Vector sent in
Receive (default)	None. Same record.
Receive + `symbolMessageHashField <name>`	Adds one new field with the symbol-pattern hash (a stable identifier for the message pattern, usable as a dedup key, metric dimension, or correlation ID).
`receiverOptimize true`	The value of the message field (`message` by default, or whatever `vectorInputMessageField` is set to) is replaced with a compact encoded form. A separate `tenx-template` event is emitted with the template needed to decode it. All other fields stay verbatim.
`receiverOptimize true` + `symbolMessageHashField <name>`	Both of the above.

The tag field stamped by Vector's ingest transform (typically from .source_type) is carried on the Forward wire as the Fluent tag, and surfaces as the event's source inside Log10x, used for rate-based grouping and emitted back to Vector on the return Forward record. Internally, Log10x's Vector input module reads the message text from the field named by vectorInputMessageField (default message); when the Receiver app is configured with k8sExtractorName: fluentK8s, the kubernetes.* sub-object is also materialized as enrichment fields for use by message-pattern and rate filtering.

Key Files

File	Purpose
`stream.yaml`	Vector socket input + Fluent Forward output stream definitions
`conf/tenx-sidecar.yaml`	Reference Vector config showing the ingest sink + egress source with no return-path transforms

Quickstart

1. Run Log10x:

tenx @run/input/forwarder/vector @apps/receiver

2. Wire up your Vector config, start from the sidecar recipe and add your real sources + destinations:

vector.yaml

sources:
  app_logs:
    type: file
    include: [/var/log/app.log]
    read_from: end

  # Receive processed events back from Log10x
  tenx_out:
    type: fluent
    mode: tcp
    address: 127.0.0.1:9001

transforms:
  # Enrichment runs here exactly once, the return path skips this block.
  ingest:
    type: remap
    inputs: [app_logs]
    source: |
      .cluster = get_env_var("CLUSTER_NAME") ?? "unset"
      .tag = .source_type

sinks:
  # Hand off to the Log10x sidecar
  tenx_in:
    type: socket
    inputs: [ingest]
    mode: tcp
    address: 127.0.0.1:9000
    encoding: { codec: json }
    framing: { method: newline_delimited }

  # Destinations consume tenx_out (not the raw sources or `ingest`) so
  # enrichment never re-fires on the return path.
  destinations:
    type: console
    inputs: [tenx_out]
    encoding: { codec: json }

For Splunk integration see the 10x for Splunk documentation. For Kubernetes deployment via the official Vector Helm chart see the Helm sidecar overlay.

Config Files

To configure the Vector module, Edit these files.

Below is the default configuration from: vector/config.yaml.

Edit Online

# 🔟❎ 'run' Vector receiver configuration
#
# To learn more see https://doc.log10x.com/run/input/forwarder/vector/

tenx: run

# =============================== Dependencies ================================

include:
  - run/input/forwarder/config.yaml
  - run/modules/input/forwarder/vector

# =============================== Vector Options ==============================

vector:

  # ----------------------------- Input Options -----------------------------

  input:

    # 'port' specifies the TCP port to listen on for events from Vector's
    #  `socket` sink (`mode: tcp`, `encoding.codec: json`,
    #  `framing.method: newline_delimited`).
    port: 9000

    # 'path' switches the input to a Unix domain socket instead of TCP.
    #  When set, takes precedence over 'port'.
    # path: /tmp/tenx-vector-in.sock

    # 'messageField' is the name of the JSON field in each Vector record
    #  carrying the log line text.
    messageField: message

  # ----------------------------- Output Options ----------------------------

  output:

    # 'host' specifies the TCP host of Vector's `fluent` source receiving
    #  processed events from Log10x.
    host: 127.0.0.1

    # 'port' specifies the TCP port of Vector's `fluent` source. MUST differ
    #  from input.port or the two would collide on the same listener.
    port: 9001

    # 'path' switches the output to a Unix domain socket instead of TCP.
    #  When set, takes precedence over 'host'/'port'.
    # path: /tmp/tenx-vector-out.sock

    # 'encodeType' is the on-wire format for the Forward record.
    #  - 'delimited': each top-level field of the rendered record becomes its
    #    own Forward record field, preserves the original record's structure.
    #  - 'json': the whole record is wrapped as one stringified field.
    encodeType: delimited

Options

Specify the options below to configure the Vector:

Name	Description	Category
vectorInputPort	TCP port to listen on for events from Vector's socket sink	Input
vectorInputPath	Unix domain socket path to listen on (Linux/macOS)	Input
vectorInputMessageField	Name of the JSON field carrying the original log line	Input
vectorOutputHost	TCP host of the Vector fluent source receiving processed events	Output
vectorOutputPort	TCP port of the Vector fluent source receiving processed events	Output
vectorOutputPath	Unix domain socket path of the Vector fluent source (Linux/macOS)	Output
vectorOutputEncodeType	Output format when outputFields are set. Possible values: [json, delimited]	Output

Input

`vectorInputPort`

TCP port to listen on for events from Vector's socket sink.

Type	Default	Category
String	9000	Input

TCP port where Log10x listens for events sent by Vector's socket sink (mode: tcp, encoding.codec: json, framing.method: newline_delimited). Match this against the address in your Vector sink.

`vectorInputPath`

Unix domain socket path to listen on (Linux/macOS).

Type	Default	Category
String	""	Input

When set, Log10x listens on a Unix domain socket instead of TCP, useful on Linux/macOS to avoid TCP overhead. Configure Vector's socket sink with mode: unix and a matching path. Ignored on Windows.

`vectorInputMessageField`

Name of the JSON field carrying the original log line.

Type	Default	Category
String	message	Input

Name of the field in each Vector record that contains the actual log message text. The default (message) matches Vector's convention for the file, kubernetes_logs, journald, and socket sources. Set to e.g. log if your sources emit the log line under a different key.

Output

`vectorOutputHost`

TCP host of the Vector fluent source receiving processed events.

Type	Default	Category
String	127.0.0.1	Output

Hostname or IP where Vector's fluent source is listening for processed events from Log10x. Pairs with vectorOutputPort.

`vectorOutputPort`

TCP port of the Vector fluent source receiving processed events.

Type	Default	Category
String	9001	Output

TCP port where Vector's fluent source is listening for processed events from Log10x. MUST differ from the port Log10x's own socket input listens on (default 9000) or the two would collide on the same listener.

`vectorOutputPath`

Unix domain socket path of the Vector fluent source (Linux/macOS).

Type	Default	Category
String	""	Output

Optional Unix domain socket path for Vector's fluent source. When set, takes precedence over vectorOutputHost / vectorOutputPort. Linux/macOS only, ignored on Windows.

`vectorOutputEncodeType`

Output format when outputFields are set. Possible values: [json, delimited].

Type	Default	Category
String	delimited	Output

Specifies how the combined output (main event field plus outputFields) is encoded when writing back to Vector. Possible values:

json: formats all fields as a JSON object
delimited: formats field values separated by the output delimiter Only takes effect when vectorOutputFields is set.

This module is defined in vector/module.yaml.