Skip to content

Integration

How 10x sits in front of Datadog, from Fluent Bit or through the Datadog Agent, how the products fit together across the Standard index, Flex Logs, S3, and metrics, and the AWS Lambda path for CloudWatch-origin logs.

Datadog Agent Integration

How do I integrate 10x with Datadog Agent

10x operates before logs reach Datadog, as a sidecar or preprocessing step. There's no Datadog agent plugin; instead, 10x processes logs via standard forwarders.

Two Integration Paths:

Path 1: Fluent Bit / Fluentd (10x sidecar container, Recommended)

10x sits between your application and Datadog's log intake, and Fluent Bit sends to Datadog directly with an API key:

Application logs → Fluent Bit (10x sidecar) → Datadog

Configuration (Fluent Bit output to Datadog):

[OUTPUT]
Name datadog
Match *
apikey ${DD_API_KEY}
json_date_key timestamp
dd_service myapp
dd_source myapp_logs

Path 2: Datadog Agent file input

10x writes optimized logs to a file; Datadog Agent reads from it:

logs:
  - type: file
    path: /var/log/app/optimized/*.log
    service: myapp
    source: myapp_logs
    # No special encoding config needed, logs are already optimized by 10x

Cost Impact:

Datadog meters log ingestion per GB, and separately meters indexing/retention per million events. tier_down moves low-value patterns into a Flex Logs index (cheaper, still queryable), cutting the index cost on the down-tiered slice; offload removes patterns from Datadog entirely. Savings depend on how much of your volume is low-value pattern noise.

Ask the MCP for a local cost POC to measure your specific reduction ratio before production deployment.

What is the performance overhead of 10x with Datadog Agent

The Receiver processes events at the edge before they reach Datadog, with resource use scaling with throughput. Memory is capped by the container limit, CPU via threadPoolSize. For resource requirements and Kubernetes resource specs, see Node counting.

Agent interaction: on the Datadog Agent path, the Receiver writes to a file or socket and the Agent reads from it asynchronously. No blocking.

Datadog Agent overhead: the agent ingests fewer bytes, since tier_down and offload remove low-value patterns from the Standard-index path, potentially reducing agent CPU.

No search-time overhead: events sent to Datadog (the hot path and the Flex tier) are standard log format. Compacted events live in S3, where the Retriever expands them, so there is no in-Datadog expansion cost.

Can I use 10x with Datadog's log sampling

Yes, they're complementary.

  • Datadog exclusion filters: query- and facet-based rules where you choose which logs to exclude from an index, at a sampling rate you configure from 0 to 100%. Excluded logs are still ingested and reduce indexing, not ingestion (lossy on the excluded slice for that index).
  • 10x: down-tier and offload are non-destructive levers. tier_down moves noisy patterns into the cheaper Flex Logs index (still queryable in Datadog); offload sends the rest to your own S3, where the Retriever fetches them back on demand. Both keep the data reachable. Filter mode (sample/drop) is a separate, lossy choice you opt into.
  • Together: down-tier and offload first, then add Datadog exclusion filters only if you still need to cut what gets indexed further.

Recommended: start with 10x. Re-evaluate exclusion filters after measuring actual reduction.

Architecture

How do the 10x products work together for Datadog

Data paths, deployed incrementally:

  1. Hot path, Receiver → Datadog Standard index: ERRORs, security alerts, SLO signals, and events powering real-time monitors go directly to Datadog in their original format. Receiver passes critical events and caps the rest with severity-aware rate limits.

  2. Down-tier path, Receiver marks low-value patterns (routeState) → Datadog Flex Logs index: the primary in-Datadog cost lever. Receiver caps each pattern and, under the action set for the service, routes the excess into a Flex Logs index, cutting the dominant index cost on that slice while the logs stay queryable in Datadog's own Log Explorer with no rehydration.

  3. Offload path, Receiver → S3 (patterns removed from Datadog entirely): patterns you want out of Datadog go to your own S3 (compacted losslessly for storage). When you need them, Retriever expands and streams them back on-demand, paying Datadog ingestion only on what you fetch.

  4. Metrics, high-volume repetitive patterns → Datadog metrics: Retriever aggregates high-volume repetitive S3 events into metric data points published to Datadog's time-series API. Dashboard visibility at metric pricing, not log pricing.

  5. Lambda path: for the functions themselves, the Extension runs the engine inside each execution environment and regulates events before they leave the function. Filtered and down-tiered events ship to the Datadog Logs API; to take patterns out of Datadog, the Receiver offload lever writes them to your own S3.

Datadog's Standard index receives only the events that justify it. See Retriever use cases for detailed workflows: incident investigation, dashboard population, compliance queries, and metric aggregation.

What the Receiver's Datadog metric output costs in custom metrics

Datadog counts a custom metric for each distinct metric name and tag set. Its price list gives $5 per 100 custom metrics a month, and its billing page bills indexed custom metrics beyond the allotment at the rate in the customer's contract and ingested custom metrics at $0.10 per 100; the allotment is 100 per host on Pro and 200 on Enterprise (both read 2026-10-11). The Receiver's Datadog metric output tags each per-pattern series with the pattern and where it ran (message_pattern, tenx_hash, tenx_user_service, severity_level, routeState, k8s_container, k8s_namespace) and with the runtime attributes (tenx_app, tenx_env), and with nothing that names one node or one process, so the same series comes from every node and the count does not grow with the fleet or with restarts.

The output publishes the 200 tag sets with the most bytes (topSeries, configurable; 0 publishes all). With the two shipped aggregators and two counters each, that is 800 custom metrics, inside a Pro allotment from ten hosts up. A pattern outside the top is published once it costs more than the smallest one inside. On the OpenTelemetry demo capture the top 200 patterns carried 98.8% of the bytes. To tag the host as well, add it as a runtime attribute, host_name:<host>, and expect the count to multiply by the number of nodes.

What log data should I send to Datadog vs S3

Datadog Standard index (via Receiver): events that need real-time indexing and monitoring, ERROR-level events, security alerts, SLO signals, and data powering active monitors and dashboards. Typically 10-30% of total volume.

Datadog Flex Logs (via tier_down): low-value patterns that still need to be searchable in Datadog but don't justify the Standard index. tier_down routes them to the cheaper Flex tier; they stay queryable in Log Explorer with no rehydration.

Your own S3 (via offload): patterns you want out of Datadog entirely, stored in your bucket. When you need specific data back in Datadog, Retriever streams it on-demand, paying Datadog ingestion only on what you fetch.

Most log data does not need to be in the Standard index at all times, it needs to be either in the cheaper Flex tier or available from S3 when you need it.

AWS Lambda

Can I optimize Datadog logs from AWS Lambda functions

Yes, for functions that ship logs through the OTel collector extension. Attach the published tenx-receive layer: the Extension runs the engine inside each function's execution environment and regulates events before they leave the function.

How it works: the collector hands each log record to the engine over loopback and gets it back carrying tenx_hash and routeState. A routing connector ships the Datadog slice through the collector's existing exporter and lands the offload slice in your own S3, where the Retriever fetches it back on-demand.

Cost levers: a mute file in your own S3 caps noisy patterns, tier_down routes low-value patterns to a Datadog Flex Logs index (cheaper, still queryable), and offload sends the rest to your own S3.

Deployment guide