Skip to content

Reporter

The Reporter identifies storage and licensing cost drivers by analyzing app/infra events before forwarders ship them to log analyzers. It deploys as a DaemonSet alongside your forwarder — not a sidecar injected into it, and not a cloud app polling your SIEM.

Overview

What is the Reporter and what metrics does it provide

The Reporter is a read-only DaemonSet that tails the same event stream your forwarder sees, pre-SIEM -- it doesn't modify, filter, or redirect any data, and it is not in the critical log path. It provides real-time metrics including:

Unlike SIEM-side analysis, the Reporter captures pre-transport metrics — you see exactly what you're sending and what it will cost before data reaches your destination.

Does the Reporter reduce my costs directly

The Reporter provides visibility -- it shows where every dollar goes by classifying events using pre-compiled log templates from your source code.

To reduce costs, pair the Reporter with the Regulator:

  • Filter mode -- budget caps, rate sampling, declarative mute files
  • Compact mode -- 50%+ lossless volume reduction with a SIEM-side expand plugin

Start with Dev to preview savings locally, then deploy the Reporter (DaemonSet) for pre-SIEM cost visibility. For agentless SIEM-side analysis without deploying a DaemonSet, the log10x-mcp server offers an on-demand log10x_poc_from_siem_submit tool.

What visibility does the Reporter provide vs. my SIEM's built-in metrics

SIEM-side metrics show what arrived at your platform but lack granular per-event-type attribution and pre-transport context. They also reflect the SIEM's post-sampling view — you can't use them to claim "we see more than our SIEM."

The Reporter captures metrics at the origin (pre-SIEM) and automatically enriches every event -- no manual regex or configuration required:

  • Message extraction -- identifies the core message pattern from each event for cost-per-event-type metrics
  • Kubernetes context -- container, pod, and namespace for per-workload cost attribution
  • Severity level -- classifies by level (DEBUG, INFO, WARN, ERROR) to surface excessive debug logging
  • Multi-line grouping -- groups stack traces as single logical events for accurate costing
  • HTTP codes and GeoIP enrichment from log content

This surfaces optimization opportunities not available from the SIEM side -- like which specific services, event types, or severity levels drive costs. Use the Console to explore breakdowns interactively.

Integration & Deployment

Which log forwarders does the Reporter support

The Reporter works with all major log forwarders:

Deployment: a DaemonSet alongside your forwarder — not a mutating webhook that injects into forwarder pods, and not a sidecar inside them. Similar pattern to datadog-agent or splunk-otel-collector. Kubernetes deployment via Helm chart. Setup time: ~20 minutes.

The Reporter tails the same source your forwarder tails. No changes to existing fluent-bit / fluentd / otel-collector configs.

Resource requirements: Same engine as the Regulator — 512 MB heap + 2 threads handles 100+ GB/day per node. The Reporter is async and never in the data path, so it adds zero latency to log shipping. See Performance FAQ for sizing details.

Does the Reporter replace my existing forwarder

No. The Reporter runs alongside your forwarder, not in place of it. Your existing fluent-bit / fluentd / otel-collector / filebeat / splunk-uf continues to ship logs to your SIEM exactly as before. The Reporter reads the same files/streams in parallel to extract cost insight metrics.

Is the Reporter in the critical log path

No. The Reporter fails independently. If it goes down, your logs continue flowing to the SIEM uninterrupted — you temporarily lose cost-visibility metrics on that node until the DaemonSet pod respawns.

Where do the metrics go

The Reporter exports metrics to any supported time-series database:

  • Prometheus (scrape endpoint or remote write)
  • Datadog
  • CloudWatch
  • Elastic
  • SignalFx

Metrics include volume by message pattern, event counts, and cost estimates. Build dashboards in your existing tools, set alerts on cost thresholds, and correlate log costs with application metrics.

All metric destinations can be configured simultaneously. The system is extensible — define custom registries to add support for additional time-series systems.

Performance

What latency does the Reporter add

Zero. The Reporter is a separate DaemonSet pod — it is not in the data path. Metrics calculation runs in parallel with forwarding. If the Reporter stops, logs continue flowing to their destination unchanged.

What happens if the Reporter fails

Insights go stale; logs continue to the SIEM uninterrupted. The DaemonSet controller respawns the pod automatically.

Scenario Behavior
Reporter pod crash or OOM Kubernetes restarts the pod; metrics resume on next collection cycle. Logs unaffected.
Volume exceeds Reporter capacity Reporter falls behind on metric emission; logs unaffected. Increase DaemonSet resources or sampling if sustained.
Forwarder crashes Unrelated — Reporter tails source logs directly and is independent of the forwarder lifecycle.
Network interruption No effect on log shipping. Reporter queues metrics until Prometheus is reachable again.
Downstream SIEM slow or unreachable Unrelated — Reporter does not ship to the SIEM.
Rollback helm uninstall removes the DaemonSet; forwarders unaffected.

Use Cases

Can I use the Reporter without other Log10x products

Yes, the Reporter is standalone. It can be used independently to understand log cost distribution without other Log10x components.

The metrics are valuable on their own for:

  • Capacity planning and trend analysis
  • Budget forecasting with real volume data
  • Chargeback allocation across teams
  • Anomaly detection on volume spikes

All metrics feed into ROI Analytics -- Grafana dashboards showing cost per application, volume by severity, and top patterns by cost. Optional: enable AI-powered analysis for ranked optimization recommendations based on your metrics.

Can the Reporter identify repeated stack traces as cost drivers

Yes. Exception stack traces spanning 50-100 lines repeated thousands of times are a common cost driver.

The Reporter measures stack trace volume pre-SIEM -- and quantifies the savings potential:

  • Original: 80-line trace x 10,000 occurrences = 800,000 lines
  • After Regulator Compact mode compaction: 80-line template + 10,000 references = ~80,100 lines
  • Projected reduction: 90%+ for repeated stack traces

Works with all languages: Python tracebacks, Java/Kotlin, Node.js, Go panic dumps, Ruby, C# .NET, PHP. Deploy the Regulator to act on the findings (filter mode to cap, compact mode to shrink).