Skip to content

Backfill

Defines a new metric in Datadog or Prometheus (via remote_write) and fills it in with historical data pulled from the Retriever archive — timestamps preserved, so the new metric looks like it has been running all along. ~15 minutes of config instead of ~2-4 weeks of Athena + remote-write Lambda engineering.

Example

"create a Datadog metric db_query_timeout by tenant, 90d backfilled"

New metric log10x.db_query_timeout_by_tenant defined in Datadog. Pulled db_query_timeout events from the archive, bucketed at 5m, grouped by tenant_id. 2,160 historical points emitted with original timestamps preserved. View: https://app.datadoghq.com/metric/explorer?metric=log10x.db_query_timeout_by_tenant

More to ask

  • "new Prometheus metric for Auth_Failed, 30d back"
  • "Datadog gauge for Payment_Retry rate per second, 90d"
  • "backfill cart_validation_failed count by service, 60d"

Prerequisites

This tool requires the Retriever deployed. Datadog and Prometheus destinations are wired; CloudWatch / Elastic / SignalFx return a clear "not yet implemented" error.

Tool schema (advanced)
Field Type Required Default Description
pattern string yes — Pattern to backfill — templateHash or symbolMessage. Use Identify first to resolve a raw log line.
metric_name string yes — Destination metric name. Follow the TSDB's naming convention (e.g., log10x.payment_retry_by_tenant).
destination string yes — One of: datadog, prometheus, cloudwatch, elastic, signalfx.
from string yes — Start of backfill window — ISO8601 or relative (now-90d).
to string no now End of backfill window.
aggregation string no count One of: count, sum_bytes, unique_values, rate_per_second.
bucket_size string no 5m One of: 1m, 5m, 1h, 1d.
group_by string[] no — Fields to group on. Each unique combo becomes its own time series.
unique_field string when aggregation=unique_values — The field whose distinct values are counted per bucket (e.g., tenant_id).
filters string[] no — Optional JS filter expressions over event payloads, passed through to the Retriever.
emit_forward boolean no false Wire the live Reporter to continue emitting going forward. Not yet implemented in the current build.
environment string no — Environment nickname (multi-env).