Index
Deploy the Retriever in the shape that fits your workload. Both modes use the same engine image, the same S3 layout, and the same four roles (indexer, query, subquery, stream) — so you can start on one and move to the other without re-indexing.
At a glance
| Lambda | Kubernetes | |
|---|---|---|
| Best for | Bursty query traffic, low ops, pay-per-invocation | Steady high-volume ingest, in-cluster neighbors |
| Always-on cost | ~$0 (optional warm pool via Provisioned Concurrency) | Always-on pods |
| Cold query p50 | ~7 s (sub-10 s with Provisioned Concurrency) | ~2 s |
| Warm query p50 | ~1.2 s | ~1 s |
| Index scaling | SQS batch × per-function concurrency | HPA on the indexer cluster |
| Event output | Pipeline emits directly to SIEM | Fluent Bit sidecar on stream pods |
| Max query time | 15 min (Lambda hard limit) | Unbounded |
| Clouds | AWS | AWS EKS (GKE, AKS, self-hosted on roadmap) |
| → | Deploy on Lambda | Deploy on Kubernetes |
Cost at 1 TB/day
Monthly, ballpark, 30-day retention, 1000 queries/day. Vendor rows are published list prices before enterprise discount.
| Stack | Monthly | Query latency |
|---|---|---|
| Retriever on Kubernetes | ~$1,500 | ~1 s warm · ~2 s cold |
| Retriever on Lambda | ~$2,945 | ~1.2 s warm · ~7 s cold |
| DIY Parquet + Athena | $2K–$3K + engineering build | seconds |
| AWS Athena on raw S3 | ~$6K | 30 s – 5 min |
| Self-managed Elasticsearch | $8K–$15K + ops | sub-second |
| AWS OpenSearch (managed) | $15K–$25K | sub-second |
| Grafana Cloud Loki | ~$15K | seconds |
| Datadog Flex Logs | ~$18K | seconds |
| Datadog Logs (Standard) | ~$75K | sub-second |
| Splunk Cloud | $60K–$250K | sub-second |
| Datadog Log Rehydration | $0 hot + $1.50/GB per query | hours |
Two axes at once:
- Mode choice: K8s is cheaper than Lambda at 1 TB/day sustained because Lambda compute is priced per second and the indexer runs near-continuously. Lambda wins the cost comparison when ingest is bursty or intermittent.
- Retriever vs alternatives: 5×–80× gap against hot SIEMs, roughly level on cost with Athena-on-S3 but with sub-second query latency instead of 30 s – 5 min scans.
Which one fits
| If you're… | Pick |
|---|---|
| Launching a new retriever and want the fewest moving parts | Lambda |
| Already running on AWS EKS and want the retriever next to your other workloads | Kubernetes |
| Bursty: hours of quiet punctuated by short periods of heavy query activity | Lambda |
| Steady: continuous high-throughput ingest across the day | Kubernetes |
| Need queries longer than 15 minutes (deep archival rehydration, compliance pulls) | Kubernetes |
| Want to minimize ops: no cluster, no HPA, no node-sizing | Lambda |
What's identical
- Same engine image — built from
pipeline/run-lambda/(Lambda) orpipeline/run-quarkus/(Kubernetes); both wrap the same pipeline binary in a different front-end. - Same S3 layout — raw logs under your chosen source prefix; bloom + reverse-index artifacts under
tenx/{target}/.... - Same query semantics — both accept the same query JSON and produce events +
_DONE.jsonmarkers identically. - Same MCP integration —
log10x_retriever_queryandlog10x_retriever_serieswork against either, transparently.
Switching later
Because the two modes read and write the same S3 layout, you can run both against the same archive during a migration. Stand up Lambda for query-only against your existing Kubernetes-indexed archive, measure, then cut over indexing last — or do the reverse.