Lambda
Lambda deploys the 10x Engine as an AWS Lambda container image that processes CloudWatch Logs subscription filter events. A thin Python handler bridges the Lambda event format to 10x's stdin input, and the built-in Fluent Bit output ships processed events to their destination (e.g., Datadog, Splunk, Elasticsearch).
This launcher employs the Cloud flavor (container image).
Use-case:
Replace log forwarder Lambdas (e.g., Datadog Forwarder) with a 10x-powered Lambda that reports, regulates, or optimizes CloudWatch Logs events before shipping to analytics platforms.
Benefits
Lambda deployment provides several technical advantages for processing CloudWatch Logs with the 10x Engine:
Zero Infrastructure
No Kubernetes, no sidecars, no persistent forwarders. The Lambda container image includes the 10x Engine, symbol library, Fluent Bit, and Python handler -- everything needed to process and ship logs. CloudWatch's subscription filter delivers batched events; Lambda's built-in retry handles reliability.
Drop-in Replacement
Subscribes to the same CloudWatch Log Groups as existing forwarder Lambdas. No changes to your application logging, CloudWatch configuration, or analytics platform setup.
Cold and Warm Starts
Lambda keeps the container warm across invocations. The 10x Engine loads once on cold start (JIT warms up over the first few invocations), then stays loaded for subsequent calls. Typical cold start overhead is comparable to any JVM-based Lambda.
Event-Driven Scaling
Lambda scales automatically with CloudWatch event volume -- no capacity planning or autoscaler tuning. Each invocation processes a batch of events independently, enabling parallel processing across log groups.
Architecture Flow
graph LR
A["CloudWatch<br/>Log Group"] -->|"Subscription<br/>Filter"| B["Python Handler<br/>(Decode + Pipe)"]
B -->|"1. Pipe events<br/>via stdin"| C["10x Engine<br/>(Report/Regulate/Optimize)"]
C -->|"2. Processed<br/>events"| D["Fluent Bit<br/>(Output Plugin)"]
D -->|"3. Ship to<br/>destination"| E["Analytics Platforms<br/>(Datadog, Splunk, etc.)"]
classDef aws fill:#ff9900,stroke:#cc7a00,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef tenx fill:#059669,stroke:#047857,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef handler fill:#8b5cf6,stroke:#7c3aed,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef forwarder fill:#3b82f6,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef destination fill:#6b7280,stroke:#4b5563,color:#ffffff,stroke-width:2px,rx:8,ry:8
class A aws
class B handler
class C tenx
class D forwarder
class E destination1. CloudWatch delivers batched log events (~256KB compressed, hundreds to thousands of events per invocation) via subscription filter.
2. Python handler decodes the gzip+base64 payload and pipes raw log lines to the 10x Engine's stdin.
3. 10x Engine processes the stream -- reporting, regulating, or optimizing events based on the selected app.
4. Fluent Bit ships processed events to the configured analytics platform using its native output plugins (Datadog, Splunk, Elasticsearch, CloudWatch). CloudWatch metadata (log group, log stream) is preserved as tags.
Deployment
Step 1: Prerequisites
| Requirement | Description |
|---|---|
| Log10x License | Your license key (get one) |
| AWS CLI | AWS CLI v2 installed and configured |
| Docker | For building the container image |
| ECR Repository | An ECR repository to push the image to |
| IAM Role | Lambda execution role with CloudWatch Logs read permissions |
Step 2: Build Container Image
The Dockerfile extends the standard 10x Docker image with the AWS Lambda Runtime Interface Client:
Step 3: Push to ECR
Replace <account> with your AWS account ID and <region> with your AWS region (e.g., us-east-1):
Step 4: Create Lambda Function
Replace the following values before running:
<account>— your AWS account ID<region>— your AWS region (e.g.,us-east-1)<lambda-execution-role>— name of an IAM role with CloudWatch Logs read and ECR pull permissions<your-10x-key>— your Log10x API key (get one)<your-dd-key>— your Datadog API key (or substitute with credentials for your destination)
aws lambda create-function \
--function-name 10x-lambda \
--package-type Image \
--code ImageUri=<account>.dkr.ecr.<region>.amazonaws.com/10x-lambda:latest \
--role arn:aws:iam::<account>:role/<lambda-execution-role> \
--memory-size 512 \
--timeout 300 \
--environment "Variables={TENX_API_KEY=<your-10x-key>,DD_API_KEY=<your-dd-key>,TENX_APP=@apps/edge/optimizer}"
Environment Variables:
| Variable | Required | Description |
|---|---|---|
TENX_API_KEY |
Yes | Log10x API key for licensing and metrics |
TENX_APP |
No | App to run: @apps/edge/reporter, @apps/edge/regulator, or @apps/edge/optimizer (default: reporter) |
TENX_EXTRA_ARGS |
No | Additional CLI arguments for the 10x engine |
Output destination is controlled by the 10x pipeline configuration, which uses Fluent Bit output plugins to ship processed events. Set the appropriate environment variables for your destination:
| Destination | Environment Variables |
|---|---|
| Datadog | DD_API_KEY, DD_SITE (default: datadoghq.com) |
| Splunk | SPLUNK_HOST, SPLUNK_PORT, SPLUNK_HEC_TOKEN |
| Elasticsearch | ELASTICSEARCH_HOST, ELASTICSEARCH_PORT, ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD |
| CloudWatch | AWS credentials via IAM role or env vars |
Resource Recommendations:
| Setting | Recommended | Notes |
|---|---|---|
| Memory | 512 MB - 1 GB | 10x engine ~50-80MB + symbol library |
| Timeout | 300 seconds | Generous for large batches; most invocations complete in seconds |
| Concurrency | Start with 10 | Scale up based on log group volume |
Step 5: Subscribe to CloudWatch Log Group
Replace the following values before running:
<your-log-group>— the CloudWatch Log Group to subscribe (e.g.,/aws/lambda/my-app)<region>— your AWS region<account>— your AWS account ID
aws logs put-subscription-filter \
--log-group-name <your-log-group> \
--filter-name 10x-optimizer \
--filter-pattern "" \
--destination-arn arn:aws:lambda:<region>:<account>:function:10x-lambda
Repeat for each log group you want to optimize. Use --filter-pattern to target specific log events if needed.
Step 6: Verify
Trigger a test invocation using the sample event included in the docker-images/lambda repository:
aws lambda invoke \
--function-name 10x-lambda \
--payload file://test-event.json \
/tmp/response.json
Check CloudWatch Logs for the Lambda function itself:
Verify no errors appear. Processed events should appear in your analytics platform shortly after invocation.
App Selection
The same edge apps work in Lambda -- choose based on your goal: