Compiler
The compiler app produces symbol library files that enable the edge/cloud apps to transform input events into well-defined TenXObjects.
The compiler scans GitHub repos and container images to find symbol values (e.g., log statements, class names, enums), capturing their the source file and context within it.
The linker packages this information into symbol library files, similar to package artifacts (e.g., npm, .jar, .NET assembly).
Compile custom symbols if:
- Using proprietary or internal logging frameworks
- Using frameworks not in the default library
- Want symbols from your specific application code
Benefit
10x apps utilize symbol library files at runtime to assign a shared TenXTemplate (i.e., Hidden class) to incoming log events, creating a shared schema structure for each event type.
Operating on well-defined objects enables direct access to symbol and variable values, eliminating the overhead of parsing repetitive JSON structures and evaluating brittle, computationally expensive regex patterns for each event.
Out-of-the-Box Ready
The 10x runtime includes a built-in symbol library with industry-standard frameworks (Kubernetes, OpenTelemetry, Spring Boot, Django, and 30+ others). You can immediately test the Dev app with OpenTelemetry sample files or try it with your own logs for out-of-the-box optimization, even without custom symbol compilation.
Workflow
The Compiler app processes source code and binaries to generate symbol libraries that enable intelligent log processing and optimization.
graph LR
A["<div style='font-size: 14px;'>🔄 CI/CD</div><div style='font-size: 10px; text-align: center;'>Trigger Process</div>"] --> B["<div style='font-size: 14px;'>📥 Pull</div><div style='font-size: 10px; text-align: center;'>Fetch Sources</div>"]
B --> C["<div style='font-size: 14px;'>🔍 Scan</div><div style='font-size: 10px; text-align: center;'>Extract Symbols</div>"]
C --> D["<div style='font-size: 14px;'>🔗 Link</div><div style='font-size: 10px; text-align: center;'>Build Library</div>"]
D --> E["<div style='font-size: 14px;'>📤 Push</div><div style='font-size: 10px; text-align: center;'>Commit Output</div>"]
E --> F["<div style='font-size: 14px;'>📦 Distribute</div><div style='font-size: 10px; text-align: center;'>to Runtime Apps</div>"]
classDef cicd fill:#7c3aed88,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef pull fill:#2563eb88,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef scan fill:#059669,stroke:#047857,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef link fill:#ea580c88,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef push fill:#16a34a88,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8
classDef distribute fill:#f59e0b,stroke:#d97706,color:#ffffff,stroke-width:2px,rx:8,ry:8
class A cicd
class B pull
class C scan
class D link
class E push
class F distribute🔄 Invoke: runs locally for development, periodically as k8s CronJob for continuous updates, or integrated as part of CI/CD pipelines (GitHub Actions) for automated symbol generation
📥 Pull: Fetches source code from GitHub, Docker, Helm, and existing symbol files for reuse
🔍 Scan: Extracts symbol values from source code and binaries across multiple programming languages to create symbol unit files
🔗 Link: Combines symbol unit files into a single symbol library file for runtime use by edge and cloud apps
📤 Push: Commits symbol library files to GitHub for distribution to runtime apps and reuse in future compilation runs
📦 Distribute: Edge and cloud apps pull symbol libraries via the @github launch macro at startup and poll the remote branch for changes (e.g., every 5 minutes). Between compilation runs, events are processed using the previous symbol library version — slightly less efficient for new log patterns, but no data is lost. The next run picks up the changes automatically.
Architecture
The compiler executes periodically (k8s CronJob) or as a CI/CD pipeline (GitHub Actions) step to pull modified repos, process changes, and push updated symbol files to GitHub.
Getting Started
The Compiler is optional — you don't need to run it to get started. The 10x runtime includes built-in support for industry-standard frameworks and implements automatic JIT (just-in-time) symbol discovery for custom log formats.
Getting started guidance: - What if I have proprietary log formats — JIT fallback creates templates automatically, achieving ~80% efficiency - When should I run the Compiler — when you need higher efficiency or control over symbol discovery - CI/CD integration — automate compilation with GitHub Actions or Kubernetes CronJobs
See the Compiler FAQ for complete details on symbol discovery, compilation workflows, and troubleshooting.
This app is defined in compiler/app.yaml.