Modules
Modules package configuration, JavaScript and resource files to form reusable building blocks for compiling and running 10x apps.
This approach is similar to other software runtimes (e.g., JVM, Node.js) which support packaging code and config files into reusable libraries.
Compile
The Compile pipeline uses extensible modules to generate symbol libraries from an array of text and binary formats:
ANTLR parses source code from 50+ programming languages.
Jackson parses text/binary files in a wide array of configuration and serialization formats.
Executable import symbols from external sources such as OS commands (e.g., strings) and custom scripts (e.g., .py).
To learn more see the Compile API.
Run
The Run pipeline uses extensible modules to process typed TenXObjects to a variety of sources:
Input modules read log/trace events from log forwarders, analyzers and object storage.
Regulator modules filter 'noisy' telemetry using local/central policies.
Output modules write TenXObjects and aggregated summaries to event and metric destinations.
To learn more see the Input and Output API.
Files
Modules comprise the following file types:
| Type | Description | Format | Example |
|---|---|---|---|
| Module | Defines the options a module can receive via config files and cli arguments | .yaml | elastic/module.yaml |
| Config | Config files specifying arguments defined by anf other this modules | .yaml | executable/strings.yaml, elastic/stream.yaml |
| JavaScript | Control aspects of the module's behavior at run time | .js | policy-regulator-object.js |
| Lookup | Lookup tables for enriching TenXObjects | .tsv, .csv, .mmdb | lookup/http.csv |
 Camel Route1 |
Apache Camel route definitions and classes to connect to remote data sources | .yaml, .jar | elastic/route.yaml |
| Log4j2 Appender1 | Log4j2.yaml and appender classes for writing TenXObjects to local/remote destinations | .yaml, .jar | forward/log4j2.yaml |
| MicroMeter Registry1 | Config and registry classes for publishing TenXSummaries to metric outputs | .yaml, .jar. | datadog/stream.yaml |
Configuration
The 10x Engine loads module files from local or GitHub folders.
GitHub provides a centralized mechanism for publishing, managing and accessing module files.
Modules can specify a default YAML config file to provide example values for any of the options defined by the module.
For example, the elastic/config.yaml config loads the Elastic input module and outlines its option values to specify.
# 🔟❎ 'run' ElasticSearch input configuration
# This configuration defines an ElasticSearch input from
# which events are read and transformed into TenXObjects for aggregation and reporting.
# Set the 10x pipeline to 'run'
tenx: run
# =============================== Dependencies ================================
include: modules/input/analyzer/elasticsearch
# =============================== Elastic Options ==============================
# Multiple Elastic inputs can be defined below:
elastic:
- name: myElastic
....