Pipelines

Each 10x Engine instance executes a configurable pipeline to read, process and output data:

• Compile

  ---

  Scan source code and binary repos to generate symbol library files.

  Learn more

• Run

  ---

  Transform event stream into TenXObjects for edge/cloud processing.

  Learn more

Units

Each pipeline is composed of units which are runtime components that read, process and output data. Units are chained together to form a pipeline.

Each pipeline unit defines its configurable values via a module file.

For example, the compile scan unit provides settings for extracting symbol values from source code/binary files, while the run input unit provides settings for reading log/trace events.

The 10x Engine supports the following unit types:

Type	Description	Compile Example	Run Example
Provider	Loads local/remote resources into the pipeline	GitHub	Symbol
Producer	Read input data into the pipeline	Source	Input
Processor	Process input data into output objects	Scan	Transform
Publisher	Write output objects to local/remote output	Link	Output
Observer	Observer and monitor pipeline execution	Console	Monitor

Upon initialization the 10x Engine scans its include paths folder for sub-folders containing pipeline definition files.

A pipeline will run until all producer units have completed, at which point it will signal all units to begin a shutdown process where each unit flushes its remaining data and the process terminates.