Skip to content

Compile

The AOT compile pipeline produces symbol library files that enable the run stream processor pipeline to transform input events into typed TenXObjects. To launch this pipeline use the compiler app.

Workflow

This pipeline chains together the following key units:

graph LR
    A["<div style='font-size: 14px;'>📥 Pull</div><div style='font-size: 10px; text-align: center;'>Fetch Sources</div>"] --> B["<div style='font-size: 14px;'>🔍 Scan</div><div style='font-size: 10px; text-align: center;'>Extract Symbols</div>"]
    B --> C["<div style='font-size: 14px;'>🔗 Link</div><div style='font-size: 10px; text-align: center;'>Build Library</div>"]
    C --> D["<div style='font-size: 14px;'>📤 Push</div><div style='font-size: 10px; text-align: center;'>Commit Output</div>"]

    classDef pull fill:#2563eb88,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef scan fill:#059669,stroke:#047857,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef link fill:#ea580c88,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8
    classDef push fill:#16a34a88,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8

    class A pull
    class B scan
    class C link
    class D push

  • Pull

    Pull input files from local disk and remote repos as well as existing symbol files to reuse.

  • Scan

    Scan input files in an array of programming languages, text, and binary formats to produce symbol unit files.

  • Link

    Link symbol unit files into a symbol library artifact for runtime use.

  • Push

    Push output files to GitHub for use by subsequent run and compile pipelines.

Pull

Pull existing symbol units, source code, and binary input files from GitHub , Helm, Docker and Artifactory repos. Pulling symbol units produced by previous invocations of the compile pipeline enables reuse to avoid parsing previously processed source code/binary input files whose checksum has not changed.

Scan

Scan input files to extract symbol tokens representing low-cardinality values shared across instances of a logical app/infra event originating from a source code location/binary executable. They are analogous to compiled artifacts (e.g., .jar, .so) and are GitHub/Artifactory committable.

This phase utilizes a configurable set of scanner modules to parse input files in various formats.

Link symbol units into a single output Symbol library file to generate shared TenXTemplate schemas at runtime for input events.

This phase is analogous to a linker that takes one or more object files (i.e., symbol units) and combines them into a single library file.

Push

Push output symbol files to a target GitHub repo for reuse by subsequent compile invocations and for cloud/edge apps at runtime from a central configuration repository.


This pipeline is defined in compile/pipeline.yaml.