Symbol Library Linker

Combine symbol units into a single output symbol library file used by the 10x run pipeline to generate shared TenXTemplates schemas for input log/trace events.

This phase is analogous to a linker that takes one or more object files (i.e., symbol units) generated during a compilation process and combines them into a single library file for use at run time.

Symbol Library

To dynamically create class and metadata information at run time, the 10x compile pipeline scans GitHub and container repos ahead-of-time to generate symbol libraries, similar to package artifacts (e.g., npm, .jar, .NET assembly).

The 10x run pipeline utilizes symbol library files to assign a shared TenXTemplate (i.e., hidden class) to input log/trace events with the same structure, creating a cached optimized schema for each event type.

Operating on typed TenXObjects enables direct access to symbol and variable values at runtime without repeatedly parsing JSON structures or evaluating complex, brittle regular expressions for each event.

File Structure

Each symbol library .10x.tar archive file contains:

A .10x.json file combining the symbol unit files generated during the scan phase.
A .10x.pb Protocol Buffer file which provides a reverse in-mem index of the symbols values contained within the JSON for fast loading and random access. To learn more see the .proto IDL.

Configuration

To configure the Symbol library linker unit, Edit these settings.

Below is the default configuration from: link/config.yaml.

Edit Online

# 🔟❎ 'compile' merge configuration

# Combine symbol units into a single output symbol library file.
# To learn more see https://doc.log10x.com/compile/link/

# Set the 10x pipeline to 'compile'
tenx: compile

# ============================ Symbol Library Output ==========================

# 'outputSymbolLibraryFile' specifies location of output symbol library file which
#  links the contents of 'outputSymbolFolder' to a single .json artifact and a
#  compressed .zip version for use at run time by 10x 'run' pipelines.
#  If a local output dir exists write to it, otherwise use the pipeline's temp folder.
#  To learn more see https://doc.log10x.com/compile/link

outputSymbolLibraryFile: $=TenXEnv.get("TENX_OUTPUT_SYMBOL_LIBRARY_FILE", path("data/shared/symbols", "<tenx.io.tmpdir>") + "/" + TenXEnv.get("runtimeName", "symbols") + ".10x.tar")


# ============================ Symbol Filter options ==========================

# The options below control which symbol files and values are merged into 'outputSymbolLibraryFile'

symbol:

  # 'fileNameFilters' — exclude test files from symbol library
  fileNameFilters:

    # ── Java ───────────────────────────────────────
    - ^.*(Test|Tests|TestCase|TestSuite)\.java$     # *Test.java, *TestCase.java, *TestSuite.java
    - ^Test.*\.java$                                # TestFoo.java (legacy)

    # ── JavaScript / TypeScript ────────────────────
    - ^.*\.(test|spec)\.(js|jsx|ts|tsx|mjs|cjs)$    # *.test.js, *.spec.tsx

    # ── Python ─────────────────────────────────────
    - ^test_.+\.py$                                 # test_utils.py
    - ^.+_test\.py$                                 # utils_test.py
    - ^test(s|case|cases|suite|suites)?\.py$        # test.py, tests.py, testcase.py, testcases.py, testsuite.py

    # ── Scala ──────────────────────────────────────
    - ^.*(Spec|Test|Suite)\.scala$                  # *Spec.scala, *Test.scala, *Suite.scala

    # ── C# ─────────────────────────────────────────
    - ^.*(Test|Tests|TestCase|TestSuite)\.cs$       # *Test.cs, *TestCase.cs, *TestSuite.cs

    # ── Go ─────────────────────────────────────────
    - ^.*_test\.go$                                 # *_test.go
    # Note: testdata.go is usually a data file, not test code → not filtered here

    # ── C++ ───────────────────────────────────────
    - ^.*(_test|Test|Tests)s?\.(cpp|cc|cxx|hpp|h|hxx)$  # *_test.cpp, *Test.hpp, *Tests.cpp

  # 'folderFilters' — patterns to exclude files by folder
  # Plain name (e.g., "test") → matches direct parent folder only
  # Regex (e.g., "[/\\]vendor[/\\]") → matches anywhere in full path
  folderFilters:
    - test           # direct parent is "test"
    - tests          # direct parent is "tests"
    - __tests__      # Jest convention
    - spec           # RSpec/Jasmine convention
    - specs          # RSpec/Jasmine convention
    - unit           # unit test folder
    - integration    # integration test folder
    - e2e            # e2e tests (Cypress, Playwright)
    - end-to-end     # alternate e2e naming
    - acceptance     # acceptance test folder
    - testdata       # Go test data folder
    - fixtures       # test fixtures folder
    - mocks          # mock implementations folder
    - "[/\\\\]vendor[/\\\\][a-z]+\\.[a-z]"   # Go vendored deps (vendor/domain.tld pattern)

  # 'types' controls the types of symbols written into 'outputSymbolLibraryFile'
  #  for reflective use at runtime.
  #  Narrowing the this list reduces the size of the output 10x symbols file. Possible values:
  types:
    - class   # source code class/interface name (e.g. 'AcmeLogic')
    - enum    # source code enum/literal name (e.g. 'severity: {error,warn,debug}')
    - log     # source code print format passed to a logging method/stream (e.g. 'err("could not connect to %s",..');
    - exec    # string value extracted from a binary executable input file (e.g. via 'strings' nix command)
    # - method # code method/function name (e.g. 'foo')
    # - const  # string constant assigned into a variable or method arg (e.g. 'greeting = "hello world"')
    # - text    # string value, usually read from a config file (e.g. JSON/XML/YAML field name)
    # - package # source code package/namespace/module name (e.g. 'com.acme.core')

Options

Specify the options below to configure the Symbol library linker:

Name	Description
outputSymbolLibraryFile	Location of output symbol library file
maxSymbolUnitFileSize	Max file size for 'outputSymbolLibraryFile' before it is split into chunks
symbolTypes	Token types read from input source/binary files written to output
symbolFileNameFilters	Regex patterns to exclude symbol files from output symbol library
symbolFolderFilters	Patterns to exclude symbol files by folder from output symbol library

`outputSymbolLibraryFile`

Location of output symbol library file.

Type	Default
File

Specifies an output symbol library file merging the symbol unit files produced for inputPaths. The symbol library file provides a single artifact to set as the symbolPaths at run time.

`maxSymbolUnitFileSize`

Max file size for 'outputSymbolLibraryFile' before it is split into chunks.

Type	Default
String	40MB

Sets the maximum file size (e.g., 40MB) for outputSymbolLibraryFile before splitting into chunks, each weighing no more than this value. This option enables large artifacts to remain GitHub committable.

`symbolTypes`

Token types read from input source/binary files written to output.

Type	Default
List	[package, class, enum, log, text, exec]

Filter symbol values to write to outputSymbolLibraryFile based on their context. This option reduces the size of the output symbol library file by omitting symbol types not required at run time. For possible values, see: contexts.

`symbolFileNameFilters`

Regex patterns to exclude symbol files from output symbol library.

Type	Default
List	[]

Filters symbol files to exclude from outputSymbolLibraryFile using regex patterns. This option is typically used to exclude test files from the symbol library, reducing output size and focusing on production code symbols.

Common patterns for excluding test files across languages:

Java: ^.*(Test|Tests|TestCase)\.java$ (JUnit/TestNG test files)
JavaScript/TypeScript: ^.*\.(test|spec)\.(js|jsx|ts|tsx|mjs|cjs)$ (Jest/Mocha/Vitest test files)
Python: ^test_.+\.py$ or ^.+_test\.py$ (pytest/unittest test files)
Scala: ^.*(Spec|Test)\.scala$ (ScalaTest/Specs2 test files)
C#: ^.*(Test|Tests)\.cs$ (xUnit/NUnit/MSTest test files)
Go: ^.*_test\.go$ (built-in testing framework test files)
C++: ^.*(_test|Test)s?\.(cpp|cc|cxx|hpp|h|hxx)$ (GoogleTest/Catch2/Boost.Test test files).

`symbolFolderFilters`

Patterns to exclude symbol files by folder from output symbol library.

Type	Default
List	[]

Filters symbol files to exclude from outputSymbolLibraryFile based on folder location. This option complements symbolFileNameFilters by excluding files based on their containing folder.

Plain folder name (e.g., test) - matches when the file's direct parent folder has this name. Only alphanumeric characters, underscores, and hyphens are treated as plain names.

Regex pattern (anything else) - matched against the full file path (case-insensitive). Use [/\\] for cross-platform path separators (Unix / and Windows \).

Examples:

test - Files directly inside a test folder
__tests__ - Files directly inside __tests__ (Jest convention)
[/\\]vendor[/\\] - Files anywhere under a vendor folder (Go vendored deps)
[/\\]node_modules[/\\] - Files anywhere under node_modules.

This unit is defined in link/unit.yaml.