Symbols

Identify the source code/binary executable origin of symbol values within TenXTemplate hidden classes.

Identifying where each symbol value originated from within TenXTemplates enables reflective programming wherein TenXObjects can inspect their structure and member types and source code/executable file origin.

Each symbol lists its file of origin (e.g., foo.js, sudo, acme.dll) and the context in which it appears in it (e.g., class/func/enum name, string format..).

Source code/executable origin information is accessible at run time via the symbol, symbolSequence, and symbolOrigin functions.

this.class = this.symbol("class");
this.sequence = this.symbolSequence("log,exec"); 
this.origin = this.symbolOrigin("log,exec");

For the following Apache Spark log event:

17/06/09 20:10:42 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.10.34.11:48069

Field	Value
class	CoarseGrainedExecutorBackend
sequence	Connecting to driver
origin	CoarseGrainedExecutorBackend.scala

For the following Unix log event:

Jun 14 15:16:02 combo sshd(pam_unix)[19937]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=218.188.2.4

Field	Value
class	""
sequence	authentication failure
origin	sshd

Contexts

Compile scanners assigns each extracted symbol value within a target source code/binary file a context which describes its meaning and position within the file.

Supported contexts include:

package

Source code package/namespace name. For example:

package com.acme.client.core;

  public class AcmeLogic {
  ...  
  }

namespace MyCompany.Proj1
{
    class MyClass1
    {
      ...
    }
}

class

Source code class/interface definition. For example:

class AcmeClient:
...

public interface AcmeServer {...}

The scanner will capture AcmeServer, AcmeClient as class symbols.

The scanner will capture com.acme.client.core, MyCompany.Proj1 as package symbols.

method

Source code method/function name. For example:

function hostError(host) {...}

void Person::get_name() {

The scanner will capture hostError, get_name as method symbols

enum

Source code enum literal value. For example:

from enum import Enum

class STATUS(Enum):
OK = 1
FAIL = 2
RETRY = 3

The scanner will capture:

• OK, FAIL, RETRY as enum symbols
• STATUS as a class symbol

log

Source code constant arg within a logging method. For example:

function hostError(host) {
  //The "error connecting" symbol passes as an indirect arg to the logger
  var msg = fmtError("error connecting", host);
  console.error(msg); 
} 

The scanner will capture error connecting as a log symbol.

const

Source code constant value assigned to a variable, parameter or code annotation. For example:

@JsonElement(key = "price")
public String price;

report("state: " + state);

this.key = "key";

The scanner will capture price, state, key as a const symbol.

exec

Symbol captured from a custom program or OS command via the executable symbol scanner (e.g., strings).

For example:

strings -a /usr/bin/sudo

The scanner will capture Authentication failure message %s as an exec symbol.

text

Config field name/value captured from file (e.g., .xml, .json) via the text symbol scanner.

For example:

spring:
  name: test
  environment: testing

The scanner will capture the field names spring, name, and environment as text symbols.

The test and testing field values are only captured as text if the textScanFieldValues is set to true.

Configuration

To configure the Origin selector unit, Edit these settings.

Below is the default configuration from: symbol/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/transform/symbol/config.yaml
• Within cloned repo
  ./pipelines/run/transform/symbol/config.yaml

Download

# 🔟❎ 'run' symbol origin configuration

# Identify the origin of symbol values within TenXTemplates.
# To learn more see https://doc.log10x.com/run/transform/symbol/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Origin Options ==============================

symbol:

  # 'maxOrigins' controls the number of source/binary origins to list 
  #  per symbol token sequence. As a series of tokens within 
  #  a target TenXTemplate (e.g., 'ERROR', 'could not connect to "{}') may appear 
  #  in multiple source/binary files within the pipeline's loaded library.
  #  The sorting algorithm configured below is used to select the 'maxSymbolOrigins' 
  #  topmost entries to list.
  maxOrigins: 64

  # 'sequenceReserved' defines a list of terms to ignore when searching
  #  for the symbol tokens that constitute an TenXTemplate's 'message' portion.
  #  For example, for an event whose text contains: 'connect success = true',
  #  the value 'true' will not be considered a part of the event's message,
  #  as the 'true' value is mostly likely the result of a variable boolean state.
  #  For more information, see: https://doc.log10x.com/api/js/#TenXObject+symbolSequence
  sequenceReserved:
    - "null"
    - "nil"
    - "true"
    - "false"
    - "to"
    - "the"
    - "a"
    - "at"
    - "for"
    - "log"
    - "info"
    - "http"  
    - TRACE
    - DEBUG
    - INFO
    - NOTICE
    - WARN
    - ERROR
    - CRITICAL
    - ALERT
    - EMERGENCY

  # ----------------------------- Debug Options -------------------------------

  debug:

    # 'symbol' debugging allows for verbose printing of the selection process
    #  for symbol tokens from the pipeline's 10x symbol files used to produce the
    #  results of 10x reflection functions.
    #  For more information, see: https://doc.log10x.com/api/js/#TenXObject+symbol


    # 'origins' outputs information about the origin selection process for an TenXTemplate symbol.
    #  For example, setting 'units' to 'foo.js' will output information about if/how it 
    #  was selected as the origin of TenXTemplate symbols.
    #  In other words, if a symbol (e.g., 'MyClass') has 'foo.java' as the source file
    #  from which it originated, adding 'foo.js' to 'units' will emit information
    #  about the selection process.
    #  Specifying '*' will emit information for all source /binary files that have
    #  been selected as the origin of any TenXTemplate objects within the pipeline.
    origins: [
    #  '*'
    ]

    # 'symbols' logs the origin selection process for symbol tokens within an TenXTemplate.
    #  For example, set 'symbols' to 'Could not connect to' to log how the 10x JavaScript 'symbolSequence' 
    #  determines the origin source code/binary file within the pipeline's symbol library. 

    #  Specifying '*' will emit information for all source code /binary files that have
    #  been selected as the origin of any TenXTemplate objects within the pipeline.
    symbols: [
     # '*'
    ]

Options

Specify the options below to configure the Origin selector:

Name	Description	Category
symbolSequenceReserved	List of terms to ignore when scanning symbol tokens	Symbol
symbolMaxOrigins	Max number of source/binary origins to list per symbol	Symbol
symbolDebugOrigins	List of source scanned file names for which to log information	Debug
symbolDebugSymbols	List of TenXTemplate symbol values for which to log information	Debug
symbolDebugLoggerName	Debug logger name	Debug

Symbol

symbolSequenceReserved

List of terms to ignore when scanning symbol tokens.

Type	Default	Category
List	[null, nil, true, false]	Symbol

Defines a list of terms that to ignore when searching for the symbol tokens constituting an TenXTemplate's 'message' portion (i.e., the longest consecutive sequence of symbol tokens). For example, for an TenXObject whose text contains: 'connect success = true', the value 'true' will not be considered a part of the instance's message, as it is mostly likely the result of a variable boolean state. To learn more see symbolSequence.

symbolMaxOrigins

Max number of source/binary origins to list per symbol.

Type	Default	Category
Number	64	Symbol

Controls the number of source code/binary origins to list per symbol token sequence. As a series of tokens within a given template (e.g., 'ERROR', 'could not connect to "{}') may appear in multiple source/binary files within the pipeline's symbol files, the sorting algorithm selects maxSymbolOrigins for comparison.

Debug

symbolDebugOrigins

List of source scanned file names for which to log information.

Type	Default	Category
List	[]	Debug

Specifies a list of input file names within the loaded symbol library (e.g., 'my.scala', 'foo.java') for which to log information on if/how they participate in selecting the origin of a target TenXTemplate symbol sequence. This option helps debug how the symbolSequence function selects origin values for a target TenXTemplate symbol sequence.

Specify '*' for all input files.

symbolDebugSymbols

List of TenXTemplate symbol values for which to log information.

Type	Default	Category
List	[]	Debug

List of symbol values such as class and method names(e.g., 'MyClass', 'foo') for which to log information on how their origin information is selected. This option helps debug how the symbolSequence function selects origin values for a target TenXTemplate symbol sequence.

Specify '*' for all symbols.

symbolDebugLoggerName

Debug logger name.

Type	Default	Category
String	consoleOut	Debug

Controls the name of the log4j logger used by debugOrigins and debugSymbols.

This unit is defined in symbol/unit.yaml.