Symbol Message Calculator

Adds a name to every event, made of the known words of the log statement that most likely wrote it, such as Receive_ListRecommendations_for_product_ids. Events from one statement share that name whatever IDs or timestamps they carry, unless a value that is itself a known word splits them. The Reporter reports cost by this name, the Receiver samples or mutes by it, and metric outputs publish counts per name.

Fields

The module writes these fields on each event:

Field	Example	Option
message_pattern	Receive_ListRecommendations_for_product_ids	symbolMessageField, on in the shipped configuration
Pattern hash	An 11-character hash of the name, for joins and metric tags	symbolMessageHashField, off by default; metrics carry it either way
Origin	recommendation_server.py:ListRecommendations	symbolOriginField, off by default
Skeleton	The message with ~ for each word of the name and $ for each value	symbolSkeletonField, off by default

A template is the exact shape of one line format, which compact uses to store events. One log statement can produce several templates. Templates of one statement that differ only in values outside the library, such as timestamps and numbers, share one name; a value made of library words is read as part of the name, as the browser names are in the Web example below.

In the engine's metrics, tenx_hash is the pattern hash. On a Splunk compact event, the field of that name holds the template hash.

Message Extraction

The module names each event in five steps, run in this order:

1. Pick the text

The module reads the field named by inputField, such as log, so envelope keys such as stream and kubernetes stay out of the name. With several fields listed, it reads the first one the event has.

For a multi-line event, it reads the first line that holds a message. In a stack trace that is the exception line. The .NET console logger writes info: cart.cartstore.ValkeyCartStore[0] and then GetCartAsync called with userId=8d3b2c1a as two records, and the second one names the event.

2. Find the source

Each phrase the symbol library knows points to the places in code that can print it. The module picks the place that best explains the line, comparing in this order:

1. Words from the message itself. A phrase matched only inside a JSON value or brackets ranks lower, so a long error string carried in an attribute does not outweigh the short message it belongs to.
2. The length of the longest phrase that place prints.
3. How many distinct words of the line that place explains, and their total length.

Any remaining tie resolves the same way on every run and every node.

3. Check the evidence

The source is accepted when at least two words of the message match it; otherwise the event is named by step 5 and gets no origin. A path such as /var/lib/app counts as one word, and parts of IDs do not count. With symbolContexts set to any, one word is enough.

4. Build the name

The name is the matched words in the order they appear, joined with _, up to symbolMaxLen characters (120 by default).

• The name stops at a timestamp, a bracket or the end of the line, so request IDs, thread names and node IDs printed before the message stay out.
• A logger or class name printed between the timestamp and the message stays in, as in oteldemo_AdService_Targeted_ad_request_received_for.
• When the message quotes an exception after :, at most half the length goes to the quoted text, so two statements quoting the same exception keep different names.
• Numbers, IDs, timestamps, repeated words and any word outside the symbol library never enter the name. A value that is itself a known word can, as the browser names do in the Web example below.

A known word that is part of a value is skipped, so lines from opensearch-0 and opensearch-1 share one name:

• a word joined by -, _, . or $ to a number or another value, as in req-b3e2 or subdir2;
• a word inside a host:port such as www.evernote.com:443, or inside a path that has a value in it;
• an upper-case code that mixes letters and digits, such as L9ECAV7KIM.

5. Fall back

• One word found: the name is the line's known words from the start of the message. Bracketed prefixes, msg=-style keys and reserved words are skipped. A logger name such as org.apache.kafka.log.LocalLog is kept, since it often tells two statements apart.
• No known word: the name is template_ plus a hash of the line's shape, its fixed words and punctuation with the values removed. The name holds while the shape holds. A stack frame or other line listed in messageNegators that arrives as its own event is named the same way.

The brackets, key=value separators, message keys and reserved words these steps use are options of the Symbol unit, set in its config file.

See the JavaScript implementation of this module on Github.

When Names Change

A name stays the same across IDs, timestamps, nodes, restarts, pod renames and the order events arrive in. The same engine version, symbol library and configuration always give the same name. A name changes when:

• the statement's text changes in code;
• the symbol library is recompiled and a word on the line changes between value and known word;
• an engine upgrade changes the naming rules.

Examples

OTel Demo .NET Java Web

{
  "stream": "stderr",
  "log": "2025-04-17 14:32:40,287 INFO [main] [recommendation_server.py:47] - Receive ListRecommendations for product ids:['L9ECAV7KIM', '0PUK6V6EV0']",
  "kubernetes": { "container_name": "recommendationservice", "namespace_name": "default" }
}

Pattern: Receive_ListRecommendations_for_product_ids

Origin: recommendation_server.py:ListRecommendations

A header record from the .NET console logger, then the message:

{ "stream": "stdout", "log": "info: cart.cartstore.ValkeyCartStore[0]", "kubernetes": { "container_name": "cart" } }
{ "stream": "stdout", "log": "      GetCartAsync called with userId=02fec73e-9f03-11f0-9b9e-a666c4b68b87", "kubernetes": { "container_name": "cart" } }

Pattern: GetCartAsync_called_with_userId

Origin: ValkeyCartStore.cs:ValkeyCartStore

{
  "stream": "stdout",
  "log": "2025-10-01 20:12:37 - oteldemo.AdService - Targeted ad request received for [accessories] trace_id=262794c5d52cea66092b4d8de7d1c6ed span_id=11dc1636e05888c3 trace_flags=01 ",
  "kubernetes": { "container_name": "ad", "namespace_name": "default" }
}

Pattern: oteldemo_AdService_Targeted_ad_request_received_for

Origin: AdService.java:getAds

{
  "stream": "stdout",
  "log": "192.168.43.96 - - [01/Aug/2025:22:21:50 +0000] \"GET /products/LensCleaningKit.jpg HTTP/1.1\" 200 101928 \"http://frontend-proxy:8080/\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/133.0.0.0 Safari/537.36\"",
  "kubernetes": { "container_name": "image-provider", "namespace_name": "default" }
}

Pattern: GET_HTTP_http_frontend_proxy_Mozilla_X_Linux_AppleWebKit_KHTML_like_Gecko_Safari

Origin: none

Config Files

To configure the Symbol Message Calculator module, Edit these files.

Below is the default configuration from: message/config.yaml.

Edit Online Edit Locally

Edit config.yaml Locally

Download and save to:

• Linux / Docker / macOS
• Custom directory
  $TENX_CONFIG/run/initialize/message/config.yaml
• Within cloned repo
  ./pipelines/run/initialize/message/config.yaml

Download

# 🔟❎ 'run' symbol lookup configuration

# Configure a symbol origin lookup to enrich TenXObjects
# To learn more see https://doc.log10x.com/run/initialize/message/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include: run/modules/initialize/message

# ============================== Symbol Options ===============================

symbol:

  # 'contexts' lists the kinds of symbols that can name an event: 'log' is text
  #  passed to a logging call in source code, 'exec' is text read from a program
  #  file. Values, in any case: package, class, method, log, enum, const, text,
  #  exec, any. Order does not matter.
  #  To learn more see https://doc.log10x.com/run/transform/symbol/#contexts
  contexts: log,exec

  # 'messageField' is the field that holds each event's name.
  messageField: message_pattern

  # 'messageHashField' is the field that holds an 11-character hash of the name.
  #  Metrics and aggregation carry the hash either way. Name a field here to also
  #  add the hash to events sent back to a log forwarder:
  #
  # messageHashField: tenx_hash

  # 'maxLen' is the longest name to write, in characters. 0 means no limit.
  maxLen: 120

# ============================== Message Options ==============================

# 'negators' lists line starts that mark a line as a continuation of the event
# above it, such as a stack frame. A continuation line that arrives as its own
# event is named by its shape, template_<hash>, and not from the symbol library.
message:
  negators:
    - " "                 # any indented line (stack frames, indented app detail, multi-line JSON, banners)
    - "\t"                # real tab byte, for msgpack/forward inputs that preserve raw whitespace
    - "\\t"               # literal backslash+t, for file/JSON inputs that keep escape sequences raw
    - "Caused by: "       # Java chained exception (column 0)
    - "Suppressed: "      # Java suppressed exception (column 0)
    - "--- End of "       # .NET inner exception marker (column 0)
    - "created by "       # Go goroutine creator (column 0)
    - "#0 "               # PHP stack frame depth 0
    - "#1 "               # PHP stack frame depth 1
    - "#2 "               # PHP stack frame depth 2
    - "#3 "               # PHP stack frame depth 3
    - "#4 "               # PHP stack frame depth 4
    - "#5 "               # PHP stack frame depth 5

Options

Specify the options below to configure the Symbol Message Calculator:

Name	Description
symbolContexts	Kinds of symbols that can name an event
symbolMaxLen	Longest name, in characters
symbolMessageField	Field that holds the name
symbolMessageHashField	Field that holds a hash of the name
symbolOriginField	Field that holds the statement's source location
symbolSkeletonField	Field that holds the message layout
messageNegators	Line starts that mark a continuation line

symbolContexts

Kinds of symbols that can name an event.

Type	Default
String	log,exec

A comma-separated list of the symbol contexts that can name an event. The default log,exec matches text passed to a logging call in source code and text read from a program file, such as the output of strings. Order in the list does not matter.

symbolMaxLen

Longest name, in characters.

Type	Default
Number	120

The longest name to write, in characters. Set 0 for no limit.

symbolMessageField

Field that holds the name.

Type	Default
String	symbolMessage

The field that holds each event's name. The shipped configuration sets message_pattern.

The name comes from the symbolSequence function.

symbolMessageHashField

Field that holds a hash of the name.

Type	Default
String	""

The field that holds an 11-character hash of the name. Unset by default.

The hash is TenXString.hash of the name: xxHash64 of its UTF-8 bytes, big-endian, in base64url without padding. Any tool can compute the same value, so it works as a join key, a metric tag or a forwarder filter.

Metrics and aggregation carry the hash whether or not this option is set. Events sent back to a log forwarder return as they arrived; set this option to add the hash to them, for example symbolMessageHashField tenx_hash.

symbolOriginField

Field that holds the statement's source location.

Type	Default
String

The field that holds where the statement is in code, such as recommendation_server.py:ListRecommendations. Unset by default.

The value comes from the symbolOrigin function.

symbolSkeletonField

Field that holds the message layout.

Type	Default
String	""

The field that holds the message with ~ in place of each word of the name and $ in place of each value. Unset by default.

A viewer holding the name and this field can print the line as it was written, punctuation and spacing included, without access to the logs.

messageNegators

Line starts that mark a continuation line.

Type	Default
List	[]

Line starts that mark a line as a continuation of the event above it, such as a stack frame. Empty by default. The shipped configuration lists:

• a space, a tab or a literal \t: any indented line;
• Caused by: and Suppressed:: Java chained exceptions;
• --- End of: the .NET inner exception marker;
• created by: the Go goroutine creator;
• #0 to #5: PHP stack frames.

A continuation line that arrives as its own event is named by its shape, template_ plus a hash, and not from the symbol library. The group module keeps such a line with the event above it, even when it holds a level word such as info.

This module is defined in message/module.yaml.