K8s Context Extractor

Enrich TenXObjects with Kubernetes context by extracting container, pod, and namespace k8s names from their enclosing text.

Example

For the event below, the log field contains raw text that is structured into a typed TenXObject.

This module extracts underlying event Kubernetes context such including container name, pod name, and namespace as named fields from a TenXObject's surrounding fullText for further processing and aggregation.

For example, for the event below this module will extract the following field values:

container_name: coredns
pod_name: coredns-7db6d8ff4d-pddxj
namespace_name: kube-system

{
  "log": "[INFO] plugin/reload: Running configuration SHA512 = f869070685748660180df1b7a47d58cdafcf2f368266578c062d1151dc2c900964aecc5975e8882e6de6fdfb6460463e30ebfaad2ec8f0c3c6436f80225b3b5b\n",
  "stream": "stdout",
  "docker": {
    "container_id": "4c195cfdbf7e41f640631629970b9af2d8a1f40f63dcffd15edca84e2e2e497e"
  },
  "kubernetes": {
    "container_name": "coredns",
    "namespace_name": "kube-system",
    "pod_name": "coredns-7db6d8ff4d-pddxj",
    "container_image": "registry.k8s.io/coredns/coredns:v1.11.1",
    "container_image_id": "docker-pullable://registry.k8s.io/coredns/coredns@sha256:1eeb4c7316bacb1d4c8ead65571cd92dd21e27359f0d4917f1a5822a73b75db1",
    "pod_id": "38b91d65-ba47-4d0f-a689-711056955842",
    "pod_ip": "10.244.0.99",
    "host": "minikube",
    "labels": {
      "k8s-app": "kube-dns",
      "pod-template-hash": "7db6d8ff4d"
    }
  },
  "tenx_tag": "kubernetes.var.log.containers.coredns-7db6d8ff4d-pddxj_kube-system_coredns-4c195cfdbf7e41f640631629970b9af2d8a1f40f63dcffd15edca84e2e2e497e.log"
}

Config Files

To configure the k8s Context Extractor module, Edit these files.

Below is the default configuration from: k8s/config.yaml.

Edit Online

# 🔟❎ 'run' k8s lookup configuration

# Configure a k8s field extractors to enrich TenXObjects
# To learn more see https://doc.log10x.com/run/initialize/k8s/

# Set the 10x pipeline to 'run'
tenx: run

# =============================== Dependencies ================================

include: run/modules/initialize/k8s

# ============================== Symbol Options ===============================

k8s:

  # 'extractorName' specifies which extractor (e.g., fluentK8s/filebeatK8s) to use for k8s metadata extraction
  extractorName: fluentK8s # extract k8s context using Fluent schema

  # 'namespaceNameField' specifies the name of the field in which to assign an extracted k8 namespace name status code, if found
  namespaceNameField: k8s_namespace

  # 'containerNameField' specifies the name of the field in which to assign an extracted k8 container name status code, if found
  containerNameField: k8s_container

  # 'podNameField' specifies the name of the field in which to assign an extracted k8 pod name.
  # Default empty. Pod names churn on every deploy (each restart mints a new pod ID), so
  # surfacing pod name as a metric-label dimension causes unbounded TSDB active-series growth.
  # The stable 'k8s_namespace' + 'k8s_container' enrichments above cover the common per-app
  # attribution case without the deploy-churn cost. To opt in to per-pod resolution, set this
  # to a concrete field name (e.g. 'k8s_pod'); the 'tenx_user_process' alias activates with it.
  podNameField: ""

  # 'teamLabel' names a pod label whose value is the team that owns the container, for
  # example 'team' or 'app.kubernetes.io/part-of'. Its value goes into 'teamField' and joins
  # the metric labels. It adds series only where one container name runs in several teams'
  # pods. Empty reads no label.
  teamLabel: ""

  # 'teamField' specifies the name of the field in which to assign the team label's value.
  # An event whose pod carries no such label gets no field, and its Datadog series carry
  # the tag with an empty value.
  teamField: tenx_team

Options

Specify the options below to configure the k8s Context Extractor:

Name	Description
k8sExtractorName	Name of extractor to use for k8s context extraction
k8sNamespaceNameField	K8s namespace target field name
k8sContainerNameField	K8s container name target field name
k8sPodNameField	K8s pod name target field name
k8sTeamLabel	Pod label key whose value names the team that owns the container
k8sTeamField	K8s team target field name

`k8sExtractorName`

Name of extractor to use for k8s context extraction.

Type	Default
String	fluentK8s

Defines the name of the extractor to use for k8s context extraction.

To learn more see k8s extractors.

`k8sNamespaceNameField`

K8s namespace target field name.

Type	Default
String	namespace

Specify the field name to assign with the extracted k8s namespace.

`k8sContainerNameField`

K8s container name target field name.

Type	Default
String	container

Specify the field name to assign with the extracted k8s container name.

`k8sPodNameField`

K8s pod name target field name.

Type	Default
String	pod

Specify the field name to assign with the extracted k8s pod name.

`k8sTeamLabel`

Pod label key whose value names the team that owns the container.

Type	Default
String

Specify the pod label key whose value names the team that owns an event's container, for example team or app.kubernetes.io/part-of. The value is assigned to the field k8sTeamField names and joins the metric labels. A container name that runs in one team's pods adds no series; a name shared by several teams' pods, such as a common init container, adds one per team. Empty reads no label.

`k8sTeamField`

K8s team target field name.

Type	Default
String	tenx_team

Specify the field name to assign with the value of the k8sTeamLabel pod label. An event whose pod carries no such label gets no field, and its Datadog series carry the tag with an empty value.

This module is defined in k8s/module.yaml.