Module

Module YAML config files define the options which pipeline modules and units can receive at run time via input command line arguments and config files.

To load a module file at run time see YAML config.

Attributes

Each module file defines the following attributes:

name

Provides a logical name for this module (e.g., 'datadogLogsInput').

Input, output and extractor configurations can reference this value using their respective foreach values to create a matching configuration object for each instance of this module (only applies if allowMultiple is true).

title

Provides a concise human-readable title for this module (e.g., 'Datadog Logs Input').

description

Provides a short form description of this module.

configFolder

Specifies a folder containing default config files for the options defined by this module.

allowMultiple

Specifies whether multiple instances of this module can be defined via command line arguments and configuration files.

For example, input and output modules support multiple instances (e.g., read events from different ElasticSearch instances and publish summaries to different Prometheus endpoints).

doc

Provides long-form documentation for this module. This value supports markdown elements.

options

Specifies an array of options which define the configuration values this module can receive at run time via CLI argument, .yaml files and JSON requests.

Each option contains the following attributes:

Attribute	Description	Required	Possible value
names	List of logical names by which this option can be specified	✔️
type	Data type of values defined by this option	✔️	[number,boolean, string, list, file]
required	Specifies whether a value must be explicitly set for this options	✔️	True/False
category	Category name for this option to divide module options into logical sub-groups
defValue	Default value this option will have if not required and not specified by user
doc	Markdown content documenting use of this option

Configuration

The example module below is derived from the ElasticSearch input module.

To configure instances of an Elastic input at run time, specify its options via CLI arguments, configuration files or JSON.

For an example configuration see elastic/config.yaml.

module:

# 'name' provides a logical name for this module .

# Input/output/extractor configurations can reference this value using their 'foreach' members
# to create a matching object for each instance of this option group (only applies to option groups with 'allowMultiple' true). 

name: elasticInput 

# 'title' provides a concise human-readable description of this module
title: ElasticSearch input

# 'description' provides a short form description of this module
description: read events from an ElasticSearch hosted/on-premises cluster

# 'configFolder' specifies a folder containing default config files for the settings defined by this module
configFolder: run/config/input/analyzer/elastic

# 'allowMultiple' is enabled for this module. Pipeline command line arguments and configuration files
#  can specify multiple groupings of the options defined below.
allowMultiple: true

# 'doc' specifies a long-form description of this module's. This value supports markdown elements.
doc: |
  Configures an [ElasticSearch](https://www.elastic.co/elasticsearch) input to read events...

# 'options' specifies a list of option specifications that define the values of this module
#  can receive at run time via command line arguments, YAML or JSON.
options:

  # 'names' specifies the different names by which to identify each option 
  #  via command line arguments or a YAML/JSON field. This array value allows
  #  for defining long-form names (e.g., 'elasticSearchHost') which are suitable
  #  for JSON/YAML fields, and short-form/acronym (e.g., 'eh') suitable for use within a shell. context.
  - names:
      elasticSearchHost
      eh

  # 'description' provides a summary of this option. 
  #  This value will appear as part of the 10x Engine's -help printout. 
  description: elastic host address

  # 'type' defines the type of values this option can accept. 
  #  Supported values: number, string, boolean, list
  type: string

  # 'required' defines whether the value of this option must be set when 
  #  specifying an instance of this module using command line arguments or YAML/JSON.
  required: false

  # 'defValue' documents the default value for this option if 'required' is false
  defValue: 127.0.0.1

  # 'category' specifies an optional label for this option within the context of this module (e.g., 'authentication') for grouping options according to logical categories.
  category: authentication

  # 'doc' provides a long-form description of this module's purpose and examples.
  #  This value supports markdown elements.
  doc: |
    set the elastic host address to connect to. Examples: ...

  ...