Introduction

The event processors provide an easy way to configure a set of functions in order to transform an event message that will be be written to a specific output.

While the event format is the de facto format used by gNMIc in case the output is influxdb or prometheus, it can be used with any other output type.

Transforming the received gNMI message is sometimes needed to accomodate the output system ( converting types, complying with name constraints,...), or simply filtering out values that you are not interested on.

The event format#

The event format is produced by gNMIc from the gNMI Notification messages received within a gNMI subscribe response update, it contains 5 fields:

• name: A string field populated by the subscription name, it is used as a part of the metric name in case of prometheus output or it can be used as the measurement name in case of influxdb output.
• timestamp: An int64 field containing the timestamp received within the gnmi Update.
• tags: A map of string keys and string values. The keys and values are extracted from the keys in the gNMI PathElement keys. gNMIc adds the subscription name and the target name/address.
• values: A map of string keys and generic values. The keys are build from a xpath representation of the gNMI path without the keys, while the values are extracted from the gNMI Node values.
• deletes: A string list built from the delete field of the gNMI Notification message.

Defining an event processor#

Event processors are defined under the section processors in gNMIc configuration file.

Each processor is identified by a name, under which we specify the processor type as well as additional fields specific to each type.

All processors support a debug field that enables extra debug log messages to help troubleshoot the processor transformation.

Below is an example of an event-delete processor, which deletes all values with a name containing multicast or broadcast

processors:
  # processor name
  my-processor:
    # processor type
    event-delete:
      value-names:
        - ".*multicast.*"
        - ".*broadcast.*"

Linking an event processor to an output#

Once the needed event processors are defined under section processors, they can be linked to the desired output(s) in the same file.

Each output can be configured with different event processors allowing flexibility in the way the same data is written to different outputs.

A list of event processors names can be added under an output configuration, the processors will apply in the order they are configured.

In the below example, 3 event processors are configured and linked to output1 of type influxdb.

The first processor converts all values type to integer if possible.

The second deletes tags with name starting with subscription-name.

Finally the third deletes values with name ending with out-unicast-packets.

outputs:
  output1:
    type: influxdb
    url: http://localhost:8086
    bucket: telemetry
    token: srl:srl
    batch-size: 1000
    flush-timer: 10s
    event-processors:
      - proc-convert-integer
      - proc-delete-tag-name
      - proc-delete-value-name

processors:
  proc-convert-integer:
    event-convert:
      value-names:
        - ".*"
      type: int

  proc-delete-tag-name:
    event-delete:
      tag-names:
        - "^subscription-name"

  proc-delete-value-name:
    event-delete:
      value-names:
        - ".*out-unicast-packets"

Event processors with cache#

In the scenario where processors are configured under an output with caching enabled, the event messages retrieved from the cache are processed as a single set by each processor. This concurrent processing facilitates the application of a logic that merges or combines messages, enabling more complex and integrated processing strategies.

Event processors pipeline#

Processors under an output are applied in a strict sequential order for each group of event messages received.

Event processors plugins#

gNMIc incorporates the capability to extend its functionality through the use of event processors as plugins. To integrate seamlessly with gNMIc, these plugins need to be written in Golang.

The communication between gNMIc and these plugins is facilitated by HashiCorp's go-plugin package, which employs netrpc as the underlying protocol for this interaction.

See some plugin examples here