Prometheus TSDB Configurations

Configuration Overview

You can enable telemetry using Prometheus TSDB on RBFS. RBFS includes a pre-packaged Prometheus Time Series Database (TSDB) and it does not require a separate Prometheus installation. However, to begin collecting and storing telemetry data, you must enable TSDB on an RBFS device.

Once enabled, the telemetry configuration allows RBFS to stream time-series data from its microservices and processes to the integrated Prometheus collector.

The configuration defines how time-series telemetry data is collected, stored, and transported to external systems. It allows you to define metric profiles, create individual metrics, and associate those metrics with specific metric profile.

You can create alerts for event notifications. Alerts profile allow you to group multiple alerts.

In the Metric configuration, you must define the attributes, filters and labels. They help in tagging, querying, filtering, and grouping the data in time-series database.

Telemetry on RBFS with Prometheus TSDB

Prometheus TSDB configurations in RBFS at a high level:

  • Metric Profile Configuration: Define a common settings such as scrape interval and append timestamp for a group of metrics.

  • Metric Configuration: Configure the parameters of the metrics which are to be collected and exported.

    • Attribute Configuration (Mandatory): Define the properties of the metric and describe how the metrics are collected and processed.

    • Define Filters (Optional): Filters the data which is included in the metric based on defined criteria.

    • Define Labels (Optional): Allows for querying, grouping, and filtering in time-series data.

  • Alert Configuration: Inform (alert) users about specific conditions in the system based on the captured telemetry data.

  • Alert Profile Configuration: Group multiple alerts into a single profile by applying the same characteristics to them.

  • Enabling TSDB Data History Retention: Retain TSDB database history.

  • Enabling TSDB Self Monitoring: Prometheus TSDB can monitor its own health state.

  • Monitoring Daemon Status Using TSDB: Monitor the status of system daemons (processes), specifically to track whether they are operational or not.

Configuration Hierarchy

The diagram illustrates the Prometheus configuration hierarchy.

TSDB Configuration Hierarchy

Configuration Syntax and Commands

The following sections describe the TSDB configuration syntax and commands.

Guidelines for Naming in TSDB Configuration

Metric and Metric Profile names must not start with "rtbrick," as this name is reserved for internal use. Also, RBFS has built-in default metric profile; therefore, the custom metric profile names should not be "default".

Metric Configuration

Metric configuration is used to configure the parameters of the metric data which is to be exported. You can create a metric by specifying various parameters.

NOTE:

  • The exact resource name to monitor may vary by platform. Refer to global.chassis_0.resource.sensor.thermal to identify the correct resource name, and update your Prometheus or Grafana configuration accordingly.

  • Every metric must be attached to a metric profile. If you do not explicitly associate a metric with a profile, it will automatically be assigned to the default metric profile.

Syntax:

set time-series metric <name> <option> <value>

Attribute Description

append-timestamp <true | false>

Set the append-timestamp to true for exporting the metric values with timestamp. By default, the value is 'false'.

attribute

Specifies the name of the attribute in the target table which is to be scraped and exported.

bds-type <object-metric | index-metric>

Specify the type of attribute, that is scraped and exported. There are two types, object-metric and index-metric.

description

Description for the metric. Allowed range of 128 characters.

filter <match-attribute-name>

Specifies the matching attribute name for the filter.

include-subscribed-tables <true | false>

Specifies whether the configuration needs to be applied on a subscribed tables as well. Default: false.

index-name

Specifies the index-name of the index-metric attribute. This is applicable for index-metric only.

prometheus-type <counter | gauge>

Configures the metric data type. Currently the supported Prometheus metric data are 'counter' and 'gauge'.

table-name

Specifies the target table from which the data is scraped and exported.

Example:

The following configuration defines a custom time-series metric named 'chassis_fan_speed_rpm'. This metric extracts data from a specific table named global.chassis_0.resource.sensor.thermal. The configuration specifies the 'include-subscribed-tables' option as 'false', indicating that this setting will not apply to subscribed tables.

The 'bds-type' is defined as 'object-metric', which means the metric is based on object-like data entries in the table. The 'prometheus-type' is specified as 'gauge', that is for numeric values that can go up or down. In this configuration, description for the metric is added as "Chassis fan speed in rpm."

A filter is applied to the 'resource_name' column, with the 'match-attribute-value' set to 'Chassis'. The match-type is defined as 'regular-expression'. This filter determines which rows in the source table (global.chassis_0.resource.sensor.thermal) will be included when generating this metric. It specifically matches rows where the value of 'resource_name' satisfies the regular expression for "Chassis."

Also, a second filter is applied to the 'resource_type' column, which only includes rows where the value is exactly 'fan', ensuring a case-sensitive, exact string match. This ensures that only fan-related resource entries are considered and filters out irrelevant sensors for temperature, voltage, and other unrelated data.

set time-series metric chassis_fan_speed_rpm
set time-series metric chassis_fan_speed_rpm table-name global.chassis_0.resource.sensor
set time-series metric chassis_fan_speed_rpm include-subscribed-tables false
set time-series metric chassis_fan_speed_rpm bds-type object-metric
set time-series metric chassis_fan_speed_rpm prometheus-type gauge
set time-series metric chassis_fan_speed_rpm description "Chassis fan speed in rpm"
set time-series metric chassis_fan_speed_rpm filter resource_name
set time-series metric chassis_fan_speed_rpm filter resource_name match-attribute-value Chassis
set time-series metric chassis_fan_speed_rpm filter resource_name match-type regular-expression
set time-series metric chassis_fan_speed_rpm filter resource_type
set time-series metric chassis_fan_speed_rpm filter resource_type match-attribute-value fan
set time-series metric chassis_fan_speed_rpm filter resource_type match-type exact
set time-series metric chassis_fan_speed_rpm attribute rpm
set time-series metric chassis_fan_speed_rpm attribute rpm label fan
set time-series metric chassis_fan_speed_rpm attribute rpm label fan label-value resource_name
set time-series metric chassis_fan_speed_rpm attribute rpm label fan label-type dynamic
supervisor@rtbrick.net: cfg> show config time-series metric chassis_fan_speed_rpm
{
  "rtbrick-config:metric": [
    {
      "name": "chassis_fan_speed_rpm",
      "table-name": "global.chassis_0.resource.sensor",
      "include-subscribed-tables": "false",
      "bds-type": "object-metric",
      "prometheus-type": "gauge",
      "description": "Chassis fan speed in rpm",
      "filter": [
        {
          "match-attribute-name": "resource_name",
          "match-attribute-value": "Chassis",
          "match-type": "regular-expression"
        },
        {
          "match-attribute-name": "resource_type",
          "match-attribute-value": "fan",
          "match-type": "exact"
        }
      ],
      "attribute": [
        {
          "attribute-name": "rpm",
          "label": [
            {
              "label-key": "fan",
              "label-value": "resource_name",
              "label-type": "dynamic"
            }
          ]
        }
      ]
    }
  ]
}

Configure Attributes

Attributes define how to extract specific values from the metric data.

When you export time-series metric data for an attribute that includes more than 50 label values (whether user-defined or default), the exported metric may be truncated.

Syntax:

set time-series metric <metric-name> attribute <attribute-name> label <label-key> [label-type / label-value]

Attribute Description

<attribute-name>

Specifies the name of the attribute in the target table which is to be scraped and exported. This attribute must be numeric or has a numeric converter.

<label-key> [label-type | label-value]

Label name for uniquely identifying time-series data.

Example:

The following configuration enables a metric named 'chassis_fan_speed_rpm' with the attribute 'rpm'. It also adds a label called 'fan' to the 'rpm' attribute, indicating that the metric 'chassis_fan_speed_rpm' will expose an attribute named 'rpm'. This setup defines the source of the label’s value. Instead of using a static value, it pulls the resource name from the system to serve as the value for the 'fan' label. This marks the label as dynamic that indicates it can have different values for each fan.

set time-series metric chassis_fan_speed_rpm attribute rpm
set time-series metric chassis_fan_speed_rpm attribute rpm label fan
set time-series metric chassis_fan_speed_rpm attribute rpm label fan label-value resource_name
set time-series metric chassis_fan_speed_rpm attribute rpm label fan label-type dynamic

Example:

supervisor@rtbrick.net: cfg> show config time-series metric chassis_fan_speed_rpm attribute
{
  "rtbrick-config:attribute": [
    {
      "attribute-name": "rpm",
      "label": [
        {
          "label-key": "fan",
          "label-value": "resource_name",
          "label-type": "dynamic"
        }
      ]
    }
  ]
}

Define Filters

Filters allow you to include or exclude specific time-series data based on the characteristics of the metric’s labels. They help refine time-series data collection or analysis, particularly when dealing with high volumes of data.

set time-series metric <metric-name> filter <option> <value>

Attribute Description

match-attribute-value

Matching attribute value for this filter. Note: The regex pattern should adhere to the POSIX standard.

match-negate [True or False]

Reverses the match logic. If specify true, it selects values that do not match the given pattern. If specify false, it selects values that do match.

match-type [regular-expression or exact]

Sets the type of match to perform. regular-expression interprets the pattern as a regex and exact indicates an exact string match.

Example:

The following configuration defines a filter called 'state' for the time-series metric named 'bgp_default_peer_state'. The option 'match-attribute-value' is set to 'Established', indicating that the system should match metrics where the attribute value is 'Established'. Also, the 'match-type' option is specified as 'exact', meaning that it should adhere to the exact match specified.

set time-series metric bgp_default_peer_state filter state
set time-series metric bgp_default_peer_state filter state match-attribute-value Established
set time-series metric bgp_default_peer_state filter state match-type exact
supervisor@rtbrick.net: cfg> sh conf time-series metric bgp_default_peer_state filter state
{
  "rtbrick-config:filter": [
    {
      "match-attribute-name": "state",
      "match-attribute-value": "Established",
      "match-type": "exact"
    }
  ]
}

Define Labels

Labels are key-value metadata on metrics. For example, it adds related labels such as CPU ID, interface name and so on.

set time-series metric <metric-name> attribute <attribute-name> label <label> [label-type / label-value]

Attribute Description

label-name

Specify the label name. For information on how to define label names, see Metric and label naming.

static or dynamic

Specifies the type of labels, a static value or dynamic value.

label_value

Specify the label value that is to be used.

Example:

The following configuration outlines how labels are applied to the metric name 'chassis_temperature_millicelsius' for the attribute 'temperature.' It adds a label with the key 'resource_name,' specifying that the label’s value should also be 'resource_name.' The configuration designates the label type as 'dynamic,' meaning the actual value will be derived dynamically from system data.

Also, this configuration enables the label key 'resource_type,' where the label value is defined as 'resource type,' and the label type is also specified as dynamic. In summary, for each temperature data (chassis_temperature_millicelsius), two labels are attached: 'resource_name' and 'resource_type.'

supervisor@rtbrick.net: cfg> show conf set time-series metric chassis_temperature_millicelsius attribute temperature label
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_name
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_name label-value resource_name
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_name label-type dynamic
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_type
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_type label-value resource_type
set time-series metric chassis_temperature_millicelsius attribute temperature label resource_type label-type dynamic
supervisor@rtbrick.net: cfg> sh conf time-series metric chassis_temperature_millicelsius attribute temperature label
{
  "rtbrick-config:label": [
    {
      "label-key": "resource_name",
      "label-value": "resource_name",
      "label-type": "dynamic"
    },
    {
      "label-key": "resource_type",
      "label-value": "resource_type",
      "label-type": "dynamic"
    }
  ]
}

Tracking Invalid Configurations for TSDB

RBFS offers functionality to track invalid TSDB configurations. This feature allows you to identify misconfigurations or invalid configurations.

You can view the list of misconfiguration using the show command:

show time-series metric rtbrick_invalid_metric_config

There are five error types for invalid configurations in RBFS:

Error Type Description

Error Type 1: invalid_filter

This error occurs due to a invalid filter name. If the configured filter attribute is not present in the table, this error is shown.

Error Type 2: invalid_attribute

This error occurs due to a invalid attribute name. If the configured attribute is not present in the table, then this error is shown.

Error Type 3: invalid_attribute_type

This error occurs due to a invalid type of attribute. If the configured attribute is present in the table and if the datatype is wrong, then this error is shown.

Error Type 4: invalid_label

This error occurs due to a invalid label name or key name. If the configured label attribute is not present in the table, then this error is shown.

Error Type 5: invalid_label_attribute_type

This error occurs due to a invalid label attribute type. If the configured label attribute is present in the table and if the datatype is wrong, this error is shown.

The following are the example configurations for invalid or misconfigurations in TSDB.

Error Type 1: Invalid Filter

In the following configuration, filter name 'invalid_filter' is used as a key instead of specifying a valid attribute. So this is not a valid configuration.

set time-series metric bgp_default_peer_state filter invalid_filter
set time-series metric bgp_default_peer_state filter invalid_filter match-attribute-value Active
set time-series metric bgp_default_peer_state filter invalid_filter match-type exact
supervisor@rtbrick.net: cfg> sh conf time-series metric bgp_default_peer_state filter invalid_filter
{
  "rtbrick-config:filter": [
    {
      "match-attribute-name": "invalid_filter",
      "match-attribute-value": "Active",
      "match-type": "exact"
    }
  ]
}
Error Type 2: Invalid Attribute

In this configuration 'invalid_attribute' is not a valid attribute for the metric 'bgp_default_peer_state'. So this is not a valid configuration.

set time-series metric bgp_default_peer_state attribute invalid_attribute
supervisor@rtbrick.net: cfg> sh conf time-series metric bgp_default_peer_state attribute invalid_attribute
{
  "rtbrick-config:attribute": [
    {
      "attribute-name": "invalid_attribute"
    }
  ]
}
Error Type 3: Invalid Attribute Type

In this configuration, though 'af_configured' exists, it is not of a supported data type for metrics. So this is not a valid configuration.

set time-series metric bgp_default_peer_state attribute af_configured
supervisor@rtbrick.net: cfg> show conf time-series metric bgp_default_peer_state attribute af_configured
{
  "rtbrick-config:attribute": [
    {
      "attribute-name": "af_configured"
    }
  ]
}
Error Type 4: Invalid Label

In this configuration the 'invalid_label' is not a valid label key. So this is not a valid configuration.

set time-series metric bgp_default_peer_state attribute state label invalid_label
{
  "rtbrick-config:label": [
    {
      "label-key": "invalid_label",
      "label-value": "invalid_label",
      "label-type": "dynamic"
    }
  ]
}

Error Type 5: Invalid Label Type

error type5: invalid_label_attribute_type

In this configuration, the label value ('af_configured') has a data type that is not allowed for label values. So this is not a valid configuration.

set time-series metric bgp_default_peer_state attribute state label invalid_label_type
set time-series metric bgp_default_peer_state attribute state label invalid_label_type label-value af_configured
set time-series metric bgp_default_peer_state attribute state label invalid_label_type label-type dynamic
supervisor@L1-STD-10-1102>bm14-tst.fsn.rtbrick.net: cfg> sh conf time-series metric bgp_default_peer_state attribute state label invalid_label
_type
{
  "rtbrick-config:label": [
    {
      "label-key": "invalid_label_type",
      "label-value": "af_configured",
      "label-type": "dynamic"
    }
  ]
}

Metric Profile Configuration

In a metric profile, you can club a collection of related metrics, which can be managed and exported together.

You can create a metric profile and associate a individual metric with the profile. It allows you to define a common setting such as scrape interval and default attributes for a group of metrics. This common settings will be applied automatically to all associated metrics.

If you do not configure a Metric Profile, the system will use a default metric profile with a predefined scrape interval.

Syntax:

set time-series metric-profile <profile-name> scrape-interval <scrape-interval>

Attribute Description

profile-name

Specify the name of the profile. Note: Metric Profile names must not begin with the name 'rtbrick' as it is reserved for internal use. Similarly, RBFS has built-in default metric profiles, therefore, do not use the name 'default' as metric profile name.

scrape-interval

The frequency (period of time) at which the system fetches (scrapes) data from a target source.

Example:

This configuration defines a metric profile called "up_metrics." The configuration sets the scrape interval to 10 seconds, meaning that metrics linked to this metric profile will be collected every 10 seconds.

set time-series metric-profile up_metrics
set time-series metric-profile up_metrics scrape-interval 10s

Example:

{
  "rtbrick-config:metric-profile": [
    {
      "profile-name": "up_metrics",
      "scrape-interval": "10s"
    }
  ]
}

Attaching a Metric with a Metric Profile

You can associate an existing metric with an existing metric profile.

Syntax:

set time-series metric <metric-name> profile-name <profile-name>

Attribute Description

<metric-name>

Specify the metric name.

<profile-name>

Specify the metric profile name.

The following command is used to associate a metric named 'process_status' with the metric profile named 'up_metrics'.

set time-series metric process_status profile-name up_metrics

Example:

supervisor@rtbrick.net: cfg> show config time-series metric process_status profile-name
{
  "rtbrick-config:profile-name": "up_metrics"
}

Alert Configuration

You can configure alerts which are notifications about the events based on the captured metrics.

Every alert must be attached to an alert profile. If you do not explicitly associate a alert with an alert profile, it will automatically be assigned to the default alert profile to avoid the loss of alert notifications. The repeat interval of the default profile is set to one hour (1h).
Syntax

set time-series alert-profile <profile-name> <attribute> <value>

Attribute Description

annotations

Key-value pairs that add metadata to the alert.

description

Alert description.

expr

PromQL Alert evaluation. It defines condition for triggering this alert.

for

Specifies the duration the condition must be true before triggering the alert.

group

Logical grouping of alerts.

interval

Defines how often this alert is evaluated (for example, every 30s, 1m, and so on).

labels

Key-value pairs used to categorize alerts, mainly for routing in Alertmanager.

level

Alert’s severity level.

summary

Explicit annotation label summary.

Example:

The following configuration creates an alert named 'daemon_crash_alert' and associates it with an alert group called 'daemons.' The alert checks every second, with an interval set to '1s,' to determine if any daemon has crashed. The 'expr' option is defined as "min_over_time(process_status{administrative_state='UP'}[5d]) == 0". This expression filters metrics that are in the 'UP' state and examines the last 5 days of time-series data. The condition 'min_over_time(…​) == 0' indicates that if the minimum operational status during that 5-day period was 0, it means the process was down at least once. It indicates a crash that triggers the alert.

The severity level of this alert is set to 2. Also, the summary is defined as "Process {{$labels.bd}} has crashed." In this summary, {{$labels.bd}} is replaced with the name of the affected process.

In a nutshell, this alert is created to check every second for any daemon process that has been DOWN at any point in the period of 5 days. If it finds that a process has been DOWN, it triggers an alert including the state information and the specific process.

set time-series alert daemon_crash_alert
set time-series alert daemon_crash_alert group daemons
set time-series alert daemon_crash_alert interval 1s
set time-series alert daemon_crash_alert expr "min_over_time(process_status{administrative_state='UP'}[5d]) == 0"
set time-series alert daemon_crash_alert level 2
set time-series alert daemon_crash_alert summary "Process {{$labels.bd}} has crashed."
set time-series alert daemon_crash_alert description "Daemon has crashed"
set time-series alert daemon_crash_alert annotations Administrative_Status:{{$labels.administrative_state}}
set time-series alert daemon_crash_alert annotations Operational_Status:{{$value}}
supervisor@rtbrick.net: cfg> show config time-series alert daemon_crash_alert
{
  "rtbrick-config:alert": [
    {
      "name": "daemon_crash_alert",
      "group": "daemons",
      "interval": "1s",
      "expr": "min_over_time(process_status{administrative_state='UP'}[5d]) == 0",
      "level": "2",
      "summary": "Process {{$labels.bd}} has crashed.",
      "description": "Daemon has crashed",
      "annotations": [
        "Administrative_Status:{{$labels.administrative_state}}",
        "Operational_Status:{{$value}}"
        ]
    }
  ]
}

Alert Profile Configuration

You can set up alert profiles for Prometheus time-series data. This configuration defines alerts, how alerts should be grouped, filtered, and repeated.

Syntax

set time-series alert-profile <profile-name> <attribute> <value>

Attribute Description

profile-name

Specify the name of the alert profile.

Priority

Specify the priority number.

group-by

Groups alerts by one or more labels.

matchers

Filters which time-series are matched for alerting.

repeat-interval

Defines how often repeating alerts are sent.

The following configuration defines a time-series alert profile named '4h_receiver'. This profile specifies how alerts that meet certain criteria should be managed.

The priority level is set to '1', indicating that this alert profile has high priority, as lower numbers indicate higher preference. The 'repeat-interval' is configured for '4h', meaning that alerts associated with this profile will be resent every 4 hours if they remain active. In essence, alerts under this profile are sent once every 4 hours.

The 'matchers' parameter is defined as 'repeat_interval=4h', which filters the alerts to which this profile applies. Only alerts with the label 'repeat_interval=4h' will be managed by this profile.

set time-series alert-profile 4h_receiver
set time-series alert-profile 4h_receiver priority 1
set time-series alert-profile 4h_receiver repeat-interval 4h
set time-series alert-profile 4h_receiver matchers repeat_interval=4h
{
  "rtbrick-config:alert-profile": [
    {
      "profile-name": "4h_receiver",
      "priority": 1,
      "repeat-interval": "4h",
      "matchers": [
        "repeat_interval=4h"
        ]
    }
  ]
}

Enabling Time-series Database History

You can enable the retention of the history of time series databases in every brick daemon. By default, time series database history is disabled.

Syntax

set time-series history-status [enable | disable]

Attribute Description

[enable|disable]

Enable time series database history by specifying 'enable'. Time-series database history is disabled, by default.

Example:

The following configuration enables timeseries history status by specifying 'enable'.

set time-series history-status enable
supervisor@rtbrick.net: cfg> show config time-series history-status
{
  "rtbrick-config:history-status": "enable"
}

Monitoring Daemon Status Using TSDB

This configuration defines TSDB metric to monitor the status of system daemons (processes), specifically whether they are operational.

Syntax:

set time-series metric <metric-name> <attribute> <value>

Attribute Description

metric

Name of the metric.

table-name

Specifies the target table from which the data is scraped and exported.

bds-type <object-metric | index-metric>

Specify the type of attribute, that is scraped and exported. There are two types, object-metric and index-metric.

prometheus-type <counter | gauge>

Configures the metric data type. The supported Prometheus metric data type are 'counter' and 'gauge'.

description

Description for the metric. Allowed range is 128 characters.

index-name

Specifies the index-name of the index-metric attribute. This is applicable for index-metric only.

include-subscribed-tables <true | false>

Specifies whether the configuration needs to be applied on a subscribed tables as well. Default: false.

filter <match-attribute-name>

Specifies the matching attribute name for the filter.

label

Specify the label key (name for the label).

label <label-value>

Value (name) of the label.

label <label-type>

Label type includes: 'dynamic' or 'static'.

Example:

This configuration defines a metric named process_status, which extracts data from a specific internal system table called global.process.status. The BDS type is defined as object-metric, indicating that the metric is derived from an object table. In the configuration, the metric type is specified as a gauge. It indicates increase or decrease, for example 1 for UP and 0 for DOWN.

The description provided is "testing UP metrics." The main attribute is defined as operational_status, and its values will be exported as part of the metric from the table. Additionally, the configuration introduces a label called administrative_state for the metric. The value for this label will be sourced from the admin_status column in the same row of the table. It is marked as dynamic, meaning the label value can vary at runtime for each row of data.

set time-series metric process_status
set time-series metric process_status table-name global.process.status
set time-series metric process_status bds-type object-metric
set time-series metric process_status prometheus-type gauge
set time-series metric process_status description "testing UP metrics"
set time-series metric process_status attribute operational_status
set time-series metric process_status attribute operational_status label administrative_state
set time-series metric process_status attribute operational_status label administrative_state label-value admin_status
set time-series metric process_status attribute operational_status label administrative_state label-type dynamic
set time-series metric process_status attribute operational_status label bd
set time-series metric process_status attribute operational_status label bd label-value process
set time-series metric process_status attribute operational_status label bd label-type dynamic

Example:

{
  "rtbrick-config:metric": [
    {
      "name": "process_status",
      "table-name": "global.process.status",
      "bds-type": "object-metric",
      "prometheus-type": "gauge",
      "description": "testing UP metrics",
      "attribute": [
        {
          "attribute-name": "operational_status",
          "label": [
            {
              "label-key": "administrative_state",
              "label-value": "admin_status",
              "label-type": "dynamic"
            },
            {
              "label-key": "bd",
              "label-value": "process",
              "label-type": "dynamic"
            }
          ]
        }
      ]
    }
  ]
}

TSDB Self-Monitoring Configuration

You can enable a self-monitoring feature that allows Prometheus to monitor its own performance and health. This self-monitoring functionality collects metrics regarding the system’s operation, such as query performance and resource usage. The scrape interval determines how often these metrics are collected.

Syntax:

set time-series self-monitoring enable <false|true> set time-series self-monitoring scrape-interval <scrape-interval[1-128]>

Attribute Description

enable <false | true>

Specify true to enable the self monitoring functionality for Prometheus TSDB.

<scrape-interval[1-128]>

The frequency (period of time) at which the system fetches (scrapes) data from a target source.

Example:

The following CLI commands enable the self-monitoring capability for the time-series database. The self monitoring is enabled by specifying 'true'. The scrape-interval is defined as 10s. It means the metrics are collected every 10 seconds.

set time-series self-monitoring enable true
set time-series self-monitoring scrape-interval 10s

Example:

{
  "rtbrick-config:self-monitoring": {
    "enable": "true",
    "scrape-interval": "10s"
  }
}

Enabling Append-Timestamp in Prometheus

RBFS supports 'append-timestamp' functionality in Prometheus. When the append-timestamp functionality is enabled, the first metric generated after an object is updated will include the actual timestamp of the update. This accurately reflects when the change took place. For all subsequent metrics for the same object (since it has not been been updated again) the current scrape time will be used instead.

You can enable append-timestamp for both metric or metric profile. If you configure append-timestamp for both, metric-level gets priority.

Syntax:

set time-series metric <metric-name> append-timestamp <false|true> set time-series metric-profile <profile-name> append-timestamp <false|true>

Attribute Description

<metric-name>

Name of the metric.

<profile-name>

Name of the metric profile.

append-timestamp <false / true>

Specify true to enable time-stamp functionality.

Example command:

The following command is used to add a timestamp to each metric associated with the metric profile called 'chassis'.

set time-series metric-profile chassis append-timestamp true

Example:

{
  "rtbrick-config:metric-profile": [
    {
      "profile-name": "chassis",
      "append-timestamp": "true"
    }
  ]
}