Cluster counter tables endpoint overview

05/08/2024 Contributors

Overview

The Counter Manager subsystem allows both manual and automated processes to access statistical information about various aspects of the ONTAP system. The information is most often utilized to assess the current performance of the system.

The data architecture is broken down into four components:

Tables
Rows
Counters / Properties
Aggregation

Tables

A table represents a collection of statistics that are grouped according to a common feature or function. An example counter manager table is for network adapters. This table would contain statistics related to the network adapter's performance such as the number of packets, rate of flow and error counters.

A table is described by its schema which includes a detailed description about the various statistics included, their format and their purpose.

The table catalog is a collection of all the statistical tables that the ONTAP REST interface supports, which can be queried to find information about a data point of interest.

Rows

Each table is populated with a list of rows. Each row is identified by a unique key and represents a specific statistical entity within the system. For example, a system may contain multiple network adapters that are represented by several records in the network adapter table.

Counter / Property

A counter is the basic 'numeric' statistical unit of the architecture.

A property is the basic 'string' statistical unit of the architecture.

Counter values can be organized as singular values or into multi-dimensional arrays. An array can be one or two dimensional; formatted as a list of label / value pairs. Addditional detail can be found in the "counter" model definition.

A table schema definition consists of multiple counters and properties.

Counters are classified according to their type. The available type options are the following:

average
rate
raw
delta
percent

Average and percent counters specify a secondary counter called the 'denominator' in the schema. The client must use the provided and secondary counters to compute the final intended value.

For example:

Determining the average wait time for a workload per visit
Query the 'wait_time' and 'visits' field from a 'qos_detail' row:
curl -X GET "https://<mgmt-ip>/api/cluster/counter/tables/qos_detail/rows/<instance-id>?fields=counters&counters.name=visits&#124;wait_time"
{
"counter_table": {
  "name": "qos_detail"
},
"id": "main-vsim1:_WAFL.CPU_ha",
"counters": [
  {
    "name": "visits",
    "value": 14631
  },
  {
    "name": "wait_time",
    "value": 167816
  }
],
"_links": {
  "self": {
    "href": "/api/cluster/counter/tables/qos_detail/rows/<instance-id>"
  }
}
}
The average wait time per visit is calculated as 167816 / 14631 = 11 micro-seconds

In the above example, the average is calculated since boot-time. Sample periods are discussed in more detail below.

Counter Computations

The statistics available through the counter tables gives you information about a specific point in time. This data can be useful, but more often you are interested in the statistics over a period of time.

The procedure for calculating a statistic over a period of time involves the following:

Collect a data sample at the beginning of the period. If the counter requires a denominator, this should be collected at the same time.
Collect a second data sample at the end of the period. If the counter requires a denominator, collect a second sample at the same time.
Calculate the final result using the collected information and the formula associated with the counter type below

All counters that are not of type 'raw' will require some computation to be useful.

----------------

'''

T1, T2 : The start and ending time of the sample period
C1, C2 : The counter value at the start and ending time of the period
D1, D2 : The denominator value at the start and ending time of the period
----------------

'''

Percentage = ((C2 - C1) x 100) / (D2 - D1)
Rate = (C2 - C1) / (T2 - T1)
Average = (C2 - C1) / (D2 - D1)
Delta = C2 - C1
----------------

'''

Aggregation

An aggregation is a logical container that consolidates the information from multiple entities into a single entity. There are two methods of aggregating tables:

Automatic
Combination.

Automatic

Tables with automatic aggregation are generated by consolidating all entities with matching identifiers. The underlying tables that contribute to the aggregated table are referenced by the following syntax: {table_name}:constituent.

Combination

Tables with combination aggregation are generated by consolidating all entities according to a unique field in the definition. The name of the combination table uses the following syntax: {table name}:{aggregation_name}.

An example combination table is 'volume:svm' table. This table aggregates all the volume statistics associated with a given vserver into a single table.

Multi-Dimensional Arrays

Numeric counters can be scalar, one-dimensional or two dimensional values. Scalars are the most common values which consist of a single numeric value. A one-dimensional array is commonly used to present histograms such as the following table:

< 1s     :  3
< 5s     : 10
< 60s    :  1

A counter endpoint response that contains the above table would be formated as follows:

{
"name": "Sample One-Dimensional Counter",
"labels": [ "< 1s", "< 5s", "< 60s" ],
"values": [3, 10, 1]
}

A two-dimensional array is used to report information about more complex relationships. An example data set is below:

            New      Used
--------------------------

'''

Car             1         2
Truck           3         4
Motorcycle      5         6

A counter endpoint response that contains the above table would be formated as follows:

{
"name": "Sample Two-Dimensional Counter",
"labels": [ "New", "Used" ],
"counters": [
  {
    "label": "Car",
    "values": [1, 2]
  },
  {
    "label": "Truck",
    "values": [3, 4]
  },
  {
    "label": "Motorcycle",
    "values": [5, 6]
  }
]
}

Filtering / Querying

The counter endpoints adhere to the same behavior as other endpoints, with exception of how queries are handled for nested array fields.

The default behavior when processing a nested array query is to return the entire array content on a match. The counter endpoints' behavior will only return entries in the array that match the query.

Counter responses can contain a significant amount of data. This behavior improves the response by only returning the information requested and eliminating extra work for the client.

For example:

Given the following array:
"list": [ "fruit_apple", "color_red" ]
When you apply the following query:
list=fruit*
The default query behavior will return the array as:
"list": [ "fruit_apple", "color_red" ]
The counter endpoints will return the array as:
"list": [ "fruit_apple" ]

Examples

Retrieving a table schema definition

This example retrieves the table description and schema definition for the qos_detail table.

# The API:
/api/support/counter/tables/{name}

# The call:
curl -X GET "https://<mgmt-ip>/api/cluster/counter/tables/qos_detail?fields=*" -H "accept: application/hal+json"

# The response:
{
"name": "qos_detail",
"description": "The qos_detail table that provides service center-based statistical information.

*Note:*
This table returns a large number of rows. Querying by row name and using wild cards may improve response times.",
"counter_schemas": [
  {
    "name": "in_latency_path",
    "description": "Determines whether or not service center-based statistics are in the latency path.",
    "type": "raw",
    "unit": "none"
  },
  {
    "name": "node.name",
    "description": "System node name",
    "type": "string",
    "unit": "none"
  },
  {
    "name": "resource.name",
    "description": "Name of the associated resource.",
    "type": "string",
    "unit": "none"
  },
  {
    "name": "service_time",
    "description": "The workload's average service time per visit to the service center.",
    "type": "average",
    "unit": "microsec",
    "denominator": {
      "name": "visits"
    }
  },
  {
    "name": "visits",
    "description": "The number of visits that the workload made to the service center; measured in visits per second.",
    "type": "rate",
    "unit": "per_sec"
  },
  {
    "name": "wait_time",
    "description": "The workload's average wait time per visit to the service center.",
    "type": "average",
    "unit": "microsec",
    "denominator": {
      "name": "visits"
    }
  }
],
"_links": {
  "self": {
    "href": "/api/cluster/counter/tables/qos_detail"
  }
}
}

Query for tables that contain a keyword in the description

This example retrieves all table definitions contain the word "security" in their description.

# The API:
/api/support/counter/tables

# The call:
curl -X GET "https://<mgmt-ip>/api/cluster/counter/tables/?fields=name,description&description=*security*" -H "accept: application/hal+json"

# The response:
{
"records": [
  {
    "name": "csm_global",
    "description": "This table reports global statistics of the Cluster Session Manager. The counters report the processing overhead of SpinNP cryptography, both encryption and decryption, as carried out by CSM as it handles cross-cluster data traffic, mostly on behalf of their data protection operations. For example, a customer might seek to know the processor time being consumed by these cryptographic operations in support of their cross-cluster traffic. That data might help them evaluate the performance impact of these security operations.",
    "_links": {
      "self": {
        "href": "/api/cluster/counter/tables/csm_global"
      }
    }
  },
  {
    "name": "file_directory",
    "description": "This table reports how many times file-directory jobs were triggered to the set the file-security ACLS or SLAG ACLS. This counter gives an indication how frequently the feature is being used to set the ACLS on file-directory/volume.",
    "_links": {
      "self": {
        "href": "/api/cluster/counter/tables/file_directory"
      }
    }
  }
],
"num_records": 2,
"_links": {
  "self": {
    "href": "/api/cluster/counter/tables?fields=name,description&description=*security*"
  }
}
}

Query for a specific property within all table rows.

This example requests the property named 'node.name' for all 'wafl' table rows.

The properties array content excludes any entries that do not match the provided query.

# The API:
/api/cluster/counter/tables/{counter_table.name}/rows

# The call:
curl -X GET "https://<mgmt-ip>/api/cluster/counter/tables/wafl/rows?properties.name=node.name&fields=properties" -H "accept: application/hal+json"

# The response:
{
"records": [
  {
    "id": "<instance id>",
    "properties": [
      {
        "name": "node.name",
        "value": "<node name>"
      }
    ],
    "_links": {
      "self": {
        "href": "/api/cluster/counter/tables/wafl/rows/<instance id>"
      }
    }
  }
],
"num_records": 1,
"_links": {
  "self": {
    "href": "/api/cluster/counter/tables/wafl/rows?properties.name=node.name&fields=properties"
  }
}
}

Query for a list of properties that match a wildcard on a specific row.

This example queries for all properties associated with a row of the volume table.