Skip to main content
Data Infrastructure Insights

What's New with Data Infrastructure Insights

Contributors netapp-alavoie netapp-chunge RSS

NetApp is continually improving and enhancing its products and services. Here are some of the latest features and functionalities available in Data Infrastructure Insights (formerly Cloud Insights).

November 2024

Analyze Configuration Changes in your Environment

Configuration changes are one of the most common causes of issues in modern IT. Data Infrastructure Insights' (DII’s) new change analysis capability lets you clearly understand changes that cause issues in your environment. Accelerate troubleshooting time by showing all changes in devices and related infrastructure components that may have led to an issue. Additionally, when you or your team perform planned changes, you can validate those changes quickly and ensure there are no unexpected effects before service levels are impacted.

Infrastructure Change Analytics example

KubeVirt Support: Monitor Virtual Machine workloads running within your Kubernetes Cluster

DII now fully supports KubeVirt, the Kubernetes-native virtualization solution used by platforms like OpenShift Virtualization and Harvester. Gain full visibility into the metrics, events, configuration changes, and network traffic of both Virtual Machines and Container workloads within your Kubernetes clusters.


October 2024

Unlock New Insights with Custom Expressions in Monitors

Expressions enable you to perform arithmetic operations in Metric and Anomaly Detection Monitors. Here are some examples:

  • Ratio: IOPS/TB to detect where service level limits are being reached on cloud storage providers.

  • Percentage: Used/Available to calculate utilization

  • Aggregation: Combine multiple types of physical port errors into one monitor

  • Comparison: Compare current resource headroom utilization with the optimal headroom point to identify resources that are not running at full capacity.

Creating an expression in a metric monitor

Minimize Alert Disruptions During Maintenance Period

Maintenance Windows allow you to suppress alert notifications during scheduled maintenance periods, helping you avoid unnecessary disruptions.

With Maintenance Windows, you can schedule specific maintenance periods during which alert notifications are suppressed, for the objects and metrics you choose. For example, you can suppress alert notifications triggered by specific storage systems, when those storage systems are in a planned upgrade period.

Note that only alert notifications are suppressed (email, webhook); the alerts themselves are still shown on the Observability > Alerts > All Alerts page.

Maintenance Windows Example

Streamline Alert Management with New Alert Notification Rules

Alert Notification Rules simplify notification management across monitors and teams.

Control alert delivery across your organization's channels, ensuring the right information reaches the right team. No need to manage separate monitors for different teams; route alerts based on related object attributes (storage name, data center, application name) or monitor attributes (group, severity).

setting filters for notification rule

Log Analytics in dashboards

You can now include log events in your dashboards, to visualize event data and have more comprehensive and contextual understanding of your environment. Investigate logs and view related metrics without leaving the dashboard!

Log Analysis example

Better VMware Observability with VMware Events

Proactively manage and troubleshoot your VMware environment with real-time events. VMware events provide insights on VM migrations, resource allocations, and host health. Now available for use in Queries, Dashboards, and Monitors. Requires VMware version 8 or higher. Simply select the logs.vmware.events source.

VMware Events are also used for DII's new Configuration Change Analysis noted above.

vmware log selection in the drop-down

Data Collector updates:

  • Pure FlashBlade: This collector gathers inventory and performance data from FlashBlade clusters that expose version 2 of their REST API.


September 2024

Introducing Data Infrastructure Insights, formerly Cloud Insights

On Tuesday, September 24, 2024, NetApp has officially changed the name of Cloud Insights to Data Infrastructure Insights (DII). This has been announced during the Insight user conference by Haiyan Song in her main stage keynote presentation and in an Insight conference product press release.

The DII service remains the same; there are no feature changes or modifications. This is a name change to better align the service name with its capabilities for all IT infrastructure.

August 2024

View Data Specific to Your Time Range

Investigating an alert? Zoomed in on a chart? These actions change the time range for those pages. Now you can lock that time range, navigate to other Cloud Insights pages, and see data specific to that locked time range. Investigating and troubleshooting just got a whole lot easier!

tooltip stating to click the icon to lock the time range for use on other pages

Change and Change Ratio (%) Analysis

Change-ratio time aggregations aid you in identifying significant changes and trends in metric values over time.
Those insights are key to understanding what changed, such as considerable capacity growth for a specific time or a change in single port performance.

  • Change - observe the change in a metric between two points within a selected period.

  • Change Ratio - observe the proportional change in a metric between two points, with respect to the initial point, within a selected period.

bar chart example showing change and change ration aggregation choices

Export Log Query Results to .CSV

When viewing log query results, easily export up to 10,000 rows to .CSV by clicking the new "Export" button. This enhances data accessibility, eases data analysis and reporting, and facilitates seamless integration with other data processing tools.

Export to CSV button on a log query page

Resolve Alerts by time

Cloud Insights now gives you the option to resolve an alert when the monitored metric stays within the acceptable range for a specified duration. This lets you focus on genuine issues, reducing the noise associated with metrics repeatedly crossing defined thresholds by consolidating multiple alerts to one.

resolving an alert based on time


July 2024

AIOps: Anomaly Detection

Cloud Insights uses machine learning to detect unexpected changes in the patterns of data in your environment, and provide proactive alerts to help you identify problems early.

A data center behaves in different ways at different times of the day, and on different days of the week. Cloud Insights uses weekly seasonality to compare historical behavior for each day and time.

Anomaly Detection Monitoring can provide alerts for situations such as when the definition of "normal" is unclear, where behavior changes over time, or when working with large amounts of data where manually defining thresholds is impractical.

New Anomaly Detection Monitors alert when anomalies such as this occur on object metrics you choose.

graph showing detected anomaly

Workload Security Improvements

NFS 4.1 support

The SVM Data Collector now supports NFS versions up to and including NFS 4.1 with ONTAP 9.15.1 or later..

New Forensics Activity API

The Forensics Activity API has a new version. When calling the API for Forensics Activity, use the cloudsecure_forensics.activities.v2 API.

Note that if you are making multiple calls to this API, for best results ensure that the calls occur sequentially, instead of in parallel. Multiple parallel calls may cause the API to time out.

Easier Dashboard Navigation

This feature is all about streamlining your operational workflows and making it easier to collaborate between teams.

Grouping your dashboards makes it easier to quickly get the visibility you need, and now with the new navigation menu, you can jump between different dashboards without losing your place, making it a breeze to explore and manage your infrastructure.
Align dashboard groups with your operational runbooks to further enhance your experience.

Drop-down to select another dashboard in the same group(s) as the current dashboard


June 2024

Operating System support

The following operating systems are supported with Cloud Insights Acquisition Units, in addition to those already supported:

  • Red Hat Enterprise Linux 8.9, 8.10, 9.4

  • Rocky 9.4

  • AlmaLinux 9.3 and 9.4

May 2024

Automatically resolve alerts based on time

Log alerts can now be resolved based on time; if the alert condition stops occurring, Cloud Insights can resolve the alert automatically after a specified time has passed. You can choose to resolve the alert in minutes, hours, or days.

Resolve an alert based on time passed


April 2024

iSCSI support for Kubernetes

Cloud Insights now has support to map the iSCSI storage associated with Kubernetes, allowing faster troubleshooting using the Kubernetes network map and the ability to provide chargeback or showback reports via Reporting.

Pod-to-storage example

Operating System support

The following operating systems are supported with Cloud Insights Acquisition Units, in addition to those already supported:

  • Oracle Enterprise Linux 8.8

  • Red Hat Enterprise Linux 8.8

  • Rocky 9.3

  • OpenSUSE Leap 15.1 through 15.5

  • SUSE Enterprise Linux Server 15, 15 SP2 through 15 SP5


March 2024

Workload Security Agent Details

Each of your Workload Security Agents has its own landing page, where you can easily see summary information about the Agent as well as the installed Data and User Directory Collectors associated with that Agent.

Agent Detail Landing Page example

Chart more data more quickly

When analyzing data on an asset's landing page, adding additional data to the Expert View charts is a snap. For each table on the landing page, if an object type has relevant data, hover over that object to display the "Add to Expert View" icon. Selecting this icon adds that object to the Additional Resources and displays it in the Expert View charts.

Add table data to expert view

Or maybe you want to see a landing page table's data in its own chart. Simply select the Show Chart icon to open the chart below the table:

Show Chart icon


February 2024

Usability Improvements

Save a snapshot of your current dashboard by selecting Export as Image from the right-corner drop-down. Cloud Insights creates a .PNG of the current widget states.

Export as Image Dropdown

Object and Metric selection is easier than ever for Widgets, Monitors, etc. Choose the object type you want, and then select a metric relevant to that object in the separate drop-down.

Object and Metric Selectors separate

Export Data Collector and Acquisition Unit lists to .CSV by selecting the icon at the top of those pages.

Export DC and AU Lists to .csv

We’ve re-organized the Help > Support page so it’s easier to find what you’re looking for, and because you asked for them, we added direct links on this page to API Swagger and user documentation.

API Links on the Help > Support Page

Links in the “triggeredOn” column on the Alerts list page will navigate to the appropriate Landing Page, if a Landing Page is available for that object.

Links in TriggeredOn alert field

See all changes in your namespace

Kubernetes Change Analysis now allows you to see a timeline of changes when selecting Cluster and Namespace. Previously, Workload must also have been selected. When filtering on Cluster and Namespace, the timeline of all workload changes in that namespace are shown on one line.

Namespace Timeline

When viewing a log alert, related log entries are shown in a new table. A log entry is related if it occurs in the same source and timeframe as the alert, and is subject to the same conditions. Select “Analyze Logs” to explore further.

Related Logs on a log alert landing page

Collect ONTAP Switch Data

Cloud Insights can collect data from the ONTAP system’s back-end switches; simply enable the collection in the data collector’s Advanced Configuration section, and ensure the ONTAP system is configured to provide switch information and has the appropriate permissions set.

Workload Security Data Collector API

In large environments, you can automate Workload Security collector creation using the new Data Collectors API. Navigate to Admin > API Access > API Documentation and select the Workload Security API type to learn more.


January 2024

Try Cloud Insights Features you haven’t used yet

In addition to your initial trial of Cloud Insights, you may also take advantage of Module Evaluations. For example, if you are subscribed to Cloud Insights and have been monitoring storage and virtual machines, when you add Kubernetes to your environment, you will automatically enter into a 30-day trial of Kubernetes Observability. Kubernetes Observability managed unit usage will not count against your subscribed entitlement until after the trial period ends.

How healthy are my workloads?

Workload health is available at a glance on the Kubernetes > Explore > Workloads page, so you can quickly see which workloads are performing well and which may need some help. Easily identify if the health issue is related to infrastructure, network, or configuration changes, and drill down to analyze the root cause.

Workload Health at a glance

Data Collector Updates

Data Domain Identification

The Data Domain collector has been improved to better identify HA systems for durability across failover events This change will cause a one time re-identification of Data Domain appliances in HA systems, which will subsequently cause any annotations on those assets to be removed (because these arrays will be re-identified). You will need to re-attach annotations to your Data Domain objects.

Enhanced Ransomware Detection ML Algorithm

Workload Security includes a new 2nd-generation ransomware detection ML algorithm to detect the most sophisticated attacks faster and more accurately.

"Seasonality" of behaviors: weekend behavior may follow different patterns from weekday, or morning behavior from afternoon. Workload Security algorithms take this seasonality into account.

Deprecated Functionality

Occasionally functionality is deprecated as features evolve. Here are some of the features and functionalities which have been deprecated in Cloud Insights:

Workload Secure REST cloudsecure_forensics.activities.v1 API is deprecated

The cloudsecure_forensics.activities.v1 API is deprecated. This API returns information on activities associated with entities in the Storage Workload Security environment. This API has been replaced with cloudsecure_forensics.activities.*v2*_.

GET for this API previously returned the following:

{
  "count": 24594,
  "limit": 1000,
  "offset": 0,
  "results": [
    {
      "accessLocation":

This API now returns:

{
  "limit": 1000,
  "meta": {
    "page": {
      "after": "lvlvk3pp.4cpzcg4kpybl",
      "before": "lvlxy3dz.4cq5ajdnl9fk",
      "size": 1000
    }
  },
  "results": [
    {
      "accessLocation": "10.249.6.220",

For more details see the Swagger documentation at “Admin > API Access > API Documentation > Workload Security”.


December 2023

Change Analytics at a glance

Kubernetes Change Analytics provide you with an all-in-one view of recent changes to your Kubernetes environment. Alerts and deployment status are at your fingertips. With Change Analytics, you can track every deployment and configuration change, and correlate it with the health and performance of K8s services, infrastructure, and clusters.

Change Analytics Dashboard

Kubernetes Workload Performance Dashboard

Workload performance is available at a glance in the comprehensive Kubernetes Workload Performance dashboard. Quickly view graphs of Volume, Throughput, Latency, and Retransmission trends, as well as a table of workload traffic for each namespace in your environment. Filters allow easy focus into areas of interest.

Workload Performance Menu

Workload Performance Dashboard

Query Details on one screen

In a query, selecting a row opens a side panel showing attribute, annotation, and metric details for the selected row, providing helpful information without needing to drill into the object's landing page. Links in the row or side panel allow for easy navigation.

Slideout Panel for Metric Query

Data Collector updates:

  • Brocade FOS REST: This collector is moved out of "Preview" and is now generally available. Some things to note:

    • FOS introduced their REST API with FOS 8.2. But some features like routing only received REST API capabilities with 9.0.

    • If you have a fabric consisting of mixed FOS assets 8.2 higher, as well as some < 8.2, the Cloud Insights FOS REST collector will fail to discover those older assets. You can edit the FOS REST collector and build a comma-delimited list of the IPv4 address of those devices for exclusion from that collector.

  • SELinux: Cloud Insights includes enhancements to the Linux Acquisition Unit initial installation to ensure robustness of operation within Linux environments with SELinux enforcement enabled. These enhancements only impact new AU deployments; if you have any SELinux issues relating to AU upgrades, contact NetApp Support to remediate your SELinux configuration.


November 2023

Workload Security: Pause/Resume a Collector

In Workload Security, you can Pause a Data Collector if the collector is in Running state. Open the "three dots" menu for the collector and select PAUSE. While the collector is paused, no data is gathered from ONTAP, and no data is sent from the collector to ONTAP. Select Resume to begin collecting again.

Storage Node Support Information

On a storage node landing page, the User Data section provides at-a-glance information about your support offering, current status, support status, and warranty end date. Note that Cloud Insights currently only auto-publishes this information for NetApp devices. Note also that these support fields are annotations, so they can be used in queries and dashboards.

Storage Node Support Information

Map VMWare tags to Cloud Insights annotations

The VMWare data collector allows you to populate Cloud Insights text annotations with same-name tags that are configured on VMWare.

Brocade CLI collector reliability enhancements for FOS 9.1.1c and higher firmware

On some Brocade Fibre Channel switches running 9.1.1c firmware, certain CLI commands’ output may be prepended with the “motd” login banner text, or warnings for users to change default passwords. The Brocade CLI collector has been enhanced to ignore these two types of extraneous text.

Prior to this enhancement, only FOS 9.1.1c switches without Virtual Fabrics present were likely discoverable with this collector type.


October 2023

Enhanced Workload Security

Workload Security has been improved with the following:

  • Access Denied: Workload Security integrates with ONTAP to receive "Access Denied" events and provide an additional analytics and automatic responses layer.

  • Allowed File Types: If a ransomware attack is detected for a known file extension, that file extension can be added to an allowed file types list to prevent unnecessary alerting.

Module Trials

In addition to your initial trial of Cloud Insights, you may also take advantage of Module Evaluations. For example, if you are already subscribed to Infrastructure Observability but are adding Kubernetes to your environment, you will automatically enter into a 30-day trial of Kubernetes Observability. You will only be charged for your Kubernetes Observability managed unit usage at the end of the evaluation period.

Restrict access to specified domains

Admins and Account Owners now have the ability to restrict Cloud Insights access to email domains they specify. Go to Admin > User Management and select the Restrict Domains button.

Restrict Domains Modal

Data Collector Updates

The following Data Collector/Acquisition Unit changes are in place:

  • Isilon / PowerScale REST: Various new attributes and metrics have been added to Cloud Insights enhanced analytics capabilities under the emc_isilon.node_pool.* name. These counters and attributes will empower users to build dashboards and monitors for node_pool capacity consumption; users with Isilon clusters built from dissimilar hardware node models will have multiple node pools, and understanding your HDD/SSD/total capacity consumption at a node pool level is useful for both monitoring and planning.

  • Rubrik “Service account” authentication support: Cloud Insights' Rubrik collector now supports both traditional HTTP Basic Authentication (username and password), and Rubrik’s Service Account approach, which requires a username + secret + Organization ID.


September 2023

Easily Find What You Want in the Logs

Log Query (Observability > Log Queries > +New Log Query) includes a number of enhancements to make log exploration easier and more informative.

Include/Exclude

When filtering for a value, you can easily choose whether to Include or Exclude results matching the filter. Selecting "Exclude" creates a "NOT <value>" filter. You can combine Include and Exclude values in a single filter.

Filter showing Exclude radio button

Advanced Query

Advanced Querying gives you the opportunity to create "free form" filters, combining or excluding values using AND, NOT, OR, wildcards, etc.

Example Log Query illustrating AND

The "Filter By" and Advanced Query are "AND"ed together to form a single query. The results are displayed in the results list and the chart.

Grouping in the Chart

When you select a log attribute to Group By, the list and chart show the results of the current filter. In the chart, columns grouped into colors. Hovering over a column in the chart will display details about the specific entries, similar to the overall information shown when you expand the chart Legend. In the legend, you can also choose to set an Include or Exclude filter for a specific grouping.

Log Query Group By example showing stacked columns in the chart

"Floating" Log Detail Panel

When exploring logs using the Log Query, selecting an entry in the list opens a detail panel for that entry. You can now choose to display that slideout panel "Floating" (i.e. displayed over the rest of the screen) or 'In Page" (i.e. displayed as its own frame within the page). To switch between these views, select the "In Page / Floating" button in the upper-right corner of the panel.

In-Page

Collapse the Menu

You can collapse the left-side Cloud Insights navigation menu by selecting the "Minimize" button below the menu. While the menu is minimized, hover over an icon to see which section it opens; selecting the icon opens the menu and takes you directly to that section.

Minimize the menu

Data Collector Improvements

Cloud Insights has made it easier to show and find data collector information:

  • Processing of data collector lists is more efficient, which means the time it takes to display and navigate these lists is greatly reduced. If you have a large environment with many data collectors, you will see a significant improvement when listing your data collectors.


August 2023

Collecting Isilon/PowerScale Logs and Advanced Analytics Data

The Isilon REST and PowerScale REST collectors contain the following improvements:

  • Isilon log events are available for use in queries and alerts

  • Isilon Advanced Analytic attributes are available for use in queries, dashboards, and alerts:

    • emc_isilon.cluster

    • emc_isilon.node

    • emc_isilon.node_disk

    • emc_isilon.net_iface

These are enabled by default for users of the Isilon REST and/or PowerScale REST collectors. NetApp strongly encourages users of the Isilon CLI-based collector to migrate to the new REST API-based collector to receive enhancements such as the above.

Improved Workload Map

The workload map is more usable and less noisy; it groups all similar external services into one node if they communicate with the same workloads, reducing the complexity of the graph and making it easier to understand how services are interconnected.

Choosing a grouped node will display a detailed table with the network traffic metrics for each external service relevant to that node.

Kubernetes Managed Unit usage adjustment

In the event of a compute resource in your Kubernetes cluster environment being counted by both the NetApp Kubernetes Monitoring Operator and an underlying infrastructure data collector (for example, VMware), your usage of these resources will be adjusted to ensure the most efficient counting of managed units. You can view the Kubernetes MU adjustments on the Admin > Subscription page, in both the Summary and Usage tabs.

Summary tab:
k8s MU Adjustment shown on the estimation calculator

Usage tab:
k8s MU Adjustment shown on the Usage tab

Collector/Acquisition changes:

The following Data Collector/Acquisition Unit changes are in place:

  • Acquisition Units now support RHEL 8.7.

Improved Menus

We have updated the left hand navigation menu to better support our customers' workflows. New top level items such as Kubernetes provide accelerated access to what the customer needs, and a consolidated administrators console supports the tenant owner role.

Here are some additional examples of the changes:

  • The top level Observability menu showcases data discovery, alerting and log queries

  • ‘API Access’ functionality for Observability and Workload Security are under one menu

  • Likewise for Observability and Workload Security ‘Notifications’ functionality, also now under one menu

Updated Left Navigation Menu

Here is a brief list of the features you can find under each menu:

Observability:

  • Explore (Dashboards, Metric Queries, Infrastructure Insights)

  • Alerts (Monitors and Alerting)

  • Collectors (Data Collectors and Acquisition Units)

  • Log Queries

  • Enrich (Annotations and Annotation Rules, Applications, Device Resolution)

  • Reporting

Kubernetes:

  • Cluster Exploration and Network Map

Workload Security:

  • Alerts

  • Forensics

  • Collectors

  • Policies

ONTAP Essentials:

  • Data Protection

  • Security

  • Alerts

  • Infrastructure

  • Networking

  • Workloads
    *VMware

Admin:

  • API Access

  • Auditing

  • Notifications

  • Subscription Information

  • User Management

July 2023

Show Recent Changes

Data Collector landing pages now include a list of recent changes. Simply click the "Recent Changes" button at the bottom of any data collector landing page to display recent data collector changes.

Recent Changes Example

Operator Improvements

The following improvements have been made to Kubernetes Operator deployment:

  • Option to bypass docker metric collection

  • Ability to add and customize tolerations to telegraf Daemonsets and Replicasets

Insight: Reclaim Cold Storage

The Reclaim ONTAP Cold Storage Insight now supports FlexGroups, and is now available to all customers.

Operator Image Signature

For customers who use a private repository for their NetApp Kubernetes Monitoring Operator, you can now copy the Image Signature Public Key during Operator installation, allowing you to confirm authenticity of the downloaded software. Select the Copy Image Signature Public Key button during the optional step to Upload the operator image to your private repository.

Download the Public Key

Aggregation, Conditional Formatting, and more for Queries

Aggregation, Unit Selection, Conditional Formatting, and Column Renaming are among the most useful features of a dashboard table widget, and now those same features are available for Queries.

Query Page results showing Aggregation

These features are available now for integration-type data (Kubernetes, ONTAP Advanced Metrics, etc.), and will be coming soon for Infrastructure objects (storage, volume, switch, etc.).

API for Audit

You can now use an API to query or export Audited events. Go to Admin > API Access and select the API Documentation link for information.

API Swagger for Audit

Data Collector: Trident Economy

Cloud Insights now supports the Trident Economy Driver, realizing these benefits:

  • Get visibility into pod-to-ONTAP Qtree mapping and performance metrics.

  • Provide seamless troubleshooting and easy navigation from Kubernetes pods to backend storage

  • Proactively detect backend performance issues with monitors


June 2023

Check out your Usage

Beginning in June, 2023, Cloud Insights provides a breakdown of Managed Unit usage based on Feature Set. Now you can quickly view and monitor managed unit (MU) usage for your Infrastructure as well as MU usage tied to Kubernetes.

Metering Usage Breakdowns

Kubernetes Network Monitoring and Map is available for all

The Kubernetes Network Performance and Map simplifies troubleshooting by mapping dependencies between Kubernetes workloads, providing real-time visibility into Kubernetes network performance latencies and anomalies to identify performance issues before they affect users. Many customers found it helpful during Preview, and now it's available for everyone to enjoy.

Collector/Acquisition changes:

The following Data Collector/Acquisition Unit changes are in place:

  • Data Domain and Cohesity MUs are metered at 40 TiB : 1 MU.

  • Acquisition Units now support RHEL and Rocky 9.0 and 9.1.

New ONTAP Essentials dashboards

The following ONTAP Essentials dashboards have been available in Preview environments, and now they are available for everyone:

  • Security Dashboard

  • Data Protection Dashboard (includes Local and Remote Protection overviews)

Additional System Monitors

The following System Monitors are included with Cloud Insights:

  • Storage VM FCP Service Unavailable

  • Storage VM iSCSI Service Unavailable


May 2023

Improved Kubernetes Monitoring Operator Installation

Installation and configuration of the NetApp Kubernetes Monitoring Operator is easier than ever with the following improvements:

  • Environment configuration settings are held in a single, self-documented config file.

  • Step-by step instructions for uploading Kubernetes Monitoring Operator images to your private repository.

  • Simple to upgrade with a single command to upgrade your Kubernetes Monitoring while keeping custom configurations.

  • More secured: API keys are securely managing secrets.

  • Easy to integrate and deploy with your CI/CD automation tools.

Storage Virtualization

Cloud Insights can differentiate between a storage array having local storage or virtualization of other storage arrays. This gives you the ability to relate cost and distinguish performance from the front-end all the way to the back-end of your infrastructure.

Storage Landing Page showing Virtual and backed storage information

New Webhook Parameters

When creating a Webhook notification, you can now include these parameters in your webhook definition:

  • %%TriggeredOnKeys%%

  • %%TriggeredOnValues%%

Reporting on Kubernetes data

Kubernetes data collected by Cloud Insights—​including Persistent Volumes (PV), PVC, Workloads, Clusters, and Namespaces—​is now available for use in Reporting, enabling chargeback, trending, forecasting, TTF calculations, and other business reporting on metrics for Kubernetes.

Default ONTAP System Monitors Enabled for New Customers

Many ONTAP System Monitors are enabled (i.e. Resumed) by default in new Cloud Insights environments. Previously, most monitors defaulted to Paused state. Because business needs vary from company to company, we always recommend taking a look at the system monitors in your environment and pausing or resuming each based on your alerting needs.


April 2023

Kubernetes Performance Monitoring and Map

The Kubernetes Network Performance and Map feature simplifies troubleshooting by mapping dependencies between Kubernetes workloads. It provides real-time visibility into Kubernetes network performance latencies and anomalies to identify performance issues before they affect users.
This capability helps organizations reduce overall costs by analyzing and auditing Kubernetes traffic flows.

Key Features:
• The Workload Map presents Kubernetes workload dependencies and flows and highlights network and performance issues.
• Monitor network traffic between Kubernetes pods, workloads, and nodes; identifies the source of traffic and latency problems.
• Reduce overall costs by analyzing ingress, egress, cross-region, and cross-zone network traffic.

Workload Map showing "Slideout" details:

Workload Map example showing "Slideout" panel with details

Kubernetes Performance Monitoring and Map is available as a Preview feature.

ONTAP Essentials Security Dashboard

The Security Dashboard gives you an instant view of your current security situation, showing charts for hardware and software volume encryption, anti-ransomware status, and cluster authentication methods. The Security Dashboard is available as a Preview feature.

ONTAP Essentials Security Dashboard

Reclaim ONTAP Cold Storage

The Reclaim ONTAP Cold Storage Insight provides data about cold capacity, potential cost/power savings, and recommended action items for volumes on ONTAP systems.

Cold Data Insight example recommednations

With this Insight, you can answer such questions as:

  • What amount of cold data on a storage cluster are sitting on (a) high-cost SSD disks, (b) HDD disks, and (c) virtual disks?

  • What workloads are the highest contributors in regards of the non-optimized storage?

  • What is the duration (in days) the data has been cold on a given workload?

Reclaim ONTAP Cold Storage is considered a Preview feature and is therefore subject to change.

Subscription Notification also controls banner messages

Setting recipients for Subscription Notifications (Admin > Notifications) now also controls who will see subscription-related in-product banner notifications.

Subscription Expiring in 2 Days banner example

Reporting has a new look

You will notice that Cloud Insights Reporting screens have a new look, and that some of the menu navigation have changed. These screens and navigation changes have been updated in the current Reporting Documentation.

New Reporting Menu look

Monitors Paused by Default

For new Cloud Insights environments, be aware that system-defined monitors do not send alert notifications by default. You will need to enable notifications for any monitor that you want alerting you, by adding one or more delivery methods for the monitor.
For existing Cloud Insights environments, the default global notification recipient list has been removed for any system-defined monitors currently in Paused state. User-defined notifications remain unchanged, as do notification settings for currently active system-defined monitors.

Looking for the API Metering tab?

API Metering has moved from the Subscription page to the Admin > API Access page.


March 2023

Cloud Connection for ONTAP 9.9+ deprecated

The Cloud Connection for ONTAP 9.9+ data collector is being deprecated. Starting April 4, 2023, Cloud Connection data collectors in your environment will no longer collect data, and will instead present an error when polling. The Cloud Connection data collector will be removed altogether from Cloud Insights in a subsequent update.

Prior to April 4, 2023, it is mandatory to configure a new NetApp ONTAP Data Management Software data collector for any ONTAP systems currently collected by Cloud Connection. Learn More.


January 2023

New Log Monitors

We've added almost two dozen additional system monitors to alert for broken interconnect links, heartbeat problems, and more. Additionally, three new Data Protection log monitors have been added, to alert on SnapMirror Auto Resync, MetroCluster Mirroring, and FabricPool Mirror Resync changes.

Note that some of these monitors will be enabled by default; you must pause them if you do not wish to alert on them. Also note that these monitors are not configured to deliver notifications; you must configure notification recipients on these monitors if you want to send alerts via email or webhook.

.CSV Export for all Dashboard Table Widgets

Ensuring accessibility to your data is essential, so we've made .CSV export available for all metric queries, dashboard table widgets, and object landing pages, regardless of the type of data (asset or integration) you're querying.

Data customizations like column selection, renaming columns, and unit conversions are also now included in the new export functionality.


December 2022

Explore Ransomware Protection and other security features during Cloud Insights Trial

Starting today, signing up for a new Trial of Cloud Insights allows you to explore Security features such as Ransomware detection and automated user-blocking response policy. If you haven't signed up for your Trial, do it today!

Kubernetes Workloads have their own landing page

Workloads are a key part of your Kubernetes environment, so Cloud Insights now provides landing pages for those workloads. From here, you can view, explore, and troubleshoot issues that affect your Kubernetes workloads.

Kubernetes Workload Landing Page Example

Check your Checksums

You asked us to provide you with checksum values during installation of the agent for Windows and Linux and we think that's a great idea. So here they are:

Agent Checksum values shown during installation

Log Alerting Improvements

Group By

When creating or editing a Log Monitor, you can now set "Group By" attributes to allow for more focused alerting. Look for the "Group By" attributes below the "filter" settings in your monitor definition.

Group By example in monitor definition

This change brings Metric Monitors and Log Monitors into feature parity by normalizing the “Group By” aspect of Monitor Definitions. This parity will allow customers to clone/duplicate all system-defined default Monitors for further customization.

Duplicating

You can now clone (duplicate) the Change Log, Kubernetes Log, and Data Collector Log monitors. This creates a new custom log monitor that you can modify to your specific definitions.

Duplicating a Log Monitor

11 New Default ONTAP Monitors covering SnapMirror for Business Continuity

We've added almost a dozen new system monitors for SnapMirror for Business Continuity (SMBC), which alert on changes to SMBC certificates and ONTAP Mediators.


November 2022

More than 40 new Security, Data Collection, and CVO monitors!

We've added dozens of new system-defined monitors to alert you to potential issues with Cloud Volumes, Security, and Data Protection. Read more about these monitors here.


October 2022

Better and more accurate Ransomware detection with ONTAP Autonomous Ransomware Protection integration

Cloud Secure improves ransomware detection through integration with ONTAP Autonomous Ransomware Protection (ARP).

Cloud Secure receives ONTAP ARP events on potential volume file encryption activity, and

  • Correlates volume encryption events with user activity to identify who is causing the damage,

  • Implements automatic response policies to block the attack,

  • Identifies which files were affected, helping to recover faster and conduct data breach investigations.


September 2022

Monitors available in Basic Edition

ONTAP Default monitors now available to use in Cloud Insights Basic Edition. This includes more than 70 infrastructure monitors and 30 workload examples.

ONTAP Power and StorageGRID dashboards

The dashboard gallery includes a new dashboard for ONTAP Power and Temperature as well as four dashboards for StorageGRID. If your environment is collecting ONTAP power metrics and/or StorageGRID data, import these dashboards by selecting +From Gallery.

At-a-glance threshold visibility in tables

Conditional Formatting allows you to set and highlight Warning-level and Critical-level thresholds in table widgets, bringing instant visibility to outliers and exceptional data points.

Conditional Formatting Example

Security Monitor

Cloud Insights can alert you when it detects that FIPS mode is disabled on the ONTAP system. Read more about System Monitors, and watch this space for more Security Monitors, coming soon!

Chat from Anywhere

Chat with a NetApp Support specialist from any Cloud Insights screen by selecting the new Help > Live Chat link. Help is available from the "?" icon in the upper right of the screen.

Help Menu with Live Chat highlighted

More visible Insights

If your environment is experiencing an Insight such as Shared resources Under Stress or Kubernetes Namespaces Running Out of Space, asset landing pages for resources affected now include links to the Insight itself, providing quicker exploration and troubleshooting.

New Data Collectors

  • Amazon S3 (available in Preview)

  • Brocade FOS 9.0.x

  • Dell/EMC PowerStore 3.0.0.0

Other Data Collector Updates

All data sources are now optimized to resume performance polling after Acquisition Unit updates and/or patches.

Operating System support

The following operating systems are supported with Cloud Insights Acquisition Units, in addition to those already supported:

  • Red Hat Enterprise Linux 8.5, 8.6


August 2022

Cloud Insights has a new look!

Starting this month, "Monitor and Optimize" has been renamed Observability. You'll find all your favorite features like Dashboards, Queries, Alerts, and Reporting here. In addition, look for Cloud Secure under the new Security menu. Note that only the menus have changed; feature functionality remains the same.

New CI Menu

Looking for the Help menu?

Help now lives in the upper right of the screen.

Help menu is in upper right corner

Not sure where to start? Check out ONTAP Essentials!

ONTAP Essentials is a set of dashboards and workflows that provide detailed views into your NetApp ONTAP inventories, workloads, and data protection, including days-to-full predictions for storage capacity and performance. You can even see if any controllers are running at high utilization. ONTAP Essentials is your ideal place for all of your NetApp ONTAP monitoring needs!

ONTAP Essentials—​available in all Editions—​is designed to be intuitive to existing ONTAP operators and administrators, easing the transition from ActiveIQ Unified Manager to service-based management tools.

Overview dashboard for ONTAP Essentials

Storage Data families are merged

You asked for it, and now you've got it. Storage base-2 and base-10 data units are now combined into one family, from bits and bytes to tebibits and terabytes, making it easier to display data your way on your dashboards. Data Rates are also now one big family of their own.

drop-dow showing merge of base-2 and base-10 data families

How much power is my storage using?

Display and monitor your ONTAP storage shelf and node power consumption, temperature, and fan speed, using the netapp_ontap.storage_shelf, netapp_ontap.system_node and netapp_ontap.cluster (power consumption only) metrics.

Storage Power Consumption metrics

Features graduated from Preview

The following features have moved out of Preview and are now available to all customers:

Feature

Description

Kubernetes Namespaces Running out of Space

The Kubernetes Namespaces Running Out of Space Insight gives you a view into workloads on your Kubernetes namespaces that are at risk of running out of space, with an estimate for the number of days remaining before each space becomes full.
Read More

Shared Resource Under Stress

The Shared Resource Under Stress insight uses AI/ML to automatically identify where resource contention is causing performance degradation in your environment, highlights any workloads impacted by it, and provides recommended actions to remediate, letting you solve performance issues more quickly.
Read More

Cloud Secure – Block user access on attack

Greater protection for your business-critical data with the ability to block user access when an attack is detected.
Access can be blocked automatically, using Automated Response Policies, or manually from the alert or user details pages.
Read More

How's my data collection health?

Cloud Insights provides two new heartbeat monitors for your Acquisition Units, as well as two monitors to alert you to data collector failures. These can be used to alert you quickly to data collection issues.

The following monitors are now available in the Data Collection monitor group:

  • Acquisition Unit Heartbeat-Critical

  • Acquisition Unit Heartbeat-Warning

  • Collector Failed

  • Collector Warning

Note that these monitors are in Paused state by default. Activate them to be alerted about data collection issues.

Auto-Renewing API Tokens

API Access Tokens can now be set for auto-renewal. By enabling this feature, new/refreshed API Access Tokens will automatically be generated for expiring tokens. Cloud Insights agents using an expiring token will automatically be updated to use the corresponding new/refreshed API Access Token, allowing them to continue to operate seamlessly. Simply check the “Renew token automatically” box when creating your token. This feature is currently supported on Cloud Insights agents running on the Kubernetes platform with the latest NetApp Kubernetes Monitoring Operator.

Basic Edition gives you more than before

Your trial is ending but you're not yet sure whether a subscription is right for you? Basic Edition has always given you a chance to continue using Cloud Insights with your current ONTAP data collector, but now you can continue capturing VMWare version, topology, and IOPS/Throughput/Latency data as well. NetApp customers with premium support on their storage systems will also be entitled to support for Cloud Insights.

Ready to learn more?

Check out the Learning Center section of the Help > Support page for links to NetApp University Cloud Insights course offerings!

Operating System support

The following operating system is supported with Cloud Insights Acquisition Units, in addition to those already supported:

  • Windows 11


June 2022

Kubernetes cluster saturation and other details

Cloud Insights makes it easier than ever to explore your Kubernetes environment, with an improved cluster detail page that provides Saturation details as well as a cleaner view into Namespaces and Workloads.

Cluster detail page

The Cluster list page also gives you a quick view of saturation, in addition to Node, Pod, Namespace, and Workload counts:

Cluster list page showing saturation numbers

How old is your Kubernetes cluster?

Is your cluster just starting in the world, or has it experienced a long digital life? Age has been added as a time metric collected for Kubernetes Nodes.

Kubernetes Node Table showing age in Days

Capacity Time-to-Full forecasting

Cloud Insights provides a dashboard to forecast the number of days until capacity runs out for each Internal Volume monitored. These values can help to significantly reduce the risk of an outage.

Internal Volume TTF forecasting dashboard

TTF counters are also available for Storage, Storage Pool, and Volume. Keep watching this space for additional dashboards for these objects.

Note that Time-to-Full forecasting is moving out of Preview and will be rolled out to all customers.

What's changed in my environment?

ONTAP change log entries can be viewed in the log explorer.

image showing change log entry examples

Operating System support

The following operating systems are supported with Cloud Insights Acquisition Units, in addition to those already supported:

  • CentOS Stream 9

  • Windows 2022

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.22.3, with performance and security improvements.
Users wishing to update can refer to the appropriate upgrade section of the Agent Installation documentation.
Previous versions of the agent will continue to function with no user action required.

Preview Features

Cloud Insights regularly highlights a number of exciting new preview features. If you are interested in previewing one or more of these features, contact your NetApp Sales Team for more information.

Feature

Description

Kubernetes Namespaces Running out of Space

The Kubernetes Namespaces Running Out of Space Insight gives you a view into workloads on your Kubernetes namespaces that are at risk of running out of space, with an estimate for the number of days remaining before each space becomes full.
Read More

Cloud Secure – block user access on attack

Greater protection for your business-critical data with the ability to block user access when an attack is detected.
Access can be blocked automatically, using Automated Response Policies or manually from the alert or user details pages.
Read More

Shared Resource Under Stress

The Shared Resource Under Stress insight uses AI/ML to automatically identify where resource contention is causing performance degradation in your environment, highlights any workloads impacted by it, and provides recommended actions to remediate, letting you solve performance issues more quickly.
Read More


May 2022

Chat live with NetApp Support

You can now chat live with NetApp Support personnel! On the Help > Support page, simply click the Chat icon or click Chat in the "Contact Us" section to start a chat session. Chat support is available US weekdays for Standard and Premium Edition users.

Chat Icon showing the blue NetApp "N" above a smile

Kubernetes Operator

We’ve made it easier to get you up and running with Cloud Insights’ advanced Kubernetes monitoring and cluster explorer.

The Kubernetes Monitoring Operator (NKMO) is the preferred method for installing Kubernetes for Cloud Insights Insights, for more flexible configuration of monitoring in fewer steps, as well as enhanced opportunities for monitoring other software running in the K8s cluster.

Click the link above for more information and pre-requisites

Manage Users and Invites with API

You can now manage users and invites using Cloud Insights' powerful API. Read more in the API Swagger Documentation.

Data Collection Alerts

Don’t miss out on critical metrics due to a failed collector!

It’s easier than ever to keep track of your data collectors with new alerts for data collector and acquisition unit failures.
Note that these Monitors are Paused by default. To enable, navigate to your monitors page and locate and resume “Acquisition Unit Shutdown” and “Collector Failed”

Alert on ONTAP storage changes

Don’t let unexpected storage changes lead to outages!

You can now configure Cloud Insights to alert when modification or removal of FlexVols, nodes and SVMs are detected on ONTAP systems.

Preview Features

Cloud Insights regularly highlights a number of exciting new preview features. If you are interested in previewing one or more of these features, contact your NetApp Sales Team for more information.

Feature

Description

Kubernetes Namespaces Running out of Space

The Kubernetes Namespaces Running Out of Space Insight gives you a view into workloads on your Kubernetes namespaces that are at risk of running out of space, with an estimate for the number of days remaining before each space becomes full.
Read More

Internal Volume and Volume Capacity Time-to-Full forecasting

Cloud Insights is able to prognose the number of days until capacity runs out for each Internal Volume and Volume monitored. This value can help to significantly reduce the risk of an outage.

Cloud Secure – block user access on attack

Greater protection for your business-critical data with the ability to block user access when an attack is detected.
Access can be blocked automatically, using Automated Response Policies or manually from the alert or user details pages.
Read More

Shared Resource Under Stress

The Shared Resource Under Stress insight uses AI/ML to automatically identify where resource contention is causing performance degradation in your environment, highlights any workloads impacted by it, and provides recommended actions to remediate, letting you solve performance issues more quickly.
Read More


April 2022

Share your Feedback!

We want your input to help shape Cloud Insights. Earn points and prizes by participating in NetApp's Insights to Action program. Sign up now!

Updated Dashboard Editor

We’ve overhauled our dashboard creation tools to make it easier for you to visualize your data even more quickly. Navigate to the “Dashboards” page of Cloud Insights to edit an existing dashboard, add one from our dashboard gallery, or create a new dashboard of your own to check it out.

Widget Editor Improved Layout

A new Count aggregation method has also been introduced. When grouping data in bar chart, column chart, and pie chart widgets, you can quickly and easily show the number of relevant objects for the selected metric.

aggregation drop-down showing Count

Additionally, line charts now allow you to select one of three interpolation methods:

  • None - No interpolation is done

  • Linear - Interpolates a data point between the existing points

  • Stair - Uses the previous data point as the interpolated data point

Enhanced Monitoring for Your Kubernetes Infrastructure

Cloud Insights keeps you on top of changes in your Kubernetes environment by alerting you when pods, daemonsets, and replicasets are created or removed, as well as when new deployments are created. Kubernetes monitors default to paused state, so you should enable only the specific ones you need.

Preview Features

Cloud Insights regularly highlights a number of exciting new preview features. If you are interested in previewing one or more of these features, contact your NetApp Sales Team for more information.

Feature

Description

Internal Volume and Volume Capacity Time-to-Full forecasting

Cloud Insights is able to prognose the number of days until capacity runs out for each Internal Volume and Volume monitored. This value can help to significantly reduce the risk of an outage.

Cloud Secure – block user access on attack

Greater protection for your business-critical data with the ability to block user access when an attack is detected.
Access can be blocked automatically, using Automated Response Policies or manually from the alert or user details pages.
Read More

Shared Resource Under Stress

The Shared Resource Under Stress insight uses AI/ML to automatically identify where resource contention is causing performance degradation in your environment, highlights any workloads impacted by it, and provides recommended actions to remediate, letting you solve performance issues more quickly.
Read More

New Data Collector

  • Cohesity SmartFiles - This REST API-based collector will acquire a Cohesity cluster, discovering the “Views” (as CI Internal Volumes), the various nodes, as well as collecting performance metrics.

Other Data Collector Updates

Collection and display of performance data has been improved on the following data collectors:

  • Brocade CLI

  • Dell/EMC VPlex, PowerStore, Isilon/PowerScale, VNX Block/Clariion CLI, XtremIO, Unity/VNXe

  • Pure FlashArray

These performance enhancements are already available in all NetApp data collectors as well as VMware and Cisco, and will be rolled out to all other data collectors over the next few months.


March 2022

Cloud Connection for ONTAP 9.9+

The NetApp Cloud Connection for ONTAP 9.9+ data collector eliminates the need to install an external acquisition unit, thereby simplifying troubleshooting, maintenance, and initial deployment.

New FSx for NetApp ONTAP Monitors

Monitoring your FSx for NetApp ONTAP environment is easy with new system-defined monitors for both infrastructure (metrics) and workloads (logs).

FSx monitors for infrastructure
FSx monitors for workloads

New Cloud Secure features available to all

Your environment is more secure than ever with the following Cloud Secure features now generally available:

Feature

Description

Data Destruction – File Deletion attack detection

Detect abnormal large-scale file deletion activity, block malicious file access by malicious users, and take automatic snapshots with automatic response policies.

Separate notifications for Warnings and Alerts

Warning and Alert notifications can be sent to separate recipients, ensuring the right team can stay informed

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.21.2, with performance and security improvements.
Users wishing to update can refer to the appropriate upgrade section of the Agent Installation documentation.
Previous versions of the agent will continue to function with no user action required.

Data Collector Updates

  • The Broadcom Fibre Channel Switches data collector has been optimized to reduce the number of CLI commands issued with each inventory poll.


February 2022

Cloud Insights addresses Apache Log4j vulnerabilities

Customer security is a top priority at NetApp. Cloud Insights includes updates to its software libraries to address the recent Apache Log4j vulnerabilities.

Please refer to the following on NetApp’s Product Security Advisory website:

You can read more about these vulnerabilities and NetApp's response at the NetApp Newsroom.

Kubernetes Namespace Detail Page

Exploring your Kubernetes environment is now better than ever, with informative detail pages for your cluster’s namespaces. The namespace detail page provides a summary of all the assets used by a namespace, including all the backend storage resources and their capacity utilizations.

Kubernetes Namespace Detail Page


December 2021

Deeper integration for ONTAP systems

Simplify alerting for ONTAP hardware failures and more with new integration with NetApp Event Management System (EMS).
Explore and alert on low-level ONTAP messages in Cloud Insights to inform and improve troubleshooting workflows and further reduce reliance on ONTAP element management tooling.

Querying Logs

For ONTAP systems, Cloud Insights Queries include a powerful Log Explorer, allowing you to easily investigate and troubleshoot EMS log entries.

Log Queries

Data Collector-level notifications.

In addition to system-defined and custom-created Monitors for alerting, you can also set alert notifications for ONTAP data collectors, allowing you to specify recipients for collector-level alerts, independent of other monitor alerts.

Greater flexibility of Cloud Secure roles

Users can be granted access to Cloud Secure features based on roles set by an administrator:

Role

Cloud Secure Access

Administrator

Can perform all Cloud Secure functions, including those for Alerts, Forensics, data collectors, automated response policies, and APIs for Cloud Secure.
An Administrator can also invite other users but can only assign Cloud Secure roles.

User

Can view and manage Alerts and view Forensics. User role can change alert status, add a note, take snapshots manually, and block user access.

Guest

Can view Alerts and Forensics. Guest role cannot change alert status, add a note, take snapshots manually, or block user access.

Operating System support

CentOS 8.x support is being replaced with CentOS 8 Stream support. CentOS 8.x will reach End-of-Life on December 31, 2021.

Data Collector Updates

A number of Cloud Insights data collector names have been added to reflect vendor changes:

Vendor/Model

Previous Name

Dell EMC PowerScale

Isilon

HPE Alletra 9000 / Primera

3PAR

HPE Alletra 6000

Nimble


November 2021

Adaptive Dashboards

New variables for attributes and the ability to use variables in widgets.

Dashboards are now more powerful and flexible than ever. Build adaptive dashboards with attribute variables to quickly filter dashboards on the fly. Using these and other pre-existing variables you can now create one high level dashboard to see metrics for your entire environment, and seamlessly filter down by resource name, type, location, and more. Use number variables in widgets to associate raw metrics with costs, for example cost per GB for storage as a service.

Drop-Down annotations in a variable
attribute filtering in a variable

Access the Reporting Database via API

Enhanced capabilities for integration with third party reporting, ITSM, and automation tools: Cloud Insights' powerful API allows users to query the Cloud Insights Reporting database directly, without going through the Cognos Reporting environment.

Pod tables on VM Landing Page

Seamless navigation between VMs and the Kubernetes Pods using them: for improved troubleshooting and performance headroom management, a table of associated Kubernetes Pods will now appear on VM landing pages.

Kubernetes Pod table on a VM landing page

Data Collector Updates

  • ECS now reports firmware for storage and node

  • Isilon has improved prompt detection

  • Azure NetApp Files collects performance data more quickly

  • StorageGRID now supports Single Sign-On (SSO)

  • Brocade CLI properly reports model for X&-4

Additional Operating Systems supported

The Cloud Insights Acquisition Unit supports the following operating systems, in addition to those already supported:

  • Centos (64-bit) 8.4

  • Oracle Enterprise Linux (64-bit) 8.4

  • Red Hat Enterprise Linux (64-bit) 8.4


October 2021

Filters on K8S Explorer pages

Kubernetes Explorer page filters give you focused control of the data displayed for your Kubernetes cluster, node, and pod exploration.

Kubernetes Explorer filtering example

K8s Data for Reporting

Kubernetes data is now available for use in Reporting, allowing you to create chargeback or other reports. In order for Kubernetes chargeback data to be passed to Reporting, you must have an active connection to, and Cloud Insights must be receiving data from, your Kubernetes cluster as well as its back-end storage. If there is no data received from the back-end storage, Cloud Insights can not send Kubernetes object data to Reporting.

Kubernetes data showin in a chargeback report

Dark Theme has arrived

Many of you asked for a dark theme, and Cloud Insights has answered. To switch between light and dark theme, click the drop-down next to your user name.
Switch to Dark Theme is available in the User drop-down
An image of a typical dashboard shown in dark theme

Data Collector Support

We’ve made some improvements in Cloud Insights Data Collectors. Here are some highlights:

  • New collector for Amazon FSx for ONTAP


September 2021

Performance Policies are now Monitors

Monitors and Alerts have supplanted Performance Policies and Violations throughout Cloud Insights. Alerting with Monitors provides greater flexibility and insight into potential problems or trends in your environment.

Autocomplete Suggestions, Wildcards, and Expressions in Monitors

When creating a monitor for alerting, typing in a filter is now predictive, allowing you to easily search for and find the metrics or attributes for your monitor. Additionally, you are given the option to create a wildcard filter based on the text you type.

Type-ahead filters in Monitors

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.19.3, with performance and security improvements.
Users wishing to update can refer to the appropriate upgrade section of the Agent Installation documentation.
Previous versions of the agent will continue to function with no user action required.

Data Collector Support

We’ve made some improvements in Cloud Insights Data Collectors. Here are some highlights:

  • Microsoft Hyper-V collector now uses PowerShell instead of WMI

  • Azure VMs and VHD collector is now up to 10 times faster due to parallel calls

  • HPE Nimble now supports federated and iSCSI configurations

And since we’re always improving Data Collection, here are some other recent changes of note:

  • New collector for EMC Powerstore

  • New collector for Hitachi Ops Center

  • New collector for Hitachi Content Platform

  • Enhanced ONTAP collector to report Fabric Pools

  • Enhanced ANF with Storage Pool and Volume performance

  • Enhanced EMC ECS with Storage Nodes and Storage performance as well as the Object Count in buckets

  • Enhanced EMC Isilon with Storage Node and Qtree metrics

  • Enhanced EMC Symetrix with volume QOS limit metrics

  • Enhanced IBM SVC and EMC PowerStore with Storage Nodes parent serial number


August 2021

New Audit Page User Interface

The Audit page provides a cleaner interface and now allows the export of audit events to .CSV file.

Enhanced User Role Management

Cloud Insights now allows even greater freedom for assigning user roles and access controls. Users can now be assigned granular permissions for monitoring, reporting, and Cloud Secure separately.

This means you can allow more users administrative access to monitoring, optimization, and reporting functions whilst restricting access to your sensitive Cloud Secure audit and activity data to only those that need it.

Find out more about the different levels of access in the Cloud Insights documentation.


June 2021

Autocomplete Suggestions, Wildcards, and Expressions in Filters

With this release of Cloud Insights, you no longer need to know all the possible names and values on which to filter in a query or widget. When filtering, you can simply start typing and Cloud insights will suggest values based on your text. No more looking up Application names or Kubernetes attributes ahead of time just to find the ones you want to show in your widget.

As you type in a filter, the filter displays a smart list of results from which you can choose, as well as the option to create a wildcard filter based on the current text. Selecting this option will return all results that match the wildcard expression. You can of course also select multiple individual values that you want added to the filter.

Wildcard Filter

Additionally, you can create expressions in a filter using NOT or OR, or you can select the "None" option to filter for null values in the field.

Read more about filtering options in queries and widgets.

APIs available by Edition

Cloud Insights' powerful APIs are more accessible than ever, with Alerts APIs now available in Standard and Premium Editions.
The following APIs are available for each Edition:

API Category Basic Standard Premium

Acquisition Unit

check mark

check mark

check mark

Data Collection

check mark

check mark

check mark

Alerts

check mark

check mark

Assets

check mark

check mark

Data Ingestion

check mark

check mark

Kubernetes PV and Pod Visibility

Cloud Insights provides visibility into the back-end storage for your Kubernetes environments, giving you insight to your Kubernetes Pods and Persistent Volumes (PVs). You can now track PV counters such as IOPS, latency, and throughput from a single Pod’s usage through a PV counter to a PV and all the way to the back-end storage device.

On a Volume or Internal Volume landing page, two new tables are displayed:

Kubernetes PV Table
Kubernetes Pod Table

Note that to take advantage of these new tables, it is recommended to uninstall your current Kubernetes agent, and install it fresh. You must also install Kube-State-Metrics version 2.1.0 or later.

On a Kubernetes Node page, you can now click to open the Node's VM page. The VM page also includes a link back to the Node itself.

Kubernetes Node Page showing VM link
Kubernetes VM Page showing Node link

Alert Monitors replacing Performance Policies

To enable the added benefits of multiple thresholds, webhook and email alert delivery, alerting on all metrics using a single interface, and more, Cloud Insights will be converting Standard and Premium Edition customers from Performance Policies to Monitors during the months of July and August, 2021. Learn more about Alerts and Monitors, and stay tuned for this exciting change.

Cloud Secure supports NFS

Cloud Secure now supports NFS for ONTAP data collection. Monitor SMB and NFS user access to protect your data from ransomware attacks.
Additionally, Cloud Secure supports Active-Directory and LDAP user directories for collection of NFS user attributes.

Cloud Secure snapshot purge

Cloud Secure automatically deletes snapshots based on the Snapshot Purge Settings, to save storage space and reduce the need for manual snapshot deletion.

Purge Settings

Cloud Secure data collection speed

A single data collector agent system can now post up to 20,000 events per second to Cloud Secure.


May 2021

Here are some of the changes we've made in April:

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.17.3, with performance and security improvements.
Users wishing to update can refer to the appropriate upgrade section of the Agent Installation documentation.
Previous versions of the agent will continue to function with no user action required.

Add Corrective Actions to an Alert

You can now add an optional description as well as additional insights and/or corrective actions when creating or modifying a Monitor by filling in the Add an Alert Description section. The description will be sent with the alert. The insights and corrective actions field can provide detailed steps and guidance for dealing with alerts and will be displayed in the summary section of the alert landing page.

Alert Corrective Actions and Description

Cloud Insights APIs for All Editions

API access is now available in all editions of Cloud Insights.
Users of Basic edition can now automate actions for Acquisition Units and Data Collectors, and Standard Edition users can query metrics and ingest custom metrics.
Premium edition continues to allow full use of all API categories.

API Category Basic Standard Premium

Acquisition Unit

check mark

check mark

check mark

Data Collection

check mark

check mark

check mark

Assets

check mark

check mark

Data Ingestion

check mark

check mark

Data Warehouse

check mark

For details on API usage, please refer to the API documentation.


April 2021

Easier Management of Monitors

Monitor Grouping simplifies the management of monitors in your environment. Multiple monitors can now be grouped together and paused as one. For example, if you have an update occurring on a stack of infrastructure, you can pause alerts from all those devices via one click.

Monitor groups is the first part of an exciting new feature bringing improved management of ONTAP devices to Cloud Insights.

Monitor Grouping

Enhanced Alerting Options Using Webhooks

Many commercial applications support Webhooks as a standard input interface. Cloud Insights now supports many of these delivery channels, providing default templates for Slack, PagerDuty, Teams, and Discord, in addition to providing customizable generic webhooks to support many other applications.

Webhooks Notifications

Improved Device Identification

To improve monitoring and troubleshooting as well as deliver accurate reporting, it is helpful to understand the names of devices rather than their IP addresses or other identifiers. Cloud Insights now incorporates an automatic way to identify the names of storage and physical host devices in the environment, using a rule-based approach called Device Resolution, available in the Manage menu.

You asked for more!

A popular ask by customers has been for more default options for visualizing the range of data, so we have added the following five new choices that are now available throughout the service via the time range picker:

  • Last 30 Minutes

  • Last 2 Hours

  • Last 6 Hours

  • Last 12 Hours

  • Last 2 Days

Multiple Subscriptions in one Cloud Insights Environment

Starting April 2, Cloud Insights supports multiple subscriptions of the same edition type for a customer in a single Cloud Insights instance. This enables customers to co-term parts of their Cloud Insights subscription with infrastructure purchases. Contact NetApp Sales for assistance with multiple subscriptions.

Choose Your Path

While setting up Cloud Insights, you can now choose whether to start with Monitoring and Alerting or Ransomware and Insider Threat Detection. Cloud Insights will configure your starting environment based on the path you choose. You can configure the other path at any time afterward.

Easier Cloud Secure Onboarding

And it is easier than ever to start using Cloud Secure, with a new step-by-step setup checklist.

Cloud Secure Checklist

As always, we love to hear your suggestions! Send them to ng-cloudinsights-customerfeedback@netapp.com.


February 2021

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.17.0, which includes vulnerability and bug fixes.

Cloud Cost Analyzer

Experience the power of Spot by NetApp with Cloud Cost, which provides a detailed cost analysis of past, present, and estimated spending, providing visibility into cloud usage in your environment. The Cloud Cost dashboard delivers a clear view of cloud expenses and a drill down into individual workloads, accounts, and services.

Cloud Cost can help with these major challenges:

  • Tracking and monitoring your cloud expenses

  • Identifying waste and potential optimization areas

  • Delivering executable action items

Cloud Cost is focused on monitoring. Upgrade to the full Spot by NetApp account to enable automatic cost saving and environment optimization.

Querying for objects having null values using filters

Cloud Insights now allows searching for attributes and metrics having null/none values through the use of filters. You can perform this filtering on any attributes/metrics in the following places:

  • On the Query page

  • In Dashboard widgets and page variables

  • On the Alerts list page

  • When creating Monitors

To filter for null/none values, simply select the None option when it appears in the appropriate filter drop-down.

Null filter in dropdown

Multi-Region Support

Starting today we offer the Cloud Insights service in different regions across the globe, which facilitates performance and increases security for customers based outside the United States. Cloud Insights/Cloud Secure stores information according to the region in which your environment is created.

Click here for more information.


January 2021

Additional ONTAP Metrics Renamed

As part of our continuing effort to improve efficiency of data-gathering from ONTAP systems, the following ONTAP metrics have been renamed.

If you have existing dashboard widgets or queries using any of these metrics, you will need to edit or re-create them to use the new metric names.

Previous Metric Name New Metric Name

netapp_ontap.disk_constituent.total_transfers

netapp_ontap.disk_constituent.total_iops

netapp_ontap.disk.total_transfers

netapp_ontap.disk.total_iops

netapp_ontap.fcp_lif.read_data

netapp_ontap.fcp_lif.read_throughput

netapp_ontap.fcp_lif.write_data

netapp_ontap.fcp_lif.write_throughput

netapp_ontap.iscsi_lif.read_data

netapp_ontap.iscsi_lif.read_throughput

netapp_ontap.iscsi_lif.write_data

netapp_ontap.iscsi_lif.write_throughput

netapp_ontap.lif.recv_data

netapp_ontap.lif.recv_throughput

netapp_ontap.lif.sent_data

netapp_ontap.lif.sent_throughput

netapp_ontap.lun.read_data

netapp_ontap.lun.read_throughput

netapp_ontap.lun.write_data

netapp_ontap.lun.write_throughput

netapp_ontap.nic_common.rx_bytes

netapp_ontap.nic_common.rx_throughput

netapp_ontap.nic_common.tx_bytes

netapp_ontap.nic_common.tx_throughput

netapp_ontap.path.read_data

netapp_ontap.path.read_throughput

netapp_ontap.path.write_data

netapp_ontap.path.write_throughput

netapp_ontap.path.total_data

netapp_ontap.path.total_throughput

netapp_ontap.policy_group.read_data

netapp_ontap.policy_group.read_throughput

netapp_ontap.policy_group.write_data

netapp_ontap.policy_group.write_throughput

netapp_ontap.policy_group.other_data

netapp_ontap.policy_group.other_throughput

netapp_ontap.policy_group.total_data

netapp_ontap.policy_group.total_throughput

netapp_ontap.system_node.disk_data_read

netapp_ontap.system_node.disk_throughput_read

netapp_ontap.system_node.disk_data_written

netapp_ontap.system_node.disk_throughput_written

netapp_ontap.system_node.hdd_data_read

netapp_ontap.system_node.hdd_throughput_read

netapp_ontap.system_node.hdd_data_written

netapp_ontap.system_node.hdd_throughput_written

netapp_ontap.system_node.ssd_data_read

netapp_ontap.system_node.ssd_throughput_read

netapp_ontap.system_node.ssd_data_written

netapp_ontap.system_node.ssd_throughput_written

netapp_ontap.system_node.net_data_recv

netapp_ontap.system_node.net_throughput_recv

netapp_ontap.system_node.net_data_sent

netapp_ontap.system_node.net_throughput_sent

netapp_ontap.system_node.fcp_data_recv

netapp_ontap.system_node.fcp_throughput_recv

netapp_ontap.system_node.fcp_data_sent

netapp_ontap.system_node.fcp_throughput_sent

netapp_ontap.volume_node.cifs_read_data

netapp_ontap.volume_node.cifs_read_throughput

netapp_ontap.volume_node.cifs_write_data

netapp_ontap.volume_node.cifs_write_throughput

netapp_ontap.volume_node.nfs_read_data

netapp_ontap.volume_node.nfs_read_throughput

netapp_ontap.volume_node.nfs_write_data

netapp_ontap.volume_node.nfs_write_throughput

netapp_ontap.volume_node.iscsi_read_data

netapp_ontap.volume_node.iscsi_read_throughput

netapp_ontap.volume_node.iscsi_write_data

netapp_ontap.volume_node.iscsi_write_throughput

netapp_ontap.volume_node.fcp_read_data

netapp_ontap.volume_node.fcp_read_throughput

netapp_ontap.volume_node.fcp_write_data

netapp_ontap.volume_node.fcp_write_throughput

netapp_ontap.volume.read_data

netapp_ontap.volume.read_throughput

netapp_ontap.volume.write_data

netapp_ontap.volume.write_throughput

netapp_ontap.workload.read_data

netapp_ontap.workload.read_throughput

netapp_ontap.workload.write_data

netapp_ontap.workload.write_throughput

netapp_ontap.workload_volume.read_data

netapp_ontap.workload_volume.read_throughput

netapp_ontap.workload_volume.write_data

netapp_ontap.workload_volume.write_throughput

New Kubernetes Explorer

The Kubernetes Explorer provides a simple topology view of Kubernetes Clusters, allowing even non-experts to quickly identify issues & dependencies, from the cluster level down to the container and storage.

A wide variety of information can be explored using the Kubernetes Explorer's drill-down details for status, usage, and health of the Clusters, Nodes, Pods, Containers, and Storage in your Kubernetes environment.

The Kubernetes Explorer


December 2020

Simpler Kubernetes Installation

Kubernetes Agent installation has been streamlined to require fewer user interactions. Installing the Kubernetes Agent now includes Kubernetes data collection.


November 2020

Additional Dashboards

The following new ONTAP-focused dashboards have been added to the gallery and are available for import:

  • ONTAP: Aggregate Performance & Capacity

  • ONTAP FAS/AFF - Capacity Utilization

  • ONTAP FAS/AFF - Cluster Capacity

  • ONTAP FAS/AFF - Efficiency

  • ONTAP FAS/AFF - FlexVol Performance

  • ONTAP FAS/AFF - Node Operational/Optimal Points

  • ONTAP FAS/AFF - PrePost Capacity Efficiencies

  • ONTAP: Network Port Activity

  • ONTAP: Node Protocols Performance

  • ONTAP: Node Workload Performance (Frontend)

  • ONTAP: Processor

  • ONTAP: SVM Workload Performance (Frontend)

  • ONTAP: Volume Workload Performance (Frontend)

Column Rename in Table Widgets

You can rename columns in the Metrics and Attributes section of a table widget by opening the widget in Edit mode and clicking the menu at the top of the column. Enter the new name and click Save, or click Reset to set the column back to the original name.

Note that this only affects the column's display name in the table widget; the metric/attribute name does not change in the underlying data itself.

Table Widget Rename Column


October 2020

Default Expansion of Integration Data

Table widget grouping now allows for default expansions of Kubernetes, ONTAP Advanced Data, and Agent Node metrics. For example, if you group Kubernetes Nodes by Cluster, you will see a row in the table for each cluster. You could then expand each cluster row to see a list of the Node objects.

Basic Edition Technical Support

Technical Support is now available for subscribers to Cloud Insights Basic Edition in addition to Standard and Premium Editions. Additionally, Cloud Insights has simplified the workflow for creating a NetApp support ticket.

Cloud Secure Public API

Cloud Secure supports REST APIs for accessing Activity and Alert information. This is accomplished through the use of API Access Tokens, created through the Cloud Secure Admin UI, which are then used to access the REST APIs. Swagger documentation for these REST APIs is integrated with Cloud Secure.


September 2020

Query Page with Integration Data

The Cloud Insights Query page supports integration data (i.e. from Kubernetes, ONTAP Advanced Metrics, etc.). When working with integration data, the query results table displays a "Split-Screen" view, with object/grouping on the left side, and object data (attributes/metrics) on the right. You can also choose multiple attributes for grouping integration data.

Query showing integration data

Unit Display Formatting in Table Widget

Unit display formatting is now available in Table widgets for columns that display metric/counter data (for example, gigabytes, MB/second, etc.). To change a metric's display unit, click the "three dots" menu in the column header and select "Unit Display". You can choose from any of the available units. Available units will vary according to the type of metric data in the display column.

Table Widget Unit Management

Acquisition Unit Detail Page

Acquisition Units now have their own landing page, providing useful detail for each AU as well as information to help with troubleshooting. The AU detail page provides links to the AU's data collectors as well as helpful status information.

Cloud Secure Docker Dependency Removed

Cloud Secure's dependency on Docker has been removed. Docker is no longer required for Cloud Secure agent installation.

Reporting User Roles

If you have Cloud Insights Premium Edition with Reporting, every Cloud Insights user in your environment also has a Single Sign-On (SSO) login to the Reporting application (i.e. Cognos); by clicking the Reports link in the menu, they will automatically be logged in to Reporting.

Their user role in Cloud Insights determines their Reporting user role:

Cloud Insights Role

Reporting Role

Reporting Permissions

Guest

Consumer

Can view, schedule, and run reports and set personal preferences such as those for languages and time zones. Consumers cannot create reports or perform administrative tasks.

User

Author

Can perform all Consumer functions as well as create and manage reports and dashboards.

Administrator

Administrator

Can perform all Author functions as well as all administrative tasks such configuration of reports and the shutdown and restart of reporting tasks.

Note Cloud Insights Reporting is available for environments of 500 MUs or more.
Important If you are a current Premium Edition customer and wish to retain your reports, read this important note for existing customers.

New API Category for Data Ingestion

Cloud Insights has added a Data Ingestion API category, giving you greater control over custom data and agents. Detailed documentation for this and other API Categories can be found in Cloud Insights by navigating to Admin > API Access and clicking the API Documentation link. You can also attach a comment to the AU in the Note field, which is displayed on the AU detail page as well as the AU list page.


August 2020

Monitoring and Alerting

In addition to the current ability to set performance policies for storage objects, VMs, EC2, and ports, Cloud Insights Standard Edition now includes the ability to configure monitors for thresholds on Integration data for Kubernetes, ONTAP advanced metrics, and Telegraf plugins. You simply create a monitor for each object metric you want to trigger alerts, set the conditions for warning-level or critical-level thresholds, and specify the email recipient(s) desired for each level. You can then view and manage alerts to track trends or troubleshoot issues.

Monitor Conditions


July 2020

Cloud Secure Take a Snapshot Action

Cloud Secure protects your data by automatically taking a snapshot when malicious activity is detected, ensuring that your data is safely backed up.

You can define automated response policies that take a snapshot when ransomware attack or other abnormal user activity is detected.
You can also take a snapshot manually from the alert page.

Automatic Snapshot taken:
Alert Action Screen

Manual Snapshot:
Alert Action Screen

Metric/Counter updates

The following capacity counters are available for use in Cloud Insights UI and REST API. Previously these counters were only available for the Data Warehouse / Reporting.

Object Type Counter

Storage

Capacity - Spare Raw
Capacity - Failed Raw

Storage Pool

Data Capacity - Used
Data Capacity - Total
Other Capacity - Used
Other Capacity - Total
Capacity - Raw
Capacity - Soft Limit

Internal Volume

Data Capacity - Used
Data Capacity - Total
Other Capacity - Used
Other Capacity - Total
Clone Saved Capacity - Total

Cloud Secure Potential Attack Detection

Cloud Secure now detects potential attacks such as ransomware. Click on an alert in the Alerts list page to open a detail page showing the following:

  • Time of attack

  • Associated user and file activity

  • Action taken

  • Additional information to assist with tracking down possible security breaches

Alerts page showing potential ransomware attack:
Ransomware Alert Example

Detail page for potential ransomware attack:
Ransomware Detail Page Example

Subscribe to Premium Edition through AWS

During your trial of Cloud Insights, you can self-subscribe through AWS Marketplace to either Cloud Insights Standard Edition or Premium Edition. Previously, you could only self-subscribe through AWS Marketplace to Standard Edition only.

Enhanced Table Widget

The dashboard/asset page Table widget includes the following enhancements:

  • "Split-Screen" view: Table widgets display the object/grouping on the left side, and the object data (attributes/metrics) on the right.

    Table Widget showing left and right panes

  • Multiple attribute grouping: For Integration data (Kubernetes, ONTAP Advanced Metrics, Docker, etc.), you can choose multiple attributes for grouping. Data is displayed according to the grouping attributes/you choose.

    Grouping with Integration Data (shown in Edit mode):
    Integration Data Grouping in Table Widget

  • Grouping for Infrastructure data (storage, EC2, VM, ports, etc.) is by a single attribute as before. When grouping by an attribute which is not the object, the table will allow you to expand the group row to see all the objects within the group.

    Grouping with Infrastructure data (shown in display mode):
    Infrastructure Data Grouping in Table Widget

Metrics Filtering

In addition to filtering on an object's attributes in a widget, you can now filter on metrics as well.

Metrics Filtering

When working with integration data (Kubernetes, ONTAP Advanced Data, etc.), metric filtering removes the individual/unmatched data points from the plotted data series, unlike infrastructure data (storage, VM, ports etc.) where filters work on the aggregated value of the data series and potentially remove the entire object from the chart.

Integration Metric Filtering

ONTAP Advanced Counter Data

Cloud Insights takes advantage of NetApp's ONTAP-specific Advanced Counter Data, which provides a host of counters and metrics collected from ONTAP devices. ONTAP Advanced Counter Data is available to all NetApp ONTAP customers. These metrics enable customized and wide-ranging visualization in Cloud Insights widgets and dashboards.

ONTAP Advanced Counters can be found by searching for "netapp_ontap" in the widget's query, and selecting from among the counters.

Searching for ONTAP Advanced Counters

You can refine your search by typing additional parts of the counter name. For example:

  • lif

  • aggregate

  • offbox_vscan_server

  • and more

ONTAP Widget Example - WAFL
ONTAP Widget Example - Cp Reads

Please note the following:

  • Advanced Data collection will be enabled by default for new ONTAP data collectors. To enable Advanced Data collection for your existing ONTAP data collectors, edit the data collector and expand the Advanced Configuration section.

  • Advanced Data collection is not available for 7-mode ONTAP.

Advanced Counter Dashboards

Cloud Insights comes with a variety of pre-designed dashboards to help get you started on visualizing ONTAP Advanced Counters for topics such as Aggregate Performance, Volume Workload, Processor Activity, and more. If you have at least one ONTAP data collector configured, these can be imported from the Dashboard Gallery on any dashboard list page.

Learn More

More information on ONTAP Advanced Data can be found at the following links:

Policies and Violations Menu

Performance Policies and Violations are now found under the Alerts menu. Policy and Violation functionality are unchanged.

Policies and Violations Menu

Updated Telegraf Agent

The agent for ingestion of telegraf integration data has been updated to version 1.14, which includes bugs fixes, security fixes, and new plugins.

Note: When configuring a Kubernetes data collector on the Kubernetes platform, you may see an "HTTP status 403 Forbidden" error in the log, due to insufficient permissions in the "clusterrole" attribute.

To work around this issue, add the following highlighted lines to the rules: section of the endpoint-access clusterrole, and then restart the Telegraf pods.

rules:
- apiGroups:
  - ""
  - apps
  - autoscaling
  - batch
  - extensions
  - policy
  - rbac.authorization.k8s.io
  attributeRestrictions: null
  resources:
  - nodes/metrics
  - nodes/proxy     <== Add this line
  - nodes/stats
  - pods            <== Add this line
  verbs:
  - get
  - list            <== Add this line

June 2020

Simplified Data Collector Error Reporting

Reporting a data collector error is easier with the Send Error Report button on the data collector page. Clicking the button sends basic information about the error to NetApp and prompts investigation into the problem. Once pressed, Cloud Insights acknowledges that NetApp has been notified, and the Error Report button is disabled to indicate that an error report for that data collector has been sent. The button remains disabled until the browser page is refreshed.

Error Report Button

Widget Improvements

The following improvements have been made in dashboard widgets. These improvements are considered Preview functionality and may not be available for all Cloud Insights environments.

  • New object/metric chooser: Objects (Storage, Disk, Ports, Nodes, etc.) and their associated metrics (IOPS, Latency, CPU Count, etc.) are now available in widgets in a single inclusive drop-down with powerful search capability. You can enter multiple partial terms in the drop-down, and Cloud Insights will list all object metrics meeting those terms.

Object/Metric Chooser

  • Multiple tags grouping: When working with integration data (Kubernetes, etc.), you may group the data by multiple tags/attributes. For example, Sum memory usage by Kubernetes Namespace and Container name.

Multiple grouping when displaying integration data


May 2020

Reporting User Roles

The following roles have been added for Reporting:

  • Cloud Insights Consumers: can run and view reports

  • Cloud Insights Authors: can perform the Consumer functions as well as create and manage reports and dashboards

  • Cloud Insights Administrators: can perform the Author functions as well as all administrative tasks

Cloud Secure Updates

Cloud Insights includes the following recent Cloud Secure changes.

In the Forensics > Activity Forensics page, we provide two views to analyze and investigate user activity:

  • Activity view, focused on user activity (What operation? Where performed?)

  • Entities view, focused on what files the user accessed.

Entities Page Example

Additionally, the Alert email notification now contains a direct link to the alert page.

Dashboard Grouping

Dashboard grouping allows better management of dashboards that are relevant to you. You can add related dashboards to a group for "one-stop" management of, for example, your storage or virtual machines.

Groups are customized per user, so one person's groups can be different from someone else's. You can have as many groups as you need, with as few or as many dashboards in each group as you like.

Dashboard Groups

Dashboard Pinning

You can pin dashboards so favorites always appear at the top of the list.

Dashboard Pins

TV Mode and Auto-Refresh

TV Mode and Auto-Refresh allow for near-real-time display of data on a dashboard or asset page:

  • TV Mode provides an uncluttered display; the navigation menu is hidden, providing more screen real estate for your data display.

  • Data in widgets on Dashboards and Asset Landing Pages Auto-Refresh according a refresh interval (as little as every 10 seconds) determined by the Dashboard Time Range selected (or widget time range, if set to override the dashboard time).

Combined, TV Mode and Auto-Refresh provide a live view of your Cloud Insights data, perfect for seamless demonstrations or in-house monitoring.


April 2020

New Dashboard Time Range Choices

Time range choices for dashboards and other Cloud insights pages now include Last 1 Hour and Last 15 Minutes.

Cloud Secure Updates

Cloud Insights includes the following recent Cloud Secure changes.

  • Better file and folder metadata change recognition to detect if the user changed Permission, Owner, or Group Ownership.

  • Export user activity report to CSV.

Cloud Secure monitors and audits all user access operations on files and folders. Activity auditing allows you to comply with internal security policies, meet external compliance requirements such as PCI, GDPR, and HIPAA, and conduct data breach and security incident investigations.

Default Dashboard Time

The default time range for dashboards is now 3 Hours instead of 24 hours.

Optimized Aggregation Times

Optimized time aggregation intervals in time-series widgets (Line, Spline, Area, and Stacked Area charts) are more frequent for 3-hour and 24-hour dashboard/widget time ranges, allowing for faster charting of data.

  • 3 hour time range optimizes to a 1 minute aggregation interval. Previously this was 5 minutes.

  • 24 hour time range optimizes to a 30 minute aggregation interval. Previously this was 1 hour.

You can still override the optimized aggregation by setting a custom interval.

Display Unit Auto-Format

In most widgets, Cloud Insights knows the base unit in which to display values, for example Megabytes, Thousands, Percentage, Milliseconds (ms), etc., and now automatically formats the widget to the most readable unit. For example a data value of 1,234,567,890 bytes would be auto formatted to 1.23 gibibytes. In many cases, Cloud Insights knows the best format for the data being acquired. In cases where the best format is not known, or in widgets where you want to override the automatic formatting, you can choose the format you want.

Auto Format in Widgets

Import Annotations Using API

With Cloud Insights Premium Edition's powerful API, you can now import annotations and assign them to objects using a .CSV file. You can also import applications and assign business entities in the same way.

Importing Annotations

Simpler Widget Selector

Adding widgets to dashboards and asset landing pages is easier with a new widget selector that shows all widget types in a single all-at-once view, so the user no longer needs to scroll through a list of widget types to find the one they want to add. Related widgets are color-coordinated and grouped by proximity in the new selector.

New widget selector


February 2020

API with Premium Edition

Cloud Insights Premium Edition comes with a powerful API that can be used to integrate Cloud Insights with other applications, such as CMDB’s or other ticketing systems.

Detailed, Swagger-based information is found in Admin > API Acccess, under the API Documentation link. Swagger provides a brief description and usage information for the API, and allows you to try each API out in your environment.

The Cloud Insights API uses Access Tokens to provide permission-based access to categories of API, such as ASSETS or COLLECTION.

API Documentation

Initial Polling After Adding A Data Collector

Previously, after configuring a new data collector, Cloud Insights would poll the data collector immediately to gather inventory data, but would wait until the configured performance poll interval (typically 15 minutes) to gather initial performance data. It would then wait for another interval before initiating the second performance poll, which meant it would take up to 30 minutes before meaningful data was acquired from a new data collector.

Data collector polling has been greatly improved, such that the initial performance poll occurs immediately after the inventory poll, with the second performance poll occurring within a few seconds after completion of the first performance poll. This allows Cloud Insights to begin showing useful data on dashboards and graphs within a very short time.

This poll behavior also occurs after editing the configuration of an existing data collector.

Easier Widget Duplication

It is easier than ever to create a copy of a widget on a dashboard or landing page. In dashboard Edit mode, click the menu on the widget and select Duplicate. The widget editor is launched, pre-filled with the original widget's configuration and with a “copy” suffix in the widget name. You can easily make any necessary changes and Save the new widget. The widget will be placed at the bottom of your dashboard, and you can position it as needed. Remember to Save your dashboard when all changes are complete.

Duplicate a widget

Single Sign-On (SSO)

With Cloud Insights Premium Edition, administrators can enable Single Sign-On (SSO) access to Cloud Insights for all users in their corporate domain, without having to invite them individually. With SSO enabled, any user with the same domain email address can log into Cloud Insights using their corporate credentials.

Note SSO is only available in Cloud Insights Premium Edition, and must be configured before it can be enabled for Cloud Insights. SSO configuration includes Identity Federation through NetApp Cloud Central. Federation allows single sign-on users to access your NetApp Cloud Central accounts using credentials from your corporate directory.

January 2020

Swagger documentation for REST API

Swagger explains each available REST API in Cloud Insights, as well as its usage and syntax. Information on Cloud Insights APIs is available in documentation.

Feature Tutorials Progress Bar

The feature tutorials checklist has been moved to the top banner and now features a progress indicator. Tutorials are available for each user until dismissed, and are always available in Cloud Insights documentation.

Tutorial Checklist Progress

Acquisition Unit Changes

When installing an Acquisition Unit (AU) on a host or VM that has the same name as an already-installed AU, Cloud Insights assures a unique name by appending the AU name with "_1", "_2", etc. This is also the case when uninstalling and reinstalling an AU from the same VM without first removing it from Cloud Insights. Want a different AU name altogether? No problem; AU's can be renamed after installation.

Optimized Time Aggregation in Widgets

In widgets, you can choose between an Optimized time aggregation interval or a Custom interval that you set. Optimized aggregation automatically selects the right time interval based on the selected dashboard time range (or widget time range, if overriding the dashboard time). The interval dynamically changes as the dashboard or widget time range is changed.

Simplified "Getting Started with Cloud Insights" process

The process for getting started using Cloud Insights has been simplified to make your first-time setup smoother and easier. Simply select an initial data collector and follow the instructions. Cloud Insights will walk you through configuring the data collector and any agent or acquisition unit required. In most cases it will even import one or more initial dashboards so you can start gaining insight into your environment quickly (but please allow up to 30 minutes for Cloud Insights to collect meaningful data).

Additional improvements:

  • Acquisition Unit installation is simpler and runs faster.

  • Alphabetical Data Collectors choices make it easier to find the one you’re looking for.

  • Improved Data Collector setup instructions are easier to follow.

  • Experienced users can skip the getting started process with the click of a button.

  • A new Progress bar shows you where you are in the process.

    Progress Bar


December 2019

Business Entity can be used in filters

Business Entity annotations can be used in filters for queries, widgets, performance policies, and landing pages.

Drill-down available for Single-Value and Gauge widgets, and any widgets rolled to by "All"

Clicking the value in a single-value or gauge widget opens a query page showing the results of the first query used in the widget. Additionally, clicking the legend for any widget whose data is rolled up by "All" will also open a query page showing the results of the first query used in the widget.

Trial period extended

New users who sign up for a free trial of Cloud Insights now have 30 days to evaluate the product. This is an increase from the previous 14-day trial period.

Managed Unit calculation

The calculation of Managed Units (MUs) in Cloud Insights has been changed to the following:

  • 1 Managed Unit = 2 hosts (any virtual or physical machine)

  • 1 Managed Unit = 4 TB of unformatted capacity of physical or virtual disks

This change effectively doubles the environment capacity that you can monitor using your existing Cloud Insights subscription.


November 2019

October 2019

Reporting

Cloud Insights Reporting is a business intelligence tool that enables you to view pre-defined reports or create custom reports. With Reporting you can perform the following tasks:

  • Run a pre-defined report

  • Create a custom report

  • Customize the report format and delivery method

  • Schedule reports to run automatically

  • Email reports

  • Use colors to represent thresholds on data

Cloud Insights Reporting can generate custom reports for areas like chargeback, consumption analysis, and forecasting, and can help answer questions such as the following:

  • What inventory do I have?

  • Where is my inventory?

  • Who is using our assets?

  • What is the chargeback for allocated storage for a business unit?

  • How long until I need to acquire additional storage capacity?

  • Are business units aligned along the proper storage tiers?

  • How is storage allocation changing over a month, quarter, or year?

Reporting is available with Cloud Insights Premium Edition.

Active IQ Enhancements

Active IQ Risks are now available as objects that can be queried as well as used in dashboard table widgets. The following Risks object attributes are included:
* Category
* Mitigation Category
* Potential Impact
* Risk Detail
* Severity
* Source
* Storage
* Storage Node
* UI Category


September 2019

New Gauge Widgets

Two new widgets are available for displaying single-value data on your dashboards in eye-catching colors based on thresholds you specify. You can display values using either a Solid Gauge or Bullet Gauge. Values that land inside the Warning range are displayed in orange. Values in the Critical range are displayed in red. Values below the Warning threshold are displayed in green.

Solid/Traditional Gauge
Bullet Gauge

Conditional Color Formatting for Single Value Widget

You can now display the Single-Value widget with a colored background based on thresholds you set.

Single-Value widgets with formatting

Invite Users During Onboarding

At any point during the onboarding process, you can click on Admin > User Management > +User to invite additional users to your Cloud Insights environment. Be aware that users with Guest or User roles will see greater benefit once onboarding is complete and data has been collected.

Data Collector Detail Page improvement

The data collector detail page has been improved to display errors in a more readable format. Errors are now displayed in a separate table on the page, with each error displayed on a separate line in the case of multiple errors for the data collector.


August 2019

All vs. Available Data Collectors

When adding data collectors to your environment, you can set a filter to show only the data collectors available to you based on your subscription level, or all data collectors.

ActiveIQ Integration

Cloud Insights collects data from NetApp ActiveIQ, which provides a series of visualizations, analytics, and other support related services to NetApp customers and their hardware / software systems. Cloud Insights integrates with ONTAP Data Management systems. See Active IQ for more information.


July 2019

Dashboard Improvements

Dashboards and Widgets have been improved with the following changes:

  • In addition to Sum, Min, Max, and Avg, Count is now an option for roll up in Single-Value widgets. When rolling up by “Count”, Cloud Insights checks if an object is active or not, and only adds the active ones to the count. The resulting number is subject to aggregation and filters.

  • In the Single-Value widget, you now have a choice to display the resulting number with 0, 1, 2, 3, or 4 decimal places.

  • Line charts show an axis label and units when a single counter is being plotted.

  • Transform option is available for Services integration data now in all time-series widgets for all metrics. For any services integration (Telegraf) counter or metric in time-series widgets (Line, Spline, Area, Stacked Area), you are given a choice of how you want to Transform the values. None (display value as-is), Sum, Delta, Cumulative, etc.

Downgrading to Basic Edition

Downgrade to Basic Edition fails with an error message if there is no available NetApp device configured that has successfully completed a poll in the last 7 days.

Collecting Kube-State-Metrics

The Kubernetes Data Collector now collects objects and counters from the kube-state-metrics plugin, greatly expanding the number and scope of metrics available for monitoring in Cloud Insights.


June 2019

Cloud Insights Editions

Cloud Insights is available in different Editions to fit your budget and business needs. Existing NetApp customers with an active NetApp support account can enjoy 7 days of data retention and access to NetApp data collectors with the free Basic Edition, or get increased data retention, access to all supported data collectors, expert technical support and more with Standard Edition. For more information on available features, see NetApp's Cloud Insights site.

New Infrastructure Data Collector: NetApp HCI

  • NetApp HCI Virtual Center has been added as an Infrastructure data collector. The HCI Virtual Center data collector collects NetApp HCI Host information and requires read-only privileges on all objects within the Virtual Center.

Note that the HCI data collector acquires from the HCI Virtual Center only. To collect data from the storage system, you must also configure the NetApp SolidFire data collector.


May 2019

New Service Data Collector: Kapacitor

  • Kapacitor has been added as a data collector for services.

Integration with Services via Telegraf

In addition to acquisition of data from infrastructure devices such as switches and storage, Cloud Insights now collects data from a variety of Operating Systems and Services, using Telegraf as its agent for collection of integration data. Telegraf is a plugin-driven agent that can be used to collect and report metrics. Input plugins are used to collect the desired information into the agent by accessing the system/OS directly, by calling third-party APIs, or by listening to configured streams.

Documentation for currently supported integrations can be found in the menu to the left under Reference and Support.

Storage Virtual Machine Assets

  • Storage Virtual Machines (SVMs) are available as assets in Cloud Insights. SVMs have their own Asset Landing Pages, and can be displayed and used in searches, queries, and filters. SVMs can also be used in dashboard widgets as well as associated with annotations.

Reduced Acquisition Unit System Requirements

  • The system CPU and memory requirements for the Acquisition Unit (AU) software have been reduced. The new requirements are:

Component

Old Requirement

New Requirement

CPU Cores

4

2

Memory

16 GB

8 GB

Additional Platforms Supported

Linux

Windows

CentOS 7.3 64-bit
CentOS 7.4 64-bit
CentOS 7.6 64-bit
Debian 9 64-bit
Red Hat Enterprise Linux 7.3 64-bit
Red Hat Enterprise Linux 7.4 64-bit
Red Hat Enterprise Linux 7.6 64-bit
Ubuntu Server 18.04 LTS

Microsoft Windows 10 64-bit
Microsoft Windows Server 2008 R2
Microsoft Windows Server 2019


April 2019

Filter Virtual Machines by Tags

When configuring the following data collectors, you can filter to include or exclude virtual machines from data collection according to their Tags or Labels.


March 2019

  • You can select recipients for email notifications when subscription-related events occur, such as upcoming trial expiration or subscribed account changes. You can choose recipients for these notifications from among following:

    • All Account Owners

    • All Administrators

    • Additional Email Addresses that you specify

Additional Dashboards

  • The following new AWS-focused dashboards have been added to the gallery and are available for import:

    • AWS Admin - Which EC2 are in high demand?

    • AWS EC2 Instance Performance by Region


February 2019

Collecting from AWS Child Accounts

  • Cloud Insights supports collection from AWS child accounts within a single data collector. Your AWS environment must be configured to allow Cloud Insights to collect from child accounts.

Data Collector Naming

  • Data Collector names can now include periods (.), hyphens (-), and spaces ( ) in addition to letters, numbers, and underscores. Names may not begin or end with a space, period, or hyphen.

Acquisition Unit for Windows


January 2019

"Owner" field is more readable

  • In Dashboard and Query lists, the data for the "Owner" field was previously an authorization ID string, instead of a user-friendly owner name. The "Owner" field now shows a simpler and more readable owner name.

Managed Unit Breakdown on Subscription Page

  • For each data collector listed on the Admin > Subscription page, you can now see a breakdown of Managed Unit (MU) counts for hosts and storage, as well as the total.


December 2018

Improvement of UI Load Time

  • The initial loading time for the Cloud Insights user interface (UI) has been significantly improved. Refresh time for the UI also benefits from this improvement in circumstances where metadata is loaded.

Bulk Edit Data Collectors

  • You can edit information for multiple data collectors at the same time. On the Observability > Collectors page, select the data collectors to modify by checking the box to the left of each and click the Bulk Actions button. Choose Edit and modify the necessary fields.

    The data collectors selected must be the same vendor and model, and reside on the same Acquisition Unit.

Support and Subscription pages are Available During Onboarding

  • During the onboarding workflow, you can navigate to the Help > Support and Admin > Subscription pages. Returning from those pages returns you to the onboarding workflow, providing you have not closed the browser tab.


November 2018

Subscribe through NetApp Sales or AWS Marketplace

  • Cloud Insights subscription and billing is now available directly through NetApp. This is in addition to the self-serve subscription available through AWS Marketplace. A new Contact Sales link is presented on the Admin > Subscription page. For customers whose environments have or are expected to have 1,000 or more Managed Units (MUs), it is recommended to contact NetApp sales via the Contact Sales link.

  • Text-type annotations can now include hyperlinks.

Onboarding Walkthrough

  • Cloud Insights now features an onboarding walkthrough for the first user (administrator or account owner) to log in to a new environment. The walkthrough takes you through installing an Acquisition Unit, configuring an initial data collector, and selecting one or more useful dashboards.

  • In addition to selecting dashboards during onboarding, you can import dashboards via Dashboards > Show All Dashboards and clicking +From Gallery.

Duplicating Dashboards

  • The ability to duplicate a dashboard has been added to the dashboard list page as a choice in the options menu for each dashboard, and on a dashboard's main page itself from the Save menu.

Cloud Central products menu

  • The menu allowing you to switch to other NetApp Cloud Central products has moved to the upper right corner of the screen.