Skip to main content
AI Data Engine

Monitor deployment health and investigate issues in AI Data Engine

Contributors netapp-dbagwell
PDFs

The AI Data Engine (AIDE) health monitor helps you identify unhealthy workloads and resource constraints that could interrupt scanning. Use the health details to assess the impact of an issue and gather information for troubleshooting.

Monitor deployment health and investigate issues

Steps

  1. In NetApp Console, go to AI data hub > AI Data Engine.

  2. Select Configuration, then select Health monitor.

  3. Check System stability under Infrastructure Status to see whether any pods are unhealthy.

    An unhealthy pod can't run its workload as expected. Unhealthy statuses include Pending, Failed, Evicted, Unknown, OOMKilled, and CrashLoopBackOff.

  4. Identify resource constraints that might affect your deployment:

    • For a single-node deployment, review Node health. If the node indicates a health issue, check the CPU, memory, and disk utilization cards for signs of resource pressure.

    • For a multi-node deployment, review Node health to identify nodes with a Critical, Warning, or Offline status. Compare available resource utilization metrics for signs of resource pressure. For offline nodes, investigate node availability.

  5. For a multi-node deployment, select View details on a utilization card to investigate the resource:

    • CPU utilization: Compare current usage with the warning and critical thresholds.

    • Memory utilization: Review current usage, out-of-memory (OOM) risk, and the eviction threshold.

    • Disk usage: Review available space, risk, and the stop threshold.

    Pay particular attention to memory and disk warnings because these conditions can interrupt scanning. Follow any recommended actions displayed in the details.

  6. To identify affected workloads, use the Pod and Status filters under Pod health to narrow the list.

    Review each affected pod's workload, status, and resource usage. Use this information to distinguish resource pressure from pods that are failing, repeatedly restarting, or waiting to run.

  7. Review Problematic workloads to identify workloads that have stopped or are frequently restarting. Note the reported times to help correlate these events with resource usage.

  8. Review resource utilization trends to see whether resource pressure is temporary or sustained:

    • For a single-node deployment, review Resource utilization distribution for the relevant time range.

    • For a multi-node deployment, review Top 20 Nodes (Utilization) for the relevant time range. Compare utilization across nodes, and inspect CPU, memory, and disk usage for affected nodes to identify brief spikes or sustained high utilization.

  9. If you need help resolving an issue, select Download in Pod health to download the pod information. Gather the relevant details from System information, including the hostname, operating system, kernel, Console version, Kubernetes version, and Console account ID, to share with your administrator or support team.

Result

After you take corrective action, return to Health monitor and review System stability, Pod health, Node health, and resource utilization to verify that the reported conditions have cleared. If scanning was interrupted, go to Data sources to verify that it has resumed. Refer to View scan status and details.