Configuring an Agent to Collect Data Edit on GitHub Request doc changes

Contributors netapp-alavoie dgracenetapp

Cloud Insights uses Telegraf as its agent for collection of integration data. Telegraf is a plugin-driven server agent that can be used to collect and report metrics, events, and logs. Input plugins are used to collect the desired information into the agent by accessing the system/OS directly, by calling third-party APIs, or by listening to configured streams (i.e. Kafka, statsD, etc). Output plugins are used to send the collected metrics, events, and logs from the agent to Cloud Insights.

Installing an Agent

This topic provides instructions for installing the Telegraf agent on the following platforms:

To install an agent, regardless of the platform you are using, you must first do the following:

  1. Log into the host you will use for your agent.

  2. Log in to your Cloud Insights site and go to Admin > Data Collectors.

  3. Click on +Data Collector and choose a data collector to install. There are several types of data collectors:

    • Host (Windows, Linux, macOS, etc.)

    • Service (integration with a wide variety of agent-collected plugins). Agents are configured and run as a service for RHEL/CentOS, Ubuntu/Debian, macOS, and Windows. For Kubernetes platforms, the agent in configured as a DaemonSet.

    • Infrastructure (collects from storage, switch, cloud platform, etc.). Infrastructure collection is done with an Acquisition Unit instead of an Agent.

  4. Choose the appropriate platform for your host (Windows, Linux, macOS, etc.)

  5. Follow the remaining steps for each platform below

Once you have installed an agent on a host, you do not need to install an agent again on that host.
Once you have installed an agent on a server/VM, Cloud Insights collects metrics from that system in addition to collecting from any data collectors you configure. These metrics are gathered as "Node" metrics.

Windows

Windows Agent Install

Pre-requisites:
  • PowerShell must be installed

Steps to install agent on Windows:
  1. If prompted, select Windows from the Platform drop-down.

  2. Choose an Agent Access Key.

  3. Copy the command block from the agent installation dialog. You can click the clipboard icon to quickly copy the command to the clipboard.

  4. Open a PowerShell window

  5. Paste the command into the PowerShell window and press Enter.

  6. The command will download the appropriate agent installer, install it, and set a default configuration. When finished, it will restart the agent service. The command has a unique key and is valid for 24 hours.

  7. Click Finish or Continue

After the agent is installed, you can use the following commands to start/stop the service:

Start-Service telegraf
Stop-Service telegraf

Uninstalling the Agent

To uninstall the agent on Windows, do the following in a PowerShell window:

  1. Stop and delete the Telegraf service:

    Stop-Service telegraf
    sc.exe delete telegraf
  2. Delete the C:\Program Files\telegraf folder to remove the binary, logs, and configuration files

Upgrading the Agent

To upgrade the telegraf agent, do the following:

  1. Stop and delete the telegraf service:

    Stop-Service telegraf
    sc.exe delete telegraf
  2. Delete the SYSTEM\CurrentControlSet\Services\EventLog\Application\telegraf key from the registry

  3. Delete C:\Program Files\telegraf\telegraf.conf

  4. Delete C:\Program Files\telegraf\telegraf.exe

  5. Install the new agent.

RHEL and CentOS

RHEL/CentOS Agent Install

Pre-requisites:
  • The following commands must be available: curl, sudo, ping, and dmidecode

Steps to install agent on RHEL/CentOS:
  1. If prompted, select RHEL/CentOS from the Platform drop-down.

  2. Choose an Agent Access Key.

  3. Copy the command block from the agent installation dialog. You can click the clipboard icon to quickly copy the command to the clipboard.

  4. Open a Bash window

  5. Paste the command into the Bash window and press Enter.

  6. The command will download the appropriate agent installer, install it, and set a default configuration. When finished, it will restart the agent service. The command has a unique key and is valid for 24 hours.

  7. Click Finish or Continue

After the agent is installed, you can use the following commands to start/stop the service:

If your operating system is using systemd (CentOS 7+ and RHEL 7+):

sudo systemctl start telegraf
sudo systemctl stop telegraf

If your operating system is not using systemd (CentOS 7+ and RHEL 7+):

sudo service telegraf start
sudo service telegraf stop

Uninstalling the Agent

To uninstall the agent on RHEL/CentOS, in a Bash terminal, do the following:

  1. Stop the Telegraf service:

    systemctl stop telegraf (If your operating system is using systemd (CentOS 7+ and RHEL 7+)
    /etc/init.d/telegraf stop (for systems without systemd support)
  2. Remove the Telegraf agent:

    yum remove telegraf
  3. Remove any configuration or log files that may be left behind:

    rm -rf /etc/telegraf*
    rm -rf /var/log/telegraf*

Upgrading the Agent

To upgrade the telegraf agent, do the following:

  1. Stop the telegraf service:

    systemctl stop telegraf (If your operating system is using systemd (CentOS 7+ and RHEL 7+)
    /etc/init.d/telegraf stop (for systems without systemd support)
  2. Remove the previous telegraf agent:

    yum remove telegraf
  3. Install the new agent.

Ubuntu and Debian

Ubuntu/Debian Agent Install

Pre-requisites:
  • The following commands must be available: curl, sudo, ping, and dmidecode

Steps to install agent on Debian or Ubuntu:
  1. If prompted, select Ubuntu/Debian from the Platform drop-down.

  2. Choose an Agent Access Key.

  3. Copy the command block from the agent installation dialog. You can click the clipboard icon to quickly copy the command to the clipboard.

  4. Open a Bash window

  5. Paste the command into the Bash window and press Enter.

  6. The command will download the appropriate agent installer, install it, and set a default configuration. When finished, it will restart the agent service. The command has a unique key and is valid for 24 hours.

  7. Click Finish or Continue

After the agent is installed, you can use the following commands to start/stop the service:

If your operating system is using systemd:

sudo systemctl start telegraf
sudo systemctl stop telegraf

If your operating system is not using systemd:

sudo service telegraf start
sudo service telegraf stop

Uninstalling the Agent

To uninstall the agent on Ubuntu/Debian, in a Bash terminal, run the following:

  1. Stop the Telegraf service:

    systemctl stop telegraf (If your operating system is using systemd)
    /etc/init.d/telegraf stop (for systems without systemd support)
  2. Remove the Telegraf agent:

    dpkg -r telegraf
  3. Remove any configuration or log files that may be left behind:

    rm -rf /etc/telegraf*
    rm -rf /var/log/telegraf*

Upgrading the Agent

To upgrade the telegraf agent, do the following:

  1. Stop the telegraf service:

    systemctl stop telegraf (If your operating system is using systemd)
    /etc/init.d/telegraf stop (for systems without systemd support)
  2. Remove the previous telegraf agent:

    dpkg -r telegraf
  3. Install the new agent.

macOS

MacOS Agent Install

Pre-requisites:
  • The "curl" command must be available

Steps to install agent on macOS:
  1. If prompted, select macOS from the Platform drop-down.

  2. Choose an Agent Access Key.

  3. Copy the command block from the agent installation dialog. You can click the clipboard icon to quickly copy the command to the clipboard.

  4. Open a Bash window

  5. Paste the command into the Bash window and press Enter.

  6. The command will download the appropriate agent installer, install it, and set a default configuration. When finished, it will restart the agent service. The command has a unique key and is valid for 24 hours.

  7. If you previously installed a Telegraf agent using Homebrew, you will be prompted to uninstall it. Once the previously installed Telegraf agent is uninstalled, re-run the command in step 5 above.

  8. Click Finish or Continue

After the agent is installed, you can use the following commands to start/stop the service:

sudo launchctl start telegraf
sudo launchctl stop telegraf

Uninstalling the Agent

To uninstall the agent on macOS, in a Bash terminal, run the following:

  1. Stop the Telegraf service:

    sudo launchctl stop telegraf
  1. Uninstall the telegraf agent:

    cp /Applications/telegraf.app/scripts/uninstall /tmp
    sudo /tmp/uninstall
  2. Remove any configuration or log files that may be left behind:

    rm -rf /usr/local/etc/telegraf*
    rm -rf /usr/local/var/log/telegraf.*

Upgrading the Agent

To upgrade the telegraf agent, do the following:

  1. Stop the telegraf service:

    sudo launchctl stop telegraf
  2. Uninstall the previous telegraf agent:

    cp /Applications/telegraf.app/scripts/uninstall /tmp
    sudo /tmp/uninstall
  3. Install the new agent.

Kubernetes

Kubernetes Agent Install

Pre-requisites:
  • The following commands must be available: curl and sudo

Steps to install agent on Kubernetes:
  1. If prompted, select Kubernetes from the Platform drop-down.

  2. Choose an Agent Access Key.

  3. Copy the command block from the agent installation dialog. You can click the clipboard icon to quickly copy the command to the clipboard.

  4. Open a Bash window

  5. Paste the command into the Bash window and press Enter.

  6. The command will download the appropriate agent installer, install it, and set a default configuration. When finished, it will restart the agent service. The command has a unique key and is valid for 24 hours.

  7. Click Finish or Continue

After the agent is installed, generate the Telegraf DaemonSet YAML and ReplicaSet YAML:

kubectl --namespace monitoring get ds telegraf-ds -o yaml > /tmp/telegraf-ds.yaml
kubectl --namespace monitoring get rs telegraf-rs -o yaml > /tmp/telegraf-rs.yaml

You can use the following commands to stop and start the Telegraf service:

kubectl --namespace monitoring delete ds telegraf-ds
kubectl --namespace monitoring delete ds telegraf-rs
kubectl --namespace monitoring apply -f /tmp/telegraf-ds.yaml
kubectl --namespace monitoring apply -f /tmp/telegraf-rs.yaml

Configuring the Agent to Collect Data from Kubernetes

For Kubernetes environments, Cloud Insights deploys the Telegraf agent as a DaemonSet and a ReplicaSet. The pods in which the agents run need to have access to the following:

  • hostPath

  • configMap

  • secrets

These Kubernetes objects are automatically created as part of the Kubernetes agent install command provided in the Cloud Insights UI. Some variants of Kubernetes, such as OpenShift, implement an added level of security that may block access to these components. The SecurityContextConstraint is not created as part of the Kubernetes agent install command provided in the Cloud Insights UI, and must be created manually. Once created, restart the Telegraf pod(s).

    apiVersion: v1
    kind: SecurityContextConstraints
    metadata:
      name: telegraf-hostaccess
      creationTimestamp:
      annotations:
        kubernetes.io/description: telegraf-hostaccess allows hostpath volume mounts for restricted SAs.
      labels:
        app: ci-telegraf
    priority: 10
    allowPrivilegedContainer: false
    defaultAddCapabilities: []
    requiredDropCapabilities: []
    allowedCapabilities: []
    allowedFlexVolumes: []
    allowHostDirVolumePlugin: true
    volumes:
    - hostPath
    - configMap
    - secret
    allowHostNetwork: false
    allowHostPorts: false
    allowHostPID: false
    allowHostIPC: false
    seLinuxContext:
      type: MustRunAs
    runAsUser:
      type: RunAsAny
    supplementalGroups:
      type: RunAsAny
    fsGroup:
      type: RunAsAny
    readOnlyRootFilesystem: false
    users:
    - system:serviceaccount:monitoring:telegraf-user
    groups: []

Installing the kube-state-metrics server

When you install the kube-state-metrics server you can enable colletction of the following Kubernetes objects: StatefulSet, DaemonSet, Deployment, PV, PVC, ReplicaSet, Service, Namespace, Secret, ConfigMap, Pod Volume, and Ingress.

Use the following steps to install the kube-state-metrics server:

Steps
  1. Create a temporary folder (for example, /tmp/kube-state-yaml-files/) and copy the .yaml files from https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard to this folder.

  2. Run the following command to apply the .yaml files needed for installing kube-state-metrics:

    kubectl apply -f /tmp/kube-state-yaml-files/

Uninstalling the Agent

To uninstall the agent on Kubernetes, do the following:

  1. If the monitoring namespace is being used solely for Telegraf:

    kubectl delete ns monitoring

If the monitoring namespace is being used for other purposes in addition to Telegraf:

  1. Stop and delete the Telegraf service:

    kubectl --namespace monitoring delete ds telegraf-ds
    kubectl --namespace monitoring delete rs telegraf-rs
  2. Delete the Telegraf ConfigMap and ServiceAccount:

    kubectl --namespace monitoring delete cm telegraf-conf
    kubectl --namespace monitoring delete cm telegraf-conf-rs
    kubectl --namespace monitoring delete sa telegraf-user
  3. Delete the Telegraf ClusterRole and ClusterRolebinding:

    kubectl --namespace monitoring delete clusterrole endpoint-access
    kubectl --namespace monitoring delete clusterrolebinding endpoint-access

Upgrading the Agent

To upgrade the telegraf agent, do the following:

  1. Remove the current the telegraf deployments:

    kubectl --namespace monitoring delete ds telegraf-ds
    kubectl --namespace monitoring delete rs telegraf-rs
  2. Back up the existing configurations:

    kubectl --namespace monitoring get cm telegraf-conf -o yaml > /tmp/telegraf-conf.yaml
    kubectl --namespace monitoring get cm telegraf-conf-rs -o yaml > /tmp/telegraf-conf-rs.yaml
  3. Install the new agent.

  4. Re-apply the configurations:

    kubectl --namespace monitoring apply -f /tmp/telegraf-conf.yaml --force
    kubectl --namespace monitoring apply -f /tmp/telegraf-conf-rs.yaml --force
  5. Restart all telegraf pods. Run the following command for each telegraf pod:

    kubectl --namespace monitoring delete pod <Telegraf_pod>

Troubleshooting Agent Installation

Some things to try if you encounter problems setting up an agent:

Problem: Try this:

I already installed an agent using Cloud Insights

If you have already installed an agent on your host/VM, you do not need to install the agent again. In this case, simply choose the appropriate Platform and Key in the Agent Installation screen, and click on Continue or Finish.

I already have an agent installed but not by using the Cloud Insights installer

Remove the previous agent and run the Cloud Insights Agent installation, to ensure proper default configuration file settings. When complete, click on Continue or Finish.

Additional information may be found from the Support page or in the Data Collector Support Matrix.