PDFs

Configuring the ONTAP SVM Data Collector

Contributors netapp-alavoie Download PDF of this page

Cloud Secure uses data collectors to collect file and user access data from devices.

Before you begin

  • This data collector is supported on Data ONTAP 9.2 and later versions.

  • Only data type SVMs are supported. SVMs with infinite/flexgroup volumes are not supported

  • SVM has several sub-types. Of these, only default and sync_source are supported.

  • An Agent must be configured before you can configure data collectors.

  • Make sure that you have a properly configured User Directory Connector, otherwise events will show encoded user names and not the actual name of the user (as stored in Active Directory) in the “Activity Forensics” page.

  • For optimal performance, you should configure the FPolicy server to be on the same subnet as the storage system.

  • You must add an SVM using one of the following two methods:

    • By Using Cluster IP, SVM name, and Cluster Management Username and Password

      • SVM name must be exactly as is shown in ONTAP and is case-sensitive.

    • By Using SVM Vserver Management IP, Username, and Password

    • If you are not able or not willing to use the full Administrator Cluster/SVM Management Username and Password, you can create a custom user with lesser privileges as mentioned in the “A note about permissions” section below. This custom user can be created for either SVM or Cluster access.

      • o You can also use an AD user with a role that has at least the permissions of csrole as mentioned in “A note about permissions” section below. Also refer to the ONTAP documentation.

  • Ensure the correct applications are set for the SVM by executing the following command:

    clustershell::> security login show -vserver <vservername> -user-or-group-name <username>

Example output:
SVM Command Output Example

  • Ensure that the SVM has a CIFS server configured:
    clustershell::> vserver cifs show

    The system returns the Vserver name, CIFS server name and additional fields.

  • Set a password for the SVM vsadmin user. If using custom user or cluster admin user, skip this step.
    clustershell::> security login password -username vsadmin -vserver svmname

  • Unlock the SVM vsadmin user for external access. If using custom user or cluster admin user, skip this step.
    clustershell::> security login unlock -username vsadmin -vserver svmname

  • Ensure the firewall-policy of the data LIF is set to ‘mgmt’ (not ‘data’). Skip this step if using a dedicated management lif to add the SVM.
    clustershell::> network interface modify -lif <SVM_data_LIF_name> -firewall-policy mgmt

  • When a firewall is enabled, you must have an exception defined to allow TCP traffic for the port using the Data ONTAP Data Collector.

    See Agent requirements for configuration information. This applies to on-premise Agents and Agents installed in the Cloud.

  • When an Agent is installed in an AWS EC2 instance to monitor a Cloud ONTAP SVM, the Agent and Storage must be in the same VPC. If they are in separate VPCs, there must be a valid route between the VPC’s.

A Note About Permissions

Permissions when adding via Cluster Management IP:

If you cannot use the Cluster management administrator user to allow Cloud Secure to access the ONTAP SVM data collector, you can create a new user named “csuser” with the roles as shown in the commands below. Use the username “csuser” and password for “csuser” when configuring the Cloud Secure data collector to use Cluster Management IP.

To create the new user, log in to ONTAP with the Cluster management Administrator username/password, and execute the following commands on the ONTAP server:

security login role create -role csrole -cmddirname DEFAULT -access none
security login role create -role csrole -cmddirname "network interface" -access readonly
security login role create -role csrole -cmddirname version -access readonly
security login role create -role csrole -cmddirname volume -access readonly
security login role create -role csrole -cmddirname vserver -access readonly
security login role create -role csrole -cmddirname "vserver fpolicy" -access all
security login role create -role csrole -cmddirname "volume snapshot" -access all
security login create -user-or-group-name csuser -application ontapi -authmethod password -role csrole

Permissions when adding via Vserver Management IP:

If you cannot use the Cluster management administrator user to allow Cloud Secure to access the ONTAP SVM data collector, you can create a new user named “csuser” with the roles as shown in the commands below. Use the username “csuser” and password for “csuser” when configuring the Cloud Secure data collector to use Vserver Management IP.

To create the new user, log in to ONTAP with the Cluster management Administrator username/password, and execute the following commands on the ONTAP server. For ease, copy these commands to a text editor and replace the <vservername> with your Vserver name before and executing these commands on ONTAP:

security login role create -vserver <vservername> -role csrole -cmddirname DEFAULT -access none
security login role create -vserver <vservername> -role csrole -cmddirname "network interface" -access readonly
security login role create -vserver <vservername> -role csrole -cmddirname version -access readonly
security login role create -vserver <vservername> -role csrole -cmddirname volume -access readonly
security login role create -vserver <vservername> -role csrole -cmddirname vserver -access readonly
security login role create -vserver <vservername> -role csrole -cmddirname "vserver fpolicy" -access all
security login role create -vserver <vservername> -role csrole -cmddirname "volume snapshot" -access all
security login create -user-or-group-name csuser -application ontapi -authmethod password -role csrole -vserver <vservername>

Configure the data collector

Steps for Configuration

  1. Log in as Administrator or Account Owner to your Cloud Insights environment.

  2. Click Admin > Data Collectors > +Data Collectors

    The system displays the available Data Collectors.

  3. Hover over the NetApp SVM tile and click *+Monitor.

    The system displays the ONTAP SVM configuration page. Enter the required data for each field.

Configuration

Field

Description

Name

Unique name for the Data Collector

Agent

Select a configured agent from the list.

Connect via Management IP for:

Select either Cluster IP or SVM Management IP

Cluster / SVM Management IP Address

The IP address for the cluster or the SVM, depending on your selection above.

SVM Name

The Name of the SVM (this field is required when connecting via Cluster IP)

Username

User name to access the SVM/Cluster
When adding via Cluster IP the options are:
1. Cluster-admin
2. ‘csuser’
3. AD-user having similar role as csuser.
When adding via SVM IP the options are:
4. vsadmin
5. ‘csuser’
6. AD-username having similar role as csuser.

Password

Password for the above user name

Filter Shares/Volumes

Choose whether to include or exclude Shares / Volumes from event collection

Enter complete share names to exclude/include

Comma-separated list of shares to exclude or include (as appropriate) from event collection

Enter complete volume names to exclude/include

Comma-separated list of volumes to exclude or include (as appropriate) from event collection

Monitor Folder Access

When checked, enables events for folder access monitoring. Note that folder create/rename and delete will be monitored even without this option selected. Enabling this will increase the number of events monitored.
After you finish

  • In the Installed Data Collectors page, use the options menu on the right of each collector to edit the data collector. You can restart the data collector or edit data collector configuration attributes.

Troubleshooting

Known problems and their resolutions are described in the following table.

In the case of an error, click on the error in the Status column for detail about the error.

CS Data Collector Error

Problem: Resolution:

Error message: "Connection to the FPolicy server <IP> is broken. ( reason: "FPolicy server is removed from external engine." )"

SVM is unable to reach the Fpolicy Server.
1. Make sure there is route available from SVM to the Fpolicy Server/Agent machine IP. Login to the cluster/SVM and ping the Fpolicy Server IP address using the following command:
net ping -lif <data_lif> -destination <agent IP> -vserver <svmname>
2. In instances where the same SVM was added in two different Cloud Secure environments (tenants), the last one will always succeed. The second collector will configure fpolicy with its own IP address and kick out the first one. So the collector in the first one will stop receiving events and its "audit" service will enter into error state.
To prevent this, configure each SVM on a single environment.

Collector reports Error Message: “No local IP address found on the connector that can reach the data interfaces of the SVM”.

This is most likely due to a networking issue on the ONTAP side. Please follow these steps:
1. Ensure that there are no firewalls on the SVM data lif or the management lif which are blocking the connection from the SVM.
2. When adding an SVM via a cluster management IP, please ensure that the data lif and management lif of the SVM are pingable from the Agent VM. In case of issues, check the gateway, netmask and routes for the lif.

You can also try logging in to the cluster via ssh using the cluster management IP, and ping the Agent IP. Make sure that the agent IP is pingable:

network ping -vserver <vserver name> -destination <Agent IP> -lif <Lif Name>

If not pingable, make sure the network settings in ONTAP are correct, so that the Agent machine is pingable.

3. While adding the collector via SVM IP and vsadmin credentials, check if the SVM Lif has Data plus Mgmt role enabled. In this case ping to the SVM Lif will work, however SSH to the SVM Lif will not work.
If yes, create an SVM Mgmt Only Lif and try connecting via this SVM management only Lif.

Message: "Failed to determine ONTAP type for [hostname: <IP Address>. Reason: Connection error to Storage System <IP Address>: Host is unreachable (Host unreachable)"

1. Verify that the correct SVM IP Management address or Cluster Management IP has been provided.
2. SSH to the SVM or the Cluster to which you are intending to connect. Once you are connected ensure that the SVM or the Cluster name is correct.

Error Message: "Connector is in error state. Service.name: audit. Reason for failure: External fpolicy server terminated."

1. It is most likely that a firewall is blocking the necessary ports in the agent machine. Verify the port range 35000-55000/tcp is opened for the agent machine to connect from the SVM. Also ensure that there are no firewalls enabled from the ONTAP side blocking communication to the agent machine.
2. Type the following command in the Agent box and ensure that the port range is open.
sudo iptables-save | grep 3500*

3. Login to SVM, enter the following commands and check that no firewall is set to block the communication with ONTAP.
system services firewall show
system services firewall policy show

Check firewall commands on the ONTAP side.

4. SSH to the SVM/Cluster which you want to monitor. Ping the Agent box from the SVM management lif (with CIFS, NFS protocols support) and ensure that ping is working:

network ping -vserver <vserver name> -destination <Agent IP> -lif <Lif Name>

If not pingable, make sure the network settings in ONTAP are correct, so that the Agent machine is pingable.

No events seen in activity page.

1. Check if ONTAP collector is in “RUNNING” state. If yes, then ensure that some cifs events are being generated on the cifs client VMs by opening some files.

2. If no activities are seen, please login to the SVM and enter the following command.
<SVM>event log show -source fpolicy
Please ensure that there are no errors related to fpolicy.

3. If no activities are seen, please login to the SVM. Enter the following command
<SVM>fpolicy show
Please check if the fpolicy policy named with prefix “metadata_service” has been set and status is “on”. If not set, then most likely the Agent is unable to execute the commands in the SVM. Please ensure all the prerequisites as described in the beginning of the page have been followed.

SVM Data Collector is in error state and Errror message is “Agent failed to connect to the collector”

1. Most likely the Agent is overloaded and is unable to connect to the Data Source collectors.
2. Check how many Data Source collectors are connected to the Agent.
3. Also check the data flow rate in the “All Activity” page in the UI.
4. If the number of activities per second is significantly high, install another Agent and move some of the Data Source Collectors to the new Agent.

SVM Data Collector shows error message as "fpolicy.server.connectError: Node failed to establish a connection with the FPolicy server "12.195.15.146" ( reason: "Select Timed out")"

Firewall is enabled in SVM/Cluster. So fpolicy engine is unable to connect to fpolicy server.
CLIs in ONTAP which can be used to get more information are:

event log show -source fpolicy which shows the error
event log show -source fpolicy -fields event,action,description which shows more details.

Check firewall commands on the ONTAP side.

Error Message: “Connector is in error state. Service name:audit. Reason for failure: No valid data interface (role: data,data protocols: NFS or CIFS or both, status: up) found on the SVM.”

Ensure there is an operational interface (having role as data and data protocol as CIFS/NFS.

The data collector goes into Error state and then goes into RUNNING state after some time, then back to Error again. This cycle repeats.

This typically happens in the following scenario:
1. There are multiple data collectors added.
2. The data collectors which show this kind of behavior will have 1 SVM added to these data collectors. Meaning 2 or more data collectors are connected to 1 SVM.
3. Ensure 1 data collector connects to only 1 SVM.
4. Delete the other data collectors which are connected to the same SVM.

If you are still experiencing problems, reach out to the support links mentioned in the Help > Support page.

CONTRIBUTE
Edit on GitHub Request doc changes Contributor's Guide