Install BlueXP classification on a host with no internet access

Contributors netapp-tonacki netapp-bcammett

Complete a few steps to install BlueXP classification on a Linux host in an on-premises site that doesn’t have internet access - also known as "private mode". This type of installation is perfect for your secure sites.

Learn about the different deployment modes for the BlueXP Connector and BlueXP classification.

Note that you can also deploy BlueXP classification in an on-premises site that has internet access.

The BlueXP classification installation script starts by checking if the system and environment meet the required prerequisites. If the prerequisites are all met, then the installation starts. If you would like to verify the prerequisites independently of running the BlueXP classification installation, there is a separate software package you can download that only tests for the prerequisites. See how to check if your Linux host is ready to install BlueXP classification.

Supported data sources

When installed private mode (sometimes called an "offline" or "dark" site), BlueXP classification can only scan data from data sources that are also local to the on-premises site. At this time, BlueXP classification can scan the following local data sources:

  • On-premises ONTAP systems

  • Database schemas

  • SharePoint On-Premises accounts (SharePoint Server)

  • Non-NetApp NFS or CIFS file shares

  • Object Storage that uses the Simple Storage Service (S3) protocol

There is no support currently for scanning Cloud Volumes ONTAP, Azure NetApp Files, FSx for ONTAP, AWS S3, or Google Drive, OneDrive, or SharePoint Online accounts when BlueXP classification is deployed in private mode.

Limitations

Most BlueXP classification features work when it is deployed in a site with no internet access. However, certain features that require internet access are not supported, for example:

  • Managing Microsoft Azure Information Protection (AIP) labels

  • Sending email alerts to BlueXP users when certain critical Policies return results

  • Setting BlueXP roles for different users (for example, Account Admin or Compliance Viewer)

  • Copying and synchronizing source files using BlueXP copy and sync

  • Receiving user feedback

  • Automated software upgrades from BlueXP

    Both the BlueXP Connector and BlueXP classification will require periodic manual upgrades to enable new features. You can see the BlueXP classification version at the bottom of the BlueXP classification UI pages. Check the BlueXP classification Release Notes to see the new features in each release and whether you want those features. Then you can follow the steps to upgrade the BlueXP Connector and upgrade your BlueXP classification software.

Quick start

Get started quickly by following these steps, or scroll down to the remaining sections for full details.

One Install the BlueXP Connector

If you don’t already have a Connector installed in private mode, deploy the Connector on a Linux host now.

Two Review BlueXP classification prerequisites

Ensure that your Linux system meets the host requirements, that it has all required software installed, and that your offline environment meets the required permissions and connectivity.

Three Download and deploy BlueXP classification

Download the BlueXP classification software from the NetApp Support Site and copy the installer file to the Linux host you plan to use. Then launch the installation wizard and follow the prompts to deploy the BlueXP classification instance.

Four Subscribe to the BlueXP classification service

The first 1 TB of data that BlueXP classification scans in BlueXP is free for 30 days. A BYOL license from NetApp is required to continue scanning data after that point.

Install the BlueXP Connector

If you don’t already have a BlueXP Connector installed in private mode, deploy the Connector on a Linux host in your offline site.

Prepare the Linux host system

BlueXP classification software must run on a host that meets specific operating system requirements, RAM requirements, software requirements, and so on.

  • BlueXP classification is not supported on a host that is shared with other applications - the host must be a dedicated host.

  • When building the host system in your premises, you can choose among three system sizes depending on the size of the data set that you plan to have BlueXP classification scan.

    System size CPU RAM (swap memory must be disabled) Disk

    Large

    16 CPUs

    64 GB RAM

    500 GiB SSD on /, or
    - 100 GiB available on /opt
    - 395 GiB available on /var/lib/docker
    - 5 GiB on /tmp

    Medium

    8 CPUs

    32 GB RAM

    200 GiB SSD on /, or
    - 50 GiB available on /opt
    - 145 GiB available on /var/lib/docker
    - 5 GiB on /tmp

    Small

    8 CPUs

    16 GB RAM

    100 GiB SSD on /, or
    - 50 GiB available on /opt
    - 45 GiB available on /var/lib/docker
    - 5 GiB on /tmp

    Note that there are limitations when using these smaller systems. See Using a smaller instance type for details.

  • When deploying a compute instance in the cloud for your BlueXP classification installation, we recommend a system that meets the "Large" system requirements above:

  • UNIX folder permissions: The following minimum UNIX permissions are required:

    Folder Minimum Permissions

    /tmp

    rwxrwxrwt

    /opt

    rwxr-xr-x

    /var/lib/docker

    rwx------

    /user/lib/systemd/system

    rwxr-xr-x

  • Operating system:

    • The following operating systems require using the Docker container engine:

      • Red Hat Enterprise Linux version 7.8 and 7.9 - the Linux kernel version must be 4.0 or greater

      • CentOS version 7.8 and 7.9 - the Linux kernel version must be 4.0 or greater

      • Ubuntu 22.04 (requires BlueXP classification version 1.23 or greater)

  • Red Hat Subscription Management: The host must be registered with Red Hat Subscription Management. If it’s not registered, the system can’t access repositories to update required 3rd-party software during installation.

  • Additional software: You must install the following software on the host before you install BlueXP classification:

  • Firewalld considerations: If you are planning to use firewalld, we recommend that you enable it before installing BlueXP classification. Run the following commands to configure firewalld so that it is compatible with BlueXP classification:

    firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=8080/tcp
firewall-cmd --permanent --add-port=443/tcp
firewall-cmd --reload

    Note that you must restart Docker whenever you enable or update firewalld settings.

Tip The IP address of the BlueXP classification host system can’t be changed after installation.

Verify BlueXP and BlueXP classification prerequisites

Review the following prerequisites to make sure that you have a supported configuration before you deploy BlueXP classification.

  • Ensure that the Connector has permissions to deploy resources and create security groups for the BlueXP classification instance. You can find the latest BlueXP permissions in the policies provided by NetApp.

  • Ensure that you can keep BlueXP classification running. The BlueXP classification instance needs to stay on to continuously scan your data.

  • Ensure web browser connectivity to BlueXP classification. After BlueXP classification is enabled, ensure that users access the BlueXP interface from a host that has a connection to the BlueXP classification instance.

    The BlueXP classification instance uses a private IP address to ensure that the indexed data isn’t accessible to others. As a result, the web browser that you use to access BlueXP must have a connection to that private IP address. That connection can come from a host that’s inside the same network as the BlueXP classification instance.

Verify that all required ports are enabled

You must ensure that all required ports are open for communication between the Connector, BlueXP classification, Active Directory, and your data sources.

Connection Type Ports Description

Connector <> BlueXP classification

8080 (TCP), 6000 (TCP), 443 (TCP), and 80

The security group for the Connector must allow inbound and outbound traffic over ports 6000 and 443 to and from the BlueXP classification instance.

Make sure port 8080 is open so you can see the installation progress in BlueXP.

Connector <> ONTAP cluster (NAS)

443 (TCP)

BlueXP discovers ONTAP clusters using HTTPS. If you use custom firewall policies, they must meet the following requirements:

  • The Connector host must allow outbound HTTPS access through port 443. If the Connector is in the cloud, all outbound communication is allowed by the predefined security group.

  • The ONTAP cluster must allow inbound HTTPS access through port 443. The default "mgmt" firewall policy allows inbound HTTPS access from all IP addresses. If you modified this default policy, or if you created your own firewall policy, you must associate the HTTPS protocol with that policy and enable access from the Connector host.

BlueXP classification <> ONTAP cluster

  • For NFS - 111 (TCP\UDP) and 2049 (TCP\UDP)

  • For CIFS - 139 (TCP\UDP) and 445 (TCP\UDP)

BlueXP classification needs a network connection to each Cloud Volumes ONTAP subnet or on-prem ONTAP system. Security groups for Cloud Volumes ONTAP must allow inbound connections from the BlueXP classification instance.

Make sure these ports are open to the BlueXP classification instance:

  • For NFS - 111 and 2049

  • For CIFS - 139 and 445

NFS volume export policies must allow access from the BlueXP classification instance.

BlueXP classification <> Active Directory

389 (TCP & UDP), 636 (TCP), 3268 (TCP), and 3269 (TCP)

You must have an Active Directory already set up for the users in your company. Additionally, BlueXP classification needs Active Directory credentials to scan CIFS volumes.

You must have the information for the Active Directory:

  • DNS Server IP Address, or multiple IP Addresses

  • User Name and Password for the server

  • Domain Name (Active Directory Name)

  • Whether you are using secure LDAP (LDAPS) or not

  • LDAP Server Port (typically 389 for LDAP, and 636 for secure LDAP)

If you are using multiple BlueXP classification hosts to provide additional processing power to scan your data sources, you’ll need to enable additional ports/protocols. See the additional port requirements.

Install BlueXP classification on the on-premises Linux host

For typical configurations you’ll install the software on a single host system. See those steps here.

A diagram showing the location of the data sources you can scan when using a single BlueXP classification instance deployed on-prem without internet access.

For very large configurations where you’ll be scanning petabytes of data, you can include multiple hosts to provide additional processing power. See those steps here.

A diagram showing the location of the data sources you can scan when using multiple BlueXP classification instances deployed on-prem without internet access.

Single-host installation for typical configurations

Follow these steps when installing BlueXP classification software on a single on-premises host in an offline environment.

Note that all installation activities are logged when installing BlueXP classification. If you run into any issues during installation, you can view the contents of the installation audit log. It is written to /opt/netapp/install_logs/. See more details here.

What you’ll need

  • Verify that your Linux system meets the host requirements.

  • Verify that you have installed the two prerequisite software packages (Docker Engine, and Python 3).

  • Make sure you have root privileges on the Linux system.

  • Verify that your offline environment meets the required permissions and connectivity.

Steps

  1. On an internet-configured system, download the BlueXP classification software from the NetApp Support Site. The file you should select is named DataSense-offline-bundle-<version>.tar.gz.

  2. Copy the installer bundle to the Linux host you plan to use in private mode.

  3. Unzip the installer bundle on the host machine, for example:

    tar -xzf DataSense-offline-bundle-v1.25.0.tar.gz

    This extracts required software and the actual installation file cc_onprem_installer.tar.gz.

  4. Unzip the installation file on the host machine, for example:

    tar -xzf cc_onprem_installer.tar.gz

  5. Launch BlueXP and select Governance > Classification.

  6. Click Activate Data Sense.

    A screenshot of selecting the button to activate BlueXP classification.

  7. Click Deploy to start the on-prem installation.

    A screenshot of selecting the button to deploy BlueXP classification on premises.

  8. The Deploy Data Sense On Premises dialog is displayed. Copy the provided command (for example: sudo ./install.sh -a 12345 -c 27AG75 -t 2198qq --darksite) and paste it in a text file so you can use it later. Then click Close to dismiss the dialog.

  9. On the host machine, enter the command you copied and then follow a series of prompts, or you can provide the full command including all required parameters as command line arguments.

    Note that the installer performs a pre-check to make sure your system and networking requirements are in place for a successful installation.

    Enter parameters as prompted: Enter the full command:

    1. Paste the information you copied from step 8:
      sudo ./install.sh -a <account_id> -c <client_id> -t <user_token> --darksite

    2. Enter the IP address or host name of the BlueXP classification host machine so it can be accessed by the Connector system.

    3. Enter the IP address or host name of the BlueXP Connector host machine so it can be accessed by the BlueXP classification system.

    Alternatively, you can create the whole command in advance, providing the necessary host parameters:
    sudo ./install.sh -a <account_id> -c <client_id> -t <user_token> --host <ds_host> --manager-host <cm_host> --no-proxy --darksite

    Variable values:

    • account_id = NetApp Account ID

    • client_id = Connector Client ID (add the suffix "clients" to the client ID if it not already there)

    • user_token = JWT user access token

    • ds_host = IP address or host name of the BlueXP classification system.

    • cm_host = IP address or host name of the BlueXP Connector system.

Result

The BlueXP classification installer installs packages, registers the installation, and installs BlueXP classification. Installation can take 10 to 20 minutes.

If there is connectivity over port 8080 between the host machine and the Connector instance, you’ll see the installation progress in the BlueXP classification tab in BlueXP.

What’s Next

From the Configuration page you can select the local on-prem ONTAP clusters and databases that you want to scan.

You can also set up BYOL licensing for BlueXP classification from the BlueXP digital wallet page at this time. You will not be charged until your 30-day free trial ends.

Multi-host installation for large configurations

For very large configurations where you’ll be scanning petabytes of data, you can include multiple hosts to provide additional processing power. When using multiple host systems, the primary system is called the Manager node and the additional systems that provide extra processing power are called Scanner nodes.

Follow these steps when installing BlueXP classification software on multiple on-premises hosts in an offline environment.

What you’ll need

  • Verify that all your Linux systems for the Manager and Scanner nodes meet the host requirements.

  • Verify that you have installed the two prerequisite software packages (Docker Engine, and Python 3).

  • Make sure you have root privileges on the Linux systems.

  • Verify that your offline environment meets the required permissions and connectivity.

  • You must have the IP addresses of the scanner node hosts that you plan to use.

  • The following ports and protocols must be enabled on all hosts:

    Port Protocols Description

    2377

    TCP

    Cluster management communications

    7946

    TCP, UDP

    Inter-node communication

    4789

    UDP

    Overlay network traffic

    50

    ESP

    Encrypted IPsec overlay network (ESP) traffic

    111

    TCP, UDP

    NFS Server for sharing files between the hosts (needed from each scanner node to manager node)

    2049

    TCP, UDP

    NFS Server for sharing files between the hosts (needed from each scanner node to manager node)
Steps

  1. Follow steps 1 through 8 from the Single-host installation on the manager node.

  2. As shown in step 9, when prompted by the installer, you can enter the required values in a series of prompts, or you can provide the required parameters as command line arguments to the installer.

    In addition to the variables available for a single-host installation, a new option -n <node_ip> is used to specify the IP addresses of the scanner nodes. Multiple node IPs are separated by a comma.

    For example, this command adds 3 scanner nodes:
    sudo ./install.sh -a <account_id> -c <client_id> -t <user_token> --host <ds_host> --manager-host <cm_host> -n <node_ip1>,<node_ip2>,<node_ip3> --no-proxy --darksite

  3. Before the manager node installation completes, a dialog displays the installation command needed for the scanner nodes. Copy the command (for example: sudo ./node_install.sh -m 10.11.12.13 -t ABCDEF-1-3u69m1-1s35212) and save it in a text file.

  4. On each scanner node host:

    1. Copy the Data Sense installer file (cc_onprem_installer.tar.gz) to the host machine.

    2. Unzip the installer file.

    3. Paste and run the command that you copied in step 3.

      When the installation finishes on all scanner nodes and they have been joined to the manager node, the manager node installation finishes as well.

Result

The BlueXP classification installer finishes installing packages, and registers the installation. Installation can take 15 to 25 minutes.

What’s Next

From the Configuration page you can select the local on-prem ONTAP clusters and local databases that you want to scan.

You can also set up BYOL licensing for BlueXP classification from the BlueXP digital wallet page at this time. You will not be charged until your 30-day free trial ends.

Upgrade BlueXP classification software

Since BlueXP classification software is updated with new features on a regular basis, you should get into a routine to check for new versions periodically to make sure you’re using the newest software and features. You’ll need to upgrade BlueXP classification software manually because there’s no internet connectivity to perform the upgrade automatically.

Before you begin

  • We recommend that your BlueXP Connector software is upgraded to the newest available version. See the Connector upgrade steps.

  • Starting with BlueXP classification version 1.24 you can perform upgrades to any future version of software.

    If your BlueXP classification software is running a version prior to 1.24, you can upgrade only one major version at a time. For example, if you have version 1.21.x installed, you can upgrade only to 1.22.x. If you are a few major versions behind, you’ll need to upgrade the software multiple times.

Steps

  1. On an internet-configured system, download the BlueXP classification software from the NetApp Support Site. The file you should select is named DataSense-offline-bundle-<version>.tar.gz.

  2. Copy the software bundle to the Linux host where BlueXP classification is installed in the dark site.

  3. Unzip the software bundle on the host machine, for example:

    tar -xvf DataSense-offline-bundle-v1.25.0.tar.gz

    This extracts the installation file cc_onprem_installer.tar.gz.

  4. Unzip the installation file on the host machine, for example:

    tar -xzf cc_onprem_installer.tar.gz

    This extracts the upgrade script start_darksite_upgrade.sh and any required third-party software.

  5. Run the upgrade script on the host machine, for example:

    start_darksite_upgrade.sh
Result

The BlueXP classification software is upgraded on your host. The update can take 5 to 10 minutes.

Note that no upgrade is required on scanner nodes if you have deployed BlueXP classification on multiple hosts systems for scanning very large configurations.

You can verify that the software has been updated by checking the version at the bottom of the BlueXP classification UI pages.