Scanning object storage that uses S3 protocol

Contributors netapp-tonacki

Complete a few steps to start scanning data within object storage directly with Cloud Data Sense. Data Sense can scan data from any Object Storage service which uses the Simple Storage Service (S3) protocol. This includes NetApp StorageGRID, IBM Cloud Object Store, Azure Blob (using MinIO), Linode, B2 Cloud Storage, Amazon S3, and more.

Quick start

Get started quickly by following these steps, or scroll down to the remaining sections for full details.

One Review object storage prerequisites

You need to have the endpoint URL to connect with the object storage service.

You need to have the Access Key and Secret Key from the object storage provider so that Cloud Data Sense can access the buckets.

Two Deploy the Cloud Data Sense instance

Deploy Cloud Data Sense if there isn’t already an instance deployed.

Three Add the Object Storage Service

Add the object storage service to Cloud Data Sense.

Four Select the buckets to scan

Select the buckets that you’d like to scan and Cloud Data Sense will start scanning them.

Reviewing object storage requirements

Review the following prerequisites to make sure that you have a supported configuration before you enable Cloud Data Sense.

  • You need to have the endpoint URL to connect with the object storage service.

  • You need to have the Access Key and Secret Key from the object storage provider so that Data Sense can access the buckets.

  • Support for Azure Blob requires that you use the MinIO service.

Deploying the Cloud Data Sense instance

Deploy Cloud Data Sense if there isn’t already an instance deployed.

If you are scanning data from S3 object storage that is accessible over the internet, you can deploy Cloud Data Sense in the cloud or deploy Data Sense in an on-premises location that has internet access.

If you are scanning data from S3 object storage that has been installed in a dark site that has no internet access, you need to deploy Cloud Data Sense in the same on-premises location that has no internet access. This also requires that the Cloud Manager Connector is deployed in that same on-premises location.

Upgrades to Data Sense software is automated as long as the instance has internet connectivity.

Adding the object storage service to Cloud Data Sense

Add the object storage service.

Steps
  1. From the Working Environments Configuration page, click Add Data Source > Add Object Storage Service.

    A screenshot of the Scan Configuration page where you can click the Add File Shares Group button.

  2. In the Add Object Storage Service dialog, enter the details for the object storage service and click Continue.

    1. Enter the name you want to use for the Working Environment. This name should reflect the name of the object storage service to which you are connecting.

    2. Enter the Endpoint URL to access the object storage service.

    3. Enter the Access Key and Secret Key so that Cloud Data Sense can access the buckets in the object storage.

      A screenshot of the dialog where you enter the values to access the object storage service.

Result

The new Object Storage Service is added to the list of working environments.

Enabling and disabling compliance scans on object storage buckets

After you enable Cloud Data Sense on your Object Storage Service, the next step is to configure the buckets that you want to scan. Data Sense discovers those buckets and displays them in the working environment you created.

Steps
  1. In the Configuration page, click Configuration from the Object Storage Service working environment.

    A screenshot of clicking Configuration for the working environment so you can choose the buckets that you want to scan.

  2. Enable mapping-only scans, or mapping and classification scans, on your buckets.

    A screenshot of selecting the object storage buckets you want to scan.

    To: Do this:

    Enable mapping-only scans on a bucket

    Click Map

    Enable full scans on a bucket

    Click Map & Classify

    Disable scanning on a bucket

    Click Off

Result

Cloud Data Sense starts scanning the buckets that you enabled. If there are any errors, they’ll appear in the Status column, alongside the required action to fix the error.