English

Getting started with Cloud Compliance for Amazon S3

Contributors netapp-bcammett netapp-tonacki Download PDF of this topic

Cloud Compliance can scan your Amazon S3 buckets to identify the personal and sensitive data that resides in S3 object storage. Cloud Compliance can scan any bucket in the account, regardless if it was created for a NetApp solution.

Pricing

You need to pay to scan your Amazon S3 buckets. Learn about pricing.

A 30-day free trial is available to scan Amazon S3 data with Cloud Compliance. A subscription to the AWS Marketplace is required to continue scanning Amazon S3 after the free trial ends. Learn how to subscribe.

Quick start

Get started quickly by following these steps, or scroll down to the remaining sections for full details.

Number 1 Create a Connector in AWS

Create a Connector in the AWS account where you want to view your Amazon S3 buckets.

Cloud Manager automatically discovers the S3 buckets in this AWS account and displays them in an Amazon S3 working environment.

Number 2 Set up your cloud environment

Ensure that your cloud environment can meet the requirements for Cloud Compliance, which includes preparing an IAM role, setting up connectivity from Cloud Compliance to S3, and more. See the complete list.

Number 3 Subscribe from the AWS Marketplace

A subscription to the AWS Marketplace is required to scan Amazon S3 after the 30-day free trial ends.

Click Settings > Credentials and click Add Subscription for the Instance Profile.

Number 4 Deploy the Cloud Compliance instance

Deploy Cloud Compliance in Cloud Manager.

Number 5 Enable Cloud Compliance in your working environment

Select the Amazon S3 working environment, click Enable Compliance, and select an IAM role that includes the required permissions.

Number 6 Configure buckets

Select the buckets that you’d like to scan and Cloud Compliance will start scanning them.

Reviewing prerequisites

Review the following prerequisites to make sure that you have a supported configuration before you enable Cloud Compliance.

Requirements specific to S3

The first two requirements are specific to scanning S3 buckets.

Set up an IAM role for the Cloud Compliance instance

Cloud Compliance needs permissions to connect to the S3 buckets in your account and to scan them. Set up an IAM role that includes the permissions listed below. Cloud Manager prompts you to select an IAM role when you enable Cloud Compliance on the Amazon S3 working environment.

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:Get*",
              "s3:List*",
              "s3:HeadBucket"
          ],
          "Resource": "*"
      },
      {
          "Effect": "Allow",
          "Action": [
              "iam:GetPolicyVersion",
              "iam:GetPolicy",
              "iam:ListAttachedRolePolicies"
          ],
          "Resource": [
              "arn:aws:iam::*:policy/*",
              "arn:aws:iam::*:role/*"
          ]
      }
  ]
}
Provide connectivity from Cloud Compliance to Amazon S3

Cloud Compliance needs a connection to Amazon S3. The best way to provide that connection is through a VPC Endpoint to the S3 service. For instructions, see AWS Documentation: Creating a Gateway Endpoint.

When you create the VPC Endpoint, be sure to select the region, VPC, and route table that corresponds to the Cloud Compliance instance. You must also modify the security group to add an outbound HTTPS rule that enables traffic to the S3 endpoint. Otherwise, Cloud Compliance can’t connect to the S3 service.

An alternative is to provide the connection by using a NAT Gateway.

You can’t use a proxy to get to S3 over the internet.

General requirements

The requirements in this section apply to Cloud Compliance in general, whether you’re scanning Amazon S3, Cloud Volumes ONTAP, or Azure NetApp Files. If you’ve already enabled Cloud Compliance (for Cloud Volumes ONTAP or Azure NetApp Files), then you can skip these requirements and Subscribe from the AWS Marketplace.

Enable outbound internet access

Cloud Compliance requires outbound internet access. If your virtual network uses a proxy server for internet access, ensure that the Cloud Compliance instance has outbound internet access to contact the following endpoints. Note that Cloud Manager deploys the Cloud Compliance instance in the same subnet as the Connector.

Endpoints Purpose

https://cloudmanager.cloud.netapp.com

Communication with the Cloud Manager service, which includes Cloud Central accounts.

https://netapp-cloud-account.auth0.com
https://auth0.com

Communication with NetApp Cloud Central for centralized user authentication.

https://cloud-compliance-support-netapp.s3.us-west-2.amazonaws.com
https://hub.docker.com
https://auth.docker.io
https://registry-1.docker.io
https://index.docker.io/
https://dseasb33srnrn.cloudfront.net/
https://production.cloudflare.docker.com/

Provides access to software images, manifests, and templates.

https://kinesis.us-east-1.amazonaws.com

Enables NetApp to stream data from audit records.

https://cognito-idp.us-east-1.amazonaws.com
https://cognito-identity.us-east-1.amazonaws.com

Enables Cloud Compliance to access and download manifests and templates, and to send logs and metrics.

Ensure that Cloud Manager has the required permissions

Ensure that Cloud Manager has permissions to deploy resources and create security groups for the Cloud Compliance instance. You can find the latest Cloud Manager permissions in the policies provided by NetApp.

Check your vCPU limits

Ensure that your cloud provider’s vCPU limit allows for the deployment of an instance with 16 cores. You’ll need to verify the vCPU limit for the relevant instance family in the region where Cloud Manager is running.

In AWS, the instance family is On-Demand Standard instances. In Azure, the instance family is Standard DSv3 Family.

For more details on vCPU limits, see the following:

Ensure that Cloud Manager can access Cloud Compliance

Ensure connectivity between the Connector and the Cloud Compliance instance:

  • The security group for the Connector must allow inbound and outbound traffic over port 80 to and from the Cloud Compliance instance.

    This connection enables deployment of the Cloud Compliance instance and enables you to view information in the Compliance tab.

  • If your AWS network doesn’t use a NAT or proxy for internet access, modify the security group for the Connector to allow inbound traffic over TCP port 3128 from the Cloud Compliance instance.

    This is required because the Cloud Compliance instance uses Cloud Manager as a proxy to access the internet.

    This port is open by default on all new Connector instances, starting with version 3.7.5. It’s not open on instances created prior to that.
Ensure that you can keep Cloud Compliance running

The Cloud Compliance instance needs to stay on to continuously scan your data.

Ensure web browser connectivity to Cloud Compliance

After Cloud Compliance is enabled, ensure that users access the Cloud Manager interface from a host that has a connection to the Cloud Compliance instance.

The Cloud Compliance instance uses a private IP address to ensure that the indexed data isn’t accessible to the internet. As a result, the web browser that you use to access Cloud Manager must have a connection to that private IP address. That connection can come from a direct connection to AWS or Azure (for example, a VPN), or from a host that’s inside the same network as the Cloud Compliance instance.

Subscribing from the AWS Marketplace

A 30-day free trial is available to scan Amazon S3 data with Cloud Compliance. A subscription to the AWS Marketplace is required to continue scanning Amazon S3 after the free trial ends.

These steps must be completed by a user who has the Account Admin role.

Steps
  1. In the upper right of the Cloud Manager console, click the Settings icon, and select Credentials.

    A screenshot of Cloud Manager’s top right banner where you can select the Settings icon.

  2. Find the credentials for the AWS Instance Profile.

    The subscription must be added to the Instance Profile. Charging won’t work otherwise.

    If you already have a subscription, then you’re all set—​there’s nothing else that you need to do.

    A screenshot from the Credentials page that shows the AWS Instance Profile with an active subscription.

  3. If you don’t have a subscription yet, hover over the credentials and click the action menu.

  4. Click Add Subscription.

    A screenshot of the menu in the Credentials page. It shows a button to add a subscription to the credentials.

  5. Click Add Subscription, click Continue, and follow the steps.

Deploying the Cloud Compliance instance

You deploy an instance of Cloud Compliance for each Cloud Manager instance.

Steps
  1. In Cloud Manager, click Compliance.

  2. Click Activate Cloud Compliance to start the deployment wizard.

    A screenshot of selecting the Activate Cloud Compliance button to deploy Cloud Compliance.

  3. The wizard displays progress as it goes through the deployment steps. It will stop and ask for input if it runs into any issues.

    A screenshot of the Cloud Compliance wizard to deploy a new instance.

  4. When the instance in deployed, click Continue to configuration to go to the Scan Configuration page.

Result

Cloud Manager deploys the Cloud Compliance instance in your cloud provider.

From the Scan Configuration page you can select the working environments, volumes, and buckets that you want to scan for compliance.

Enabling Cloud Compliance

Enable Cloud Compliance on Amazon S3 after you verify the prerequisites.

Steps
  1. At the top of Cloud Manager, click Working Environments.

  2. Select the Amazon S3 working environment.

    screenshot s3 we

  3. In the pane on the right, click Enable Compliance.

    screenshot s3 enable compliance

  4. When prompted, assign an IAM role to the Cloud Compliance instance that has the required permissions.

    screenshot s3 compliance iam role

  5. Click Enable Compliance.

You can also enable compliance scans for a working environment from the Scan Configuration page by clicking the screenshot gallery options icon and selecting Activate Compliance.
Result

Cloud Manager assigns the IAM role to the instance.

Configuring buckets

After Cloud Manager enables Cloud Compliance on Amazon S3, the next step is to configure the buckets that you want to scan.

When Cloud Manager is running in the AWS account that has the S3 buckets you want to scan, it discovers those buckets and displays them in an Amazon S3 working environment.

Steps
  1. Select the Amazon S3 working environment.

  2. In the pane on the right, click Configure Buckets.

    screenshot s3 configure buckets

  3. Enable compliance on the buckets that you want to scan.

    screenshot s3 select buckets

Result

Cloud Compliance starts scanning the S3 buckets that you enabled. If there are any errors, they’ll appear in the Status column, alongside the required action to fix the error.

Scanning buckets from additional AWS accounts

You can scan S3 buckets that are under a different AWS account by assigning a role from that account to access the existing Cloud Compliance instance.

Steps
  1. Go to the target AWS account where you want to scan S3 buckets and create an IAM role by selecting Another AWS account.

    screenshot iam create role

    Be sure to do the following:

    • Enter the ID of the account where the Cloud Compliance instance resides.

    • Change the Maximum CLI/API session duration from 1 hour to 12 hours and save that change.

    • Attach the Cloud Compliance IAM policy. Make sure it has the required permissions.

      {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:Get*",
                    "s3:List*",
                    "s3:HeadBucket"
                ],
                "Resource": "*"
            },
        ]
      }
  2. Go to the source AWS account where the Cloud Compliance instance resides and select the IAM role that is attached to the instance.

    1. Change the Maximum CLI/API session duration from 1 hour to 12 hours and save that change.

    2. Click Attach policies and then click Create policy.

    3. Create a policy that includes the "sts:AssumeRole" action and the ARN of the role that you created in the target account.

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": "sts:AssumeRole",
                  "Resource": "arn:aws:iam::<ADDITIONAL-ACCOUNT-ID>:role/<ADDITIONAL_ROLE_NAME>"
              },
              {
                  "Effect": "Allow",
                  "Action": [
                      "iam:GetPolicyVersion",
                      "iam:GetPolicy",
                      "iam:ListAttachedRolePolicies"
                  ],
                  "Resource": [
                      "arn:aws:iam::*:policy/*",
                      "arn:aws:iam::*:role/*"
                  ]
              }
          ]
      }

      The Cloud Compliance instance profile account now has access to the additional AWS account.

  3. Go to the Amazon S3 Scan Configuration page and the new AWS account is displayed. Note that it can take a few minutes for Cloud Compliance to sync the new account’s working environment and show this information.

    screenshot activate and select buckets

  4. Click Activate Compliance & Select Buckets and select the buckets you want to scan.

Result

Cloud Compliance starts scanning the new S3 buckets that you enabled.