Skip to main content
BlueXP copy and sync

Creating a new data broker in Google Cloud

Contributors netapp-ivanad netapp-bcammett

When you create a new data broker group, choose Google Cloud Platform to deploy the data broker software on a new virtual machine instance in a Google Cloud VPC. BlueXP copy and sync guides you through the installation process, but the requirements and steps are repeated on this page to help you prepare for installation.

You also have the option to install the data broker on an existing Linux host in the cloud or on your premises. Learn more.

Supported Google Cloud regions

All regions are supported.

Root privileges

The data broker software automatically runs as root on the Linux host. Running as root is a requirement for data broker operations. For example, to mount shares.

Networking requirements

  • The data broker needs an outbound internet connection so it can poll the BlueXP copy and sync service for tasks over port 443.

    When BlueXP copy and sync deploys the data broker in Google Cloud, it creates a security group that enables the required outbound communication.

    If you need to limit outbound connectivity, see the list of endpoints that the data broker contacts.

  • NetApp recommends configuring the source, target, and data broker to use a Network Time Protocol (NTP) service. The time difference between the three components should not exceed 5 minutes.

Permissions required to deploy the data broker in Google Cloud

Ensure that the Google Cloud user who deploys the data broker has the following permissions:

- compute.networks.list
- compute.regions.list
- deploymentmanager.deployments.create
- deploymentmanager.deployments.delete
- deploymentmanager.operations.get
- iam.serviceAccounts.list

Permissions required for the service account

When you deploy the data broker, you need to select a service account that has the following permissions:

- logging.logEntries.create
- resourcemanager.projects.get
- storage.buckets.get
- storage.buckets.list
- storage.objects.create
- storage.objects.delete
- storage.objects.get
- storage.objects.getIamPolicy
- storage.objects.list
- storage.objects.setIamPolicy
- storage.objects.update
- iam.serviceAccounts.signJwt
- pubsub.subscriptions.consume
- pubsub.subscriptions.create
- pubsub.subscriptions.delete
- pubsub.subscriptions.list
- pubsub.topics.attachSubscription
- pubsub.topics.create
- pubsub.topics.delete
- pubsub.topics.list
- pubsub.topics.setIamPolicy
- storage.buckets.update
- cloudkms.cryptoKeys.list
- cloudkms.keyRings.list

Notes:

  1. The "iam.serviceAccounts.signJwt" permission is required only if you're planning to set up the data broker to use an external HashiCorp vault.

  2. The "pubsub.*" and "storage.buckets.update" permissions are required only if you plan to enable the Continuous Sync setting on a sync relationship from Google Cloud Storage to another cloud storage location. Learn more about the Continuous Sync option.

  3. The "cloudkms.cryptoKeys.list" and "cloudkms.keyRings.list" permissions are required only if you plan to use a customer-managed KMS key on a target Google Cloud Storage bucket.

Creating the data broker

There are a few ways to create a new data broker. These steps describe how to install a data broker in Google Cloud when you create a sync relationship.

Steps
  1. Select Create New Sync.

  2. On the Define Sync Relationship page, choose a source and target and select Continue.

    Complete the steps until you reach the Data Broker Group page.

  3. On the Data Broker Group page, select Create Data Broker and then select Google Cloud Platform.

    A screenshot of the Data Broker page that enables you to choose between an AWS, Azure, Google Cloud, and On-Prem data broker.

  4. Enter a name for the data broker and select Continue.

  5. If you're prompted, log in with your Google account.

    The form is owned and hosted by Google. Your credentials are not provided to NetApp.

  6. Select a project and service account and then choose a location for the data broker, including whether you want to enable or disable a public IP address.

    If you don't enable a public IP address, then you'll need to define a proxy server in the next step.

    A screenshot that shows the information required to deploy a data broker in Google Cloud.

  7. Specify a proxy configuration, if a proxy is required for internet access in the VPC.

    If a proxy is required for internet access, then the proxy must be in Google Cloud and use the same service account as the data broker.

  8. Once the data broker is available, select Continue in BlueXP copy and sync.

    The instance takes approximately 5 to 10 minutes to deploy. You can monitor the progress from the BlueXP copy and sync service, which automatically refreshes when the instance is available.

  9. Complete the pages in the wizard to create the new sync relationship.

Result

You've deployed a data broker in Google Cloud and created a new sync relationship. You can use this data broker with additional sync relationships.

Providing permissions to use buckets in other Google Cloud projects

When you create a sync relationship and choose Google Cloud Storage as the source or target, BlueXP copy and sync enables you to choose from the buckets that the data broker's service account has permissions to use. By default, this includes the buckets that are in the same project as the data broker service account. But you can choose buckets from other projects if you provide the required permissions.

Steps
  1. Open the Google Cloud Platform console and load the Cloud Storage service.

  2. Select the name of the bucket that you'd like to use as a source or target in a sync relationship.

  3. Select Permissions.

  4. Select Add.

  5. Enter the name of the data broker's service account.

  6. Select a role that provides the same permissions as shown above.

  7. Select Save.

Result

When you set up a sync relationship, you can now choose that bucket as the source or target in the sync relationship.

Details about the data broker VM instance

BlueXP copy and sync creates a data broker in Google Cloud using the following configuration.

Node.js compatibility

v21.2.0

Machine type

n2-standard-4

vCPUs

4

RAM

15 GB

Operating system

Rocky Linux 9.0

Disk size and type

20 GB HDD pd-standard