Configuring the search integration service

Configuring the search integration service for an S3 bucket enables the grid to send object metadata to a destination Elasticsearch index. You can configure the search integration service using the Tenant Manager.

Before you begin

About this task

After you configure the search integration service for a source bucket, creating an object or updating an object's metadata or tags triggers object metadata to be sent to the destination endpoint. If you enable the search integration service for a bucket that already contains objects, metadata notifications are not automatically sent for existing objects. You must update these existing objects to ensure that their metadata is added to the destination search index.

Steps

  1. Enable search integration for your source bucket:
    1. Use a text editor to create the metadata notification XML required to enable search integration.
      See "Configuration XML for search integration" for more information. When configuring the XML, use the URN of a search integration endpoint as the destination.
      Example
      <MetadataNotificationConfiguration>
          <Rule>
              <Status>Enabled</Status>
              <Prefix></Prefix>
              <Destination>
                 <Urn>arn:aws:es:us-east-1:11111111111111:domain/mydomain/myindex/mytype</Urn>
              </Destination>
          </Rule>
      </MetadataNotificationConfiguration>
      
    2. In the Tenant Manager go to S3 > Buckets.
    3. Select the source bucket, then click Configure Search Integration.
    4. Paste the metadata notification configuration into the text box, and click Save.

      screenshot of search integration configuration dialog box
      Note: Platform services must be enabled for each tenant account by a StorageGRID Webscale administrator using the Grid Manager or Management API. Contact your grid administrator if an error occurs when you save the configuration XML.
  2. Verify that the search integration service is configured correctly:
    1. Add an object to the source bucket that meets the requirements for triggering a metadata notification as specified in the configuration XML.
      In the example shown earlier, all objects added to the bucket trigger a metadata notification.
    2. Confirm that a JSON document that contains the object's metadata and tags was added to the search index specified in the endpoint.
    You have configured your source bucket to support the search integration service.