Skip to main content

Configuration XML for search integration

Contributors netapp-perveilerk ssantho3 netapp-madkat netapp-lhalbert
PDFs

The search integration service is configured using a set of rules contained within <MetadataNotificationConfiguration> and </MetadataNotificationConfiguration> tags. Each rule specifies the objects that the rule applies to, and the destination where StorageGRID should send those objects' metadata.

Objects can be filtered on the prefix of the object name. For example, you could send metadata for objects with the prefix images to one destination, and metadata for objects with the prefix videos to another. Configurations that have overlapping prefixes aren't valid, and are rejected when they are submitted. For example, a configuration that includes one rule for objects with the prefix test and a second rule for objects with the prefix test2 is not allowed.

Destinations must be specified using the URN of a StorageGRID endpoint that has been created for the search integration service. These endpoints refer to an index and type defined on an Elasticsearch cluster.

<MetadataNotificationConfiguration>
    <Rule>
        <ID>Rule-1</ID>
        <Status>rule-status</Status>
        <Prefix>key-prefix</Prefix>
        <Destination>
           <Urn>arn:aws:es:region:account-ID:domain/mydomain/myindex/mytype</Urn>
        </Destination>
    </Rule>
    <Rule>
        <ID>Rule-2</ID>
         ...
    </Rule>
     ...
</MetadataNotificationConfiguration>

The table describes the elements in the metadata notification configuration XML.

Name Description Required

MetadataNotificationConfiguration

Container tag for rules used to specify the objects and destination for metadata notifications.

Contains one or more Rule elements.

Yes

Rule

Container tag for a rule that identifies the objects whose metadata should be added to a specified index.

Rules with overlapping prefixes are rejected.

Included in the MetadataNotificationConfiguration element.

Yes

ID

Unique identifier for the rule.

Included in the Rule element.

No

Status

Status can be 'Enabled' or 'Disabled'. No action is taken for rules that are disabled.

Included in the Rule element.

Yes

Prefix

Objects that match the prefix are affected by the rule, and their metadata is sent to the specified destination.

To match all objects, specify an empty prefix.

Included in the Rule element.

Yes

Destination

Container tag for the destination of a rule.

Included in the Rule element.

Yes

Urn

URN of the destination where object metadata is sent. Must be the URN of a StorageGRID endpoint with the following properties:

  • es must be the third element.

  • The URN must end with the index and type where the metadata is stored, in the form domain-name/myindex/mytype.

Endpoints are configured using the Tenant Manager or Tenant Management API. They take the following form:

  • arn:aws:es:region:account-ID:domain/mydomain/myindex/mytype

  • urn:mysite:es:::mydomain/myindex/mytype

The endpoint must be configured before the configuration XML is submitted, or configuration will fail with a 404 error.

URN is included in the Destination element.

Yes

Use the sample metadata notification configuration XML to learn how to construct your own XML.

Metadata notification configuration that applies to all objects

In this example, object metadata for all objects is sent to the same destination.

<MetadataNotificationConfiguration>
    <Rule>
        <ID>Rule-1</ID>
        <Status>Enabled</Status>
        <Prefix></Prefix>
        <Destination>
           <Urn>urn:myes:es:::sgws-notifications/test1/all</Urn>
        </Destination>
    </Rule>
</MetadataNotificationConfiguration>

Metadata notification configuration with two rules

In this example, object metadata for objects that match the prefix /images is sent to one destination, while object metadata for objects that match the prefix /videos is sent to a second destination.

<MetadataNotificationConfiguration>
    <Rule>
        <ID>Images-rule</ID>
        <Status>Enabled</Status>
        <Prefix>/images</Prefix>
        <Destination>
           <Urn>arn:aws:es:us-east-1:3333333:domain/es-domain/graphics/imagetype</Urn>
        </Destination>
    </Rule>
    <Rule>
        <ID>Videos-rule</ID>
        <Status>Enabled</Status>
        <Prefix>/videos</Prefix>
        <Destination>
           <Urn>arn:aws:es:us-west-1:22222222:domain/es-domain/graphics/videotype</Urn>
        </Destination>
    </Rule>
</MetadataNotificationConfiguration>
Related information

Use S3 REST API

Object metadata included in metadata notifications

JSON generated by search integration service

Configure search integration service