Skip to main content

CopyObject

Contributors netapp-pcarriga
PDFs

You can use the S3 CopyObject request to create a copy of an object that is already stored in S3. A CopyObject operation is the same as performing GetObject followed by PutObject.

Resolve conflicts

Conflicting client requests, such as two clients writing to the same key, are resolved on a "latest-wins" basis. The timing for the "latest-wins" evaluation is based on when the StorageGRID system completes a given request, and not on when S3 clients begin an operation.

Object size

The maximum recommended size for a single PutObject operation is 5 GiB (5,368,709,120 bytes). If you have objects that are larger than 5 GiB, use multipart upload instead.

The maximum supported size for a single PutObject operation is 5 TiB (5,497,558,138,880 bytes).

Note If you upgraded from StorageGRID 11.6 or earlier, the S3 PUT Object size too large alert will be triggered if you attempt to upload an object that exceeds 5 GiB. If you have a new installation of StorageGRID 11.7 or 11.8, the alert won't be triggered in this case. However, to align with the AWS S3 standard, future releases of StorageGRID won't support uploads of objects larger than 5 GiB.

UTF-8 characters in user metadata

If a request includes (unescaped) UTF-8 values in the key name or value of user-defined metadata, StorageGRID behavior is undefined.

StorageGRID does not parse or interpret escaped UTF-8 characters included in the key name or value of user-defined metadata. Escaped UTF-8 characters are treated as ASCII characters:

  • Requests succeed if user-defined metadata includes escaped UTF-8 characters.

  • StorageGRID does not return the x-amz-missing-meta header if the interpreted value of the key name or value includes unprintable characters.

Supported request headers

The following request headers are supported:

  • Content-Type

  • x-amz-copy-source

  • x-amz-copy-source-if-match

  • x-amz-copy-source-if-none-match

  • x-amz-copy-source-if-unmodified-since

  • x-amz-copy-source-if-modified-since

  • x-amz-meta-, followed by a name-value pair containing user-defined metadata

  • x-amz-metadata-directive: The default value is COPY, which enables you to copy the object and associated metadata.

    You can specify REPLACE to overwrite the existing metadata when copying the object, or to update the object metadata.

  • x-amz-storage-class

  • x-amz-tagging-directive: The default value is COPY, which enables you to copy the object and all tags.

    You can specify REPLACE to overwrite the existing tags when copying the object, or to update the tags.

  • S3 Object Lock request headers:

    • x-amz-object-lock-mode

    • x-amz-object-lock-retain-until-date

    • x-amz-object-lock-legal-hold

      If a request is made without these headers, the bucket default retention settings are used to calculate the object version mode and retain-until-date. See Use S3 REST API to configure S3 Object Lock.

  • SSE request headers:

    • x-amz-copy-source​-server-side​-encryption​-customer-algorithm

    • x-amz-copy-source​-server-side-encryption-customer-key

    • x-amz-copy-source​-server-side-encryption-customer-key-MD5

    • x-amz-server-side-encryption

    • x-amz-server-side-encryption-customer-key-MD5

    • x-amz-server-side-encryption-customer-key

    • x-amz-server-side-encryption-customer-algorithm

      See Request headers for server-side encryption

Unsupported request headers

The following request headers aren't supported:

  • Cache-Control

  • Content-Disposition

  • Content-Encoding

  • Content-Language

  • Expires

  • x-amz-checksum-algorithm

    When you copy an object, if the source object has a checksum, StorageGRID doesn't copy that checksum value to the new object. This behavior applies whether or not you try to use x-amz-checksum-algorithm in the object request.

  • x-amz-website-redirect-location

Storage class options

The x-amz-storage-class request header is supported, and affects how many object copies StorageGRID creates if the matching ILM rule uses the Dual commit or Balanced ingest option.

  • STANDARD

    (Default) Specifies a dual-commit ingest operation when the ILM rule uses the Dual commit option, or when the Balanced option falls back to creating interim copies.

  • REDUCED_REDUNDANCY

    Specifies a single-commit ingest operation when the ILM rule uses the Dual commit option, or when the Balanced option falls back to creating interim copies.

    Note If you are ingesting an object into a bucket with S3 Object Lock enabled, the REDUCED_REDUNDANCY option is ignored. If you are ingesting an object into a legacy Compliant bucket, the REDUCED_REDUNDANCY option returns an error. StorageGRID will always perform a dual-commit ingest to ensure that compliance requirements are satisfied.

Using x-amz-copy-source in CopyObject

If the source bucket and key, specified in the x-amz-copy-source header, are different from the destination bucket and key, a copy of the source object data is written to the destination.

If the source and destination match, and the x-amz-metadata-directive header is specified as REPLACE, the object's metadata is updated with the metadata values supplied in the request. In this case, StorageGRID does not re-ingest the object. This has two important consequences:

  • You can't use CopyObject to encrypt an existing object in place, or to change the encryption of an existing object in place. If you supply the x-amz-server-side-encryption header or the x-amz-server-side-encryption-customer-algorithm header, StorageGRID rejects the request and returns XNotImplemented.

  • The option for Ingest Behavior specified in the matching ILM rule is not used. Any changes to object placement that are triggered by the update are made when ILM is re-evaluated by normal background ILM processes.

    This means that if the ILM rule uses the Strict option for ingest behavior, no action is taken if the required object placements can't be made (for example, because a newly required location is unavailable). The updated object retains its current placement until the required placement is possible.

Request headers for server-side encryption

If you use server-side encryption, the request headers you provide depend on whether the source object is encrypted and on whether you plan to encrypt the target object.

  • If the source object is encrypted using a customer-provided key (SSE-C), you must include the following three headers in the CopyObject request, so the object can be decrypted and then copied:

    • x-amz-copy-source​-server-side​-encryption​-customer-algorithm: Specify AES256.

    • x-amz-copy-source​-server-side-encryption-customer-key: Specify the encryption key you provided when you created the source object.

    • x-amz-copy-source​-server-side-encryption-customer-key-MD5: Specify the MD5 digest you provided when you created the source object.

  • If you want to encrypt the target object (the copy) with a unique key that you provide and manage, include the following three headers:

    • x-amz-server-side-encryption-customer-algorithm: Specify AES256.

    • x-amz-server-side-encryption-customer-key: Specify a new encryption key for the target object.

    • x-amz-server-side-encryption-customer-key-MD5: Specify the MD5 digest of the new encryption key.

    Caution The encryption keys you provide are never stored. If you lose an encryption key, you lose the corresponding object. Before using customer-provided keys to secure object data, review the considerations for using server-side encryption.

  • If you want to encrypt the target object (the copy) with a unique key managed by StorageGRID (SSE), include this header in the CopyObject request:

    • x-amz-server-side-encryption

      Note The server-side-encryption value of the object can't be updated. Instead, make a copy with a new server-side-encryption value using x-amz-metadata-directive: REPLACE.

Versioning

If the source bucket is versioned, you can use the x-amz-copy-source header to copy the latest version of an object. To copy a specific version of an object, you must explicitly specify the version to copy using the versionId subresource. If the destination bucket is versioned, the generated version is returned in the x-amz-version-id response header. If versioning is suspended for the target bucket, then x-amz-version-id returns a "null" value.