Skip to main content

NAS data requirements for S3 client access

Contributors netapp-aherbin
PDFs

It is important to understand that there are some inherent incompatibilities when mapping NAS files and directories for S3 access. It might be necessary to adjust NAS file hierarchies before serving them using S3 NAS buckets.

An S3 NAS bucket provides S3 access to a NAS directory by mapping that directory using S3 bucket syntax, and the files in the directory tree are viewed as objects. The object names are the slash-delimited pathnames of the files relative to the directory specified in the S3 bucket configuration.

This mapping imposes some requirements when files and directories are served using S3 NAS buckets:

  • S3 names are limited to 1024 bytes, so files with longer pathnames are not accessible using S3.

  • File and directory names are limited to 255 characters, so an object name cannot have more than 255 consecutive non-slash (‘/’) characters

  • An SMB pathname that is delimited by backslash (‘\’) characters will appear to s3 as an object name containing forward-slash (‘/’) characters instead.

  • Some pairs of legal S3 object names cannot coexist in the mapped NAS directory tree. For example, the legal S3 object names “part1/part2” and “part1/part2/part3” map to files that cannot simultaneously exist in the NAS directory tree, as “part1/part2” is a file in the first name and a directory in the other.

    • If “part1/part2” is an existing file, an S3 creation of “part1/part2/part3” will fail.

    • If “part1/part2/part3” is an existing file, an S3 creation or deletion of “part1/part2” will fail.

    • An S3 object creation that matches the name of an existing object replaces the pre-existing object (in unversioned buckets); that holds in NAS but requires an exact match. The examples above will not cause removal of the existing object because while the names collide, they do not match.

While an object store is designed to support a very large number of arbitrary names, a NAS directory structure can experience performance problems if a very large number of names are placed in one directory. In particular, names with no slash (‘/’) characters in them will all be placed into the root directory of the NAS mapping. Applications that make extensive use of names that are not “NAS-friendly” would be better hosted on an actual object store bucket rather than a NAS mapping.