Skip to main content
XCP

Configure the HDFS connector

Contributors netapp-pcarriga netapp-aoife
PDFs

For XCP NFS, the Hadoop Distributed File System (HDFS) connector (hdfs://) gives XCP the capability to access any HDFS file system that is available with different vendors.

Supported features

The copy command operation from HDFS to NFS is supported for HDFS connectors.

Path Syntax

The path syntax for a HDFS connector is hdfs://[user@host:port]/full-path.

Note If you do not specify a user, host, and port, XCP calls hdfsConnect with the host set to default and the port set to 0.
Set up a HDFS connector

To run the HDFS copy command, you must set the HDFS client on the Linux system, and based on the Hadoop vendor, follow the setup configuration available on the internet. For example, you can set the client for a MapR cluster by using https://docs.datafabric.hpe.com/60/AdvancedInstallation/SettingUptheClient-redhat.html.

After you complete the HFDS client setup, you must complete the configuration on the client. To use the HDFS paths with XCP commands, you must have the following environment variables:

  • NHDFS_LIBHDFS_PATH

  • NHDFS_LIBJVM_PATH

In the following examples, the settings work with MapR and java-1.8.0-openjdk-devel on CentOS:

export JAVA_HOME=$(dirname $(dirname $(readlink $(readlink $
(which javac)))))
export NHDFS_LIBJVM_PATH=`find $JAVA_HOME -name "libjvm.so"` export
NHDFS_LIBHDFS_PATH=/opt/mapr/lib/libMapRClient.so
[demo@mapr0 ~]$ hadoop fs -ls Found 3 items
drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d1
drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d2
drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d3