Configure the HDFS connector
The Hadoop Distributed File System (HDFS) connector (hdfs://) gives XCP the capability to access any HDFS file system that is available with different vendors.
copy command operation from HDFS to NFS is supported for HDFS connectors.
The path syntax for a HDFS connector is
If you do not specify a user, host, and port, XCP calls
To run the HDFS
copy command, you must set the HDFS client on the Linux system and, based on the Hadoop vendor, follow the setup configuration available on the internet. For example, you can set the client for a MapR cluster by using
After you complete the HFDS client setup, you must complete the configuration on the client. To use the HDFS paths with XCP commands, you must have the following environment variables:
In the following examples, the settings work with MapR and java-1.8.0-openjdk-devel on CentOS:
export JAVA_HOME=$(dirname $(dirname $(readlink $(readlink $ (which javac))))) export NHDFS_LIBJVM_PATH=`find $JAVA_HOME -name "libjvm.so"` export NHDFS_LIBHDFS_PATH=/opt/mapr/lib/libMapRClient.so
[demo@mapr0 ~]$ hadoop fs -ls Found 3 items drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d1 drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d2 drwxr-xr-x - demo mapr 0 2021-01-14 00:02 d3