How to resolve Unable to load native Hadoop library for your platform in Hadoop with commands

I am trying to install the Hadoop cluster on Cloud but when I run daemons the getting below error with the snapshot.



Warning[Error]: Unable to load native Hadoop library for your platform

Full error:

Warn util.NativeCodeLoader: unable to load native Hadoop library for your platform... using builtin java classes where applicable

What is Hadoop Native  Libraries?

Basically, Hadoop native library means the implementation of certain components for performance for the non-availability of Java implementation. The implementation components are available in a single, dynamically-linked native library. It is called “Hadoop Native Library”.

Solution 1:

I added below two environment variables in Hadoop-env.sh

export HADOOP_OPTS = "$HADOOP_OPTS"-Djava.library.path = /usr/local/hadoop/lib
export HADOOP_COMMON_LIB_NATIVE_DIR = "/usr/local/hadoop/lib/native"

In case the above commands  are not working properly try this below one :

export HADOOP_OPTS = "$HADOOP_OPTS"-Djava.library.path = "$HADOOP_HOME/lib/native"

Solution 2:





Sometimes this type of warnings belongs to version compatible issues. In case the below file in the  Hadoop library path then updates it that library file.

$HADOOP_HOME/lib/native/libhadoop.so.1.0.0





Summary:  The above resolutions are very simple to resolve this error: “unable to load native Hadoop library for your platform… using builtin java classes where applicable”. It may cayuse version issue, for example, I installed Hadoop 2..x version on top of the Java 1.7 version. After that, I updated my Java 1.7 to 1.8 version but my Hadoop is still the previous version so at the time I faced this type of warning. This type of warning not a big problem but showing while running, MapReduce, Hive and Spark jobs on the Hadoop cluster. Most of the time this waring showing in edge node server at the time daemons running. In a single-node Hadoop cluster this warning, not a problem. In Development, Production environment nodes it may be caused to execution jobs take more time. So try to resolve the issue using the above two resolution, just update the Hadoop native library files in the Hadoop directory.

Leave a Reply

Your email address will not be published. Required fields are marked *