In Hadoop Eco-System we preferable mostly three Big data distributions:
1.Cloudera Distribution Hadoop
2.Horton Works Data Platform
3.MapR Distributions Platform
In Cloudera, Distribution Platform is a free version, express, and enterprise edition up to 60 days trial version.
Coming to Hortonworks Data Platform completely open source platform for production, developing and testing environment.
Then finally MapR distribution platform is a complete enterprise edition but in MapR 3 is free version is available with fewer features to compare to MapR 5 and MapR 7.
How to install MapR free version on Pseduo Cluster:
Before the install of MapR, we configured prerequisites as below:
1.Configure hostname like FQDN by using the setup command (mapr.hadoop.com) after that check your hostname using hostname -f
3.hostname < your Fully Qualified Domain>
4. vim/etc/selinux/config ===> SELinux = disabled
——-Disable Firewalls and IPTables——-
If you enable firewalls and iptables doesn’t allow some ports so we must and should disable it.
1.service iptables save
2.service iptables stop
3.chkconfig iptables off
4.service ip6table save
5.service ip6tables stop
6.chkconfig ip6tables off
—– Enable NTP service for machines —–
NTP is a Network Time Protocol is a networking protocol for time synchronization between computers and packet switched data.
1.yum -y install ntp ntpupdate ntp-doc
2.chkconfig ntpd on
8.date ( All machines have the same date otherwise it will showing error)
—— Install some additional packages in Linux OS —-
Here will install JAVA 1.8 and Python
1.yum -y install java-1.8.0 -openjdk-devel
2.yum -y install python perl expect expectk
—- setup passwordless SSH On all nodes form master node ——
For passwordless authentication in between master and slave nodes
1.ssh-keygen -t rsa
2.cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
3.ssh-copy-id [email protected]<FQDN1, FQDN2>
—–Additional Linux configuration or Transparent Huge Pages(THP)—-
1. echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
2.echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
— set up EPEL repository for installing additional packages on the system –
Here EPEL repository for installing the additional packages in centos machine
1.Install -uvh the EPEL repository
2.wget http://http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release -6.8.norach.rpm