How to Install IntelliJ IDEA on Linux/Ubuntu with Pictures




Simple Installations of IntelliJ IDEA on Ubuntu/Linux:

IntelliJ IDEA is one of the best IDE (Integrated Development Environment) tool for developing programs having some functions by developed by Jetbrains. With simple syntax to develop the code either JAVA, SCALA, Python, Ruby, Node JS and PHP etc in the help of JDK 1.7 or above version of (JDK) Java Development Kit on Linux/Ubuntu operating systems like Sublime, Atom IDEs with different feature functions. simple steps to installation with step by step process:

Prerequisites to Installation of IntelliJ IDEA on Linux:

1. Linux 64-bit version with minimum 2GB RAM and recommended 8 GB.

2.Free space on your Linux operating system at least 5 GB Hard Disk or more.

3.Java Development Kit (JDK 1.7) or above version is mandatory for standalone edition.

Standalone mode installation on Linux/Ubuntu:

Step 1: Download tarball from Jetbrain official website like below preference with the below snapshot:

Step 2: For simple practice programming and Andriod programs download to free, open-source community edition is enough otherwise will go with the Ultimate edition for web developing related. Then download “IntelliJ IDEA tar ball”

Step 3: Extract directly from that tarball or using below simple command for tarball extraction:
“tar -xzvf  ideaIC-20XX.X.X.tar.gz” on your location.

Step 4: Go to idea-IC- folder in the bin folder then run the idea.sh then simple to open IntelliJ IDEA otherwise set the environment variables and path for “IntelliJIDEA_HOME ” than simple to open the IDE




Step 5: To check Path Variables for set correctly or not by using below:
“echo $IntelliJIDEA_HOME”.

Step 6: Then simply execute the command on the terminal using “./idea.sh”

Summary: The above simple steps for IntelliJ IDEA installations in Ubuntu/Linux operating systems. It is an open-source and as well as community edition with some features for web-developing. Nowadays IntelliJ IDEA for Spark with Scala programming, Java and JavaScript programming,  Andriod and iOS developing for programmers.

Latest(Updated) Hadoop Admin interview Questions for experienced





1. Explain how to execute the Spark job in Resource manager?

2. Explain how to build a Multi-node cluster on Linux with prerequisites? do you need to install Hadoop on Datanodes also?

3. Explain recently resolved Hadoop issue on your project?

4. How to use Kerberos in your cluster and how it is work?

5.Explain how to install Hortonwork on Google Cloud Platform with step by step

6.Difference between GCP Vs AWS and Linux machine while installing of Hadoop cluster?

7.If one Talend (ETL Tool) job is failed on your Hadoop environment with socket connection issue then how to resolve that issue with simple steps?

8. Explain about Spark SQL jobs with a real-time scenario including impala queries?

9. What is the difference between Spark with Scala and PySpark can you explain briefly?

10. How much daily data will get it from your MapR/Cloudera/Hortonwork Cluster and explain about active and passive nodes?

11. Do you know Python script? If you know explain about which scripts were implemented?

12. How to set the replication factor manually in a single node Hadoop cluster?

Error while running Sqoop jobs in Hadoop Cluster





Apache Sqoop: Sqoop is one of the import/export large data from HDFS to RDBMS and RDBMS to HDFS vice versa.
Talend: Talend is an open-source ETL Tool to provide Data Integration and Big Data environment and Cloud storage based Tool. Nowadays most popular for Data Integration and Big Data.
While running the Talend jobs in the Hadoop cluster environment getting Sqoop connection error like below.

Caused by:  java.sql.SQLRecoverableException: IO Error: Connection reset
at oracle.jdbc.driver.T4Cconnection.login(T4CConnection.java.498)
at.oracle.jdbc.driver.PhyscicalConnection.<init>(PhysicalConnection.java:553)
at.oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at.java.sql.DriverManager.getConnection(DriverMAnager.java:571)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
... 46 more
Caused by: java.net.SocketException:Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java.196)

The above issue belongs to the connection (network) issue between Integration tools and Database Management System.

Resolution 1:

Step 1: We use Java Database Connectivity JDBC driver.
Step 2: Then configure DNS on both systems 
Step 3: Restart the DB and Tools in the Hadoop cluster.

Resolution: 2

Step 1: $JAVA_HOME/jre/lib/security/java.security
Step 2: securerandom.source=file:/dev/urandom

After completion of all steps then restart the Apache Sqoop.

How to install WhatsApp on Windows 7/8/10 in Laptop/Computer Without Bluestacks




Simple steps to Install WhatsApp on Windows in Laptop/Computer without Bluestacks:

WhatsApp Messenger is one of the topmost communication Application for Android, iOS and Windows laptop for all users. Here is a step to step installation of WhatsApp Messenger on the Windows or Mac os in your computer/ Laptop without Bluestack application.

Here is the complete installation of WhatsApp Messenger without Bluestacks:

Step 1: Go to WhatsApp official website WhatsApp for downloading WhatsApp.exe file for Windows.

Step 2: Then download “Download For Windows (64-BIT)”

Step 3: After downloading WhatsApp Messenger .exe file then install it.

Step 4: Give your mobile number then will get OTP. Give your OTP then click on “Next”.

After successful installation of WhatsApp in your computer/laptop. Default in your desktop created a shortcut of WhatsApp.

Another option for WhatsApp on your Laptop in below steps of all users.

How to open WhatsApp Web on your computer/laptop:

Step 1: First Go to WhatsAppWeb on your Google chrome browser/Firefox in your laptop/computer.

Step 2: Then open WhatsApp on your mobile either Android or iOS

Step 3: On top of Right: click on Menu -> click on WhatsApp Web

Step 4: Take your phone to this screen to capture the QR code like below snapshot in your computer/laptop

How to Logout from WhatsAppWeb from your computer:

Step 1: After successfully open WhatsAppWeb in your laptop/computer on the left of your screen select Menu -> click on Logout then automatically closed the WhatsApp from your browser.

If you want to change your Setting in your WhatsAppWeb click on Settings then change simply.




Summary: WhatsApp Messenger installation and WhatsAppWeb in your computer/Laptop on Windows 7/8/10 operating system without Bluestacks. Above steps for all users with simple guidance. WhatsApp Messenger almost using on Android, iOS, and Windows mobiles but sometimes we need to installation on Laptop/Computer for sharing or communication purpose. So here is simple for all users.

HBase error: KeeprErrorCode = ConnectionLoss for /hbase in Cluster

On top of Hadoop Cluster Installed HBase (one kind of NoSQL database within Hadoop) service for real-time random reads/random writes in aginst to sequential file accessing of Hadoop Distributed File System (HDFS).




HBase used for better storage but we can’t use HBase to process data with some business logic for some other services like HIVE, Map-Reduce, PIG, andSQOOP, etc.

After Installed Spark server getting below error with HBase Snapshot from Hadoop cluster CLI

Below is the error in the HBase node:

at org.jruby.Ruby.runScript(Ruby.java:697)
at org.jruby.Ruby.runNormally(Ruby.java:597)
at org.jruby.Ruby.runFromMain(Ruby.java:446)
at org.jruby.Ruby.internalRun(Main.Ruby.java:258)
ERROR [ main] client.ConnectManager$HConnectionImplementation: Can't get connection to Zookeeeper: KEeperErrorCode = ConnectionLoss for /hbase
Error: KeeperErrorCode = ConnectionLoss for /hbase
Here is some help for this command:
List all tables in hbase. Optional regualr expression paramete could be used to filter the output. Examples:

How to resolve the below error in HBase Master node?

Resolutions for KeeprErrorCode = ConnectionLoss for /HBase in Cluster:

Above error code means HBase Master is not running on Hadoop cluster:

Resolution 1:

Step 1: First will check the HBase Master node is running or not by using "jps" commands.
Step 2: using "stop-all.sh" command to stop the all running services on Hadoop cluster
Step 3: using "start-all.sh" command to start all running services.
Step 4: using "jps" command to check the services if it showing HBase master working then fine otherwise will do below steps:
Step 5: Goto root user using "sudo su"
Step 6:  Goto hbase shell file path: "cd /usr/lib/habse-1.2.6-hadoop/bin/start-hbase.sh"
Step 7: Open the hbase shell using "hbase shell" command
Step 8: use "list" command.

Resolution 2:





It may cause Zookeeper issue while HBase Master node tries to get the list from Zookeeper then it fails.

Step 1: First check zookeeper service is running or not using "ps -ef | grep zookeeper"
Step 2: Using "sudo service zookeeper stop" command to stop the Zookeeper service in Haodop cluster and stop the HBase service as well.
Step 3: Then HBase xml file to increase the number of connection to Zookeeper services using"hbase.zookeeper,property.maxClientCnxns"
Step 4: start the zookeeper service first then start the HBase service.

Connection issues in Cassandra and HBase

What is Apache Cassandra?




Cassandra is an open-source, distributed, Not Only SQL database management system designed to handle a large amount of across data.

How to install Cassandra?Cassandra simple to install on Ubuntu/Linux with step by step processing and why should be using Apache Cassandra in Data handled:

Install Cassandra on ubuntu linux

What is Apache HBase?

Hadoop + DataBase runs on top of Hadoop eco-system. It is a Database which is an open-source, distributed, NoSQL database related. It provides random access and data stores in HDFS files that are indexed by key, values

How to install Apache HBase on Linux/Ubuntu system?

It is simple to the installation of HBase on the Linux operating system with step by step processing.
Installation of HBase on Ubuntu

Cassandra Connection error:

Error: Exception encountered during startup

java.lang.Illegal exceptionArgumentException is already in reerseMap to (Username)

at org.apache.cassandra.utils.concurrentBiMap.put(concurrentBiMap.java:97)

at org.apache.cassandra.config.schema.load(schema.java:406)

at org.apache.cassandra.config.schema.load(schema.java:117)

HBase Connection Error:

Client.ConnectionManager$HConnection Implementation: Can't get  connection to Zookeeper service connection loss for /hbase

After installation of Cassandra and HBase services on top of Hadoop Eco-system I got this type of error.  Anyone have found resolution please post it here

How to install PyCharm(Python) on Linux/Ubuntu and How to create project




PyCharm Installation on Linux:

Step 1: Download tarball from PyCharm official website from JetBrains for Professionals(For both Scientific and Web Python development) and Community (For Python development) edition than simply Download the Community edition.

Step 2: After downloaded tarball then extract it by using ” command and then find out the folder in the directory

tar -xzvf pycharm-community-2019.2.tar.gz"

Step 3: Go to /home/sreekanth/Downloads/pycharm-community-2019.2/bin folder

Step 4: Run the “pycharm.sh” file using below command in the bin folder.

"bash pycharm.sh "

 

Step 5: Then automatically open the PyCharm window like below image:

If you want to check every Tip click on “Next Tip” otherwise directly click on “Close”

Step 6: Go to Top left side click on File -> New Project for creating a new project

Step 7: After clicking on the “New Project” option then choose the Project Location where you will choose for applications like below snapshot.




If choosing a location then click on the “Create” button.

Step 7: After created your project and completed the programming then execute will check the results in the “Python Console”. If you want to write sample programs then directly write and execute simple.

Example:  >> print(“Hello”) then click on the enter button will see the results

Summary: Above steps are to install the PyCharm community edition on Linux or Ubuntu operating system. Here is provided step by step processing with pictures.

Permission Denied error in Hive while creating Database in Hadoop eco-system

I have installed Hive service on top Hadoop eco-system then trying to create a database but I got below error and find out a solution as well.



Permission Denied Error in Hive:

FAILED: Execution Error, return code1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive> set hive.auto.convert.join.nonconditional task = false:
hive> create database myhive:
FAILED: Error in metadata: MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied user = hadoop access = WRITE, inode*/user*: hdfs : supergroup : drwxr-rx-r
at org.apache.hadoop.hdfs.server.namenode.FSPErmissionChecker.check(FSPermissionChecker.java:224)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4891)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:669)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java.453)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>

Above error belongs to Permission issue in Hive component:

Resolution:

To resolve the permission issue to the user Hadoop in HDFS. Can you please follow step for the solutions is just change permission to the user using chmod commands:

Step 1: Login to as hduser then execute the below commands one by one.
Step 2: sudo - u hdfs hadoop fs -mkdir /user/hive/warehouse
Step 3: sudo -u hdfs hadoop fs -chmod g+w /tmp
Step 4: sudo -u hdfs hadoop fs -chmod g+w /user/hive/warehouse
Step 5: sudo - u hdfs hadoop fs  -chown -R /user/hive/warehouse
Step 6: sudo chmod 777 /var/lib/hive/metastore
Step 7: cd /var/lib/hive/metastore/metastore_db/
Step 8 :sudo rm *.lck

Summary: I have tried above resolutions then working fine now for above error in Hive.

What is Apache Spark Eco-System | Spark SQL | Spark Streaming | GrapX





What is Apache Spark?
Spark is a fast, easy to use and flexible data processing and in-memory compute framework. It can run on top of Hadoop eco-system, and Cloud accessing diverse data sources including HDFS, HBase, and other services.

Different Key Features of Spark:
1.Fast

2.In General Purpose

3.Scalable

4.Fault-Tolerant

What is the Spark Engine?

Spark engine is for scheduling, distributing and monitoring the large data applications.

What is RDD?

RDD means that Resilient Distribution DataSets. Designed to be fault-tolerant and represents data distributed across the cluster. If node failing is proportional to the number of nodes in a cluster.

RDD supports two operations:
1. Transformations

2.Actions




What is Hive on Spark?

Hive support for Apache Spark, wherein Hive execution is configured to Spark below configurations:

hive > set spark.home=/location/ to /Spark_Home

hive > set hive.execution.engine=spark;

Hive on Spark supports Spark on yarn mode by default

Spark Eco-System

1.Spark SQL – For developing

2.Spark Streaming – For live data streaming

3.GraphX for computing graphs

4.MLib for Machine learning

5.SparkR for Spark engine.

What is Spark SQL?

Spark SQL called as a Shark is a novel module. It introduced that Spark with structured data and processing. Spark executes relational SQL queries on data. The core of the Spark SQL is to supports the RDDs.

What is Spark Streaming?

Apache Spark streaming supports live data processing. It is an extension to the Spark API, allowing stream processing of continuous live data streams. For example data from different sources like HDFS, Flume services are streamed and finally processed to file systems.

What is Spark GraphX?

Spark GraphX means that processing the graphs to build and transform capable graphs. And its component enables programmers to reason about structured data at small.

What is Spark MLib?

Spark MLib is a scalable machine learning library provided by an Apache Spark. It provides easy to understand with algorithms and use different use cases like clustering, filtering, etc.

 

Opera Failed to Uninstall on Windows 10





In Windows 10 operating system automatically installed Opera browser so now I am trying to uninstall the Opera browser on Windows 10

Opera Browser Uninstall on Windows 10

Click on Windows button it showing Opera Browser below :

While uninstalling time I get below error with snapshot for understanding

Opera failed to uninstall: Unable to uninstall Opera.exe.
Please make sure Opera is not running and try again.

Resolutions:

Step 1: Click on the Windows button then search on Opera then right-click on the application. After that select the uninstall option. It will redirect to the Control Panel uninstall the program.

Step2: If it will not get the above option then directly ho with Control panel uninstall programs then choose Opera Browser application to right-click on Uninstall button then select the options. Click on Delete my Opera user data and click on Uninstall button.






Step 3: After clicking on Uninstall button then will get Yes/no window. Choose Yes option will get completely uninstall from Windows 10 operating system. If it is not uninstalled then restart your Windows 10. Start with every step from scratching.