Apache Spark Setup(Install) on Windows10 with Pictures





In this article, we will explain the Apache Spark installation on Windows 10 with simple steps by using the “Wintuils.exe” file. Some of the professionals’ Spark installation on Linux but some professionals need to install Spark on Windows 10 for their comfort.

Pre-requisites:

How To Install Apache Spark On Windows 10:

Step 1: Download winutils.exe Windows 10 operating system from git hub below link:

Wintulins from Github

After downloading the Wintulins file and create your own convenient. I put that file into Windows C Drive.

Step 2: Next step update the environment variable, open the “Advanced system setting:

Step 3: Click on “Environment Variables” to update the Hadoop_Home Path

Step 4: In System Variable give Variable Name like “HADOOP_HOME” after that give Variable value like “C:/wintuils” path then click on the “OK” button.

Step 5: Download the Spark latest version zip from Spark official website. Extract the Spark zip file by using WinRar.

Step 6: Then update the  “SPARK_HOME”  Path in the “Environment_Variable”. Then click on the “OK” button and save it everything.

Step 7: Open new Command prompt then type below command:

spark-shell

Congratulations! Spark successfully completed on Windows 10 operating system. In case if you do not get the Spark-shell then restart your machine.




Summary: The above steps are showing how to set up Spark on Windows/ Windows 10 operating systems with simple steps for Spark developers and Hadoop Developers/Admins. Basically Apache Spark no needs to install on top of Hadoop but needs to Hadoop Distributed File System for the large data sets.  Some servers also need to Spark like Sqoop, Hive, and Map Reduce so Hadoop is mandatory. In Big Data environment Spark is used for large data processing with a cache-based access system. In a single node cluster, Spark installation is enough for Spark developers. For Spark installation, Java is pre-requisite because it is developed by Java code. The Spark zip file once downloaded then extract that file and set up the “SPARK_HOME” in the environment variables and update the Path also. Once you update the path then open a new command prompt, type the spark-shell will get spark prompt for developing mode. If you want more comfortable then install IntelliJ IDEA, Eclipse for more import packages, and try to write code simply.



Leave a Reply

Your email address will not be published. Required fields are marked *