In this article, we will explain how to create a Spark jar file for Scala programming in Eclipse using Maven with simple steps for developers.

Spark Jar file creation in Eclipse using Maven:

Step 1: Open Eclipse IDE, then click on File –> New –> Other

Step 2: Click on “Maven Project” then click on “Next”.

Step 3: Browse the location then click on “Next”.

Step 4: select the maven type version, here selected version 1.1. After that click on the “Next” button.

Step 5: Give your parameters Group Id, Artifact Id for your convenience. Click on the “Finish” button.

Step 6: In this step add Scala nature like below snapshot.

Step 7: After adding the Scala nature, we need to check the hierarchy. Whether Scala is added or not. In case Scala is not there then go to below




Step 8: “Java Build Path” then click on “Add Folder”

Step 9: Then we will get “Source Folder Selection” –> Create New Folder

Step 10: Give the folder name “Scala” after that Add Inclusion Pattern like “**/*.scala” click on “Ok” after that click on “Finish” then Apply and close

Step 11: Then right-click on Scala folder create Scala packages and Scala object

Step 12: In this hierarchy showing Java, Scala libraries then check the  Maven dependencies. Most of the cases by default Junit is there, we need to add Spark library in the pom.xml file.

Step 13: In this step add Spark library dependency in the pom.xml file. Go to Google then search” Maven repository” after that search Spark core with Scala compatible version then simply save it the pom.xml file.

Step 14: Once it is done verify the jar files in Maven Dependencies like below Hadoop, java, Spark related jar files.

Step 15:Then start your simple Spark program on Eclispse the run the Scala application




Step 16: If you want Spark to build jar file then How to build a jar file “Run As” –> “Maven build…” –> “Goals” –> give your’s goals and then “Run” it

Step 17: If you getting like “Build Success” then verify the jar file in the hierarchy like below: