Spark / Scala project setup

Hi All,

I am a very beginner trying to build a Spark / Scala project on intellij Idea. This seems quite tedious since a simple program to load a CSV file working on the spark-shell doesn't even compile in Intellij.

I have imported all spark libraries spark-core_2.11, spar-sql_2.11, ...

I surely might have missed it completely since I can't have intellij recognized the method ""?????? It recognizes the first bit spark (org.apache.spark) but can't figure out the rest (read.format). What's going on there?

Can anyone point me to the right direction on how to setup Intellij Idea with a simple spark Scala program?

Jeezzz! this seems so complicated...

Thanks in advance,


Comment actions Permalink
Official comment

Use File / New / Project, then Scala / Activator and search for "Spark" in the list.

"Hello, Apache Spark!" project seems to work just fine.

Comment actions Permalink

Exactly same problem over here. The problem appears with maven/scala/spark project.

Comment actions Permalink

How do you do this in IntelliJ 2018.3?


Please sign in to leave a comment.