How to setup Intellij 14 Scala Worksheet to run Spark

I'm trying to create a SparkContext in an Intellij 14 Scala Worksheet.
here are my dependencies

name := "LearnSpark"

version := "1.0"
scalaVersion := "2.11.7"
// for working with Spark API
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.0"


Here is the code i run in the worksheet


import org.apache.spark.{SparkContext, SparkConf}
val conf = new SparkConf().setMaster("local").setAppName("spark-play")
val sc = new SparkContext(conf)


error


15/08/24 14:01:59 ERROR SparkContext: Error initializing SparkContext.
java.lang.ClassNotFoundException: rg.apache.spark.rpc.akka.AkkaRpcEnvFactory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)


When I run Spark as standalone app it works fine. For example

Can someone provide some guidance on where I should look to resolve this exception?

Thanks.

2 comments
Comment actions Permalink

This is a forum for discussing the development of IntelliJ IDEA plugins. Please ask your question in the Scala forum instead.

0
Comment actions Permalink

Thanks Dmitry,
I have posted question on Scala forum
Do i remove the question from here?

0

Please sign in to leave a comment.