How to setup Intellij 14 Scala Worksheet to run Spark

I'm trying to create a SparkContext in an Intellij 14 Scala Worksheet.
here are my dependencies

name := "LearnSpark"

version := "1.0"
scalaVersion := "2.11.7"
// for working with Spark API
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.0"

Here is the code i run in the worksheet

import org.apache.spark.{SparkContext, SparkConf}
val conf = new SparkConf().setMaster("local").setAppName("spark-play")
val sc = new SparkContext(conf)


15/08/24 14:01:59 ERROR SparkContext: Error initializing SparkContext.
java.lang.ClassNotFoundException: rg.apache.spark.rpc.akka.AkkaRpcEnvFactory
    at Method)
    at java.lang.ClassLoader.loadClass(
    at sun.misc.Launcher$AppClassLoader.loadClass(
    at java.lang.ClassLoader.loadClass(
    at java.lang.Class.forName0(Native Method)

When I run Spark as standalone app it works fine. For example

Can someone provide some guidance on where I should look to resolve this exception?


Comment actions Permalink

This is a forum for discussing the development of IntelliJ IDEA plugins. Please ask your question in the Scala forum instead.

Comment actions Permalink

Thanks Dmitry,
I have posted question on Scala forum
Do i remove the question from here?


Please sign in to leave a comment.