sbt run task missing provided libraries

I add a provided library to my project, such as spark.

"org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",


It works fine when I make the project.

But when I run some scala program depends on spark-streaming lib, it gives me following error message:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/StreamingContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2444)
    at java.lang.Class.getMethod0(Class.java:2687)
    at java.lang.Class.getMethod(Class.java:1620)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:125)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.StreamingContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    ... 5 more

I tried to change this dependency to

"org.apache.spark" %% "spark-streaming" % sparkVersion,

It works.

It seems that sbt plugin does not support provided libbrary in Run task ?

My env:

IDEA 14 community edition with Scala plugin.


Thanks!

4 comments
Comment actions Permalink

Provided library should be normally loaded by a container where your app is running (like Tomcat JSP/Servlet container providing JSP/servlet classes).

If you have configured a library as provided, it's now your job to add it to classpath.

0
Comment actions Permalink

Thanks for your reply.

Got it!

0
Comment actions Permalink

Sound like a even more newbie Q, but how do I configure the classpath to do that? 

Also, should this go on SPARK_CLASSPATH ? On ubuntu.

Thanks,

0
Comment actions Permalink

It is perfectly working,

Thanks a lot.

0

Please sign in to leave a comment.