sbt run task missing provided libraries

I add a provided library to my project, such as spark.

"org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",

It works fine when I make the project.

But when I run some scala program depends on spark-streaming lib, it gives me following error message:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/StreamingContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    at java.lang.Class.getMethod0(
    at java.lang.Class.getMethod(
    at com.intellij.rt.execution.application.AppMain.main(
Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.StreamingContext
    at Method)
    at java.lang.ClassLoader.loadClass(
    at sun.misc.Launcher$AppClassLoader.loadClass(
    at java.lang.ClassLoader.loadClass(
    ... 5 more

I tried to change this dependency to

"org.apache.spark" %% "spark-streaming" % sparkVersion,

It works.

It seems that sbt plugin does not support provided libbrary in Run task ?

My env:

IDEA 14 community edition with Scala plugin.


Comment actions Permalink

Provided library should be normally loaded by a container where your app is running (like Tomcat JSP/Servlet container providing JSP/servlet classes).

If you have configured a library as provided, it's now your job to add it to classpath.

Comment actions Permalink

Thanks for your reply.

Got it!

Comment actions Permalink

Sound like a even more newbie Q, but how do I configure the classpath to do that? 

Also, should this go on SPARK_CLASSPATH ? On ubuntu.


Comment actions Permalink

It is perfectly working,

Thanks a lot.


Please sign in to leave a comment.