sbt project import problem

Avatar
Klasch42
Created

Hi 

I have problems importing a Scala+Spark project in IDEA CE 2016.3 on macOS. When refreshing the sbt project IDEA cannot resolve dependencies. Commandine the project build just fine. All dependencies are locally available in Ivy cache.

I stripped down the problem to a very simple setup:

build.sbt:

name := "idea-sbt"

version := "1.0"

scalaVersion := "2.11.8"

libraryDependencies ++= Vector(
  "org.apache.spark"            %% "spark-core"                 % "2.0.2" ,
  "org.apache.avro"             %  "avro"                       % "1.8.1"
)

project/build.properties:

sbt.version = 0.13.13

when refreshing the sbt project in IDEA I get following error:

Error:Error while importing SBT project:<br/>...<br/><pre>[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#run;0.13.13 ...
[info] Resolving org.scala-sbt#task-system;0.13.13 ...
[info] Resolving org.scala-sbt#tasks;0.13.13 ...
[info] Resolving org.scala-sbt#tracking;0.13.13 ...
[info] Resolving org.scala-sbt#cache;0.13.13 ...
[info] Resolving org.scala-sbt#testing;0.13.13 ...
[info] Resolving org.scala-sbt#test-agent;0.13.13 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.13 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.13 ...
[info] Resolving org.scala-sbt#command;0.13.13 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-sbt#logic;0.13.13 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.13 ...
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11;2.0.2: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11;2.0.2: not found
[error] Total time: 2 s, completed 24.01.2017 18:49:31</pre><br/>See complete log in <a href="/Users/c5254131/Library/Logs/IdeaIC2016.3/sbt.last.log">/Users/c5254131/Library/Logs/IdeaIC2016.3/sbt.last.log</a>

Strange thing is: If I comment out the Spark dependency in build.sbt, the refresh works fine.

Any ideas?

 

Klaus

Votes

0

Share

4 comments
Avatar
Justin Kaeser
Created

Could you attach the sbt.last.log from the link in the error message?

I can't reproduce the issue, but I do get warnings that some sources can't be resolved, so one thing to try is going to the sbt preferences in IntelliJ and disabling all the options under "Download:"

0
Avatar
Klasch42
Created

An update to this problem:

Seems like it has to to with our corporate http proxy. I just tried from home without setting the the http.proxy in the sbt VM parameters and then the refresh of the project works as expected.

1
Avatar
Klasch42
Created

Here is the sbt.last.log from the run with errors:

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
[info] Loading project definition from /Users/c5254131/git/idea-sbt/project
[info] Set current project to idea-sbt (in build file:/Users/c5254131/git/idea-sbt/)
> [info] Defining *:shellPrompt
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to idea-sbt (in build file:/Users/c5254131/git/idea-sbt/)
[info] Defining */*:sbtStructureOutputFile
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to idea-sbt (in build file:/Users/c5254131/git/idea-sbt/)
[info] Defining */*:sbtStructureOptions
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to idea-sbt (in build file:/Users/c5254131/git/idea-sbt/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from /Users/c5254131/Library/Application Support/IdeaIC2016.3/Scala/launcher/sbt-structure-0.13.jar
[info] Reapplying settings...
[info] Set current project to idea-sbt (in build file:/Users/c5254131/git/idea-sbt/)
[info] Updating {file:/Users/c5254131/git/idea-sbt/}idea-sbt...
[info] Resolving org.scala-lang#scala-library;2.11.8 ...
[info] Resolving org.apache.spark#spark-core_2.11;2.0.0 ...
[info] Resolving org.apache.spark#spark-core_2.11;2.0.0 ...
[warn] module not found: org.apache.spark#spark-core_2.11;2.0.0
[warn] ==== local: tried
[warn] /Users/c5254131/.ivy2/local/org.apache.spark/spark-core_2.11/2.0.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.0.0/spark-core_2.11-2.0.0.pom
[info] Resolving org.apache.avro#avro;1.8.1 ...
[info] Resolving org.codehaus.jackson#jackson-core-asl;1.9.13 ...
[info] Resolving org.codehaus.jackson#jackson-mapper-asl;1.9.13 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.7 ...
[info] Resolving org.xerial.snappy#snappy-java;1.1.1.3 ...
[info] Resolving org.apache.commons#commons-compress;1.8.1 ...
[info] Resolving org.tukaani#xz;1.5 ...
[info] Resolving org.slf4j#slf4j-api;1.7.7 ...
[info] Resolving org.scala-lang#scala-compiler;2.11.8 ...
[info] Resolving org.scala-lang#scala-reflect;2.11.8 ...
[info] Resolving org.scala-lang.modules#scala-xml_2.11;1.0.4 ...
[info] Resolving org.scala-lang.modules#scala-parser-combinators_2.11;1.0.4 ...
[info] Resolving jline#jline;2.12.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.spark#spark-core_2.11;2.0.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn] Note: Unresolved dependencies path:
[warn] org.apache.spark:spark-core_2.11:2.0.0 (/Users/c5254131/git/idea-sbt/build.sbt#L7-11)
[warn] +- default:idea-sbt_2.11:1.0
[info] Resolving org.scala-sbt#sbt;0.13.13 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-sbt#main;0.13.13 ...
[info] Resolving org.scala-sbt#actions;0.13.13 ...
[info] Resolving org.scala-sbt#classpath;0.13.13 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
[info] Resolving org.scala-sbt#interface;0.13.13 ...
[info] Resolving org.scala-sbt#io;0.13.13 ...
[info] Resolving org.scala-sbt#control;0.13.13 ...
[info] Resolving org.scala-sbt#launcher-interface;1.0.0-M1 ...
[info] Resolving org.scala-sbt#completion;0.13.13 ...
[info] Resolving org.scala-sbt#collections;0.13.13 ...
[info] Resolving jline#jline;2.13 ...
[info] Resolving org.fusesource.jansi#jansi;1.11 ...
[info] Resolving org.scala-sbt#api;0.13.13 ...
[info] Resolving org.scala-sbt#classfile;0.13.13 ...
[info] Resolving org.scala-sbt#logging;0.13.13 ...
[info] Resolving org.scala-sbt#process;0.13.13 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.13 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.13 ...
[info] Resolving org.scala-sbt#relation;0.13.13 ...
[info] Resolving org.scala-sbt#compile;0.13.13 ...
[info] Resolving org.scala-sbt#persist;0.13.13 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.13 ...
[info] Resolving org.scala-sbt#ivy;0.13.13 ...
[info] Resolving org.scala-sbt#cross;0.13.13 ...
[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-2cf13e211b2cb31f0d3b317289dca70eca3362f6 ...
[info] Resolving com.jcraft#jsch;0.1.50 ...
[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...
[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...
[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#run;0.13.13 ...
[info] Resolving org.scala-sbt#task-system;0.13.13 ...
[info] Resolving org.scala-sbt#tasks;0.13.13 ...
[info] Resolving org.scala-sbt#tracking;0.13.13 ...
[info] Resolving org.scala-sbt#cache;0.13.13 ...
[info] Resolving org.scala-sbt#testing;0.13.13 ...
[info] Resolving org.scala-sbt#test-agent;0.13.13 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.13 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.13 ...
[info] Resolving org.scala-sbt#command;0.13.13 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-sbt#logic;0.13.13 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.13 ...
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11;2.0.0: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11;2.0.0: not found
[error] Total time: 3 s, completed 24.01.2017 18:54:54

0
Avatar
Justin Kaeser
Created

Thanks for the update! Indeed, IDEA doesn't currently pass the correct proxy settings to sbt. Here's the open issue: https://youtrack.jetbrains.com/issue/SCL-9856

0

Please sign in to leave a comment.