I have a project that's running in EMR, so there's a number of provided dependencies for hadoop/spark. For cross cloud reasons, we're not depending on the components provided by AWS for builds. Because the AWS components differ slightly from some of the public jars we're depending on with `provided` we have a few minor tweaked jars that we place in sbt `unmanagedBase`. So to summarize it looks like like:
"org.apache.hadoop" % "hadoop-aws" % "2.8.3" % Provided,
unmanagedBase := baseDirectory.value / "scala_lib",
$ ./coursier resolve --tree org.apache.hadoop:hadoop-aws:2.8.3
$ ls scala_lib/
Okay, so the unmanaged version of aws-java-sdk is > the version included in the provided dependency, and the default resolution for this is to go with the higher version. From command line sbt builds all work this way. However in intellij, it looks like the unmanaged dependencies are not being resolved.
I went to project structure -> modules -> dependencies and I checked the list, I do see my scala_lib directory added in the dependencies list under unmanaged dependencies.
So I think that the issue is that conflicting dependency versions here are not being resolved in an expected way.
Can anyone offer me any help here on how to fix this?