How to customize sbt build command especially for Profiles

I need to set up some profiles - as part of building spark.  My problem is:  after selecting "Import project" there is no way to CUSTOMIZE the profile settings. Specifically i need:

    sbt -Pyarn -Phadoop-2.5 -Dhadoop-2.5 assembly

Two things:

  • how to customize the initial project import build process to use the custom command line / profile settings
  • for all subsequent builds - how to ensure the correct sbt command line is used

Six weeks ago I had already asked about this on SOF. Here is the question:


Hi! We support a subset of sbt launcher script options because we call sbt-launcher.jar without any script, but I do not see '-P' option amongst them. Is this option standard or is it from some custom sbt launcher? Could you please tell me where I can find information about this option and profiles?


This option is used by apache spark . Given the HUGE importance of Spark it would make sense to be able to build it in Intellij.  I do not know more about -P than this.  So .. how do we proceed?


OK, I have found this "-P" option.

The only solution for now is to store list of profiles you'd like to use in SBT_MAVEN_PROFILES environment variable. For example, you wrote sbt -Pyarn -Phadoop-2.5 -Dhadoop-2.5 assembly which means that you're going to use yarn and hadoop-2.5 profiles, so you should set SBT_MAVEN_PROFILES="yarn hadoop-2.5" in your .bashrc or something similar. Then you can start IDEA and import the project.

The problem is that Spark uses its own sbt launcher script which is extended with "-P" option. Its source code is here. The one thing this "-P" option do is collecting profiles into environment variable I've mentioned earlier. Another problem is that currently we do not support modification of environment variables in project import dialog. I've submitted issue to our tracker, you can watch its progress here.


Hi this SBT_MAVEN_PROFILES can be set - how? From the .bashrc file?

What is the process to use it?   here is my 'guess':

1. add

     export SBT_MAVEN_PROFILES="-Pyarn -Phadoop-2.4"

   to .bashrc

2. Source .bashrc.   Start IJ from the command line

3. Import the project.

Is that the process?




Yes, these steps are right. As an alternative you could write a wrapper shell script for IDEA. Something like this:

#! /usr/bin/env sh
export SBT_MAVEN_PROFILES="-Pyarn -Phadoop-2.4"


Please sign in to leave a comment.