How to customize sbt build command especially for Profiles
I need to set up some profiles - as part of building spark. My problem is: after selecting "Import project" there is no way to CUSTOMIZE the profile settings. Specifically i need:
sbt -Pyarn -Phadoop-2.5 -Dhadoop-2.5 assembly
Two things:
- how to customize the initial project import build process to use the custom command line / profile settings
- for all subsequent builds - how to ensure the correct sbt command line is used
Six weeks ago I had already asked about this on SOF. Here is the question:
http://stackoverflow.com/questions/28143410/how-to-customize-sbt-build-command-in-intellij
Please sign in to leave a comment.
Hi! We support a subset of sbt launcher script options because we call sbt-launcher.jar without any script, but I do not see '-P' option amongst them. Is this option standard or is it from some custom sbt launcher? Could you please tell me where I can find information about this option and profiles?
This option is used by apache spark . Given the HUGE importance of Spark it would make sense to be able to build it in Intellij. I do not know more about -P than this. So .. how do we proceed?
OK, I have found this "-P" option.
The only solution for now is to store list of profiles you'd like to use in SBT_MAVEN_PROFILES environment variable. For example, you wrote sbt -Pyarn -Phadoop-2.5 -Dhadoop-2.5 assembly which means that you're going to use yarn and hadoop-2.5 profiles, so you should set SBT_MAVEN_PROFILES="yarn hadoop-2.5" in your .bashrc or something similar. Then you can start IDEA and import the project.
The problem is that Spark uses its own sbt launcher script which is extended with "-P" option. Its source code is here. The one thing this "-P" option do is collecting profiles into environment variable I've mentioned earlier. Another problem is that currently we do not support modification of environment variables in project import dialog. I've submitted issue to our tracker, you can watch its progress here.
Hi this SBT_MAVEN_PROFILES can be set - how? From the .bashrc file?
What is the process to use it? here is my 'guess':
1. add
export SBT_MAVEN_PROFILES="-Pyarn -Phadoop-2.4"
to .bashrc
2. Source .bashrc. Start IJ from the command line
3. Import the project.
Is that the process?
thanks
stephenb
Yes, these steps are right. As an alternative you could write a wrapper shell script for IDEA. Something like this:
#! /usr/bin/env sh
export SBT_MAVEN_PROFILES="-Pyarn -Phadoop-2.4"
./PATH/TO/IDEA/INSTALLATION/bin/idea.sh