Override pom.xml dependency with local module source

Given a project consisting of two Maven-based modules, foo-common and foo-server, how can I make foo-server resolve classes based on the locally modified foo-common rather than the foo-common dependency declared in its pom? I want to be able to make refactorings across foo-server and foo-common locally, then when I'm ready to commit the code, release foo-common, and update foo-server's pom to point to the new foo-common version before releasing it as well. I assume this is a typical pattern, but I can't quite figure out how to implement it properly.

I tried going to Poject structure -> foo-server -> Dependencies, then clicking the plus sign to add a module dependency on the local foo-common. This worked, but periodically the dependency disappears from foo-server's configuration, and I have to add it again. Possibly this is happening when I make updates to third-party dependencies in foo-server's pom.

9 comments
Comment actions Permalink

A workaround that actually does work is to literally make foo-server dependendent on the "placeholder" version of the dependency, e.g.:

        <dependency>
            <groupId>org.foo.bar</groupId>
            <artifactId>foo-common</artifactId>
            <version>${env.generatedVersion}</version>
        </dependency>

This means the workflow above needs one extra step for each dependency:

  • Release foo-common, and observe the version that the build server generated for it, e.g. 1.0.2.
  • Update the version in the entry for the foo-common dependency in foo-server.
  • Release foo-server.
  • Temporarily change the entry for the foo-common dependency in foo-server to the placeholder variable so foo-server will be able to see local foo-common changes again.


One way to sum all this up is that IDEA apparently wants me to set all the versions and version dependencies to the actual values I expect them to be the next time the code is released. But this causes a chicken-and-egg problem because the build server assigns the version as each module is released. I can't manually update the next version in advance by "adding one" to the previous version because the version qualifier (the part after the dash) is assigned from the source control change number (which, I think, is not an uncommon practice), e.g.:

1.0.3-xxx

This is why I propose that IDEA should not look at the versions of dependencies that exist within the project (or at least have an option not to). It serves no useful purpose, and it actually complicates continuous delivery workflows such as the above.

3
Comment actions Permalink

Hello.

Yes, it's correct that importing or re-importing of pom.xml overrides the settings (made in Project Structure / the module / Dependencies) of the module based on that pom. That's by design.

I have not got the idea, why dependencies in pom.xml are not good for your task. Say, you've got updated code in server and common. So, server uses updated version of source code of common. Why not to change the common version in the dependency?

In other words: how developers using Maven do it without IDE?

Thank you,
Alexander.

0
Comment actions Permalink

Hi Alexander. Thanks for the quick response.

The last time I used IDEA was before Maven was in widespread use, so I might be missing something about how to correctly configure an IDEA project consisting of multiple Maven-based modules.

The issue has to do with being able to treat all the source code for the modules in a particular project as one unit for purposes of working on the code until I have completed some piece of functionality that's ready to be committed to the version control system (and released to the continuous integration environment). In other words, I want to do what anyone would do with an IDEA project, which is to perform refactorings that affect the source code in all modules without necessarily thinking about the depedencies or the order in which they must be released to make the final product ready for delivery.

Here's a concrete example:

Suppose I add method2() to FooBase in the foo-common module: In my IDEA environment I expect to be able to call method2() right away from FooChild in the foo-server module. But I can't because I imported foo-common and foo-server as Maven projects, so IDEA thinks foo-server is dependent on foo-common at a fixed binary version declared in the pom, e.g. 1.0.0. Thus, it tells me method2() does not exist when I try to access it from FooChild:

foo-common: FooBase.java

public void method2() - new


foo-server: FooChild.java

method2() - error


An inefficient solution would be to build foo-common every time I make a change that needs to be visible in foo-server, and release the new version to the binary repository, e.g. 1.0.1. Then I could update foo-server's pom in my IDEA project to point to foo-common 1.0.1.

Or I could temporarily change foo-server's pom to point to a snapshot dependency of foo-common. Presumably there is a way to tell idea to build and install the snapshot dependency of foo-common locally every time the IDEA project is built. But that's not really what I'm looking for either. I want the projects to depend on each other at the source level when I'm working on them, so that if I drill into FooBase.method2() from FooChild I don't end up in the class file.

Another solution would be to stop treating these modules as Maven modules in my IDEA project. But this isn't good either because each declares a large number of third-party dependencies in its pom, and I don't want to manually keep an IDEA dependency list in sync with the Maven pom for each module. What's needed is a way to treat local dependencies whose source is part of the same IDEA project differently than other binary dependencies.

IDEA comes close to solving this problem (although it isn't intuitive to configure): As I described in my original post, if I go into the configuration for the foo-server module and add foo-common as a module dependency, IDEA prefers the local changes in foo-common over the foo-common binary declared in the pom. The problem is that, as soon as I make any change to foo-server's pom, the local module dependency on foo-common is removed. I actually have several interdependent modules, so re-adding the configuration is tedious because it's necessarily to press the "up" button after adding each local dependency until it is moved above its binary dependency in the list.

I think what's needed is something similar to Eclipse's scheme for explicitly adding a local dependency that overrides a pom dependency. I believe it is intended to solve this very problem. When I recently switched back to IDEA I was certain a similar feature must exist since the situation I am describing is common for Maven developers, but so far I have not found it.

0
Comment actions Permalink

Thank you for the explanation.

Seems that I observe different behavior in the following case:

> Or I could temporarily change  foo-server's pom to point to a snapshot dependency of foo-common.
> Presumably there is a way to tell idea to build and install the snapshot  dependency of foo-common locally every time the IDEA project is built.
> But that's not really what I'm looking for either. I want the projects  to depend on each other at the source level when I'm working on them,
> so  that if I drill into FooBase.method2() from FooChild I don't end up in  the class file.

For me, if common module, in project (not in repository) has version X, and server module specifies exactly the same version X in dependency to common, then all references are resolved into the project _sources_, not into repository. IDEA resolves references in the editor, and IDEA compilation is successful. Maven tasks ("compile" and dependent, like "test") fails, however, if X is not installed.

So, you don't "end up in the class file" in this case. Can you confirm or deny this?

Regards,
Alexander.

0
Comment actions Permalink

As a test, I created a project containing foo-server and foo-common, both imported from trivial pom files. I did NOT explicitly make foo-server dependent on foo-common in Project structure->Modules->Dependencies, hoping IDEA would figure this out at the source level since foo-server's pom depends on version 1.0.0 of foo-common, and foo-common's pom is at version 1.0.0.

This worked. When I make changes to foo-common they're immediately visible to foo-server at the source level (without installing the versioned foo-common Maven artifact).

If I change foo-common's version to something else, e.g. 1.0.1, foo-server no longer sees any foo-common changes at the source level. References are resolved via Maven using the binary version I specify.

So it works as you described: Maven dependencies within the same project are resolved automatically at the source level as long as the versions in the poms match. This is actually very nice because it requires no explicit configuration in the IDEA project.

However, my situation has a twist that I failed to mention because I didn't realize Intellij was comparing the versions in the poms. Unfortunately my poms do not contain hard-coded versions. Instead, the version field looks something like this:

<version>${env.generatedVersion}</version>

The version is generated automatically by the build server when the code is committed. Thus, when I'm ready to release a set of changes to foo-server and foo-common, I must first commit foo-common and note the version it was assigned when it was published to the binary repository. I can then put that version in the corresponding dependency element in foo-server and commit those changes.

I suppose this is a somewhat uncommon situation, although it works well for continuous delivery pipelines in which any set of related artifacts could potentially make their way to production (if they pass all the tests at a given set of related versions). But it breaks IDEA's automatic Maven resolution because the version of foo-common specified in foo-server's pom is always the version that was assigned during its last publication, e.g. 3.1.200-123456. But the version in foo-common's pom is just a placeholder like the one above.

I didn't know it at the time, but this is why I had to explitly add a dependency at the IDEA project level between foo-server and foo-common to get things to resolve at the source level. And this actually works until I happen to make any kind of update to foo-server's pom, and IDEA imports it, removing the manually-added dependencies in the process.

The most intuitive way I can think to fix this with a project-wide configuration option would be a couple of checkboxes like the following:

     x Automatically resolve dependencies between Maven modules in the project at the source level.
          _ Require the versions in the poms to match exactly (forces resolution through Maven rather than the source for pom versions that don't match).

0
Comment actions Permalink

Hi Alexander,

Any thoughts on this? I hope I didn't make it sound more complicated than it is. The only difference between the current behavior and what I'm proposing is that there needs to be a way for IDEA to use the source from a local module in the project in preference to the Maven binary, irrespective of the pom versions. I would think this would be the expected behavior by most users.

In other words, as I'm developing the modules in a project, I expect IDEA to think of them as a whole at the source level. It's very clever and useful that the pom dependencies implicitly inform IDEA which modules in the project depend on which other modules in the project (without requiring this to be explicitly expressed in the project configuration). But paying attention to the actual binary versions seems incorrect for the typical use case. I suppose someone might want to override the local source with some particular binary version from Maven, but that seems like a corner case. Not supporting what would seem to be the more typical case of always preferring the local source seems incorrect.

This may be the only thing I can think of that Eclipse handles more logically than IDEA :).

0
Comment actions Permalink

I experimented with this. One possible hack is to go to Settings / Maven / Importing / VM options for importer, and add something like:
-Denv.generatedVersion=123
Where "123" is just a fake value. Then apply "Reimport" to both modules.

With this setting a reference to ${env.generatedVersion} in server pom is red, but source references (like method2() call) are successfully resolved. Also, Maven tasks like "package" seems to be not affected with this fake value.

If I have not miss a detail, this way the references will be Ok in IDEA, while packaging artifacts will go by Maven rules. Maybe it can help.

Regards,
Alexander.

0
Comment actions Permalink

Anyway, I'll show your case to our developer. Then we'll create a feature request or find some logical solution.

Regards,
Alexander.

0
Comment actions Permalink

Thanks for the suggested workaround. Unfortunately I wasn't able to get it work (see attached example).

Just to be clear, foo-server's pom declares that it depends on foo-common at some fixed version (not a variable) at any given point in time, e.g.:

foo-server pom.xml

        <dependency>
            <groupId>org.foo.bar</groupId>
            <artifactId>foo-common</artifactId>
            <version>1.0.1</version>
        </dependency>

Meanwhile, the version that foo-common declares as its own version (not to mention the version foo-server and all the other modules declare as their own versions) is contained in a variable:

foo-common pom.xml

<project>

    <groupId>org.foo.bar</groupId>

    <artifactId>foo-common</artifactId>

    <version>${env.generatedVersion}</version>

</project>

The related "continuous deployment" workflow is:

  • Release foo-common, and observe the version that the build server generated for it, e.g. 1.0.2.
  • Update the version in the entry for the foo-common dependency in foo-server.
  • Release foo-server.


The problem, of course, is that once both modules have been released, as soon as I make a new change to foo-common in the IDEA project, foo-server won't see it because it will only look at the 1.0.2 binary dependency (not the updated local foo-common source).



Attachment(s):
idea-mvn-dep-example.zip
0

Please sign in to leave a comment.