Intellij 2017.1.5 and gradle 4.1 milestone upgrade broke this project
I have this project https://github.com/deanhiller/webpiecesexample-all
which is pretty small. Recently, I upgraded from 2.4 to 4.1 so I might be able to play with jdk9 a bit. Anyways, the ugprade works for eclipse and gradle itself so I am recently forced to use eclipse again rather than intellij. It would be nice to be able to use intellij though.
The issue is there are html files co-located with *.java controllers in src/main/java. I am looking for a workaround and in the IDE, it seems when I try to add src/main/java a second time (even if I want it to load *.html resources and *.json), it says can't have duplicate sources.
These lines are now in the build.gradle and used to force the gradle plugin in intellij to import the gradle project correct.....
//For debugging in Intellij IDE only(for tests and server), we need to put the html files on the intellij classpath
//which are co-located with the controllers. NOTE: gradle nor eclipse need this test/resources code here
//BUT apparently, Intellij is broken now with the latest gradle update
//This used to be purely for intellij but this no longer works either(Intellij is broken :( )
//test {
// resources {
// //This ADDS src/main/java/**/*.html and we still read in src/test/resources/**
// srcDirs = ["src/main/java", "src/test/resources"]
// excludes = ["**/*.java"]
// }
//}
Any ideas how to work around this? To reproduce is more steps then I would like
1. 'git clone https://github.com/deanhiller/webpiecesexample-all.git'
2. modify compiler options to be "-parameters" and "Rebuild Project"
* Open Preferences, expand "Build, Execution, and Deployment",
* expand 'Compiler', and click on 'Java Compiler'. Add -parameters to the 'Additional Command Line Parameters'
* Click Ok to close dialogue
* Click Build menu and click Rebuild Project
3. Run all the tests and they fail with can't find *.html file (gradle/eclipse work fine here)
I guess the short issue is while gradle/eclipse work, in intellij the resources in src/main/java are not on the classpath. The tests have to use an on-demand html to groovy to *.class file compiler so users do not need to run a build on every html file change to run tests in the IDE(which would be pretty annoying).
Anyways, looking for a fix so we do not need to permanently axe Intellij use.
Please sign in to leave a comment.
Is there some change in the Gradle logic that now processes resources in the source roots? As far as I remember, you always had to modify Gradle configuration so that it processes resources in the source roots and copies it to the classpath.
I wanted to try your sample project, but it doesn't build in the command line.
By the way, please use the release Gradle versions, unreleased versions may have issues. There were also some fixes for Gradle 4.x support in the upcoming 2017.2 IntelliJ IDEA version: https://www.jetbrains.com/idea/nextversion/. Give it a try.
If the issue is still reproducible, the proper place to report it would be YouTrack: https://youtrack.jetbrains.com/issues/IDEA. Make sure to provide the sample project that works from the command line, but not from IntelliJ IDEA.
ooops, I forgot to mention, you have to be on branch "branchWithVersion" after the git clone so it uses a real released version instead of the development version.
The tests fail with the following now:
When you say "tests fail with the following now", are you referring to the gradle build or the intellij one? (ie. I did not get that in either :( ). Do you somehow reconfigure your 'user.dir' property or the directory where tests are run OR perhaps intellij changed the location which would definitely break that code...hmmm, but I did not see the same issue.
There is this method which results in what you got and I modified it and pushed it with the 'full path' that you are in...(as I am not sure why you go there). As you can see in the method and comment below, I took a painstakingly long time to test every scenario in eclipse and intellij....
I hate to ask, but could you update, run and send me that error?
in the meantime, I can downgrade gradle to 4.0 official release and get a working version(except for that weird path issue you are hitting...not sure I can repro that).
/**
* I like things to work seamlessly but user.dir is a huge issue in multiple environments I am working in.
* main server has to work in N configurations that should be tested and intellij is a PITA since
* it is inconsistent. PP means runs in the project the file is in and eclipse is consistent with
* gradle while intellij is only half the time....
*
* app dev - The environment when you generate a project and import it into an IDE
* webpieces - The environment where you test the template directly when bringing in webpieces into an IDE
*
*
* * app dev / eclipse -
* * PP - running myapp/src/tests - user.dir=myapp-all/myapp
* * PP - DevServer - user.dir=myapp-all/myapp-dev
* * PP - SemiProductionServer - user.dir=myapp-all/myapp-dev
* * PP - ProdServer - user.dir=myapp-all/myapp
* * app dev / intellij (it's different paths than eclipse :( ). user.dir starts as myapp directory
* * PP - running myapp/src/tests - myapp-all/myapp
* * NO - DevServer - user.dir=myapp-all :( what the hell! different from running tests
* * NO - SemiProductionServer - user.dir=myapp-all
* * NO - ProdServer - user.dir=myapp-all
* * webpieces / eclipse - same as app dev because eclipse is nice in this aspect
* * webpieces / intellij - ANNOYING and completely different. Runs out of webpieces a few levels down from actual subproject
* * PP - tests in webpieces gradle - myapp-all/myapp
* * PP - tests in myapp's gradle run - myapp-all/myapp
* * NO - production - user.dir=from distribution myapp directory which has subdirs bin, lib, config, public
* * Future? - run DevSvr,SemiProdSvr,ProdSvr from gradle?....screw that for now..it's easy to run from IDE so why bother(it may just work though too)
*
* - so in production, the relative paths work from myapp so 'public/' is a valid location for html files resolving to myapp/public
* - in testing, IF we want myapp-all/myapp/src/dist/public involved, it would be best to run from myapp-all/myapp/src/dist so 'public/' is still a valid location
* - in devserver, semiprodserver, and prod server, the same idea follows where myapp-all/myapp/src/dist should be the user.dir!!!
*
* - sooooo, algorithm is this
* - if user.dir=myapp-all, modify user.dir to myapp-all/myapp/src/dist (you are in intellij)
* - else if user.dir=myapp-dev, modify to ../myapp/src/dist
* - else if myapp has directories bin, lib, config, public then do nothing
* - else modify user.dir=myapp to myapp/src/dist
*/
private String modifyUserDirForManyEnvironmentsImpl(String filePath) {
File f = new File(filePath);
String name = f.getName();
if("webpiecesexample-all".equals(name)) {
return new File(filePath, "webpiecesexample/src/dist").getAbsolutePath();
} else if("webpiecesexample-dev".equals(name)) {
File parent = f.getParentFile();
return new File(parent, "webpiecesexample/src/dist").getAbsolutePath();
} else if(!"webpiecesexample".equals(name)) {
if(filePath.endsWith("webpiecesexample/src/dist"))
return filePath; //This occurs when a previous test ran already and set user.dir
else if(filePath.endsWith("webpieces")) //
return filePath+"/webserver/webpiecesServerBuilder/templateProject/webpiecesexample/src/dist";
throw new IllegalStateException("bug, we must have missed an environment="+name);
}
File bin = new File(f, "bin");
File lib = new File(f, "lib");
File config = new File(f, "config");
File publicFile = new File(f, "public");
if(bin.exists() && lib.exists() && config.exists() && publicFile.exists()) {
return filePath;
}
return new File(f, "src/dist").getAbsolutePath();
}
> When you say "tests fail with the following now", are you referring to the gradle build or the intellij one?
Both have the same error.
Error from the command line Gradle and the new version:
thanks much!!! I will have to get a windows box to test on this week then!!! It seems on windows, gradle is using a different working directory when running tests on windows(which seems odd) and I did not account for that :(. I will need to fix that first or at least reproduce it first.
thanks,
Dean
ohhhhh, I had endsWith("blah/blah/blah") instead of endsWith("blah"+File.separator+"blah"+File.separator+"blah"). I fixed and pushed but will need to find a windows box to make sure that works now(terribly sorry for the trouble....I very much prefer having things just work everywhere). Fun with differences in OS.
Some tests still fail, but with the different error:
windows fun times. I really need a windows box now. that one is weird since getCananicalPath is only supposed to get the full path but it was a fix I had to put in for macs that were not finding the directory when done relative until getCanonical was called.....nice OS nuances here, ugh. terribly terribly sorry for the trouble..I will be back once I have fully tested on windows first.
ok, I updated the branch/version/etc. and I downloaded on a windows box to match yours. One must add '-parameters' to the compiler though. Perhaps I can trim this down to a simple example instead?
"gradlew build" passes fine now (still on the git branchWithVersion branch)
Let me see if I can create a smaller example though on github sometime this week otherwise you have to set the -parameters option and it's harder to see clearly.
Sample run in IntelliJ IDEA master version (should also work the same in 2017.2 from https://www.jetbrains.com/idea/nextversion/):
Serge, thanks for testing it out!!!! I am getting a new different result with 2017.2(I think it's just a candidate release) than you which 'I think' may be because you need to clean perhaps?
Do you know about the quirk in loading persistence.xml in many JPA implementations(hibernate being this case)? Just in case, the quirk is that many JPA implementations for performance reason use the class loader to list out all META-INF/persistence.xml files and iterate over them. For each one found, it scans ONLY the classes in that folder where it is found or the jar file where it is found so rather than scanning 100 jars, it scans only the 1 jar with the JPA entities.
Sooooooo, that was a long winded way of saying my persistence.xml file when I run the tests is found in a directory with no *.class files so now my tests fail with this exception
java.lang.RuntimeException: Startup hook=PopulateDatabase failed
at org.webpieces.router.impl.AbstractRouterService.runStartupHook(AbstractRouterService.java:130)
at org.webpieces.router.impl.AbstractRouterService.runStartupHooks(AbstractRouterService.java:119)
at org.webpieces.router.impl.ProdRouterService.lambda$start$0(ProdRouterService.java:49)
at org.webpieces.router.impl.PluginSetup.wireInPluginPoints(PluginSetup.java:61)
at org.webpieces.router.impl.RouteLoader.loadImpl(RouteLoader.java:112)
at org.webpieces.router.impl.RouteLoader.load(RouteLoader.java:73)
at org.webpieces.router.impl.ProdRouterService.start(ProdRouterService.java:49)
at org.webpieces.webserver.impl.WebServerImpl.startAsync(WebServerImpl.java:68)
at org.webpieces.webserver.impl.WebServerImpl.startSync(WebServerImpl.java:54)
at org.webpieces.Server.start(Server.java:234)
at org.webpieces.TestLesson1Json.setUp(TestLesson1Json.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.IllegalArgumentException: No query defined for that name [findAllUsers]
at org.hibernate.internal.AbstractSharedSessionContract.buildQueryFromName(AbstractSharedSessionContract.java:752)
at org.hibernate.internal.AbstractSharedSessionContract.createNamedQuery(AbstractSharedSessionContract.java:730)
at org.hibernate.internal.AbstractSessionImpl.createNamedQuery(AbstractSessionImpl.java:23)
at org.webpieces.base.libs.UserDbo.findAll(UserDbo.java:126)
at org.webpieces.base.PopulateDatabase.createSomeData(PopulateDatabase.java:34)
at org.webpieces.base.PopulateDatabase.start(PopulateDatabase.java:29)
at org.webpieces.router.impl.AbstractRouterService.runStartupHook(AbstractRouterService.java:127)
... 40 more
IDEALLY, since all of a projects resources are going to ONE jar file, It would be nice to really have the output of intellijoutput/classes be all resources/*.class files like it will be in production rather than a divided set of folders which does not match the true behavior in production. ( I am not sure that is even possible though).
I am curious, on my machine, I end up with these files
webpiecesexample-all/webpiecesexample/out/production/resources/META-INF/persistence.xml (wrong location so it will not work!!!)
webpiecesexample-all/webpiecesexample/out/production/classes/ (no META-INFO/persistence.xml found here so hibernate tests now fail)
How is it that you are getting a META-INF/persistence.xml in classes directory?(That is the only way your tests will pass). Can you clean that out and re-run it?
thanks,
Dean
The important part of that post was the last few lines(sorry, I should have said those few lines at the top as they are the critical piece that needs to work for tests to pass).
oh, I should note, gradle has an issue in it's latest release that tells me the following is deprecated....(but is critical for persistence.xml to work)....
//ok, this is the deal here. JPA/hibernate made the decision to look for a persistence.xml file
//and scan for classes with @Entity in the directory OR jar with that xml file only
//maven(and I hate this) a long time ago separated src/main/java and src/main/resources but
//this screws tools in many situations like this one so this puts it back so the output is at
//least put to the same location
sourceSets.main.output.resourcesDir = sourceSets.main.output.classesDir
compileJava.doLast { tasks.processResources.execute() }
Basically, I needed the output of resources to go to the classes directory so the persistence.xml ends up in the same folder as the *.class files.
Perhaps, there is some gradle foo that I can do to tell intellij to copy persistence.xml file over?
I did clean and rebuild in the internal master build, and the tests still pass with "d:\work\attaches\webpiecesexample-all\webpiecesexample\out\production\classes\META-INF\persistence.xml" in the correct location.
However, it fails as described in 2017.2 RC2. So, there is probably some fix pushed for that in master which is not available in 2017.2 yet.
I've asked the responsible developer to comment.
ah, that makes more sense! thanks! Hopefully a new release is out soon or is there a location I can download an even more recent one. I just did this pages download https://www.jetbrains.com/idea/nextversion/
2017.2 RC2 172.3317.53
thanks,
Dean
Not sure why it worked in master for me, but after the second try I got the same problem as you. However, I found that you can use this option as a workaround:
Since 2017.2 IntelliJ IDEA doesn't use gradle output folders, see details at https://youtrack.jetbrains.com/issue/IDEA-175172#comment=27-2281329
You can try the following configuration:
if (System.getProperty('idea.active')) {
// to add src/main/java/**/*.html files for the IntelliJ resource builder
sourceSets {
main {
resources {
srcDirs += ["src/main/java"]
}
}
}
// to have classes and resources in the same output directory
idea {
module {
outputDir file("out/production/classes")
}
}
}
Or delegate the build/run action to Gradle as Serge suggested.
@Vladislav yeah!!!, that rocks, IDE should NOT use the same output folders as the build systems(I hate that when it does). I find people run ./gradlew clean build and then work in their IDE during a 3 minute build. REAL bad idea when they share output folders and you run tests and such from the IDE and classes are missing due to building.
I think that is a big win, but I am curious as to why not put all output from a project into ONE output folder as it will all go to ONE jar file so it is more one to one with the way production will run. (Eclipse does it that way and over the years, they seem to break less projects going that route.....or at least it has seemed more stable to me.....I flip between both IDE's quite a bit). I will play with the code snippet tonight so I can get intellij working again.
@serge, your workaround eventually worked!!!!! so I can at least at that step but I will try the gradle snippet above this week as well. At first, there was a weird exception that seems to have gone away around gradle initialization. I wish I would have captured it, but all is working now
thanks much!!!!
IDEA tries to follow gradle conventions here to have build results similar to gradle build results. Of course it's not possible at edges.
Btw, you can also have all output from all projects under the same parent folder using the following option:
Oh, how to do this now for the test directories? (I am having that issue now as well since we have some test entities in test dir and xml file in src/test/resources and need them to end up in same output directory)
thanks,
Dean