Latest version 2019.1 - Tomcat server hangs on deployment startup - unable to ping server at localhost
I have searched all over the place and seen this issue reported multiple times under different versions. I am using:
- Mac OS High Sierra 10.13.6
- Tomcat 7.0.65 or 7.0.93 (tried both - same issue)
- Java 8 - installed at: /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home
Upon starting the Tomcat server with IJ, the following show up in the logs...
INFO: Starting ProtocolHandler ["http-bio-8083"]
Mar 27, 2019 10:30:46 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Mar 27, 2019 10:30:46 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 143 ms
[2019-03-27 10:30:54,663] Artifact myappservices:war: Waiting for server connection to start artifact deployment...
Mar 27, 2019 10:30:56 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /Users/mcs_macbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/webapps/manager
Mar 27, 2019 10:30:57 PM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Mar 27, 2019 10:30:57 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deployment of web application directory /Users/mcs_macbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/webapps/manager has finished in 418 ms
[2019-03-27 10:31:30,302] Artifact myappservices.war: Waiting for server connection to start artifact deployment...
If you hit the Refresh Deployment Status button, all you get is: Artifact myappservices.war: Waiting for server connection to start artifact deployment...
It just sits here hanging until I hit the stop button in the IDE's "Application Servers" Window... and upon doing that, a yellow message pops up that says: Unable to ping server at localhost:1999 and it also appears in the idea.log file
In looking at that log file, I see this:
2019-03-27 22:52:25,601 [ 260543] WARN - n.process.BaseOSProcessHandler - Process hasn't generated any output for a long time.
If it's a long-running mostly idle daemon process, consider overriding OSProcessHandler#readerOptions with 'BaseOutputReader.Options.forMostlySilentProcess()' to reduce CPU usage.
Command line: /Users/mcs_macbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/bin/catalina.sh run
java.lang.Throwable: Process creation:
at com.intellij.execution.process.BaseOSProcessHandler.<init>(BaseOSProcessHandler.java:33)
at com.intellij.execution.process.OSProcessHandler.<init>(OSProcessHandler.java:89)
at com.intellij.javaee.run.execution.LocalJavaeeServerProcessHandler.<init>(LocalJavaeeServerProcessHandler.java:40)
at com.intellij.javaee.run.execution.PatchedLocalState$ScriptBasedLocalJavaeeServerProcessHandler.<init>(PatchedLocalState.java:170)
at com.intellij.javaee.run.execution.PatchedLocalState.startJ2EEProcess(PatchedLocalState.java:94)
at com.intellij.javaee.run.execution.J2EEProcessHandlerWrapper.lambda$new$0(J2EEProcessHandlerWrapper.java:90)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:220)
at com.intellij.util.concurrency.BoundedTaskExecutor.access$100(BoundedTaskExecutor.java:26)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.lambda$run$0(BoundedTaskExecutor.java:198)
at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:224)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.run(BoundedTaskExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-27 22:52:27,111 [ 262053] WARN - erver.JavaeeServerInstanceImpl - Unable to ping server at localhost:1999
For the record, I DO NOT have JAVA_OPTS set anywhere (neither in the IDE, VM args text box is blank) or in the catalina.sh or startup.sh. There is NO setenv.sh found in this version of Tomcat (neither 7.0.65 or an even later version, 7.0.93). I know this issue of JAVA_OPTS fouling up JMX ports has been pointed to on a number of other posts. However, that does not seem to apply here and I am at a loss after fighting with this new set up for almost 8 hours now.
Also, Intellij DEFAULTS the JMX port to 1099... after that failed every single time, I changed it to 1999, hence the value shown in the log messages. It made no difference. I have checked for running Java processes... there are none other than the IDE itself. There is not another instance of the server running.
Any insight would be appreciated.
Thanks
Please sign in to leave a comment.
Could you please check if the same issue occurs with the brand new Tomcat installation that doesn't have any modifications to the configs/logging/startup scripts, etc?
Is that the new problem for 2019.1? Does 2018.3.6 from https://www.jetbrains.com/idea/download/previous.html work without issues?
Do you have spaces in username or any special symbols in the server's path?
@Serge Baranov - Thanks for your response. I have tried using the previous version you suggested. 2018.3.6 , plus a clean Tomcat install this time of an older Tomcat 7.0.65 which works fine on another machine.
Since Tomcat comes with the "manager" app in /webapps, you can see from the logs it goes ahead and starts that up. Then when it gets to my app WAR, it all hangs up...
Mar 28, 2019 12:52:00 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8083"]
Mar 28, 2019 12:52:00 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8009"]
Mar 28, 2019 12:52:00 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 705 ms
Mar 28, 2019 12:52:00 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Mar 28, 2019 12:52:00 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.65
Mar 28, 2019 12:52:00 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8083"]
Mar 28, 2019 12:52:00 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Mar 28, 2019 12:52:00 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 196 ms
Mar 28, 2019 12:52:10 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /Users/mcsmacbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/webapps/manager
Mar 28, 2019 12:52:10 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deployment of web application directory /Users/mcsmacbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/webapps/manager has finished in 287 ms
[2019-03-28 12:52:37,609] Artifact myappservices:war exploded: Waiting for server connection to start artifact deployment...
I waited 6 minutes and clicked the icon to updated deployment status:
[2019-03-28 12:58:37,225] Artifact myappservices:war exploded: Waiting for server connection to start artifact deployment...
Still stuck... I then clicked the red button to stop the server and checked the idea.log file to find this:
2019-03-28 12:54:10,614 [ 438909] WARN - n.process.BaseOSProcessHandler - Process hasn't generated any output for a long time.
If it's a long-running mostly idle daemon process, consider overriding OSProcessHandler#readerOptions with 'BaseOutputReader.Options.forMostlySilentProcess()' to reduce CPU usage.
Command line: /Users/mcsmacbook/ApacheSoftwareFoundation/apache-tomcat-7.0.65/bin/catalina.sh run
java.lang.Throwable: Process creation:
at com.intellij.execution.process.BaseOSProcessHandler.<init>(BaseOSProcessHandler.java:34)
at com.intellij.execution.process.OSProcessHandler.<init>(OSProcessHandler.java:89)
at com.intellij.execution.process.OSProcessHandler.<init>(OSProcessHandler.java:44)
at com.intellij.execution.process.KillableProcessHandler.<init>(KillableProcessHandler.java:49)
at com.intellij.execution.process.ColoredProcessHandler.<init>(ColoredProcessHandler.java:38)
at com.intellij.execution.process.KillableColoredProcessHandler.<init>(KillableColoredProcessHandler.java:38)
at com.intellij.execution.process.KillableColoredProcessHandler.<init>(KillableColoredProcessHandler.java:31)
at com.intellij.javaee.run.localRun.ColoredCommandLineExecutableObject.createProcessHandler(ColoredCommandLineExecutableObject.java:18)
at com.intellij.javaee.run.localRun.CommandLineExecutableObject.createProcessHandler(CommandLineExecutableObject.java:70)
at org.jetbrains.idea.tomcat.TomcatStartupPolicy$ExecutableCreator$1.createProcessHandler(TomcatStartupPolicy.java:230)
at com.intellij.javaee.run.execution.PatchedLocalState.startJ2EEProcess(PatchedLocalState.java:89)
at com.intellij.javaee.run.execution.J2EEProcessHandlerWrapper.lambda$new$0(J2EEProcessHandlerWrapper.java:90)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:227)
at com.intellij.util.concurrency.BoundedTaskExecutor.access$100(BoundedTaskExecutor.java:26)
at com.intellij.util.concurrency.BoundedTaskExecutor$2$1.run(BoundedTaskExecutor.java:200)
at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:229)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.run(BoundedTaskExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2019-03-28 13:00:21,512 [ 809807] WARN - erver.JavaeeServerInstanceImpl - Unable to ping server at localhost:1099
That ping error popped up briefly in a yellow alert box immediately upon hitting stop but disappeared. However, the idea log captured it.
We need the full text output from the Run tool window.
Also add #com.intellij.javaee.oss.server in Help | Debug Log Settings and share the new idea.log.
You can zip and upload all the details privately at https://uploads.services.jetbrains.com/ .
When Tomcat is started, can you verify with `sudo lsof -i -P | grep 1099` that it listens on port 1099? Does you machine have the normal ipv4 configuration or only ipv6?
Thanks... I've uploaded a pair of files (idea.log and the content of console output ) that defaults to /Users/mcsmacbook/tmp based on the option to "save output to file:" on the Edit Server Run Configuration. (not sure I did this right). Filename: mcs130_IJ_TomcatIssue_2018.3.6_tomcat7.0.65.zip
After the start up reaches the point where it hangs w/ that message that says:
Artifact myappservices:war exploded: Waiting for server connection to start artifact deployment...
I did as you said and ran the command 'sudo lsof -i -P | grep 1099' in a Terminal window... NOTHING shows up.
Found one other thing... on the machine that works with the same version of IJ and Tomcat,
that lsof -i -P | grep 1099 command DOES show a line:
java 40849 mcsmacbook 30u IPv6 0x6273f765d8fc1ab9 0t0 TCP *:1099 (LISTEN)
The TCP/IP tab on the Network settings of BOTH Macbooks shows both IPV4 and IPV6 enabled
Configure IPV4 shows Using DHCP
Configure IPV6 shows Automatically
According to the log, JMX port is set twice to different values:
Do you override it via the run configuration or some environment variable like CATALINA_OPTS?
Can you verify with lsof that Tomcat listens on port 8999 instead of 1099? Does it help if you update the Run configuration JMX port to 8999?
OMG... you sir, are a GENIUS...:-)
I have no idea where that 8999 setting is coming from but sure enough,
I checked the command:
sudo lsof -i -P | grep 8999
Got back an entry THIS time...
java 26068 mcsmacbook 28u IPv6 0xc245f730895f4cad 0t0 TCP *:8999 (LISTEN)
when I go and set the JMX Port value to 8999 in the Server tab of the Edit Server Run Configuration under the section: Tomcat Server Settings ..... and restarted the server in the Application Servers window... IT WORKED!!
Many thanks. Now to figure out where 2 values would be coming from. I downloaded a NEW copy of the Tomcat server 7.0.65 from the Apache and did not alter anything. It's just OOTB.
Sincerely appreciate your help in resolving this.
Mark
Try searching for 8999 in your environment:
In the the Run configuration options:
(in the project root). If the configuration is shared, check the XML files under .idea/runConfigurations directory.
See if creating a new run configuration helps (instead of copying/modifying the existing one).
I too was seeing this error however, the issue stated above was not the case for me. The reason I was seeing the error was related to my use of a setenv.bat file in which I set the debug level for Tomcat with:
If you're using Tomcat plugin then make sure that in Tomcat Server configuration "Before Launch" doesn't contain the 'Build:artifcat' option as that's what makes it run every time even when you've done clean install.
@... thanks a lot for your last post :)
I had similar issue and it turned out that setenv.sh caused same issue