Massive thread leak in svn+ssh

Avatar
Permanently deleted user
Created

Since perhaps a few weeks ago, I've noticed severe performance issues and out of swap space java crashes.  Today I figured out what it is: a massive thread leak.  Since opening this project 5 minutes ago, idea is up to 1,427 threads!  The vast majority of them seem to original in com.trilead.ssh2.*.  This working copy has about a dozen svn externals.  Indeed, lsof shows 118 connections to ssh!? Even if I remove the externals properties I still don't see any improvement so it must get cached in the project file somewhere?

Any suggestions would be most appreciated,

-c

9.0.3 (95.429) on MacOS X 10.6.4 and java 1.6.0_20.  Against svn server 1.6.6.


"Thread-902" daemon prio=4 tid=17176c000 nid=0x1a424d000 runnable [1a424c000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
    at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
    at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
    at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
    at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
    at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:676)
    at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:475)
    at java.lang.Thread.run(Thread.java:637)

"Thread-901" daemon prio=4 tid=1716a9000 nid=0x1a3e33000 in Object.wait() [1a3e32000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <11cdf9b00> (a com.trilead.ssh2.channel.Channel)
    at java.lang.Object.wait(Object.java:485)
    at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:987)
    - locked <11cdf9b00> (a com.trilead.ssh2.channel.Channel)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
    at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47)

"Thread-900" daemon prio=4 tid=160191800 nid=0x1a3814000 in Object.wait() [1a3813000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <11a558de0> (a com.trilead.ssh2.channel.Channel)
    at java.lang.Object.wait(Object.java:485)
    at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:987)
    - locked <11a558de0> (a com.trilead.ssh2.channel.Channel)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
    at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47)

"Thread-899" daemon prio=4 tid=160211800 nid=0x1a3711000 in Object.wait() [1a3710000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <127d38090> (a com.trilead.ssh2.channel.Channel)
    at java.lang.Object.wait(Object.java:485)
    at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:987)
    - locked <127d38090> (a com.trilead.ssh2.channel.Channel)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
    at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47)

"Thread-898" daemon prio=4 tid=15e3e1000 nid=0x1a360e000 in Object.wait() [1a360d000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <127d38148> (a com.trilead.ssh2.channel.Channel)
    at java.lang.Object.wait(Object.java:485)
    at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:987)
    - locked <127d38148> (a com.trilead.ssh2.channel.Channel)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
    at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47)

"Thread-897" daemon prio=4 tid=1606c5800 nid=0x1a33ef000 in Object.wait() [1a33ee000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <126ed8150> (a com.trilead.ssh2.channel.Channel)
    at java.lang.Object.wait(Object.java:485)
    at com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:987)
    - locked <126ed8150> (a com.trilead.ssh2.channel.Channel)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)
    at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)
    at com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47)

Votes

0

Share

1 comment
Avatar
Dmitry Jemerov
Created

Hello Christopher,

This problem is fixed in 9.0.4 EAP build: http://confluence.jetbrains.net/display/IDEADEV/Maia+EAP

Since perhaps a few weeks ago, I've noticed severe performance issues
and out of swap space java crashes.  Today I figured out what it is: a
massive thread leak.  Since opening this project 5 minutes ago, idea
is up to 1,427 threads!  The vast majority of them seem to original in
com.trilead.ssh2.*.  This working copy has about a dozen svn
externals.  Indeed, lsof shows 118 connections to ssh!? Even if I
remove the externals properties I still don't see any improvement so
it must get cached in the project file somewhere?

Any suggestions would be most appreciated,

-c

"Thread-902" daemon prio=4 tid=17176c000 nid=0x1a424d000 runnable

java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInp
utStream.java:41)
at
com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherI
nputStream.java:52)
at
com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputS
tream.java:79)
at
com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStrea
m.java:108)
at
com.trilead.ssh2.transport.TransportConnection.receiveMessage(Transpor
tConnection.java:232)
at
com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManag
er.java:676)
at
com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.jav
a:475)
at java.lang.Thread.run(Thread.java:637)
"Thread-901" daemon prio=4 tid=1716a9000 nid=0x1a3e33000 in
Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <11cdf9b00> (a com.trilead.ssh2.channel.Channel)
at java.lang.Object.wait(Object.java:485)
at
com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.
java:987)
- locked <11cdf9b00> (a com.trilead.ssh2.channel.Channel)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:58)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:70)
at
com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47
)
"Thread-900" daemon prio=4 tid=160191800 nid=0x1a3814000 in
Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <11a558de0> (a com.trilead.ssh2.channel.Channel)
at java.lang.Object.wait(Object.java:485)
at
com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.
java:987)
- locked <11a558de0> (a com.trilead.ssh2.channel.Channel)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:58)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:70)
at
com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47
)
"Thread-899" daemon prio=4 tid=160211800 nid=0x1a3711000 in
Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <127d38090> (a com.trilead.ssh2.channel.Channel)
at java.lang.Object.wait(Object.java:485)
at
com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.
java:987)
- locked <127d38090> (a com.trilead.ssh2.channel.Channel)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:58)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:70)
at
com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47
)
"Thread-898" daemon prio=4 tid=15e3e1000 nid=0x1a360e000 in
Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <127d38148> (a com.trilead.ssh2.channel.Channel)
at java.lang.Object.wait(Object.java:485)
at
com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.
java:987)
- locked <127d38148> (a com.trilead.ssh2.channel.Channel)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:58)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:70)
at
com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47
)
"Thread-897" daemon prio=4 tid=1606c5800 nid=0x1a33ef000 in
Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <126ed8150> (a com.trilead.ssh2.channel.Channel)
at java.lang.Object.wait(Object.java:485)
at
com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.
java:987)
- locked <126ed8150> (a com.trilead.ssh2.channel.Channel)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:58)
at
com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.ja
va:70)
at
com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:47
)
---
Original message URL:
http://devnet.jetbrains.net/message/5270422#5270422

--
Dmitry Jemerov
Development Lead
JetBrains, Inc.
http://www.jetbrains.com/
"Develop with Pleasure!"


0

Please sign in to leave a comment.