PyCharm, unable to set up Deployment with a remote server.
Hi, I'm getting this error whenever I hit the `Test SFTP Connection` button. I'm trying to set up Deployment for a Ubuntu machine I have that's running off a server hosted by Digital Ocean using these settings:
```
SFTP Host: ***.***.***.***
Port: 22
Root path: /home/ubuntu/
User name: ubuntu
Auth type: OpenSSH config and authentication agent
Web server root URL: http://***.***.***.***/
```
This is the `idea.log`:
```
2018-10-31 15:41:02,931 [ 93832] DEBUG - t.connections.RemoteConnection - Opening connection to sftp://***.***.***.***:22/ in @7c9de93b
2018-10-31 15:41:02,934 [ 93835] DEBUG - ntellij.ssh.impl.SshConnection - Failed to create SSH channel using existing SSH sessions, creating new SSH session
2018-10-31 15:41:02,934 [ 93835] DEBUG - ntellij.ssh.impl.SshConnection - Establishing new SSH session to ubuntu@***.***.***.***:22
2018-10-31 15:41:02,967 [ 93868] DEBUG - ellij.ssh.SshConnectionService - Creating authentication agent connector
2018-10-31 15:41:02,968 [ 93869] DEBUG - ellij.ssh.SshConnectionService - UNIX-domain socket /private/tmp/com.apple.launchd.HXK3XSYeM2/Listeners defined in environment variable SSH_AUTH_SOCK will be used to connect to authentication agent
2018-10-31 15:41:02,969 [ 93870] DEBUG - intellij.ssh.SshConnectionUtil - Setting up ssh connection to ***.***.***.***:22 for ubuntu, with identityRepositoryFactory com.intellij.ssh.legacy.AuthAgentIdentityRepositoryFactory@8ffb32a, without identities and password, adding identities from OpenSSH config, with socketFactory com.intellij.ssh.sockets.MySftpSocketFactory@0, with keepAliveTimeout 300000, with serverAliveCountMax 1000, with timeout 10000, with proxyCommand /usr/local/bin/krssh %h %p, with daemonThread true
With openSSH config , cipher.c2s=aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes256-cbc, cipher.s2c=aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes256-cbc, compression.c2s=none,zlib@openssh.com,zlib, compression.s2c=none,zlib@openssh.com,zlib
With config StrictHostKeyChecking = ask, PreferredAuthentications = publickey,keyboard-interactive,password,
2018-10-31 15:41:02,969 [ 93870] DEBUG - ins.plugins.webDeployment.sftp - Connecting to ***.***.***.*** port 22
2018-10-31 15:41:02,974 [ 93875] DEBUG - ins.plugins.webDeployment.sftp - Connection established
2018-10-31 15:41:03,035 [ 93936] DEBUG - ins.plugins.webDeployment.sftp - Remote version string: SSH-2.0-OpenSSH_7.6p1 Ubuntu-4
2018-10-31 15:41:03,035 [ 93936] DEBUG - ins.plugins.webDeployment.sftp - Local version string: SSH-2.0-JSCH-0.1.54
2018-10-31 15:41:03,036 [ 93937] DEBUG - ins.plugins.webDeployment.sftp - CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
2018-10-31 15:41:03,037 [ 93938] DEBUG - ins.plugins.webDeployment.sftp - CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - SSH_MSG_KEXINIT sent
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - SSH_MSG_KEXINIT received
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: none,zlib@openssh.com
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server: none,zlib@openssh.com
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server:
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: server:
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: client: aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes256-cbc
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: client: aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes256-cbc
2018-10-31 15:41:03,076 [ 93977] DEBUG - ins.plugins.webDeployment.sftp - kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client: none,zlib@openssh.com,zlib
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client: none,zlib@openssh.com,zlib
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client:
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client:
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: server->client aes256-gcm@openssh.com hmac-sha1 none
2018-10-31 15:41:03,077 [ 93978] DEBUG - ins.plugins.webDeployment.sftp - kex: client->server aes256-gcm@openssh.com hmac-sha1 none
2018-10-31 15:41:03,086 [ 93987] DEBUG - ins.plugins.webDeployment.sftp - SSH_MSG_KEX_ECDH_INIT sent
2018-10-31 15:41:03,086 [ 93987] DEBUG - ins.plugins.webDeployment.sftp - expecting SSH_MSG_KEX_ECDH_REPLY
2018-10-31 15:41:03,103 [ 94004] DEBUG - ins.plugins.webDeployment.sftp - ssh_rsa_verify: signature true
2018-10-31 15:41:03,104 [ 94005] DEBUG - ins.plugins.webDeployment.sftp - Host '***.***.***.***' is known and matches the RSA host key
2018-10-31 15:41:03,104 [ 94005] DEBUG - ins.plugins.webDeployment.sftp - SSH_MSG_NEWKEYS sent
2018-10-31 15:41:03,104 [ 94005] DEBUG - ins.plugins.webDeployment.sftp - SSH_MSG_NEWKEYS received
2018-10-31 15:41:03,104 [ 94005] DEBUG - ins.plugins.webDeployment.sftp - Disconnecting from ***.***.***.*** port 22
2018-10-31 15:41:03,104 [ 94005] WARN - t.connections.RemoteConnection - Retrying connecting to sftp://***.***.***.***:22/ in @7c9de93b: 1 iterations
org.apache.commons.vfs2.FileSystemException: Could not determine the type of file "sftp://***.***.***.***/".
at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1510)
at org.apache.commons.vfs2.provider.AbstractFileObject.isFile(AbstractFileObject.java:1695)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doListChildrenResolved(SftpFileObject.java:462)
at org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1148)
at org.apache.commons.vfs2.provider.AbstractFileObject.getChild(AbstractFileObject.java:1101)
at com.jetbrains.plugins.webDeployment.connections.RemoteConnectionPool.lambda$createConnection$0(RemoteConnectionPool.java:259)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:314)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.commons.vfs2.FileSystemException: Could not connect to SFTP server at "sftp://***.***.***.***/".
at com.jetbrains.plugins.webDeployment.connections.PatchedSftpFileSystem.getChannel(PatchedSftpFileSystem.java:107)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.statSelf(SftpFileObject.java:177)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doGetType(SftpFileObject.java:112)
at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1501)
... 11 more
Caused by: com.intellij.ssh.SshTransportException: java.lang.NullPointerException
at com.intellij.ssh.impl.SshConnection.a(SshConnection.kt:120)
at com.intellij.ssh.impl.SshConnection.openChannel(SshConnection.kt:68)
at com.intellij.ssh.SshConnectionService.sftp(SshConnectionService.kt:182)
at com.jetbrains.plugins.webDeployment.connections.PatchedSftpFileSystem.getChannel(PatchedSftpFileSystem.java:57)
... 14 more
Caused by: com.jcraft.jsch.JSchException: java.lang.NullPointerException
at com.jcraft.jsch.Session.updateKeys(Session.java:1214)
at com.jcraft.jsch.Session.receive_newkeys(Session.java:1098)
at com.jcraft.jsch.Session.connect(Session.java:367)
at com.jcraft.jsch.Session.connect(Session.java:183)
at com.intellij.ssh.impl.SshConnection.b(SshConnection.kt:142)
at com.intellij.ssh.impl.SshConnection.a(SshConnection.kt:109)
... 17 more
Caused by: java.lang.NullPointerException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.jcraft.jsch.Session.updateKeys(Session.java:1155)
... 22 more
2018-10-31 15:41:03,105 [ 94006] WARN - loyment.ui.WebServerConfigForm - Could not determine the type of file "sftp://***.***.***.***/".
org.apache.commons.vfs2.FileSystemException: Could not determine the type of file "sftp://***.***.***.***/".
at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1510)
at org.apache.commons.vfs2.provider.AbstractFileObject.isFile(AbstractFileObject.java:1695)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doListChildrenResolved(SftpFileObject.java:462)
at org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1148)
at org.apache.commons.vfs2.provider.AbstractFileObject.getChild(AbstractFileObject.java:1101)
at com.jetbrains.plugins.webDeployment.connections.RemoteConnectionPool.lambda$createConnection$0(RemoteConnectionPool.java:259)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:314)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.commons.vfs2.FileSystemException: Could not connect to SFTP server at "sftp://***.***.***.***/".
at com.jetbrains.plugins.webDeployment.connections.PatchedSftpFileSystem.getChannel(PatchedSftpFileSystem.java:107)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.statSelf(SftpFileObject.java:177)
at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doGetType(SftpFileObject.java:112)
at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:1501)
... 11 more
Caused by: com.intellij.ssh.SshTransportException: java.lang.NullPointerException
at com.intellij.ssh.impl.SshConnection.a(SshConnection.kt:120)
at com.intellij.ssh.impl.SshConnection.openChannel(SshConnection.kt:68)
at com.intellij.ssh.SshConnectionService.sftp(SshConnectionService.kt:182)
at com.jetbrains.plugins.webDeployment.connections.PatchedSftpFileSystem.getChannel(PatchedSftpFileSystem.java:57)
... 14 more
Caused by: com.jcraft.jsch.JSchException: java.lang.NullPointerException
at com.jcraft.jsch.Session.updateKeys(Session.java:1214)
at com.jcraft.jsch.Session.receive_newkeys(Session.java:1098)
at com.jcraft.jsch.Session.connect(Session.java:367)
at com.jcraft.jsch.Session.connect(Session.java:183)
at com.intellij.ssh.impl.SshConnection.b(SshConnection.kt:142)
at com.intellij.ssh.impl.SshConnection.a(SshConnection.kt:109)
... 17 more
Caused by: java.lang.NullPointerException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.jcraft.jsch.Session.updateKeys(Session.java:1155)
... 22 more
2018-10-31 15:41:05,644 [ 96545] WARN - t.connections.RemoteConnection - Forced closing file system of sftp://***.***.***.***:22/ in @7c9de93b
2018-10-31 15:41:05,644 [ 96545] DEBUG - t.connections.RemoteConnection - File system of sftp://***.***.***.***:22/ in @7c9de93b is closed.
```
Using macOS Mojave Version 10.14, SFTP via terminal works.
Please sign in to leave a comment.
Hi,
Could you please upload the whole log zipped to https://uploads.services.jetbrains.com/ and let me know the name of that zip file?
I'm having a nearly identical issue in PhpStorm Build #PS-183.4588.67 on Mojave, let me know if you need logs. Thanks!
Hi Brian,
Yes, please add the following lines to Help | Debug Log Settings…, reproduce the problem and upload logs folder zipped (Help | Compress Logs and Show in...) to https://uploads.services.jetbrains.com/ and let me know the name of that zip file.
#com.intellij.ssh
#com.jetbrains.plugins.webDeployment
logs-20181211-114126.zip
Thanks!
Are you able to ssh to that server outside of IDE?
Please check that username and password are saved under "Settings | Build... | Deployment ". Try using different auth type, try using IP as a host to connect.
Does Test Connection button work?
I can, as long as I'm connected to the VPN. I can even SSH into the box using PhpStorm's terminal emulator.
I'm connecting via the auth agent, but the username (not pw) is saved. No change when using the IP address. Test connection button fails with the "Could not connect to sftp server at..." message. The log I uploaded should have had all those different scenarios included, with the exception of the different auth type.
Interestingly, when I switch to key/passphrase, it claims my private key file is corrupt or in a bad format (it's RSA, works fine elsewhere). "Invalid privatekey: [B@723754ff"
Do you have any custom algorithms (for Ciphers or KexAlgorithms) declared in ~/.ssh/config SSH client configuration file?
I do, in fact:
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
Ciphers aes256-gcm@openssh.com,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
Seems to be similar to https://youtrack.jetbrains.com/issue/PY-32594 then. Please try this workaround.
That does look similar, yeah. I switched over to the config values of a coworker that has this working, but it still fails in a similar-looking way. It looked like it thought about it for just a little bit longer though. Do you want a fresh set of logs?
In fact, I have found a similar YouTrack bug report:
https://youtrack.jetbrains.com/issue/WI-44100
According to the thread last comments, there is no way to change timeout via GUI but you could alter it by editing JVM settings.
This is the option you may want to try by adding the following line to Help | Edit Custom VM Options... and restarting IDE:
-Dcom.jetbrains.plugins.webDeployment.sftp.timeout=30000
The failure is still quite rapid, within 1 second, but I did up that setting and I'm still seeing the same behavior.
>I switched over to the config values of a coworker that has this working, but it still fails in a similar-looking way
Please remove them completely from the config to make sure no custom algorithms are being used.
Behavior is the same even with a completely empty ~/.ssh/config file
That's odd. May I ask you to try again with empty config and then upload updated logs?
Here you go:
Hm,
Please also check that no KexAlgorithms are declared in system config /etc/ssh/ssh_config
Just comment it out if it's there.
There were a few in there, took those out too. Still no luck. But after I also commented out the Cipher and MAC lines, I was able to connect. Thanks for the help troubleshooting this.
So yeah, unsupported algo seems quite likely. Do you have a list of the ones IntelliJ won't choke on, or is there a patch coming that I can expect?
Great, glad it works now.
>Do you have a list of the ones IntelliJ won't choke on, or is there a patch coming that I can expect?
I will ask our developer if there is such list, but for KexAlgorithms we use the following as default:
ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1
From developer:
We use JSch library, you can refer to http://www.jcraft.com/jsch/
- Key exchange: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
- Cipher: blowfish-cbc,3des-cbc,aes128-cbc,aes192-cbc,aes256-cbc,aes128-ctr,aes192-ctr,aes256-ctr,3des-ctr,arcfour,arcfour128,arcfour256
- MAC: hmac-md5, hmac-sha1, hmac-md5-96, hmac-sha1-96
- Host key type: ssh-dss,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
Brilliant, thanks.
Had a similar issue setting up deployment with an active vpn connection after upgrading to ubuntu 18.04 LTS. I guess there is some conflict with openvpn>=2.4 . Adding "ncp-disable" to my openvpn configuration solved the problem for me.
Hi! I had a couple of perfectly working ssh server connections. This morning I got an error about the connection, delete it and now a cannot set any connection. When trying to connect it gives me the error: "Index 11 out of bounds for length 11". I am able to connect to that server through other method.
As mentioned in the comments before I've zipped the log files and uploaded them: 2020_10_07_Bn9xrPRfbZX7JTwA.
Any suggestions? Thanks
Aliraloarca
Please delete ~/.pycharm_helpers directory from the remote host, then recreate the interpreter and associated deployment and ssh config.
If this doesn't help, please add the following lines to **Help | Diagnostic Tools | Debug Log Settings...**:
#com.intellij.ssh
#com.jetbrains.plugins.webDeployment
(keep the # character)
Reproduce the issue and provide logs again.
I'm getting the same error as Aliraloarca, "Index 11 out of bounds for length 11". I removed the pycharm_helper from the serve, but it didn't help. I also get it connected sometimes, but the folders browsing on the server hanging in "loading" forever.
I uploaded the log file and the upload id is
2020_11_20_7oqnxUFrpZqGvePv
FYI for anyone else who comes across this issue. I encountered the exact same issue, but I am not running any custom SSH algorithms. In ~/.ssh/config I have the following:
Host myhost.example.com
HostName 202.202.202.202
Port 22
User myusername
IdentityFile ~/.ssh/id_rsa
IdentitiesOnly yes
I was able to resolve the issue by adding the following to /etc/hosts
202.202.202.202 myhost.example.com
So, it seems like whatever function is being called by PyCharm, or other IntelliJ tool, is ignoring the "HostName" line from `/.ssh/config and instead doing a DNS lookup using presumably a function call to the TCP/IP stack.
Hope this helps someone.