Anything with SSH Changed Recently, Specifically Agent Forwarding?

Previously I was able to run a python script on a remote server that connected to remote switch and did "things".   For some reason this is no longer working.  It appears the agent is not being forwarded to my remote server.  Specifically the "SSH_AUTH_SOCK" var is not being set during the execution of the remote ssh interpreter.   

For clarity:

[local-host] --> [remote-host] --> [network device]

I do develop of network scripting stuff on the "remote-host".  It has access to the "network device".   If I normally, from the command line, ssh into the "remote host" I can then ssh into the "network device".   I've been using pycharm to successfully develop in this environment for a long while.  I can "Run" my python scripts on the "remote host" through pycharm and they connect to the "network device" just fine.  Everything works great.

For some reason this has stopped working.  I notice that when I ssh from the cli into network device I can run "ssh-add -L" on the "remote host" and see all my identities.   However when done through pycharm's remote interpreter its not even connecting to the authentication agent.   If I try and run "ssh-add -L" using subprocess.Popen() python reports back that it cannot connect to the authentication agent.

If I print out os.environ I don't see the "SSH_AUTH_SOCK" being set.   

So, normal ssh works fine, through pycharm it does not.   I've tried this in two completely separate environments.  The only thing common is my local MacOS laptop and pycharm.   But with ssh working from my laptop and pycharm seems this may be an issue with pycharm or its interpretation of some subsystem that I'm unable to trace down.

It's been a couple weeks since I worked on these tools so I don't know if Pycharm has been updated in that time.  It's largely likely that this isn't specifically a pycharm issue, but something else.  I need some help in figuring out what though since my dev workflow is now no longer working.


Happy to post any additional output, troubleshooting, logs, whatever, just let me know what's needed.



Comment actions Permalink


- Tried reverting to 2020.1.2 and 2020.1.1 and behavior did not change so not likely tied to an update.

- Tried a scorched earth approach:

Re-installed pycharm, setup remote server connection, using OpenSSH and Agent.   Still have the same issue.   So...I dunno

Comment actions Permalink

Turning on some debug stuff It looks like pycharm is picking up the right stuff:

2020-07-09 15:18:08,932 [ 11751] INFO - z.sshj.transport.TransportImpl - Client identity string: SSH-2.0-SSHJ_0.27.0
2020-07-09 15:18:08,947 [ 11766] INFO - z.sshj.transport.TransportImpl - Server identity string: SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
2020-07-09 15:18:09,087 [ 11906] INFO - om.intellij.ssh.impl.sshj.sshj - Authentication log: SSH connection to chip@netman-new:22
* With overridden host from OpenSSH config:
* Compression methods from config: none
* With signature factories: ssh-rsa, ssh-dss, ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521, ssh-ed25519
* With keep alive interval 120 seconds
* With keep alive count max 3
* With OpenSSH-like host key verifier (knownHostsFile=[/Users/cgwyn/.ssh/known_hosts], strictHostKeyChecking=ASK, hashKnownHosts=false)
* With identity from OpenSSH config {/Users/cgwyn/.ssh/id_rsa.priv.home, without passphrase}
* With 2 identities from ssh-agent[/private/tmp/]
* With authentications allowed by the client: publickey, publickey, publickey, password, keyboard-interactive
* Connected
=> none (failure, new auth methods allowed by the server: publickey, password)
=> publickey /Users/cgwyn/.ssh/id_rsa.priv.home (success)

However if I start a Python Console on the remote host (from within pycharm)

for k, v in os.environ.items():
... if 'SSH' in k:
... print("{} --> {}".format(k, v))
SSH_CONNECTION --> 56753 22
SSH_CLIENT --> 56753 22
SSH_TTY --> /dev/pts/1

There's no "SSH_AUTH_SOCK", not sure why it's not able to work with the agent to forward keys.

Comment actions Permalink

I have the exact same issue. Suddenly cannot debug code in a professional setting. Major blocker.


Please sign in to leave a comment.