Anything with SSH Changed Recently, Specifically Agent Forwarding?
Previously I was able to run a python script on a remote server that connected to remote switch and did "things". For some reason this is no longer working. It appears the agent is not being forwarded to my remote server. Specifically the "SSH_AUTH_SOCK" var is not being set during the execution of the remote ssh interpreter.
For clarity:
[local-host] --> [remote-host] --> [network device]
I do develop of network scripting stuff on the "remote-host". It has access to the "network device". If I normally, from the command line, ssh into the "remote host" I can then ssh into the "network device". I've been using pycharm to successfully develop in this environment for a long while. I can "Run" my python scripts on the "remote host" through pycharm and they connect to the "network device" just fine. Everything works great.
For some reason this has stopped working. I notice that when I ssh from the cli into network device I can run "ssh-add -L" on the "remote host" and see all my identities. However when done through pycharm's remote interpreter its not even connecting to the authentication agent. If I try and run "ssh-add -L" using subprocess.Popen() python reports back that it cannot connect to the authentication agent.
If I print out os.environ I don't see the "SSH_AUTH_SOCK" being set.
So, normal ssh works fine, through pycharm it does not. I've tried this in two completely separate environments. The only thing common is my local MacOS laptop and pycharm. But with ssh working from my laptop and pycharm not...it seems this may be an issue with pycharm or its interpretation of some subsystem that I'm unable to trace down.
It's been a couple weeks since I worked on these tools so I don't know if Pycharm has been updated in that time. It's largely likely that this isn't specifically a pycharm issue, but something else. I need some help in figuring out what though since my dev workflow is now no longer working.
Happy to post any additional output, troubleshooting, logs, whatever, just let me know what's needed.
Please sign in to leave a comment.
Updates:
- Tried reverting to 2020.1.2 and 2020.1.1 and behavior did not change so not likely tied to an update.
- Tried a scorched earth approach:
Re-installed pycharm, setup remote server connection, using OpenSSH and Agent. Still have the same issue. So...I dunno
Turning on some debug stuff It looks like pycharm is picking up the right stuff:
However if I start a Python Console on the remote host (from within pycharm)
There's no "SSH_AUTH_SOCK", not sure why it's not able to work with the agent to forward keys.
I have the exact same issue. Suddenly cannot debug code in a professional setting. Major blocker.
Any update on this? I am facing the same issue.
Hello,
First of all, please vote for the corresponding feature request https://youtrack.jetbrains.com/issue/PY-45347 in order to increase its priority.
As a possible workaround, please check the SSH_AUTH_SOCK in terminal and add it to Run/Debug Configuration environment variable section manually.
Another workaround is to set the correct value for $SSH_AUTH_SOCK in a way so that it would be seen by all applications started from your Desktop session. For example, see this discussion: https://stackoverflow.com/questions/135688/setting-environment-variables-on-os-x
There is no workaround for the case where SSH_AUTH_SOCK changes dynamically, unfortunately.
I'm not sure the feature request captures the issue.
The issue appears to be PyCharm does not seem to honour ForwardAgent setting in SSH config.
Thus an available agent on my dev box is used to authenticate my SSH Interpreter, however the SSH agent is NOT passed to the SSH interpreter session - which is crucial. Thus when I try to do a second SSH-hop from there it fails:
dev box [PyCharm] -> remote SSH interpreter -> 3rd box used by my python script
The problem is not the dynamic nature of SSH_AUTH_SOCK on the dev box, this should stay relatively static (in my case) - however Agent Forwarding dictates that a new auth sock is created on EACH run of the remote SSH interpreter *on the remote interpreter box*. This is not happening in PyCharm despite my SSH config specifying "ForwardAgent yes".
If I do this from a regular linux console it works without issue.
To demonstrate the problem I can replace my remote interpreter with a bash script that echos out the value of the auth sock before executing python:
PyCharm log gives us:
As you can see no value is propagated. Now if I run the same thing from a console window:
But from a terminal it's fine:
This value will be different with each run, by design.
I don't consider this a feature request - it's a bug - PyCharm SSH is obeying some (eg Host), but not all the config (eg ForwardAgent) of SSH Client Config, and these exceptions are not explained in the documentation.
Thank you for clarification. I do agree this issue should be treated as bug. Could you please submit a bug report to our issue tracker at https://youtrack.jetbrains.com/issues/py ?
Thanks - this is done:
https://youtrack.jetbrains.com/issue/PY-49182