Pycharm does not use system PYTHONPATH

In our lab we use "module" to dynamically switch versions of the software needed by different projects / users on different servers.

My current config, executing in .bashrc among other things sets about 10 paths in PYTHONPATH environment variable.

When running python from an SSH session, I get
>> len(sys.path)
76
and e.g. import matplotlib works fine

When Running from PyCharm I get

import sys
print(len(sys.path))
46

and import matplotlib fails.

Looking at the project interpreter configuration *does* list all correct packages, including matplotlib. Only at run time they are not found.

 

The solution to duplicate all settings from PYTHONPATH in the Pycharm IDE via setting interpreter paths as described in the docs
https://www.jetbrains.com/help/pycharm/installing-uninstalling-and-reloading-interpreter-paths.html

does not seem appropriate -- I have multiple projects and I might want to test versus different versions of the software. Not to mention that duplicating the settings that are already available via PYTHONPATH is a redundancy that complicates and confuses things.

 

I found some relatively old posts about related issues, e.g.

https://intellij-support.jetbrains.com/hc/en-us/community/posts/206594065-PyCharm-remote-debugging-doesn-t-respect-PYTHONPATH-settings

Have the current status not changed? 

Please clarify:

1) Does PyCharm completely ignores system PYTHONPATH?

2) Is there any workaround to get PYTHONPATH used without having to copy all paths from there manually? Is it possible to create a python wrapper script like discussed in the linked post that actually works?

 

PyCharm 2018.3.4 (Professional Edition)
Build #PY-183.5429.31, built on January 29, 2019
JRE: 1.8.0_152-release-1343-b26 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.10.5

The remote python interpreter is configured through a deployment configuration

Thanks

2 comments
Comment actions Permalink

Hi,

>Does PyCharm completely ignores system PYTHONPATH?

No, Pycharm use PYTHONPATH as one of the sources to construct sys.path

 

Maybe you run PyCharm from a different environment than the one where you have set PYTHONPATH? I suggest the following test:

1. Open system terminal and source your .bashrc , ensure you have all the paths in PYTHONPATH

2. Run PyCharm from this same terminal session.

Please share the result

-1
Comment actions Permalink

Dear Andrey,

 

I think in you suggestion you missed the point that "The remote python interpreter is configured through a deployment configuration". The settings of the .bashrc do not have effect on the system environment for pycharm which is running on the local machine.

I have now upgraded to the Pycharm 2019.1.4. The problem still persists

This is what I get: all packages are listed here correctly:

But:

When running from pycharm using this interpreter:

Traceback (most recent call last):
File "/home.zam/shekhovt/datagrid/stereo-vol-code/inference3.py", line 11, in <module>
import torch
ModuleNotFoundError: No module named 'torch'

Process finished with exit code 1

 

What I said in the initial post about that the minimal working example to reproduce the problem still applies. I made a test with fewer paths to illustrate the problem now. So this is the test code:

import sys
print(len(sys.path))
import os
#
print('\n'.join(os.environ['PYTHONPATH'].split(':')))
print('\n'.join(sys.path))

This is my remote's PYTHONPATH:

/home.others/eb/easybuild/software/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages:/home.others/eb/easybuild/software/Pillow/5.3.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages:/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/easybuild/python:/home.zam/shekhovt/datagrid/lib/site-packages

That is set automatically by load module, executed from remote .bashrc.

Running the test from SSH session:

 

11

PYTHONPATH:
/home.others/eb/easybuild/software/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages
/home.others/eb/easybuild/software/Pillow/5.3.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/easybuild/python
/home.zam/shekhovt/datagrid/lib/site-packages

sys.path:
/mnt/datagrid/personal/shekhovt/stereo-vol-code
/home.others/eb/easybuild/software/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages
/home.others/eb/easybuild/software/Pillow/5.3.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/easybuild/python
/home.zam/shekhovt/datagrid/lib/site-packages
/home.others/eb/easybuild/software/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/cffi-1.12.3-py3.7-linux-x86_64.egg
/home.others/eb/easybuild/software/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/tqdm-4.33.0-py3.7.egg
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python37.zip
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7/lib-dynload
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7/site-packages

 

Configuring remote interpreter in pycharm and running from there:

7

PYTHONPATH:
/mnt/home.stud/shekhovt/.pycharm_helpers/pycharm_matplotlib_backend
/mnt/home.stud/shekhovt/.pycharm_helpers/pycharm_display

sys.path:
/mnt/datagrid/personal/shekhovt/stereo-vol-code
/mnt/home.stud/shekhovt/.pycharm_helpers/pycharm_display
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python37.zip
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7/lib-dynload
/home.others/eb/easybuild/software/Python/3.7.2-fosscuda-2019a/lib/python3.7/site-packages
/mnt/home.stud/shekhovt/.pycharm_helpers/pycharm_matplotlib_backend

 

At the same time, I see the interpreter paths are automatically populated like this, which seems correct,

But somehow this does not have effect??

0

Please sign in to leave a comment.