Pycharm Jupyter notebook remote interpreter: What's the purpose of installing packages in PyCharm?

I'm using PyCharm and connecting it a remote server...

I have pyspark also installed on my machine and the following code works....

However, you notice that PyCharm is asking to isntall the Jupyter package and that it can't resolve the pyspark module.

When I use a managed server, it makes sense that you install packages through PyCharm so that your managed server has all the appropriate packages you need.  However, when using an "external" or "configured server", what are the packages for?  I noticed that the "findspark" module couldn't be resolved and I clicked the "Install package findspark" hover option.  It installed and PyCharm could then resolve it.  What is it doing when I install a package in PyCharm that has already been installed on the external server?  For the findspark package, it doesn't really matter, it is a very small package.  Pyspark on the other hand isn't.  Do I really need to install a 250 MB package into every project just to resolve modules in PyCharm and get autocomplete working?  I feel like I'm missing something (and I'm 93.2% sure it is due to my ignorance).

Thanks for the help.  Here is my PyCharm installation details. 

 

PyCharm 2019.2.1 (Professional Edition)
Build #PY-192.6262.63, built on August 21, 2019
Licensed to PyCharm Evaluator
Expiration date: September 26, 2019
Runtime version: 11.0.3+12-b304.39 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep
Memory: 1981M
Cores: 8
Registry:
Non-Bundled Plugins:

3 comments
Comment actions Permalink

Hi, I believe you're not missing anything. When working with remote Jupyter server, PyCharm has no way to know which packages are installed there and can't resolve them unless you also install them locally. 

Here's the issue for "Jupyter package is not installed" message: https://youtrack.jetbrains.com/issue/PY-36487

And here's the general ticket to improve workflow with remote Jupyter, which should also resolve the issue with packages lookup and resolution.

0
Comment actions Permalink

Thanks for the answer Andrey - The first link you posted doesn't seem to work though and is there supposed to be a second link to the "general ticket"?

0
Comment actions Permalink

Sorry about that. The first ticket appears to be internal. Here's the link to the other one: https://youtrack.jetbrains.com/issue/PY-36323

Basically it covers alot of issues with remote Jupyter including the one you've mentioned.

0

Please sign in to leave a comment.