I'm using PyCharm and connecting it a remote server...
I have pyspark also installed on my machine and the following code works....
However, you notice that PyCharm is asking to isntall the Jupyter package and that it can't resolve the pyspark module.
When I use a managed server, it makes sense that you install packages through PyCharm so that your managed server has all the appropriate packages you need. However, when using an "external" or "configured server", what are the packages for? I noticed that the "findspark" module couldn't be resolved and I clicked the "Install package findspark" hover option. It installed and PyCharm could then resolve it. What is it doing when I install a package in PyCharm that has already been installed on the external server? For the findspark package, it doesn't really matter, it is a very small package. Pyspark on the other hand isn't. Do I really need to install a 250 MB package into every project just to resolve modules in PyCharm and get autocomplete working? I feel like I'm missing something (and I'm 93.2% sure it is due to my ignorance).
Thanks for the help. Here is my PyCharm installation details.
PyCharm 2019.2.1 (Professional Edition)
Build #PY-192.6262.63, built on August 21, 2019
Licensed to PyCharm Evaluator
Expiration date: September 26, 2019
Runtime version: 11.0.3+12-b304.39 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep