Cannot find pyspark library using Docker in python interpreter

I create a remote docker image and then I create a new interpreter in this way: 

Add new interpreter → On Docker .. → Pull or Use existing 

and everything is ok.

Unfortunately I notice that pyspark library is not shown as a package: 

 I try using the library through the docker image and everything works. 
These are the installaed packages: 

root@30a9f44932ed:/opt/bitnami/spark# pip list
DEPRECATION: Loading egg at /opt/bitnami/python/lib/python3.12/site-packages/pip-23.3.2-py3.12.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Package         Version
--------------- -----------
boto3           1.35.42
botocore        1.35.42
distlib         0.3.8
filelock        3.16.1
jmespath        1.0.1
numpy           2.1.2
pandas          2.2.3
pip             24.2
pip             23.3.2
platformdirs    4.3.6
py4j            0.10.9.7
pyspark         3.5.3
python-dateutil 2.9.0.post0
pytz            2024.2
s3transfer      0.10.3
setuptools      70.3.0
six             1.16.0
tzdata          2024.2
urllib3         2.2.3
 

How is it possible?

Can you help me?

Thanks

0

Hi Basilico1994, I am wondering, which version is in use?

Have you tried File | Invalidate Caches | Invalidate and restart? Does it help by any way?

If not, would you please share logs from Help | Collect Logs and Diagnostic Data for in-depth analysis of the issue? Please upload it to https://uploads.services.jetbrains.com and let me know the upload ID.
 

0

请先登录再写评论。