Memory on remote server increasing when trying to "view as DataFrame"
Answered
Hi,
I am using pycharm(2020.3 (Professional Edition) Build #PY-203.5981.165)
on windows and remote interperter(Centos 7,16 CPU's 32GB)
When i am trying to "view as DataFrame" my DataFrame(42_282_470,2)(size 4.5 GB),
The memory on the remote server increasing to limit and kernel died
Process finished with exit code 137,
and the IDE is frozen.
It happened to me 6 month ago so i didn't use pyCharm, I have used spyder with spyder kernels and it works well.
Actually, I can't work right now
Please sign in to leave a comment.
Hi,
Is there any chance you could provide a reproducible example?
I tried reproducing with a code and data from https://youtrack.jetbrains.com/issue/PY-27027 and it seems to work fine.
Hi Sergey,
Actually,I dont know how you could reproduce it.
I have loaded the 'NCDB_1999_to_2017.csv' file from the link you provided
size after bad lines removed - (6772563, 23) - 2.15 GB
After clicking on the ">" sign next to the data frame variable
The memory increase to 10.7 and the IDE freezed for 2 minutes~
After the IDE back to life the memory decrease to 7.2
it's not all, there is no stable behavior with the IDE, 1 cpu on the remote server always running and the memory always goes up and down by 1 GB.
Really frustrating.
Thanks.
I filed an issue to our tracker https://youtrack.jetbrains.com/issue/PY-46066, please follow it for further progress.
Probably the developer will have some ideas.
Hi Sergey,
Any updates?
I am not using the IDE because of it, I think is better ti cancel the subscription.
Thanks