Out of Memory Error while working with large datasets


I am using PyCharm Pro as my editor of choice for working with Python.

I am currently working with Python Pandas, and trying to load very large data files in CSV format. One CSV can have more than 50'000 rows and many cells in it have strings with over 10'000 characters.

I can load it in the console and work on it with Pandas, but the editor becames clunky and slow, and often crashes with "Out of Memory" error.

My specs should be fine, I am running on

AMD Ryzen 3950x processor
32GB 3200mhz RAM

Log files:

Upload id: 2022_09_17_saEizx7s5TWh4kmGemVarU (files: stacktrace.txt and 2 more)



1 comment

Hi, could you please submit a performance issue to https://youtrack.jetbrains.com/issues/py and attach logs, along with the memory snapshot according to https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems  ?


Please sign in to leave a comment.