On Windows, when using conda and conda-build extensively, PyCharm spends all of its time indexing directories making it unusable.
Hi, I am an Anaconda Distribution developer at Continuum Analytics Inc and I'd love to use PyCharm more but I'm running into a total blocker for that.
On Windows, when I point PyCharm at my conda Python, and run (or have previously run lots of) conda-build sessions (so it is writing files to the same file hierarchy as python.exe), the indexer restarts itself all of the time and takes about half an hour to complete and, once there are enough files in the hierarchy, gives up the ghost entirely (the progress bar doesn't move). When that happens, in idea.log I see:
2016-12-13 12:04:34,497 [2615945] INFO - .openapi.util.LowMemoryWatcher - Low memory signal received.
2016-12-13 12:04:38,515 [2619963] INFO - .diagnostic.PerformanceWatcher - High memory usage (free 0 of 494 MB) while dumping threads to C:\Users\ray\.PyCharmCE2016.3\system\log\threadDumps-20161213-112100-PC-163.8233.8\20161213-114923\threadDump-20161213-120438.txt
This may be related to PyCharm using a 32-bit JDK and me having a 24 core machine? Question 1: Can I limit the number of threads that PyCharm will use?
If I get in there before this happens, then using Procmon, I can see PyCharm is indexing the millions of files that you often end up with when using conda and conda-build, for example:
C:\Users\ray\mc-x64-3.5\pkgs\.trash\0d01edc7-38dd-42eb-8cc2-93c9024ff303\_b_env\R\library\IRkernel\help\IRkernel.rdb
..
C:\Users\ray\mc-x64-3.5\conda-bld\r-rjsonio_1481629654036\_b_env\Library\mingw-w64\lib\gcc\x86_64-w64-mingw32\5.3.0\adalib\g-shsh32.ali
..
Question 2: Is/would it be possible to exclude paths by wildcard from the indexer? I want to exclude *\.trash\* and *\conda-bld\* here.
Many thanks for the great IDE.
Please sign in to leave a comment.
> Question 1: Can I limit the number of threads that PyCharm will use?
Unfortunately, it's not possible. Please vote for https://youtrack.jetbrains.com/issue/IDEA-116686 and related issues to increase their priority and be notified about updates.
> Question 2: Is/would it be possible to exclude paths by wildcard from the indexer? I want to exclude *\.trash\* and *\conda-bld\* here.
Yes, you could exclude them in File| Settings| Project structure| Mark Folder as Excluded.
Also please check https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems and feel free to submit new performance problem to PyCharm issue tracker: https://youtrack.jetbrains.com/issues/PY.
> Yes, you could exclude them in File| Settings| Project structure| Mark Folder as Excluded
The problem with this is that these files and folders are *not* part of my project.
My guess is that PyCharm on Windows considers the directory where python.exe lives to be a file hierarchy that it should index, rather than starting at the Lib sub-folder which, when using conda, results in the entire conda prefix being scanned (which is more or less equivalent to an entire computer sysroot). Of course I am only guessing this is why it happens.