"Large" (1500 line) files are slow with inspections, CPU heats up
PyCharm works well and very smoothly for the most part on my i7-8550U, 16GB, 1TB SSD Ubuntu notebook. This issue is filesize dependent, and I have no idea what's causing it (I don't use any extra plugins, everything is stock as setup when installing PyCharm except for some basic visual theme changes).
On my project, I can open a .py file that's say only 50 lines. If i have:
test = 'Hello World'
print(test)
and replace the 'test' in the print command with 'testt' (or any other incorrect value), it gets underlined instantly as an unresolved reference. Removing the extra character instantly fixes the inspection, no lag, no high CPU usage.
On another python file, opened in the exact same project, I have a 1200 line JSON dictionary indexing a dataset message received from an external source and titling each entry with a name. another ~300 lines for overhead, but nothing heavy there. Basic example below:
json_msg =
{
'start-flag': ext_source[0],
'length-of-package': ext_source[1],
'command-number': ext_source[2],
...
'end-flag': ext_source[1200]
}
Simple stuff. Albeit large for a JSON dict, nothing to cry about I'd think. Thing is, if I change anything, like a single ext_source to external_source (creating an unresolved reference), it takes about 5 seconds for PyCharm to notice the inspection issue and underline it in red, and if I fix it, it takes about 10-25 seconds before it finally notices the variable name is correct again. CPU usage and heat is never an issue otherwise, but when editing this simple 1500 line file it starts to run heavily and heat up my notebook a lot. Especially when typing new info into the file.
Doing the above 'test' --> 'testt' example is also just as slow. If I comment out the 1200 line JSON dict, the file is immediately faster, more responsive, and has no issues with any highlighting, nor is the CPU running high.
Memory indicator has a max heap size of 1902MB, Allocated 682MB, usage between 150-350MB throughout process.
Disabling inspections or going into power saving mode fixes most of the issues, but I really do like the inspections and find them useful. I removed half of the inspection types that I don't use in case that would affect the speed, but no change. Disabling language injections also did nothing. What's the deal? Why can't stock PyCharm handled simple syntax and variable name (among all other types) inspections when the file is of a medium size? The slowness is annoying but not the end of the world, but the CPU usage boosts randomly and heats up more than what is reasonable, which caused me to start exploring for a solution.
Thanks in advance!
Please sign in to leave a comment.
Hello,
Thank you for your detailed description.
For further investigation, please:
1. Start CPU Usage Profiling (https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems)
2. perform the actions to reproduce the performance lack
3. save the snapshot file
4. compile logs folder zipped from ***Help | Compress Logs and Show in…***
5. upload all gathered information to https://uploads.services.jetbrains.com/ and please let me know the folder name, so I will be able to find it.
Uploaded, folder names are:
Just as an update: Any time I'm running PyCharm now my CPU overheats and uses much more resources than normal. Simply having "large" (1500 line) files open seems to make PyCharm run hot. The laggy behavior is most evident in those files, though.
Thank you for your update,
for some reason, I cannot see the mentioned files on FTP. Please try to upload it again and make sure the process finished successfully.
I did so on Friday, re-uploaded and confirmed that it was successful
I have managed to get the uploaded snapshots, however, "compile logs folder zipped from ***Help | Compress Logs and Show in…***" seems to be not included, please upload it as well and let me know.
Thank you!
Just uploaded the zipped logs folder as well
I apologies for not mentioning that the folder name is also required.
Thank you.
pycharm-logs-20191028-144018.zip
Hello,
Thank you for the uploaded logs, I have investigated it and appeared that a number of custom plugins is installed :
ANSI Highlighter (1.2.4), Datalore (0.1.2-192), Ideolog (192.0.12.0)
Please disable it and monitor the behavior, if it appears there is no effect and the performance is still low I would ask you to repeat the steps to collect data for investigation:
1. Start CPU Usage Profiling (https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems)
2. perform the actions to reproduce the performance lack
3. save the snapshot file
4. compile logs folder zipped from ***Help | Compress Logs and Show in…***
5. upload all gathered information to https://uploads.services.jetbrains.com/ and please let me know the folder name, so I will be able to find it.
Removed all third part plugins. I also made a new partition with Manjaro instead of Ubuntu, and as a clean install, Pycharm still was acting slowly. New files uploaded are:
Thank you for the provided information!
I have created a ticket on YouTrack https://youtrack.jetbrains.com/issue/IDEA-226402, please proceed with communication directly with developers there.
Also, it would be great if you could provide the JSON file with 1500 lines, but we are ok if you cannot.
Antonin,
I have uploaded two more snapshots. Both are while doing just some light editing, no third-part plugins, nothing. When Phcharm is not not being used but is active, it behaves normally with 0% to 1% CPU usage. The second i start editing files, even 500 line ones, (not the ~1500 line JSON file), CPU usage starts jumping crazily anywhere from 20% to 55-60%. Moments after I stop editing/typing, back to normal. It has nothing to do with the JSON file it seems. If I use power saving mode then Pycharm doesn't overheat my computer and overuse my CPU, but the limited functionality of powersaver mode is frustrating to work with, but it beat my laptop overheating. Being forced to use power saving mode is unacceptable such a service to be frank.
I do love Pycharm, I really do; and I've grown to know the shortcuts and love the features. But with this continued behavior every time I do anything on Pycharm (my projects aren't even that big compared to some folks!), I'm debating on switching to an alternative like VSCode, where I can just get some python inspection and git extensions and hopefully not have to worry about straining my CPU for 8 hours a day typing simple code.
Other than disabling inspections, what does power saver mode disable? I'd be really surprised and disappointed if inspections cause Pycharm to overwork so much. I'm hoping if there are other things it disables that I don't use anyways I can disable them for good in the settings in hope it solves this once and for all. I'd like try find a fix for this now rather than waiting months or forever for the ticket to be addressed (I feel since this is a single user issue as I can tell, it's fairly low priority and won't be addressed, but I truly can't keep using this setup as is).
The new files are:
Hello, thank you for the uploaded information.
Please proceed with the communication directly here with an assigned developer https://youtrack.jetbrains.com/issue/IDEA-226402 , it was created dedicatedly for your issue.
Hi, I am also experiencing this issue.
Is there a resolution? any additional update on this?
Any news on this? I found PyCharm totally useless because of this issue.
Hi,
Every performance issue should be investigated separately with an individual approach.
Please feel free to report it here https://youtrack.jetbrains.com/issues/PY with logs/snapshots attached, see https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems
I have the same issue as lisharan - my experiences are 100% identical to his.
It seems there are many of us out there, as the notice boards on StackOverflow and Reddit have many posts lamenting the same issues.
Could we not get any feedback on the resolution (or non-resolution) of lisharan's issue. That would help immensely.
Going through the process of logging snapshots and logs seems very onerous, and judging by the number of posts on these boards it seems that many have done so already. There is however no feedback on any of their issues. Is it not possible to get feedback on these issues?
I too love PyCharm, although I recently started using it, and would like to continue using it. The Vim integration is particularly helpful to me, as I love Vim. I will however also need to move to another editor if I cannot resolve this issue, as the time to edit simply takes too long.
Would love to hear your ideas on this.
Thanks.
@Lourens Walters
The last comment from the developer was left with no response from the user https://youtrack.jetbrains.com/issue/PY-38937#focus=Comments-27-3859008.0-0
Please file a new issue and do the following:
1) Clarify if the issue reproduces in a brand new project with a new and clean virtualenv.
2) Attach your logs folder (Help | Collect Logs and Diagnostic Data).
3) Ideally, also provide a CPU snapshot as mentioned above to help us narrow the issue down.
Hi Sergey Karpov
Thank you for your prompt response, much appreciated. I am busy gathering the required information and will file the issue later today.
Regards
Lourens
Lourens Walters, this issue can be easily solved by buying a powerful gaming PC, preferably with 64-core AMD Ryzen Threadripper 3990X :D
lol, I think I will do that in any case - as a Covid lockdown/ Christmas gift to myself. Always good to have some horsepower to throw at problems!
Hi Sergey Karpov,
I managed to create the issue and perform the CPU profiling as requested. PY-45336
Please let me know if you require anything else.
Thank you.
Regards
Lourens
I have the same problem. Switching tabs under / over 1500 lines of code results in ~idle to 500% CPU usage.
About:
PyCharm 2021.3.1 (Community Edition)
Build #PC-213.6461.77, built on December 28, 2021
Runtime version: 11.0.13+7-b1751.21 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Linux 5.13.0-25-generic
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 8
Non-Bundled Plugins:
org.jetbrains.fortran (2021.3)
eu.osimowicz.plugins.intellij.CodeBlocksSorter (0.1.2)
Current Desktop: ubuntu:GNOME
Hello Antti97b,
Please feel free to report it here https://youtrack.jetbrains.com/issues/PY with logs/snapshots attached, see https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems