"Large" (1500 line) files are slow with inspections, CPU heats up

PyCharm works well and very smoothly for the most part on my i7-8550U, 16GB, 1TB SSD Ubuntu notebook. This issue is filesize dependent, and I have no idea what's causing it (I don't use any extra plugins, everything is stock as setup when installing PyCharm except for some basic visual theme changes).

 

On my project, I can open a .py file that's say only 50 lines. If i have:

test = 'Hello World'
print(test)

and replace the 'test' in the print command with 'testt' (or any other incorrect value), it gets underlined instantly as an unresolved reference. Removing the extra character instantly fixes the inspection, no lag, no high CPU usage.

 

On another python file, opened in the exact same project, I have a 1200 line JSON dictionary indexing a dataset message received from an external source and titling each entry with a name. another ~300 lines for overhead, but nothing heavy there. Basic example below:

json_msg = 
{
'start-flag': ext_source[0],
'length-of-package': ext_source[1],
'command-number': ext_source[2],
...
'end-flag': ext_source[1200]
}

Simple stuff. Albeit large for a JSON dict, nothing to cry about I'd think. Thing is, if I change anything, like a single ext_source to external_source (creating an unresolved reference), it takes about 5 seconds for PyCharm to notice the inspection issue and underline it in red, and if I fix it, it takes about 10-25 seconds before it finally notices the variable name is correct again. CPU usage and heat is never an issue otherwise, but when editing this simple 1500 line file it starts to run heavily and heat up my notebook a lot. Especially when typing new info into the file.

Doing the above 'test' --> 'testt' example is also just as slow. If I comment out the 1200 line JSON dict, the file is immediately faster, more responsive, and has no issues with any highlighting, nor is the CPU running high.

Memory indicator has a max heap size of 1902MB, Allocated 682MB, usage between 150-350MB throughout process.

Disabling inspections or going into power saving mode fixes most of the issues, but I really do like the inspections and find them useful. I removed half of the inspection types that I don't use in case that would affect the speed, but no change. Disabling language injections also did nothing. What's the deal? Why can't stock PyCharm handled simple syntax and variable name (among all other types) inspections when the file is of a medium size? The slowness is annoying but not the end of the world, but the CPU usage boosts randomly and heats up more than what is reasonable, which caused me to start exploring for a solution.

Thanks in advance!

 

 

24 comments
Comment actions Permalink

Hello, 

Thank you for your detailed description. 

For further investigation, please: 

1. Start CPU Usage Profiling (https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems)  

2.  perform the actions to reproduce the performance lack

3.  save the snapshot file 

4. compile logs folder zipped from ***Help | Compress Logs and Show in…***

5. upload all gathered information to https://uploads.services.jetbrains.com/  and please let me know the folder name, so I will be able to find it. 

0
Comment actions Permalink

Uploaded, folder names are:

PY-192.6817.19_ivan_16.10.2019_12.03.49.zip (7.1 mb
PY-192.6817.19_ivan_16.10.2019_12.42.30.zip (10.3 mb)

Just as an update: Any time I'm running PyCharm now my CPU overheats and uses much more resources than normal. Simply having "large" (1500 line) files open seems to make PyCharm run hot. The laggy behavior is most evident in those files, though.

0
Comment actions Permalink

Thank you for your update, 

 

for some reason, I cannot see the mentioned files on FTP. Please try to upload it again and make sure the process finished successfully. 

0
Comment actions Permalink

I did so on Friday, re-uploaded and confirmed that it was successful 

0
Comment actions Permalink

I have managed to get the uploaded snapshots, however, "compile logs folder zipped from ***Help | Compress Logs and Show in…***" seems to be not included, please upload it as well and let me know.

 

Thank you!

1
Comment actions Permalink

Just uploaded the zipped logs folder as well

0
Comment actions Permalink

I apologies for not mentioning that the folder name is also required. 

 

Thank you. 

0
Comment actions Permalink

pycharm-logs-20191028-144018.zip

0
Comment actions Permalink

Hello,

Thank you for the uploaded logs, I have investigated it and  appeared that   a number of custom plugins is installed : 

 ANSI Highlighter (1.2.4), Datalore (0.1.2-192), Ideolog (192.0.12.0)

 

Please disable it and monitor the behavior, if  it appears there is no effect and the performance is still low I would ask you to repeat the steps to collect data for investigation: 

1. Start CPU Usage Profiling (https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems)  

2.  perform the actions to reproduce the performance lack

3.  save the snapshot file 

4. compile logs folder zipped from ***Help | Compress Logs and Show in…***

5. upload all gathered information to https://uploads.services.jetbrains.com/  and please let me know the folder name, so I will be able to find it. 

 

0
Comment actions Permalink

Removed all third part plugins. I also made a new partition with Manjaro instead of Ubuntu, and as a clean install, Pycharm still was acting slowly. New files uploaded are:

PY-192.6817.19_ivan_04.11.2019_09.59.16.zip
pycharm-logs-20191104-100015.zip
 
It's a short (2-4 min) run, where all I did is edit text/comment lines, yet my CPU usage was between 25% and 50% while writing simple comments. Goes to 5% when idle otherwise. Changed a variable name again to be something incorrect, changing back had it underlined for 30 ish seconds with CPU at ~50% before it finally fixed itself. Same issues on the new installation on Manjaro. Idle background, 30-50% CPU (or even higher) usage when editing anything.
 
Edit: Setting PyCharm to "Power save mode" calms down everything, and normal CPU usage is seen. CPU usage to goes back to 50% the moment i disable power save mode, and on the system monitor you can directly see that. Hell, you can feel the temp difference too. This is annoying though because of the decreased functionality even when plugged in, like code inspections, which are useful for me, all are disabled just so I don't overwork my PC using PyCharm.
 
1
Comment actions Permalink

Thank you for the provided information!

 

I have created a ticket on YouTrack https://youtrack.jetbrains.com/issue/IDEA-226402, please proceed with communication directly with developers there. 

 

Also, it would be great if you could provide the JSON file with 1500 lines, but we are ok if you cannot. 

0
Comment actions Permalink

Antonin,

I have uploaded two more snapshots. Both are while doing just some light editing, no third-part plugins, nothing. When Phcharm is not not being used but is active, it behaves normally with 0% to 1% CPU usage. The second i start editing files, even 500 line ones, (not the ~1500 line JSON file), CPU usage starts jumping crazily anywhere from 20% to 55-60%. Moments after I stop editing/typing, back to normal. It has nothing to do with the JSON file it seems. If I use power saving mode then Pycharm doesn't overheat my computer and overuse my CPU, but the limited functionality of powersaver mode is frustrating to work with, but it beat my laptop overheating. Being forced to use power saving mode is unacceptable such a service to be frank.

I do love Pycharm, I really do; and I've grown to know the shortcuts and love the features. But with this continued behavior every time I do anything on Pycharm (my projects aren't even that big compared to some folks!), I'm debating on switching to an alternative like VSCode, where I can just get some python inspection and git extensions and hopefully not have to worry about straining my CPU for 8 hours a day typing simple code.

Other than disabling inspections, what does power saver mode disable? I'd be really surprised and disappointed if inspections cause Pycharm to overwork so much. I'm hoping if there are other things it disables that I don't use anyways I can disable them for good in the settings in hope it solves this once and for all. I'd like try find a fix for this now rather than waiting months or forever for the ticket to be addressed (I feel since this is a single user issue as I can tell, it's fairly low priority and won't be addressed, but I truly can't keep using this setup as is).

 

The new files are:

PY-193.5233.109_ivan_04.12.2019_15.07.06.zip (8.6 mb)
PY-193.5233.109_ivan_06.12.2019_11.11.19.zip (5.5 mb)
pycharm-logs-20191206-111218.zip (137 kb)
1
Comment actions Permalink

Hello, thank you for the uploaded information. 

 

Please proceed with the communication directly here with an assigned developer https://youtrack.jetbrains.com/issue/IDEA-226402 , it was created dedicatedly for your issue. 

 

 

0
Comment actions Permalink

Hi, I am also experiencing this issue.

Is there a resolution? any additional update on this?

1
Comment actions Permalink

Any news on this? I found PyCharm totally useless because of this issue.

1
Comment actions Permalink

Hi, 

 

Every performance issue should be investigated separately with an individual approach. 

 

Please feel free to report it here https://youtrack.jetbrains.com/issues/PY  with logs/snapshots  attached, see https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems

-1
Comment actions Permalink

I have the same issue as lisharan - my experiences are 100% identical to his. 

It seems there are many of us out there, as the notice boards on StackOverflow and Reddit have many posts lamenting the same issues. 

Could we not get any feedback on the resolution (or non-resolution) of lisharan's issue. That would help immensely. 

Going through the process of logging snapshots and logs seems very onerous, and judging by the number of posts on these boards it seems that many  have done so already. There is however no feedback on any of their issues. Is it not possible to get feedback on these issues?

I too love PyCharm, although I recently started using it, and would like to continue using it. The Vim integration is particularly helpful to me, as I love Vim. I will however also need to move to another editor if I cannot resolve this issue, as the time to edit simply takes too long. 

Would love to hear your ideas on this. 

Thanks. 

 

1
Comment actions Permalink

@Lourens Walters

The last comment from the developer was left with no response from the user https://youtrack.jetbrains.com/issue/PY-38937#focus=Comments-27-3859008.0-0

Please file a new issue and do the following:

1) Clarify if the issue reproduces in a brand new project with a new and clean virtualenv.

2) Attach your logs folder (Help | Collect Logs and Diagnostic Data).

3) Ideally, also provide a CPU snapshot as mentioned above to help us narrow the issue down.

0
Comment actions Permalink

Hi Sergey Karpov

Thank you for your prompt response, much appreciated. I am busy gathering the required information and will file the issue later today. 

Regards

Lourens

1
Comment actions Permalink

Lourens Walters, this issue can be easily solved by buying a powerful gaming PC, preferably with 64-core AMD Ryzen Threadripper 3990X :D

 

0
Comment actions Permalink

lol, I think I will do that in any case - as a Covid lockdown/ Christmas gift to myself. Always good to have some horsepower to throw at problems!

1
Comment actions Permalink

Hi Sergey Karpov

I managed to create the issue and perform the CPU profiling as requested. PY-45336

Please let me know if you require anything else. 

Thank you. 

Regards

Lourens

0
Comment actions Permalink

I have the same problem. Switching tabs under / over 1500 lines of code results in ~idle to 500% CPU usage. 

About:

PyCharm 2021.3.1 (Community Edition)
Build #PC-213.6461.77, built on December 28, 2021
Runtime version: 11.0.13+7-b1751.21 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Linux 5.13.0-25-generic
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 8
Non-Bundled Plugins:
org.jetbrains.fortran (2021.3)
eu.osimowicz.plugins.intellij.CodeBlocksSorter (0.1.2)

Current Desktop: ubuntu:GNOME

0

Please sign in to leave a comment.