High Memory Usage
I have a new MacBook pro that actually has 64 GB of RAM. However, I'm still running into memory issues. My project is huge (>50K lines of code).
First I'd like to ask a clarifying question. My project is built with Gradle through IntelliJ. It is a kotlin project with multiple subprojects. How is memory shared/distributed between all of these processes? I know that IntelliJ, the kotlin daemon, the gradle daemon, and each gradle fork are all seperate processes. Do these all have their own independent memory usage? And if so, how much do they all need?
I have chosen some settings somewhat arbitrarily. They are:
IntelliJ "Custom VM Options"
-Xmx25000m
gradle.properties
org.gradle.jvmargs=-Xmx20g
kotlin.daemon.jvm.options=-Xmx20g
So it would seem I've split my memory into 3 roughly equal portions for IntelliJ, Gradle, and Kotlin.
However, I have no idea if this is optimal. I'm still running into out-of-memory issues in IntelliJ even though it has 25GB.
Please sign in to leave a comment.
Hello,
There is not an exact number. It is sum of consumption for each process. And Xmx is not the limit for every of them. Some memory is consumed by JVM staff, sub-processes etc.
Yes.
There is no exact value. It depends on the project type, size and frameworks used.
At first, let's make IDE's Xmx less. 4G would be enough. If you allocate more space for the Java process, it does not run GC in time and more space is consumed. 4G is enough even for 2M lines of code with Gradle, Kotlin, and IntelliJ IDEA .
Then, let's disable all non-bundled plugins in IntelliJ IDEA. They could have memory leaks or other bugs, especially when they are installed on the first EAP version of IntelliJ IDEA (not all plugins adopted their code base for new version).
If the out-of-memory error still occurs, please capture a memory snapshot according to https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems#memory_snapshot and upload it to https://uploads.jetbrains.com/ for analysis.
Hi @Konstantin this is all really helpful, thank you! I had no idea that the IDE can handle 2M lines of code with 4GB. I must have a memory leak in a plugin as you are suggesting.
Please let me take some time to try different things based on your advice.
I'm thinking more about your advice about Xmx. Does this advice extend to the kotlin daemon and gradle daemon? Are you suggesting 20g might be too much for those and that there may be GC issues there as well?
Yes, I'd configure lower Xmx for them too. GC is working simultaneously here.
The basic troubleshooting steps could be like
1. Build the project manually from the command line and make sure build is working (no memory leaks in the project itself).
2. Build the project in a freshly installed IntelliJ IDEA with all defaults.
3. Enable plugins one by one and check how they affect the build.
Thank you again. One more question, because in addition to practical advice I'm also trying gain a deeper understanding.
You said:
This made me curious so I started reading a bit about how GC works. I understand now that GC is triggered when different spaces (like young generation, survivor space, etc.) are filled up. So I understand that setting xmx too high will decrease the frequency of GC. But, to be clear, this couldn't cause a memory leak on its own right? Even if xmx is set way too high, wouldn't GC still eventually be triggered and clear up the memory to prevent out-of-memory issues?
If my understanding above is correct, it would seem there must be a memory leak in some plugin or something?
You are absolutely right.
Thank you again. One more question, because in addition to practical advice I'm also trying gain a deeper understanding.
You said:
This made me curious so I started reading a bit about how GC works. I understand now that GC is triggered when different spaces (like young generation, survivor space, etc.) are filled up. So I understand that setting xmx too high will decrease the frequency of GC. But, to be clear, this couldn't cause a memory leak on its own right? Even if xmx is set way too high, wouldn't GC still eventually be triggered and clear up the memory to prevent out-of-memory issues?
If my understanding above is correct, it would seem there must be a memory leak in some plugin or something?
yes man you are right
Which one it was?
I cannot simply disable all of my non-bundled plugins and add them back 1 by 1.
The issue is that the out-of-memory issue happens sparsely and is not easy to reproduce. I need my plugins to work.
I have disabled as many as I can. And I set `xmx` to 4g. However, I still get OOM errors occasionally.
My plugins are now:
I will update this thread as I come closer to a solution. Please allow me some time to experiment with disabling different plugins.
I can only guess here, any of them could have a memory leak. Moreover, IntelliJ IDEA or your project could have it as well.
So, a bit more troubleshooting is needed (at least 3 steps I mentioned earlier): https://intellij-support.jetbrains.com/hc/en-us/community/posts/7913165395474/comments/7924499992978