Why is my R session in PyCharm IDE slowing down despite sufficient available RAM?
I have the below code:
cur_data <- readRDS("./TCGA-LUAD.rds")
mRNA <- cur_data$mRNAUnstranded
clinical <- cur_data$clinical
mRNATPM <- cur_data$mRNATPM
miRNA <- cur_data$miRNA
cnv <- cur_data$cnv
methyl <- cur_data$meth450
rm(cur_data)
gc()
After running these lines, my R session gets very slow that even printing a string takes 10-15 seconds. I have checked the object sizes:
> object.size(cur_data)
3654078952 bytes
> object.size(cnv)
99061912 bytes
> object.size(methyl)
1826984576 bytes
> object.size(miRNA)
9215840 bytes
> object.size(mRNA)
166234336 bytes
> object.size(mRNATPM)
305757664 bytes
I have the below code:
cur_data <- readRDS("./TCGA-LUAD.rds")
mRNA <- cur_data$mRNAUnstranded
clinical <- cur_data$clinical
mRNATPM <- cur_data$mRNATPM
miRNA <- cur_data$miRNA
cnv <- cur_data$cnv
methyl <- cur_data$meth450
rm(cur_data)
gc()
After running these lines, my R session gets very slow that even printing a string takes 10-15 seconds. I have checked the object sizes:
> object.size(cur_data)
3654078952 bytes
> object.size(cnv)
99061912 bytes
> object.size(methyl)
1826984576 bytes
> object.size(miRNA)
9215840 bytes
> object.size(mRNA)
166234336 bytes
> object.size(mRNATPM)
305757664 bytes
On the machine that I am running my code, we have around 230GB memory available. Below I have included the memory info as well:
library(pryr)
library(ps)
# Check memory before loading data
cat("Memory before loading data: \n")
print(mem_used())
print(ps::ps_memory_info())
cur_data <- readRDS("./TCGA-LUAD.rds")
# Check memory after loading data
cat("Memory after loading data: \n")
print(mem_used())
print(ps::ps_memory_info())
The output is:
Memory before loading data:
70.4 MB
rss vms shared text lib data
151760896 19957977088 27262976 11669504 0 18425253888
dirty
0
Memory after loading data:
3.58 GB
rss vms shared text lib data
3647713280 23454457856 27262976 11669504 0 21921734656
dirty
0
Also running below command:
system("free -b", intern = TRUE)
returns:
[[1]]
[1] " total used free shared buff/cache available"
[[2]]
[1] "Mem: 270086418432 19110584320 212453797888 1396736 38522036224 248626864128"
[[3]]
[1] "Swap: 262143995904 22240501760 239903494144"
Even though I have enough RAM available, my question is why the R session gets slow? A note that I am using PyCharm (Jetbrains Client using SSH) IDE to run my codes. (Also each list item in the cur_data are a list themselves and if asking to work with one of them at the time, I need to load multiple of them at once to be able to do some processing.)
*Another quick point is that I have tried already to run the script through command line, and there is no issue with that.
请先登录再写评论。
I've copied this thread to a bug report for R plugin developers so they can examine this problem. Please join the discussion: https://youtrack.jetbrains.com/issue/R-1546/Why-is-my-R-session-in-PyCharm-IDE-slowing-down-despite-sufficient-available-RAM