My rant on Indexing again

I've been a PyCharm user since it's first released. For personal projects within a relatively small PYTHONPATH it works great.

Now I work in a company with 15+ years of Python code base. Hundreds of packages in PYTHONPATH on a shared network path.

99% of those packages NEVER change. Also, I'm working on multiple projects at a time and every time I open PyCharm it starts indexing THE SAME hundreds of packages AGAIN.

There are dozens of complaints on indexing but JetBrains for some reason is completely ignorant about this problem. 

We're not asking to make it faster, or make the problem magically disappear, just PLEASE give us the ability to control this process:

- Exclude paths from indexing. Super important! We still need some paths to be in PYTHONPATH, so they can't be removed, but they NEVER change, there is absolutely reason to reindex them every single time.

- Disable the indexing whatsoever on demand.

- Allow the indexing to run once and DO NOT re-index on restart unless explicitly asked.

- Prioritize other IDE actions over indexing. While PyCharm is indexing the IDE is pretty much useless.

Please make PyCharm great again by giving us control over the indexing!

10 comments

Hi,

I agree with you that the issue exists, and you're not the first one to ask this, but I haven't found a related feature request in our system.

You are welcome to submit a feature request at https://youtrack.jetbrains.com/issues/PY

Information on how to use YouTrack: https://intellij-support.jetbrains.com/hc/en-us/articles/207241135-How-to-follow-YouTrack-issues-and-receive-notifications

It is best to explain the use case, and why this feature is important, in the ticket's comments on YouTrack

0

I also have major issues regarding indexing. I use IntelliJ Ultimate mainly for Python work. I work with environments that have huge number of files which have nothing to do with code base. Opening these environments brings IntelliJ to its knees. For me I'd just like to work and not get into all the indexing and memory heap discussions. Why doesn't Jetbrains would set out some guidelines or profiles for optimizing IntelliJ based on project size. Like best practices.

0

Have you tried to exclude specific paths using Project Structure settings? You can add directories from outside of your project and mark them as excluded, which should stop PyCharm indexing them.

0

I've only excluded directories within my project using the 'Mark Directory As'. For excluding paths from the Project Structure settings I assumed when I created a project anything under the Global Libraries section was need to run the project and only that project.

0

You can add external content root and mark them as excluded. The code will be able to refer to them if needed, but they shouldn't be indexed by PyCharm.

0

Hey Andrey, thank you for your suggestion, but unfortunately, this didn't work. Well, sort off.... I added a few huge network mounted directories as extra project roots and excluded them. After File->Invalidate Cache and restart it's indexing them again, however it seems finished much quicker. I'll see how it behaves for a few days. Thanks.

0

Another suggestion (actually I should have suggested this in the first place):

Open your project interpreter settings, click gear icon > show all, then select your interpreter and click on interpreter path icon (it's a little buried, I know)

If you see any network paths there - delete them, and they should never bother you again (in terms of indexing)

0

And my project won't find modules :) The idea is to stop indexing and generating skeletons for some paths.

I know it sounds weird but this for historical reasons we've been versioning up packages in sys.path. So we ended up with hundreds of packages in sys.path each of them may have dozens of versions compiled for different OS architectures. So when I run my project, all I care about is my project files + common system paths like /usr/lib/pythonx.x, they should be indexed and tracked, while those network paths should be just available to the interpreter and IDE should not care about them whatsoever.

 

0

Thanks Andrey,

In my set up I create a new python SDK for each new project. When I open up a project and view the project structure I see all the python SDKs I've created. Is that the way it should be? And can I exclude those paths?

Also in your comment 'Open your project interpreter settings, click gear icon > show all' When I look Project Structure -> Platform Settings -> SDKs I don't see any of the icons you mention.

0

@C T Matsumoto

Are you using IDEA? PyCharm doesn't use the term "SDK" for it's interpreter, so I assume you're using IDEA, but IDEA's interface is different in regards to setting up Python. They are located in Project structure > SDK > Select your Python SDK > Classpath tab

0

Please sign in to leave a comment.