How to deal with "collecting data..." in debugger?

I have a function with a large array computation (about 100k objects). After its computation it's impossible to continue debugging due to terribly slow "collecting data..." operation in variables frame of debugger. It takes about 40-60 seconds after each debugger step to "collect data" and then it says "Unable to display frame variables" (however, sometimes it's evaluated eventually). Then if i click "show execution point" debugger will show me variables anyway. But if I click on "show execution point" before it says "Unable to display frame variables" it will not show me variables after appearing of error message even if i click again :(

I think the problem is connected with evaluating of text representation of my large array because array of the same size but without text representation of its objects doesn't slow debugger down.

I've googled that there should be an option that will allow to disable evaluating of text representation for specified types of objects but I can't find it :(
Source: https://www.jetbrains.com/idea/features/compiler.html#link28 (Configurable set of types to show as toString() by default, if I got it right)

I'm using PyCharm 4.0.5
9 comments
Official comment

Hi! We had this problem for a long time, but in PyCharm 2017.3 the option "Load Values Asynchronously" was implemented: https://youtrack.jetbrains.com/issue/PY-12987 (and it's turned on by default). 
If you still have the same problem with this option enabled, please create an issue in our issue tracker and provide steps for reproducing. Thank you!

By the way, it can be reproduced with following code:

class TestClass():
    field = 42

    def __repr__(self):
        return str([self.field ** .5] * 100)

def test():
    arr = []
    for i in range(300000):
        arr.append(TestClass())
    print('x')

test()

Put a breakpoint after cycle in test and you'll see
1

Still occurs, with both "Show return values" and "Show values inline toggled to off.

Debugger reaches breakpoint once. no data are readable.

Upon Resume Program (F8), debugger goes into an unknown state. Not paused, not taking requests.

1

Any updates on this issue? Unfortunately this makes PyCharm debugger useless for other than very simple and small programs.

There should be some option to disable detailed view for variables until specifically requested (view values of variables only if clicked). Or option to disable variable view completely, but still allow working with console (that's all I need).

For example the following code freezes debugger (remains in "collecting data..."). With N=100 or so everything still works. I have also enabled Gevent and disabled almost everything else with no help.

import numpy as np
N=10000
X={}
for i in range(0,N):
X[str(i)]=np.random.rand(200)

 PS. I'm running PyCharm 2017.2.3 with Python 3.6.2 on Win10.

 

 

 

1

I have the same problem. An official solution from the PyCharm team would be very appreciated!

0

The same problem in Idea Ultimate 2017.3.4 for Java debugger. Very frustratingly.

1

I have this problem as well on Ultimate 2017.2.5 for Java. Please consider optimizing it.

0

Do you have Memory view opened in debugger session? Try with it hidden if you have it opened.

Try also if it makes it better to disable toString() and alternative views in File | Settings | Build, Execution, Deployment | Debugger | Data Views | Java settings.

If it doesn't help please create new issue on YouTRack attaching idea.log ("Help | Show Log in..."): http://youtrack.jetbrains.com/issues/IDEA.

0

It has sped up quite a bit when toString() is disabled. Thanks for your help!

0

Please sign in to leave a comment.