[PyCharm Community 2019.1.3] Debug view for pytest freeze and can't access variables

When I wants to debug some pytest, the view appear, the app is working, I can set breakpoint and step into it.

Except I can't see variables values, I cant' access a debug console with the proper context.

There is something looping forever somewhere, see attached file.

Any idea?

8 comments
Comment actions Permalink

Does the issue reproduce with any test script? Please try some simple example, to determine if it's related to your scripts.

Then, please set `PYCHARM_DEBUG=True` env variable in your pytest configuration, run the debugger - it should show the extended output in the console, please paste here.

0
Comment actions Permalink

When trying to reproduce, I created a new project dedicated to this issue.

And I remember that my original source was on a project that is mainly a Java app, with this folder containing some python helpers.

So I modified the original project to have a separate Python env and just imported pytest (and boto3 because it is a AWS helper script)

And now everything works.

I'll try later to reproduce in another project, using my system python env. Maybe there is something wrong in it.

But I have a solution to my initial issue.

0
Comment actions Permalink

Ok, glad to hear it's working. If the issue reproduces again, you can try disabling cython speedups, which can cause peformance issues in rare cases. To do that, set the following env variables to your test configuration:

PYDEVD_USE_CYTHON=NO
PYDEVD_USE_FRAME_EVAL=NO

1
Comment actions Permalink

I have the same issue on PyCharn pro 2020.2. Starting with a fresh project and a very simple dummy test:

def test_var():

t = 1
assert t == 1

Setting a breakpoint works, the program hold, but PyCharm can't resolve variables (hangs on "Collecting data...") and the console won't print. Here is the console output when trying to do things:

 

/home/mowglie/.pyenv/versions/py3/bin/python /home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 41727 --file /home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/pycharm/_jb_pytest_runner.py --target test_dummy.py::test_var -- -s
Executing PyCharm's `sitecustomize`
User doesn't have a custom `sitecustomize`
Custom matplotlib backend was set for SciView
Using Cython speedups
Unable to load jupyter_debug plugin
Executing file /home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/pycharm/_jb_pytest_runner.py
arguments: ['/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/pycharm/_jb_pytest_runner.py', '--target', 'test_dummy.py::test_var', '--', '-s']
PYDEVD_FILTER_LIBRARIES False
Started in multiproc mode
('Connecting to ', '127.0.0.1', ':', '41727')
('Connected.',)
debugger: received >>99 -1 35547
<<
Received command: ??? 99 -1 35547
Received port 35547
pydev debugger: process 12882 is connecting
('Connecting to ', '127.0.0.1', ':', '35547')
('Connected.',)
debugger: received >>501 1 0.1 UNIX
<<
Received command: CMD_VERSION 501 1 0.1 UNIX
sending cmd --> CMD_VERSION @@BUILD_NUMBER@@
Connected to pydev debugger (build 202.7319.64)
debugger: received >>111 3 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
<<
Received command: CMD_SET_BREAK 111 3 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
Added breakpoint:/home/mowglie/PycharmProjects/pythonProject/test_dummy.py - line:5 - func_name:b'test_var'
debugger: received >>170 11 SET_UNIT_TESTS_DEBUGGING_MODE
146 7 SHOW_RETURN_VALUES 0
101 9
122 5 python-BaseException None None 0 1 0
<<
Received command: ??? 170 11 SET_UNIT_TESTS_DEBUGGING_MODE
Received command: CMD_SHOW_RETURN_VALUES 146 7 SHOW_RETURN_VALUES 0
Show return values: False
Received command: CMD_RUN 101 9
Received command: CMD_ADD_EXCEPTION_BREAK 122 5 python-BaseException None None 0 1 0
Exceptions to hook on terminate: {'BaseException': <_pydevd_bundle.pydevd_breakpoints.ExceptionBreakpoint object at 0x7fdc06f38e80>}
sending cmd --> CMD_THREAD_CREATE <xml><thread name="MainThread" id="pid_12882_id_140583053184448" /></xml>
IDE_PROJECT_ROOTS ['/home/mowglie/PycharmProjects/pythonProject']
LIBRARY_ROOTS ['/home/mowglie/.pyenv/versions/3.8.5/lib/python3.8', '/home/mowglie/.pyenv/versions/3.8.5/lib/python3.8/lib-dynload', '/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages', '/home/mowglie/disk/Dropbox/HomeDocs/git/salem', '/home/mowglie/disk/Dropbox/HomeDocs/git/oggm-fork', '/home/mowglie/.cache/JetBrains/PyCharm2020.2/python_stubs/551278590', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/python-skeletons', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/typeshed/stdlib/3.7', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/typeshed/stdlib/3', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/typeshed/stdlib/2and3', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/typeshed/third_party/3', '/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/typeshed/third_party/2and3']
Launching pytest with arguments -s test_dummy.py::test_var in /home/mowglie/PycharmProjects/pythonProject
Patching args: ['/sbin/ldconfig', '-p']
Process is not python, returning.
sending cmd --> CMD_PROCESS_CREATED <process/>
=== Scientific View debug enabled ===
sending cmd --> CMD_PROCESS_CREATED <process/>
('Connecting to ', '127.0.0.1', ':', '41727')
('Connected.',)
("The debugger thread 'DispatchReader' is traced which may lead to debugging performance issues.",)
debugger: received >>99 -1 43425
<<
Received command: ??? 99 -1 43425
PYDEVD_FILTER_LIBRARIES False
('Connecting to ', '127.0.0.1', ':', '43425')
('Connected.',)
debugger: received >>501 1 0.1 UNIX
<<
Received command: CMD_VERSION 501 1 0.1 UNIX
sending cmd --> CMD_VERSION @@BUILD_NUMBER@@
debugger: received >>111 3 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
<<
Received command: CMD_SET_BREAK 111 3 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
debugger: received >>111 13 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
<<
Received command: CMD_SET_BREAK 111 13 python-line /home/mowglie/PycharmProjects/pythonProject/test_dummy.py 5 test_var THREAD None None
Added breakpoint:/home/mowglie/PycharmProjects/pythonProject/test_dummy.py - line:5 - func_name:b'test_var'
debugger: received >>122 5 python-BaseException None None 0 1 0
<<
Received command: CMD_ADD_EXCEPTION_BREAK 122 5 python-BaseException None None 0 1 0
Added breakpoint:/home/mowglie/PycharmProjects/pythonProject/test_dummy.py - line:5 - func_name:b'test_var'
Exceptions to hook on terminate: {'BaseException': <_pydevd_bundle.pydevd_breakpoints.ExceptionBreakpoint object at 0x7fdbada823a0>}
debugger: received >>146 7 SHOW_RETURN_VALUES 0
170 11 SET_UNIT_TESTS_DEBUGGING_MODE
101 9
<<
Received command: CMD_SHOW_RETURN_VALUES 146 7 SHOW_RETURN_VALUES 0
Show return values: False
Received command: ??? 170 11 SET_UNIT_TESTS_DEBUGGING_MODE
Received command: CMD_RUN 101 9
debugger: received >>122 15 python-BaseException None None 0 1 0
170 17 SET_UNIT_TESTS_DEBUGGING_MODE
146 19 SHOW_RETURN_VALUES 0
<<
Received command: CMD_ADD_EXCEPTION_BREAK 122 15 python-BaseException None None 0 1 0
Exceptions to hook on terminate: {'BaseException': <_pydevd_bundle.pydevd_breakpoints.ExceptionBreakpoint object at 0x7fdbf64072e0>}
Received command: ??? 170 17 SET_UNIT_TESTS_DEBUGGING_MODE
Received command: CMD_SHOW_RETURN_VALUES 146 19 SHOW_RETURN_VALUES 0
Show return values: False
sending cmd --> CMD_THREAD_CREATE <xml><thread name="Thread-11" id="pid_12912_id_140581488043632" /></xml>
sending cmd --> CMD_THREAD_CREATE <xml><thread name="Thread-12" id="pid_12912_id_140581488045600" /></xml>
sending cmd --> CMD_THREAD_KILL pid_12912_id_140581488045600
sending cmd --> CMD_THREAD_CREATE <xml><thread name="Thread-13" id="pid_12912_id_140581488045696" /></xml>
sending cmd --> CMD_THREAD_KILL pid_12912_id_140581488045696
sending cmd --> CMD_THREAD_CREATE <xml><thread name="Thread-14" id="pid_12912_id_140581633657008" /></xml>
sending cmd --> CMD_THREAD_KILL pid_12912_id_140581633657008
sending cmd --> CMD_THREAD_CREATE <xml><thread name="MainThread" id="pid_12882_id_140583053184448" /></xml>
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1 -- /home/mowglie/.pyenv/versions/py3/bin/python
cachedir: .pytest_cache
Matplotlib: 3.3.2
Freetype: 2.6.1
rootdir: /home/mowglie/PycharmProjects/pythonProject
plugins: mpl-0.122, oggm-1.3.2.dev95+gc05d330
collecting ... collected 1 item
test_dummy.py::test_var Suspending at breakpoint in file: /home/mowglie/PycharmProjects/pythonProject/test_dummy.py on line 5
sending cmd --> CMD_THREAD_SUSPEND <xml><thread id="pid_12882_id_140583053184448" stop_reason="111" message="" suspend_type="frame_eval"><frame id="140582185093536" name="test_var" file="/home/mowglie/PycharmProjects/pythonProject/test_dummy.py" line="5"></frame><frame id="140582185125248" name="pytest_pyfunc_call" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/python.py" line="180"></frame><frame id="94234090882784" name="_multicall" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/callers.py" line="187"></frame><frame id="140582185028848" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="84"></frame><frame id="140582185028384" name="_hookexec" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="93"></frame><frame id="140582185054272" name="__call__" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/hooks.py" line="286"></frame><frame id="140582185120640" name="runtest" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/python.py" line="1570"></frame><frame id="94234090877504" name="pytest_runtest_call" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="153"></frame><frame id="94234090882144" name="_multicall" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/callers.py" line="187"></frame><frame id="140582185027920" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="84"></frame><frame id="140582185027456" name="_hookexec" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="93"></frame><frame id="140582185026992" name="__call__" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/hooks.py" line="286"></frame><frame id="140582185070144" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="247"></frame><frame id="94234090804336" name="from_call" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="294"></frame><frame id="140582185091136" name="call_runtest_hook" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="246"></frame><frame id="140582185041248" name="call_and_report" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="207"></frame><frame id="94234090878144" name="runtestprotocol" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="117"></frame><frame id="140582185066560" name="pytest_runtest_protocol" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/runner.py" line="100"></frame><frame id="94234090876864" name="_multicall" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/callers.py" line="187"></frame><frame id="140582184987888" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="84"></frame><frame id="94234090777440" name="_hookexec" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="93"></frame><frame id="140582185300960" name="__call__" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/hooks.py" line="286"></frame><frame id="94234090730336" name="pytest_runtestloop" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/main.py" line="321"></frame><frame id="94234090876224" name="_multicall" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/callers.py" line="187"></frame><frame id="140582185078000" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="84"></frame><frame id="140582185077536" name="_hookexec" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="93"></frame><frame id="140582185077072" name="__call__" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/hooks.py" line="286"></frame><frame id="140582671781216" name="_main" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/main.py" line="296"></frame><frame id="94234068782320" name="wrap_session" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/main.py" line="240"></frame><frame id="140582904662944" name="pytest_cmdline_main" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/main.py" line="289"></frame><frame id="94234090491968" name="_multicall" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/callers.py" line="187"></frame><frame id="140582904656400" name="&lt;lambda&gt;" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="84"></frame><frame id="94234090684240" name="_hookexec" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/manager.py" line="93"></frame><frame id="94234090554848" name="__call__" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/pluggy/hooks.py" line="286"></frame><frame id="94234067792464" name="main" file="/home/mowglie/.pyenv/versions/py3/lib/python3.8/site-packages/_pytest/config/__init__.py" line="157"></frame><frame id="140582986219072" name="&lt;module&gt;" file="/home/mowglie/.pycharm-2020.2.2/plugins/python/helpers/pycharm/_jb_pytest_runner.py" line="43"></frame></thread></xml>
debugger: received >>114 13 pid_12882_id_140583053184448 140582185093536 FRAME
<<
Received command: CMD_GET_FRAME 114 13 pid_12882_id_140583053184448 140582185093536 FRAME
debugger: received >>114 15 pid_12882_id_140583053184448 140582185093536 FRAME
<<
Received command: CMD_GET_FRAME 114 15 pid_12882_id_140583053184448 140582185093536 FRAME
print(t)
debugger: received >>121 17 pid_12882_id_140583053184448 140582185093536 FRAME print(t)@LINE@
<<
Received command: CMD_CONSOLE_EXEC 121 17 pid_12882_id_140583053184448 140582185093536 FRAME print(t)@LINE@

 

 

0
Comment actions Permalink

Note that it must have something to do with my python environment, because running the same test with a much simpler pyenv environment (with just python 3.8 and pytest) works fine...

0
Comment actions Permalink

By comparing the two logs (with simple env and more complex env), one of the main differences in the log is that for the complex env (where I have numpy, matplotlib, etc, installed), the log contains a section about `=== Scientific View debug enabled ===` which is not there for the simple one. Could that be the reason?

0
Comment actions Permalink

Fabien Maussion

Have you tried recreating the problematic interpreter and environment?

0
Comment actions Permalink

Andrey Resler thanks - fater some tries I have been able to isolate the pytest plugin which would cause the error. Now I have to understand what exactly in this plugin is causing problems in pycharm - I wonder if it's something with multiprocessing... I will see if I can test more.

0

Please sign in to leave a comment.