Attaching to a running gevent monkey patched process [PyCharm 2018.1]

Answered

I'm on PyCharm 2018.1 and I am trying to debug a local process that utilizes monkey patching by Gevent. I have enabled Gevent support in the IDE, however I still cannot attach. Whenever I try to attach to the process, I get the following stack, at which point pydevd in PyCharm exits with code 0.

Traceback (most recent call last):
File "/path/lib/python3.6/site-packages/pycharm-debug-py3k.egg/_pydevd_bundle/pydevd_comm.py", line 539, in start_client
s.connect((host, port))
File "/path/lib/python3.6/site-packages/gevent/_socket3.py", line 290, in connect
r = getaddrinfo(address[0], address[1], self.family)
File "/path/lib/python3.6/site-packages/gevent/_socketcommon.py", line 288, in getaddrinfo
return get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
File "/path/lib/python3.6/site-packages/gevent/resolver_thread.py", line 65, in getaddrinfo
return self.pool.apply(_socket.getaddrinfo, args, kwargs)
File "/path/lib/python3.6/site-packages/gevent/pool.py", line 325, in apply
return self.spawn(func, *args, **kwds).get()
File "/path/lib/python3.6/site-packages/gevent/event.py", line 382, in get
self._wait_core(timeout, ())
File "/path/lib/python3.6/site-packages/gevent/event.py", line 106, in _wait_core
result = self.hub.switch()
File "/path/lib/python3.6/site-packages/gevent/hub.py", line 629, in switch
switch_out()
File "/path/lib/python3.6/site-packages/gevent/hub.py", line 633, in switch_out
raise BlockingSwitchOutError('Impossible to call blocking function in the event loop callback')
gevent.hub.BlockingSwitchOutError: Impossible to call blocking function in the event loop callback

I saw this bug thread, but it pertains to remote debugging. In my case I have a local process to attach to.
https://youtrack.jetbrains.com/issue/PY-18929

Surprisingly, adding these 3 lines right before monkey.patch_all() does allow me to attach the debugger, although the process a little weird: when a breakpoint is hit, all gevent loops pause.

import os
os.environ["GEVENT_SUPPORT"] = "True"
import pydevd

I do have "Suspend=Thread" instead of All set. I am not sure why all threads pause - that does not seem like expected behavior.

 

Is there a more reasonable way to work around all of this without requiring very specific imports, or pausing all threads? I would have thought Gevent Support being set in the IDE would work fine.

3 comments
Comment actions Permalink

Hi! PyCharm's debugger can't work with gevent process which has already been pached, that's why adding 

os.environ["GEVENT_SUPPORT"] = "True"

is necessary before monkey patching. 
Could you please share a code sample, where your error appears?

0
Comment actions Permalink

Hi Elizabeth!

That makes sense then as I am attaching to an already running process that I invoked from a console. Unfortunately I am not sure how much effort it would be to craft a code sample as the code I am working with is not trivial. The short of it is it's a simple Python HTTP API being served by the reference WSGI implementation for local testing with some gevent-based asynchronous loops. However, since I am running outside of PyCharm, by the time I am trying to attach the process has already been monkey-patched.

So do I need that "import pydevd" statement there as well? What is the most concise way to allow post-monkey-patching attachment?

0
Comment actions Permalink

Follow-up: it appears I may have answered my own question. I went digging for usages of that environ variable and ended up here:

def protect_libraries_from_patching():
"""
In this function we delete some modules from `sys.modules` dictionary and import them again inside
`_pydev_saved_modules` in order to save their original copies there. After that we can use these
saved modules within the debugger to protect them from patching by external libraries (e.g. gevent).
"""
# Truncated

 

That's a pretty nifty trick. Looks like this gets called when pydevd is imported, which makes the import necessary. 

0

Please sign in to leave a comment.