Hidden errors caused by possible decoration added by PyCharm to boto3, prevent this to run the Concurrency Diagram Diagram.

已回答

创建于 2017年08月31日 15:27

I cannot manage to get the Concurrency Diagram working because it fails with the python `boto3` library, while trying to requesting a resource.

The following command:

self.__s3_resource = boto3.resource('s3')

will fail with the following

 File "/usr/local/lib/python3.5/dist-packages/boto3/__init__.py", line 92, in resource
 return _get_default_session().resource(*args, **kwargs)
 File "/usr/local/lib/python3.5/dist-packages/boto3/session.py", line 389, in resource
 aws_session_token=aws_session_token, config=config)
 File "/usr/local/lib/python3.5/dist-packages/boto3/session.py", line 263, in client
 aws_session_token=aws_session_token, config=config)
 File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 836, in create_client
 client_config=config, api_version=api_version)
 File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 64, in create_client
 cls = self._create_client_class(service_name, service_model)
 File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 86, in _create_client_class
 base_classes=bases)
 File "/usr/local/lib/python3.5/dist-packages/botocore/hooks.py", line 227, in emit
 return self._emit(event_name, kwargs)
 File "/usr/local/lib/python3.5/dist-packages/botocore/hooks.py", line 210, in _emit
 response = handler(**kwargs)
 File "/usr/local/lib/python3.5/dist-packages/boto3/utils.py", line 61, in _handler
 module = import_module(module)
 File "/usr/local/lib/python3.5/dist-packages/boto3/utils.py", line 52, in import_module
 __import__(name)
 File "/usr/local/lib/python3.5/dist-packages/boto3/s3/inject.py", line 15, in <module>
 from boto3.s3.transfer import create_transfer_manager
 File "/usr/local/lib/python3.5/dist-packages/boto3/s3/transfer.py", line 127, in <module>
 from s3transfer.exceptions import RetriesExceededError as \
 File "/usr/local/lib/python3.5/dist-packages/s3transfer/__init__.py", line 428, in <module>
 class ShutdownQueue(queue.Queue):
TypeError: function() argument 1 must be code, not str

Only when trying to run the Thread Concurrency Visualization; it looks like PyCharm is decorating internally some classes and transforming them to functions.

It works fine in any other case (Debug/Run/Profile)

It took me quite sometime to find this issue. I basically had to inserts sleeps to narrow the cause since breakpoints are not working and I could manage to enable the console to see the output of a Thread Concurrency Visualisation session and had to write to file.

so my question are.

Do you know if it possible to prevent PyCharm to decorate some portion of the code which can cause issues?
Do you know if it is possible to see the console in the Thread Concurrency Visualisation session so that identify such cases it is easier?
Are the debug breakpoints supposed to work with the Thread Concurrency Visualisation?

Thanks for your help!!

3 条评论

Elizabeth Shashkova

创建于 2017年09月04日 09:17

正式评论

Thank you for the feedback! I've created an issue in our issue tracker: https://youtrack.jetbrains.com/issue/PY-25946

Elizabeth Shashkova

创建于 2017年09月01日 09:44

Hi tetractius!

1. Yes, you're right: Thread Concurrency Visualization is based on patching some threading modules and libraries. Unfortunately, sometimes (if, for example, some other library makes similar changes) it can lead to errors. Could you please provide a full code sample for reproducing the bug?

2. This is known problem, we're going to fix it: https://youtrack.jetbrains.com/issue/PY-18409

3. No, breakpoints are turned off by design.

Thank you for your feedback!

tetractius

创建于 2017年09月02日 09:32

Hi Elizabeth,

For reproducing the issues it is enough to use boto3 resource within the main thread or withing any other thread:

The following code will work if run, debug or profile:

import boto3
import logging
import traceback


# Configuring a logger to write errors on files as we cannot see the console yet
log = logging.Logger(name="test_concurrent_view")
log.setLevel(level=logging.INFO)
streamHandler = logging.FileHandler(filename="./concurrent_view_error")
log.addHandler(streamHandler)


try:
    s3 = boto3.resource('s3')
    for bucket in s3.buckets.all():
        print(bucket.name)
except BaseException as be:
    log.error(traceback.format_exc())
    log.error(str(be))

But if you run in "Concurrent Diagram"

Instead it will fail and you can see the stacktrace already mentioned in the previous post, in the file concurrent_view_error.

Thanks!

请先登录再写评论。