Stub Module Directory Shadowing Issue in Site-Packages Follow

Answered

Created February 19, 2020 06:49

I have a shared object binary module with dynamic typing, however there are certain structures that will typically be available to the developer. I have a "fake" module directory structure similar to:

foo

|- __init__.py

import bar

import fuzz

import buzz

|- bar.py

class A ...

class B ...

|-buzz

|- __init__.py

import cash

import money

|- cash.py

|- money.py

|-fuzz

|- __init__.py

import apple

import banana

|- apple.py

|- banana.py

When attempting to simply have both the true module (foo.so) and the fake foo directory in site packages, pycharm defaults to reading from the fake foo directory which leaves me with no functionality, but does allow me to have a full set of code completion.

I attempted to rename all .py files in the fake module directory structure to .pyi which has nearly the desired effect except that nested .py definitions are not available,

ie I have access to foo.bar.A and foo.bar.B, but not foo.buzz.cash or foo.fuzz.banana in code completion, though I still can call the types and run the code since my interpreter still has access to foo.so.

I tried renaming all except the __init__.py files to .pyi which led to an import error (e.g. __init__.py in foo could not import bar because it does not recognize the pyi file extension).

If there is a way to indicate to pycharm to recursively find all stubs in the directory I have not found it, and am open to work-arounds, I was just hoping to avoid having to write something to combine the directory structure into a single stub file.

Thanks for your help and suggestions!

5 comments

Sergey Karpov

Created February 19, 2020 09:02

Is there a reason to keep it in site-packages instead of configuring it as Content Root in PyCharm or adding to Interpreter Paths?

Natefehrenbach

Created February 24, 2020 06:05

Hi Sergey, adding it as a content root or as part of interpreter paths with all of the files (except __init__.py) files renamed to .pyi worked in that I had access to the written name spaces, but they still collide with the shared object library in site_packages, ie I can run the code if I switch the root level __init__.pyi to __init__.py, however then I have to switch it back to get code completion back again. So far running with this in the content root and renaming whenever I need to do a test run is the best work around I've found.

Sergey Karpov

Created February 24, 2020 09:21

If there is a name conflict between some modules/packages, I don't think there is anything PyCharm can do about it.

It is advised in general to avoid the usage of generic names.

Natefehrenbach

Created February 25, 2020 02:26

When I created a toy version the naming conflict between my .pyi file and the true .py module didn't stop the interpreter from running the code and I had all the code completion from the pyi file:

Project Structure:

test:

|-mm.py

|-mm.pyi

|-mod.py

Code:

mm.py:

class mc1:
    def __init__(self, c_num):
        self.a = 5
        self.b = 6
        self.c = c_num

    def calc(self):
        return self.a + self.c


valmm = 10

mm.pyi

class mc1:
    def __init__(self, c_num) -> None: ...
    a = None
    b = None
    c = None
    def calc(self) -> object: ...\

class mc2:
    def __init__(self, d) -> None: ...
    a = None
    b = None
    c = None
    d = None
    def calc2(self) -> object: ...


valmm = None

mod.py

import mm

mc = mm.mc1(5)
# mc_2 = mm.mc2(5)
print(mc.a)
print(mc.calc())
print(mm.valmm)

The code runs as long as I comment out the unimplemented class mc2, but I have code completion for the mc2 class in pycharm.

Sergey Karpov

Created February 25, 2020 11:31

PyCharm is taking stub files into account first. So if the stub file lacks some objects, then likely there will be no code completion.

In your case, it's the opposite. You have stubs for a class that doesn't have an implementation in Python code. So the behavior is correct.

Please sign in to leave a comment.