Manage a pycharm project with a dependent wheel project
I am working on a project A that depends on a wheel B that I also develop.
When I have to make changes to B's interface, I need to reflect the change into the main project A, rebuild the wheel, install it and debug both. It's important to me to keep B as a wheel instead of a local subproject because the imports may behave differently, and when I debug I want the specific behaviour I get if I ran the tool with the wheel installed without access to any local files.
I have created a workflow to make this a bit less tedious, but it is fairly error-prone:
I keep both projects in two separate PyCharm instances. My main project A contains the installed wheel of my last working version of B. Whenever I need to refactor or change something in B, I do it right in the A project, which means I adapt the files of B in site-packages directly (dangerous, I know). When I am happy with the changes, I use Beyond Compare to move everything over from site-packages to the actual B project codebase, rebuild the wheel and reinstall it into A.
I don't particularly like this workflow and would prefer something like Visual Studio does in C++ where I can have multiple projects in one large solution and all the dependencies resolve on their own whenever I run the main project.
In PyCharm's case that would mean that I could work on project B right from the same instance that also holds the main project A, but PyCharm understands that B is not a local subproject of B, but instead needs to be treated as a wheel before actually having access to the changes I made. Any changes in B would automatically trigger a wheel rebuild and install when I run A. And while coding, the dependencies between A and B should also be understood so that if I need to e.g. change a function signature in B, it will refactor into A as well.
Is something like this available? Is it planned? Am I asking too much or make this issue overly complicated?
Please sign in to leave a comment.
There is no exact feature to handle your specific scenario but an overall suggestion can be set up a CI Pipeline - Every time you make changes to your B project and push those changes to your git repository, the pipeline can automatically build and publish the new version of your B package. So in your A project, you just need to pull the updated version of the B package.
JetBrains has TeamCity for that and there is plugin available in PyCharm.
Thank you, but my use case is much more local.
Your suggestion works in case I can write code that works without actually testing it, but I would like to synch the two projects every time I run and test a change. By going via git I would keep commiting a lot of broken code until I got to a place where all my changes work well.
The important takeaway from your answer is that this is not a thing in Pycharm and I don't blame you at all. It's corner case and setting it up seems like a lot of trouble for such a specific scenario.
While typing this I am getting an idea that might help me do what I need. If I set up a virtual environment in my A project and add project B as a submodule in a way that it checks out into the venv's site-packages, I could work directly on B as it's integrated into A and at the same time I could commit changes to B right into its own repo.
Not sure if that would work and if it's worth the hassle, but I might test it.
Create a New Project:
Open PyCharm and create a new project. Choose a location for your project directory.
Create Virtual Environment:
Inside the project directory, create a virtual environment to isolate dependencies. You can do this via the terminal:
python -m venv venv
On Windows: venv\Scripts\activate TellPopeyes
On macOS/Linux: source venv/bin/activate
Thank you for your input. I'm not sure whether your answer is incomplete but this is not what I need. I already know how to set up and work with a virtual environment from Pycharm.
But what I need here is a virtual environment that understands that one if its packages is actually part of the project so that if I modify code from within the subproject it needs to reflect to the venv.
At this point I believe this is out of scope for Pycharm.