How to remote deploy a package (python setup.py develop)?

Answered

I'm working on some code that is usually installed as a package. It has some entry points that will not work without installation due to the way Python's imports work. This is pretty normal for packaged code, but perhaps less common for top-level applications.

Without PyCharm, a developer would first run `python setup.py develop` to enable the entry points, and then debug/make edits, the results of which are immediately available. Because the package is "installed", they can run the entry points and debug the code as a typical Python developer would.

Now I wish to use PyCharm Professional to "remote deploy" this code to a remote system, yet still edit/debug it. This means that it's not enough to simply copy the code as-is from my local machine to the remote host - it must be "installed" with `python setup.py develop` first.

Is there a way to work on this kind of codebase with PyCharm 2008.2's Remote Deployment feature? 

5 comments
Comment actions Permalink

To follow up - I still require this. The main issue is that imports differ depending on whether the package is installed or present within a source tree. The "proper" way to handle this is to use the "develop" mode to make the package available to the interpreter, rather than relying on the filesystem. At the moment, I need to manually connect to the remote target and run `python setup.py develop` in the synched directory before PyCharm can work remotely on the code base. I'm asking if there's a better way to do this that PyCharm supports.

0
Comment actions Permalink

Hi meowsqueak,

Have you tried to use Run/Debug Configuration for that? You could run it with Parameters.

Or do I miss something?

0
Comment actions Permalink

Hi Sergey, thank you for replying.

Here's the thing - let's say you're developing a Python package with PyCharm. It has a setup.py, a directory of source code, and maybe a 'bin' directory with some related tools. Setup.py would install those 'bin' tools if run with `python setup.py install` or `python setup.py develop`, but if you don't, then the tools in bin won't work, because the import mechanism in Python relies on the package being installed. You can get around this with hacks to PYTHONPATH but that's not the right way to do it. The same problem occurs if you are working on a project that contains multiple Python packages that depend on each other - they can only import each other if they are installed first.

The other useful thing about `python setup.py develop` is that it installs the package and keeps the sources as editable - any changes you make to the sources are automatically available to the imported package. This works well with PyCharm's deployment because it simply keeps the directories synced.

The problem is that PyCharm Professional can do a good job of remote deployment, copying the sources to the remote system, but it does not install the package in the remote environment (or if it does I can't see how to do that). So what I have to do currently is use Remote Deployment for the first time, then ssh into the remote system, run `python setup.py develop` manually, then jump back to PyCharm and continue my development. This isn't terrible, but once you start working on multiple packages at once, including testing of package deployment, it quickly becomes tedious, and it's also difficult to automate for new developers.

 

PyCharm knows about remote environments. For example I can create a remote venv and point PyCharm at it and it will use it, which is fantastic. Ideally, PyCharm would have also have the ability to deploy and install (i.e. run `python setup.py develop`) a project into that remote environment.

I hope this makes more sense? Please let me know if I'm not clear enough, I'd like to find a good solution to this as it is a usability issue for my team.

0
Comment actions Permalink

Hi meowsqueak, do you use a remote interpreter in PyCharm? I believe everything should work from the box with it, if I correctly understand your usecase. E.g.

  • I have a remote interpeter configured:

  • and a sample project with setup.py

  • the source code is auto-sync with remote machine as I have Tools | Deployment | Automatic Upload option enabled (which is a default option)
  • I can run Tools | Run setup.py Task action to install my package in the editable mode

  • at this point one has to restart PyCharm to make it aware of a newly installed package due to a known issue (PY-23156)
  • now everything should be ok

Did I miss some pitfalls?

0
Comment actions Permalink

Thanks Pavel, that looks promising. I'll give it a try tomorrow.

0

Please sign in to leave a comment.