how to run 'python -m' command in pycharm?

Hi,

Try creating a run configuration in PyCharm, specify `-m torch.distributed.launch --nproc_per_node=2` as interpreter options, and `TEST.IMS_PER_BATCH 16` as script parameters. Set test_net.py as a script path.

Then debug using this configuration.

hi ,thanks for your reply!

i set the above configuration as you said ,and i can run it in pycharm but i still can not debug it,and the error message is the following:

/home/xwj/anaconda3/envs/MR/bin/python -m torch.distributed.launch --nproc_per_node=2 /home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 43655 --file /media/xwj/Programm/Python/maskrcnn-benchmark/train_copy.py MODEL.RPN.FPN_POST_NMS_TOP_N_TRAIN 2000 SOLVER.IMS_PER_BATCH 2
Traceback (most recent call last):
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py", line 1574, in main
setup = process_command_line(sys.argv)
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=1
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]
Traceback (most recent call last):
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py", line 1574, in main
setup = process_command_line(sys.argv)
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=0
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]

Process finished with exit code 0

what does this mean and what should i do?

thank you!!

hello?

Sorry, my last recommendation wasn't correct. Please try the following instead:

In your run/debug configuration, specify Module name (instead of script path), and enter the module you want to run, in our case `torch.distributed.launch`.

Everything else goes to parameters, namely:

--nproc_per_node=2 test_net.py TEST.IMS_PER_BATCH 16

Kindly try it and let me know the results.

I don't know if this was solved for the original poster, but it hasn't worked for me.

First, I tried this:
ScriptPath:  test.py

Parameters:  (test.py arguments)

Interpreter Options:  -m torch.distributed.launch --nproc_per_node=1

I do this as I want to test this locally on my single GPU system.  However, when I run it this way, I get the following error:

Traceback (most recent call last):
File "/home/pycharm/helpers/pydev/pydevd.py", line 1501, in main
setup = process_command_line(sys.argv)
File "/home/pycharm/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=0
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]

I'm confused, because the line above it shows the complete parameter list, but local_rank is not among any of the parameters in the string.  It isn't there at all.

If I take the advice above and set to run torch.distributed.launch as a module instead of my test.py as a script, I set the parameters like so:

Module name:  torch.distributed.launch

Parameters:  --nproc_per_node=1 test.py  (remaining parameters to test.py)

I get the error "No Module Named test.py"

I'm on Pycharm Version 2019.2

@SpaceCowboy851

Most likely the path to the module isn't correct.
Could you share screenshots of your file location in the project and of your run configuration?

Here is what I have set up.  Not sure how the module path is incorrect.  Do I need to specify the full path when set to module mode?  I've never used this mode of PyCharm before.

>Do I need to specify the full path when set to module mode?

Yes, but depending on where it is regarding your project.

If the module is inside the project, then just start with the top-level folder containing the module.

Okay, I just switched it back to "script" and directly referenced the torch.distributed.launch (full path to the .py file) and that works.  But I'm left wondering what the "module" option is for and how to use it.  I thought it would work like "python -m" and you would pass in the module parameter the same thing you would pass to -m on the command line, but it doesn't seem to be that it works that way.

There are already 2 issues on this one:

https://youtrack.jetbrains.com/issue/PY-36164

https://youtrack.jetbrains.com/issue/PY-42231

Probably we should fill another one

I ran into the same error @... is having, on PyCharm Professional 2021.1.1.

It seems that if instead of running it as 'module.submodule' I set it to run as only 'submodule', it'll find the module itself, but it causes import errors for files in other directories..

The second solution from Andrey Resler,

put `torch.distributed.launch` into module and Everything else goes to parameters

 works very well.

I would like to provide another solution:

1. at the script path, put the pytorch launch.py, like this, /opt/conda/lib/python3.8/site-packages/torch/distributed/launch.py
2. remove '-m torch.distributed.launch' section
3. everything else, including the options for launch.py, and your own script, its options, put them into the parameters.

An example pictrure:

The question asks how you can use the -m switch specifically @Gouchangjiang so this is completely off topic

The issues above **are still open** - now, `-m` being the standard way of running python modules I wonder *are they ever gonna be fixed*?

According to Std09200' comment, I focus too much on the given example, but not the general scenario. An intuitive walkaround is to find the exact module .py file, and then put it into the script path.

how you can use the -m switch

It's not obvious, but in the Run/Debug Configuration the "Script Path" label is actually a dropdown.

If you change "Script Path" to "Module Name" then you can specify a module and it will work as you would expect.

@...: I tried module name as you suggested:

runfile('parsecodetree.ustitles.insert504', wdir='/home/malikarumi/Projects/uscode/parsecodetree/ustitles', is_module=True)
No module named parsecodetree.ustitles.insert504

For me all solutions failed, on Pycharm Professional (via remote Gateway):

RuntimeError: The server socket has failed to listen on any local network address. The server socket has failed to bind to [::]:29500 (errno: 98 - Address already in use). The server socket has failed to bind to ?UNKNOWN? (errno: 98 - Address already in use).