how to run 'python -m' command in pycharm?

Answered

for example, i need to debug a python file in pycharm ,and the full command to run this script in terminal is  :

python -m torch.distributed.launch --nproc_per_node=2 test_net.py TEST.IMS_PER_BATCH 16

 what and where should i set or edit  in pycharm so that i can debug this script in pycharm?

 

thank you !!

10 comments
Comment actions Permalink

Hi,

Try creating a run configuration in PyCharm, specify `-m torch.distributed.launch --nproc_per_node=2` as interpreter options, and `TEST.IMS_PER_BATCH 16` as script parameters. Set test_net.py as a script path.

Then debug using this configuration.

0
Comment actions Permalink

hi ,thanks for your reply!

i set the above configuration as you said ,and i can run it in pycharm but i still can not debug it,and the error message is the following:

 

/home/xwj/anaconda3/envs/MR/bin/python -m torch.distributed.launch --nproc_per_node=2 /home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 43655 --file /media/xwj/Programm/Python/maskrcnn-benchmark/train_copy.py MODEL.RPN.FPN_POST_NMS_TOP_N_TRAIN 2000 SOLVER.IMS_PER_BATCH 2
Traceback (most recent call last):
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py", line 1574, in main
setup = process_command_line(sys.argv)
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=1
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]
Traceback (most recent call last):
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/pydevd.py", line 1574, in main
setup = process_command_line(sys.argv)
File "/home/xwj/pycharm-2018.3.6/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=0
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]

Process finished with exit code 0

 

what does this mean and what should i do?

thank you!!

 

0
Comment actions Permalink

Sorry, my last recommendation wasn't correct. Please try the following instead:

In your run/debug configuration, specify Module name (instead of script path), and enter the module you want to run, in our case `torch.distributed.launch`.

Everything else goes to parameters, namely:

--nproc_per_node=2 test_net.py TEST.IMS_PER_BATCH 16

Kindly try it and let me know the results.

1
Comment actions Permalink

I don't know if this was solved for the original poster, but it hasn't worked for me.

First, I tried this:
ScriptPath:  test.py

Parameters:  (test.py arguments)

Interpreter Options:  -m torch.distributed.launch --nproc_per_node=1

I do this as I want to test this locally on my single GPU system.  However, when I run it this way, I get the following error:

Traceback (most recent call last):
File "/home/pycharm/helpers/pydev/pydevd.py", line 1501, in main
setup = process_command_line(sys.argv)
File "/home/pycharm/helpers/pydev/_pydevd_bundle/pydevd_command_line_handling.py", line 145, in process_command_line
raise ValueError("Unexpected option: " + argv[i])
ValueError: Unexpected option: --local_rank=0
Usage:
pydevd.py --port N [(--client hostname) | --server] --file executable [file_options]

I'm confused, because the line above it shows the complete parameter list, but local_rank is not among any of the parameters in the string.  It isn't there at all.

If I take the advice above and set to run torch.distributed.launch as a module instead of my test.py as a script, I set the parameters like so:

Module name:  torch.distributed.launch

Parameters:  --nproc_per_node=1 test.py  (remaining parameters to test.py)

I get the error "No Module Named test.py"

I'm on Pycharm Version 2019.2

0
Comment actions Permalink

@SpaceCowboy851

Most likely the path to the module isn't correct.
Could you share screenshots of your file location in the project and of your run configuration?

0
Comment actions Permalink

Here is what I have set up.  Not sure how the module path is incorrect.  Do I need to specify the full path when set to module mode?  I've never used this mode of PyCharm before.

0
Comment actions Permalink

>Do I need to specify the full path when set to module mode?

Yes, but depending on where it is regarding your project.

If the module is inside the project, then just start with the top-level folder containing the module.

1
Comment actions Permalink

Okay, I just switched it back to "script" and directly referenced the torch.distributed.launch (full path to the .py file) and that works.  But I'm left wondering what the "module" option is for and how to use it.  I thought it would work like "python -m" and you would pass in the module parameter the same thing you would pass to -m on the command line, but it doesn't seem to be that it works that way.

0
Comment actions Permalink

There are already 2 issues on this one:

https://youtrack.jetbrains.com/issue/PY-36164

 

https://youtrack.jetbrains.com/issue/PY-42231

 

Probably we should fill another one

 

 

0

Please sign in to leave a comment.