How to run 100 Python scripts with Selenium in parallel?

I have a Python script that goes through a hundred web sites using Selenium. It is a massive script with 20+ thousand lines, and it takes about 30 minutes to complete per URL. This means it takes a total of 3000 minutes to go through all one hundred URLs. So instead of going through all the URLs in a loop in one script, I would like to create a hundred scripts, one per each URL, and run them all simultaneously. What is the best way to do it? So far I created three scripts like that, put them under the same project in PyCharm, and manually right clicked on each of them and selected Run. Is there any way to run all Python files from a single project with fewer clicks? 

Another thing is that the CPU usage (i7-6600U @ 2.60GHz) went to almost 100% when those three scripts were running in parallel. What is the best way to approach this considering that each script will need to have its own instance of Chrome running?

Please sign in to leave a comment.