Thinning out created JAR

In my project, I have many classes with main. I want to create a runnable JAR, that included only 1 class, and out of it, only one class from an apache.commons dependency.
I can make IntelliJ create a JAR that includes all my project classes, and all the classes in apache.common, which makes the created JAR big for no reason (and then I have to open the JAR in winrar and delete them manually).
How can I create a JAR that includes only the nececcacery classes?