Delegate build/run actions to Grade: Task 'run MyMainClass' not found

Hi

I'm trying to use IDEA's feature to let Gradle do all building. However, this feature seems to be unusable (I'm sure, I'm missing something – however, the only docs on this feature is the label of the checkbox). Building gets delegated OK, but whenever I try to run a run configuration, I get:

FAILURE: Build failed with an exception.

* What went wrong:
Task 'run MyMainClass' not found in project ':my:project'.

What am I missing there? Do I really have to define a task for the run configuration manually? If so, how can I use different run configurations with the same main class? What about debugging?