Running JUnit test multiple times; stop on first failure

While searching for the answer myself, I stumbled upon this unanswered question. Goto "Run / Debug Configuration" and select the desired test run of choice or create a new test run. Under Configuration you can set it up to repeat "N Times", "Until Failure" or "Until Success".

Is there an way to run only the failed test every time after the first run? Would be useful, because if an Appium test was successful it shouldn't fail the next time.

Therefore I don't want to run already successful tests again

Hello,

Does the "rerun failed tests" solves the issue for you?

Please see: https://www.jetbrains.com/help/idea/performing-tests.html#rerun-failed-tests

I need to run the test automated without user interaction, so it should be started one time and continue running the failed test until a specified number of round has finished.

Say start the test with the definition run 3 Times, first run all test, 2nd round run the failed tests from the first run, and in the third round run the remaining failed test from the 2nd round.

So in the end, only the tests that really don't work should remain red.

The remaining red test should be checked at the end, why they are failing and how to make them green afterwards.

Is ther any solution to do this? Would be happy about some cli commands too.

Hello,

Unfortunately there is no ready tool to rerun the failed tests automatically after the first run. 

Please feel free to create a feature request here: https://youtrack.jetbrains.com/issues/IDEA

Thank you 

Would be a useful feature to have. Great suggestion Franz.

Franz,  Olga Klisho Is there a feature request for this already? 

Hello, please see the issue created: https://youtrack.jetbrains.com/issue/IDEA-290035

Thank you 

Goto "Run / Debug Configuration" and select the desired test run of choice or create a new test run. Under Configuration you can set it up to repeat "N Times", "Until Failure" or "Until Success".

I'm using RubyMine and I don't see this option.  Olga Klisho is this only available for intellij?

I just want the simple case of running a cucumber selenium test 10 times for example, not doing anything special on failure, just so I can see common breaking points to resolve

Nicholas Shaw, there's indeed no such possibility in RubyMine. Do you need for all the frameworks or only Cucumber?

Olga Kuvardina cucumber is my only use case right now but I could certainly see it being useful for others.

I'm guessing this is an unlikely change?

Nicholas Shaw we have a request for adding it for Cucumber tests, so could you please add your vote there: https://youtrack.jetbrains.com/issue/RUBY-25056/Allow-running-repeating-cucumber-tests-Until-failure

Olga Kuvardina  that seems to relate to intelliJ, it's also from three years ago so I cannot say I have much hope.

Should I raise another request for adding the run x times feature that exists within intelliJ today?

Nicholas Shaw yes, if you'd like to see it added then please submit a request on our tracker: https://youtrack.jetbrains.com/issues/RUBY