IDEA 2019.2.2 (Ultimate Edition) ignores @Disabled for JUnit5 tests having a single @Test method

Answered

I'm using IDEA 2019.2.2 (Ultimate Edition) and it ignores `@Disabled` for JUnit5 tests having a single `@Test` method when running something like "All Tests".

IntelliJ IDEA 2019.2.2 (Ultimate Edition)
Build #IU-192.6603.28, built on September 6, 2019
Runtime version: 11.0.3+12-b304.56 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 7 6.1
GC: ParNew, ConcurrentMarkSweep
Memory: 4029M
Cores: 8
Registry: compiler.automake.allow.when.app.running=true
Non-Bundled Plugins: Batch Scripts Support, CMD Support, Insert Final Modifier, Lombook Plugin, MyBatisLogPlugin, String Manipulation, google-java-format, com.atlassian.bitbucket.references, ink.organics.pojo2json, io.codearte.props2yaml, io.protostuff.protostuff-jetbrains-plugin, liwey.Json2PojoWithLombok, mobi.hsz.idea.gitignore, cn.wuzhizhan.plugin.mybatis, MavenRunHelper, com.intellij.plugin.adernov.powershell, Pythonid, com.vladsch.idea.multimarkdown, org.sonarlint.idea

 

The issue is similar to what's noted here: https://stackoverflow.com/questions/52602351/junit-5-disabled-is-ignored

I've tried running both from right-clicking a class and from Run/Debug Configurations > JUnit > All in package and both have the issue.

 

Sample class

package xxx;

import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;

import static org.assertj.core.api.Assertions.assertThat;

@ExtendWith(SpringExtension.class)
@SpringBootTest(classes = XyzBoot.class)
public class MyTest {

@Test
@Disabled
public void testSomething() {
assertThat(true).isTrue();
}
}

 

10 comments
Comment actions Permalink

Hello,

Please tell if the issue is reproduced for you with IntelliJ IDEA 2019.3 (https://www.jetbrains.com/idea/nextversion/) when you run all tests in package? Thank you 

0
Comment actions Permalink

Olga Klisho - do you mean 2019.2.3 Build 192.681714 ?  I don't see a 2019.3 yet.

0
Comment actions Permalink

I mean EAP version, you may find it by link:

https://www.jetbrains.com/idea/nextversion/

0
Comment actions Permalink

Hi,

 

@Disabled at class level does not work for me, on Intellij Idea 2020.1.2.

It is not an easy topic, it takes a while to make it work with Maven in console. This is the link I followed.

https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit-platform.html

and it works.

```JAVA

package com.xo.junit_5;

import static org.junit.jupiter.api.Assertions.assertEquals;

import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;

@Disabled("Disabled until bug #xxxx has been fixed!")
public class DisabledClass2Test {

@Test
void test1Plus1() {
assertEquals(2, 1 + 1);
if (true) {
throw new RuntimeException("disabled, should not run");
}
}

@Test
void test2Plus2() {
assertEquals(4, 2 + 2);
if (true) {
throw new RuntimeException("disabled, should not run");
}
}

}

```

The example is  taken from here:

https://mkyong.com/junit5/junit-5-how-to-disable-tests/

And they also ask if we know a reason for the bug I am presenting here:

```TXT

Note
However, run the above test under Intellij IDE, the @Disabled on class level is NOT working as expected, no idea why?

```

 

```XML

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
<relativePath/>
</parent>
<groupId>com.xo</groupId>
<artifactId>junit_5</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>junit_5</name>
<description>Demo project for Spring Boot</description>

<properties>
<java.version>1.8</java.version>
<junit.jupiter.version>5.7.0-M1</junit.jupiter.version>
<mockito.version>3.3.3</mockito.version>
</properties>

<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<artifactId>junit</artifactId>
<groupId>junit</groupId>
</exclusion>
<exclusion>
<artifactId>org.mockito</artifactId>
<groupId>mockito-core</groupId>
</exclusion>
<exclusion>
<artifactId>org.hamcrest</artifactId>
<groupId>hamcrest-core</groupId>
</exclusion>
<exclusion>
<artifactId>org.hamcrest</artifactId>
<groupId>hamcrest-library</groupId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.junit.jupiter</groupId>
<!-- <artifactId>junit-jupiter</artifactId>-->
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.jupiter.version}</version>
<scope>test</scope>
</dependency>


<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.12</version>
<scope>provided</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

</project>

```

In intellij I am using an external Maven, 

 

```TXT

D:\GitLab\work\learning\junit_5>mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: c:\install\apache-maven-3.6.3\bin\..
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: c:\Program Files\Java\jdk1.8.0_181\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

D:\GitLab\work\learning\junit_5>mvn test
....

[INFO] BUILD SUCCESS

```

Same versions I am using in intellij Idea Ultimate.

Yet the tests are executed, even though the class is marked as `@Disabled`

0
Comment actions Permalink

Hello Emilc77,

 

Can you please clarify, do you face the incorrect behaviour while running the test with IntelliJ Runner or with Maven?

For the IntelliJ Runner the correct behaviour will be as follows:

In case you run the disabled test in the test suite or within the test package, disabled test class will be disabled. In case you explicitly run the test, the `@Disabled` annotation will not be taken to consideration. 

Do you expect the different behaviour or the experienced behaviour is different? 

Thanks

 

0
Comment actions Permalink

If you take a look at the method, if it runs, if fails.

 

When running from Maven command line, that test is disabled, the methods are not executed, therefore the "mvn test" passes.

When executed from Intellij Idea Ultimate, that @Disabled test class still runs the test methods. When placed at class level, the @Disabled annotation does not ignore any of the test method.

I want to emphases that @Disabled annotation works as expected when placed at method level, those methods are ignored.

 

0
Comment actions Permalink

Emilc77,

The behaviour that you describe is expected. 

In case the disabled class is run explicitly, we recon that user understands that it's disabled but still wants to run the tests in it. In case such a class is run in suite or within the package, it'll be correctly ignored.

Here is the issue: https://youtrack.jetbrains.com/issue/IDEA-194561

The same works for methods: in case you run the test annotated with `@Disabled` annotation explicitly, it will be run; in case you run the class with such a method, it will be correctly ignored.

Here is the issue for it:

https://youtrack.jetbrains.com/issue/IDEA-204955

 

0
Comment actions Permalink

When running `All in X` (Project-wide tests), I'd expect the method level with `@Disabled` to be skipped. Currently these are executed, but if the class has `@Disabled` the class is skipped when running project-wide tests.

1
Comment actions Permalink

Hello Larchy,

If possible can you please share the sample project where the case is reproduced for you (you may upload it here: https://uploads.jetbrains.com/ and share the upload id or attach the project to the issue created at YouTrack: https://youtrack.jetbrains.com/).

Thank you 

0
Comment actions Permalink

I  added a comment to the task:

https://youtrack.jetbrains.com/issue/IDEA-204955

Maybe you make it more clear to us, not to bother you in the future with this unclear feature.

0

Please sign in to leave a comment.