compileOnly dependencies from Gradle are not picked up by IntelliJ

The following is my IDEA plugin setup which does not seem to pick up any dependencies I declared as `compileOnly`. They are all shown as missing after running a `./gradlew clean cleanIdea idea` too.

idea {
    module {
        inheritOutputDirs = false
        outputDir = file("$buildDir/classes/main/")
        testSourceDirs += file("src/test/java")
    }

    project {
        languageLevel = "1.8"
        jdkName = "1.8"
    }
}