"Download sources" download source that mismatch class file

First of all, I am pretty new to Gradle and I am not a power user of Intellij, but I have been working with software development for 15+ yrs.

I was about to debug code from a gradle dependency. Pressing ctrl+n and entering the class' full path gave me the decompiled .class file. When I clicked "Download sources" just above the class file in Intellij it downloaded something that mismatched the bytecode I just saw (Intellij wrote "Library source does not match the bytecode for class xxx"). Why was Intellij not able to download the "right" java file? How does all this magic with downloading the java file work?