What's wrong with the EAP/Beta/Release Candidate process?
At the risk of “rubbing it in”, “kicking while you are down”, or just plain sounding like whining…
This major bug somehow managed to slip through 6 EAPs, 2 Betas, and 1 Release Candidate:
Inspections do not re-evaluate/refresh during editing : IJPL-28967 (jetbrains.com)
This was a completely obvious productivity killing issue. I hard for me to imagine how JetBrains didn't notice and fix it themselves, even without any user feedback. Don't a lot of people within the company use their own IDE?
Anyways… what has been learned from this failure? What kinds of fundamental changes need to made to the process to prevent this from happening again?
By the way, I do sympathize with the difficulties created by the closure of the Moscow office. A Reddit user offered that explanation about the quality problems here, and it makes a lot of sense:
Has Anyone Else Noticed Declining Quality in JetBrains' Products? : r/Jetbrains (reddit.com)
If that's the main issue, then hopefully just the passage of time will allow JetBrains to recover a little bit? Should I hold my breath for that, or not???
请先登录再写评论。
Hi, Tcfurrer,
Thanks for the raising this issue.
We deeply regret the inconvenience caused by the bug. We will organize a post-mortem analysis of this issue to understand why we failed to fix it in the 2024.1 release, what we can learn from it, and what we need to do to prevent/reduce similar issues in the future after we fix this issue.
Before we conduct a thorough analysis, I can share some rough answers from my opinion regarding your issue:
> This major bug somehow managed to slip through 6 EAPs, 2 Betas, and 1 Release Candidate:
I hard for me to imagine how JetBrains didn't notice and fix it themselves, even without any user feedback.
We noticed an increased occurrence of this highlighting issues in the 2024.1 EAP version, probably due to some significant refactoring in our highlighting area. Then we made some fixes for it in 2024.1: https://youtrack.jetbrains.com/issue/IJPL-5236
However, later we discovered that we still haven't fixed all the cases, which is causing IJPL-28967. Additionally, we don't have a stable reproducer for this issue.
> Don't a lot of people within the company use their own IDE?
I have no data, but from my observation, almost all of our developers use our own IDEs.
For this specific issue, most affected users are from the PHP, JS, TS and Go users. But a large portion of our internal people use our own IDE to write Kotlin/Java/C# code so we don't have a lot of internal users using/testing other languages compared to JVM/C# languages, which means we miss the awareness of this issue a bit.
> By the way, I do sympathize with the difficulties created by the closure of the Moscow office. A Reddit user offered that explanation about the quality problems here, and it makes a lot of sense:
I can't comment on anything related to this topic, and I also lack some information about it.
But I think one reason for the quality issues in our IDE product is that we are attempting to make more changes in our IDE platform and evolve our IDE codebase. This means that there are some significant fundamental changes happening behind the scenes, but these changes can also break something and lead to significant problems.
I believe that implementing some workflow changes from our QA/support/developer side can greatly contribute to improving our quality areas, and we can do better here.
Lejia Chen Thanks for the reply! I'm glad to hear that at least some efforts towards improvement are happening. We recognize the monumental challenges that you face, and are sincerely rooting for you!
It's news to me that the inspection/spellcheck refreshing issue is more common for PHP/JS/TS/Go. I was assuming it was merely a coincidence that people working in those languages reported it more. I'm working on a purely Java project, and everyone on the team was seeing this issue very frequently on 2024.1 (before moving to 2024.1.1). Since spellcheck is also involved, it can impact anyone anywhere, as far as I can tell. I still don't understand how large teams of internal developers would see this any less than my team. Solving that mystery may or may not help for improving quality control.
Thanks again!
> I still don't understand how large teams of internal developers would see this any less than my team. Solving that mystery may or may not help for improving quality control.
I just found one more reason here: we froze the code change in the 2024.1 branch near the beginning of March and uses the master branch for 2024.2 code base. So most of the internal people are switching to the internal 2024.2 nightly IDE builds.
Then we made some related changes to fix this issue in master branch, but some fixes like https://github.com/JetBrains/intellij-community/commit/37096609736fceadf2dc51d85d09220aa2f42a4c is not backported into the 2024.1 branch.
Because the master branch addressed more cases of this issue, it was less observable for us internally.
We will do a deep internal analysis to avoid a similar case in the future.
Sounds good. Thanks for sharing that learning.