failed assertions
Hello,
A couple of times now I have submitted "assertion failed" errors to the
tracker, only to find a couple of days later that the issue was moved to
the "non repeatable" state. Somehow this makes me feel uneasy - it seems
as if the problem was being ignored, even though it was clearly there,
and experience tells us that unless it is dealt with, it will certainly
reappear sooner or later. Any comments?
regards,
Christian.
请先登录再写评论。
Christian,
If the assertion itself allows to fix than it's usually fixed. However often
the situation is the following: we have no clue how this could happen. If we
have no steps to reproduce we can't fix it. So we mark them as "not
repeatable". If you are aware of the reproducing method, please share it
with us.
--
Best regards,
Mike Aizatsky.
-
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"
If I had had better clues, I would have posted them. I assumed that the
assertion itself carried enough information to be useful - after all,
thats why it is there. Anyway, I understand your position. Lets just
hope (if thats all we can do) the problems dont reappear in GA.
Mike Aizatsky wrote:
I don't think JetBrains programmers spend too much time trying to repro the issues.
In fact sometimes they were not able to repro the problems with precise steps
given to them
( http://www.intellij.net/tracker/idea/viewSCR?publicId=3164 ,
http://www.intellij.net/tracker/idea/viewSCR?publicId=4000 ,
http://www.intellij.net/tracker/idea/viewSCR?publicId=4992 )
After that it's not surprising that the difficult-to-repro issue is not opened after
many reports that the problem does indeed happen
( http://www.intellij.net/tracker/idea/viewSCR?publicId=2422 )
I think it's because the programmers are pressed to close as many bugs as possible
and they close them and very reluctant to reopen the issues
( http://www.joelonsoftware.com/news/20020715.html )
and don't try hard to repro the issues.
I wish I could tell to my customer:
1) I spent 1 minute trying to think how it happened and I don't know the reason.
2) I spent 10 minutes trying to repro it and it worked for me.
3) Come back when you have more precise steps to repro, I can't fix it because I can't repro it.
Instead I have to work hard trying to repro the issue because, well, it's my problem, not the customer's.
Timur Zambalayev
"Mike Aizatsky" <mike@intellij.com> wrote in message news:anuu9o$mup$1@is.intellij.net...
>
>
>
>
well, currently we are not customers (at least not with regard to the
EAP), and as long as that is the case, let the Jetbrainers take their
freedom. Afterwards, its a different story..
But I agree, getting a "non repeatable" reply half a day after posting
the item just dont feel good
Timur Zambalayev wrote:
>>Christian,
>>
>>If the assertion itself allows to fix than it's usually fixed. However often
>>the situation is the following: we have no clue how this could happen. If we
>>have no steps to reproduce we can't fix it. So we mark them as "not
>>repeatable". If you are aware of the reproducing method, please share it
>>with us.
>>
>>--
>>Best regards,
>>Mike Aizatsky.
>>----
>>JetBrains, Inc / IntelliJ Software
>>http://www.intellij.com
>>"Develop with pleasure!"
>>
>>
I think we're still customers even when we use the EAP.
We're either former or current customers (I paid $200 in January
for a personal license; I almost didn't use 2.5, used the latest
6xx EAP build from the very start of the 6xx EAP) or future customers
(a coworker spent $400 in August after several months of using 6xx EAP builds;
he didn't use 2.x and of course he still uses an 6xx EAP build).
"Christian Sell" <christian.sell@netcologne.de> wrote in message news:3DA31FD3.40204@netcologne.de...
>
I respectfully disagree.
What is the use of an EAP if it isn't to make a more stable product before
GA? Actually since it isn't a custom product and that you pay only $300 for
it, I wager that JetBrains doesn't have the man power to send a very special
patch for a bug that cripples a too few of us and that might be difficult to
repeat. So right now is the time to get our nasty bugs fixed or you will
have to wait for 1 month 1/2 to get the next EAP.
"Christian Sell" <christian.sell@netcologne.de> wrote in message
news:3DA31FD3.40204@netcologne.de...
>
>
>
the issues.
steps
opened after
as possible
reason.
because I can't repro it.
it's my problem, not the customer's.
news:anuu9o$mup$1@is.intellij.net...
often
If we
>
>
Timur,
I just tried to reproduce the first 3 bugs (well, 2 - 4992 is marked as
fixed) just of curiosity. I mast admit that they are also irreproducible on
my box.
--
Best regards,
Mike Aizatsky.
-
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"
Timur,
It's impossible to fix the problem if you have no ideas how to reproduce it.
Moreover many bug submitters also do not know how they got some errors. Often
stacktraces are not sufficient to understand the source of the problem.
This means that these requests are REALLY not repeatable. Moreover state
"Nor repeatable" doesn't mean "We'll never ever fix it even if a way to reproduce
it is found". It only means that it's impossible (at least right now) to reproduce it.
This state should provoke to submit as more information as it possible to help the
developers rather then wait when "Open" state will be changed to "Fixed"
Sincerely yours,
Vladimir Kondratyev
_____________________
JetBrains
Timur Zambalayev wrote:
>>Christian,
>>
>>If the assertion itself allows to fix than it's usually fixed. However often
>>the situation is the following: we have no clue how this could happen. If we
>>have no steps to reproduce we can't fix it. So we mark them as "not
>>repeatable". If you are aware of the reproducing method, please share it
>>with us.
>>
>>--
>>Best regards,
>>Mike Aizatsky.
>>----
>>JetBrains, Inc / IntelliJ Software
>>http://www.intellij.com
>>"Develop with pleasure!"
>>
>>
Mike,
The first 3 bugs (3164, 4000, 4992; 3164 is a dupe of 4000; 4000 was really fixed in #656) are fixed in #656/657
but it took several attempts because the instructions how to reproduce were not read carefully initially.
Timur Zambalayev
"Mike Aizatsky" <mike@intellij.com> wrote in message news:ao0td7$gjf$1@is.intellij.net...
>
>
>
>
"State of this request was changed from "Submitted" to "More info needed"
very good idea
Vladimir Kondratyev wrote:
>> I don't think JetBrains programmers spend too much time trying to
>> repro the issues.
>> In fact sometimes they were not able to repro the problems with
>> precise steps
>> given to them
>> ( http://www.intellij.net/tracker/idea/viewSCR?publicId=3164 ,
>> http://www.intellij.net/tracker/idea/viewSCR?publicId=4000 ,
>> http://www.intellij.net/tracker/idea/viewSCR?publicId=4992 )
>>
>> After that it's not surprising that the difficult-to-repro issue is
>> not opened after
>> many reports that the problem does indeed happen
>> ( http://www.intellij.net/tracker/idea/viewSCR?publicId=2422 )
>>
>> I think it's because the programmers are pressed to close as many bugs
>> as possible
>> and they close them and very reluctant to reopen the issues
>> ( http://www.joelonsoftware.com/news/20020715.html )
>> and don't try hard to repro the issues.
>>
>> I wish I could tell to my customer:
>> 1) I spent 1 minute trying to think how it happened and I don't know
>> the reason.
>> 2) I spent 10 minutes trying to repro it and it worked for me.
>> 3) Come back when you have more precise steps to repro, I can't fix it
>> because I can't repro it.
>>
>> Instead I have to work hard trying to repro the issue because, well,
>> it's my problem, not the customer's.
>>
>>
>> Timur Zambalayev
>>
>>
>>
>> "Mike Aizatsky" <mike@intellij.com> wrote in message
>> news:anuu9o$mup$1@is.intellij.net...
>>
>>> Christian,
>>>
>>> If the assertion itself allows to fix than it's usually fixed.
>>> However often
>>> the situation is the following: we have no clue how this could
>>> happen. If we
>>> have no steps to reproduce we can't fix it. So we mark them as "not
>>> repeatable". If you are aware of the reproducing method, please share it
>>> with us.
>>>
>>> --
>>> Best regards,
>>> Mike Aizatsky.
>>> -
>>> JetBrains, Inc / IntelliJ Software
>>> http://www.intellij.com
>>> "Develop with pleasure!"
>>>
>>>
>>
>>
>>
Timur,
I feel obliged to join this discussion.
I have been handling one of the issues (SCR#4992), which was, in your
opinion, rejected because of developer's (i.e. mine) reluctance to open
an issue under management pressure.
While I regret that our work leaves such an impression, I also feel that
it is rather undeserved.
We are trying very hard to process all the issues that come to us
promptly and carefully. However, accidents happen.
For example, a quick search in Tracker reveals that you have submitted,
since the Tracker was opened, 248 issues. You report that of those
issues three (3) items were not reviewed correctly. My calculator tells
me that this is 1.2% of all issues you have submitted. In each case,
after you pointed the mistake out in forum, the request was reviewed
again, and, with appropriate apologies, fixed. It didn't take us a lot
of time to fix each of them either.
Does this seem, in your opinion, a sympthoms of purposefuly rejecting
bug reports? Or just some mistakes that everyone is bound to make?
Deliberately sweeping flaws in one's product under the carper is a
serious violation of this industry's code of ethic (as e.g. in
http://www.acm.org/constitution/code.html).
Please consider well before you charge anyone with it.
Yours,
Dmitry
-
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"
Timur Zambalayev wrote:
>I don't think JetBrains programmers spend too much time trying to repro the issues.
>In fact sometimes they were not able to repro the problems with precise steps
>given to them
>( http://www.intellij.net/tracker/idea/viewSCR?publicId=3164 ,
>http://www.intellij.net/tracker/idea/viewSCR?publicId=4000 ,
>http://www.intellij.net/tracker/idea/viewSCR?publicId=4992 )
>
>After that it's not surprising that the difficult-to-repro issue is not opened after
>many reports that the problem does indeed happen
>( http://www.intellij.net/tracker/idea/viewSCR?publicId=2422 )
>
>I think it's because the programmers are pressed to close as many bugs as possible
>and they close them and very reluctant to reopen the issues
>( http://www.joelonsoftware.com/news/20020715.html )
>and don't try hard to repro the issues.
>
>I wish I could tell to my customer:
>1) I spent 1 minute trying to think how it happened and I don't know the reason.
>2) I spent 10 minutes trying to repro it and it worked for me.
>3) Come back when you have more precise steps to repro, I can't fix it because I can't repro it.
>
>Instead I have to work hard trying to repro the issue because, well, it's my problem, not the customer's.
>
>
>Timur Zambalayev
>
>
>
>"Mike Aizatsky" <mike@intellij.com> wrote in message news:anuu9o$mup$1@is.intellij.net...
>
>>Christian,
>>
>>If the assertion itself allows to fix than it's usually fixed. However often
>>the situation is the following: we have no clue how this could happen. If we
>>have no steps to reproduce we can't fix it. So we mark them as "not
>>repeatable". If you are aware of the reproducing method, please share it
>>with us.
>>
>>--
>>Best regards,
>>Mike Aizatsky.
>>----
>>JetBrains, Inc / IntelliJ Software
>>http://www.intellij.com
>>"Develop with pleasure!"
>>
>>
>
>
Dmitry,
Below I'll try to give the reasons for my earlier post:
1) I saw examples when JetBrains programmers were not able to repro the problem initially or
didn't read the messages carefully enough
(other examples: http://www.intellij.net/tracker/idea/viewSCR?publicId=2709 ,
http://www.intellij.net/tracker/idea/viewSCR?publicId=4180 ).
SCR#4992 was only an example of "JetBrains programmers don't spend too much time to evaluate issues".
2) I was upset with the way http://www.intellij.net/tracker/idea/viewSCR?publicId=2422 ,
http://www.intellij.net/tracker/idea/viewSCR?publicId=5197 were handled.
#5197 was closed without giving any explanations in either #2422 and #5197
(I explicitly said in #5197 that this is a reopening of #2422) leaving me with the feeling that I'm just wasting
my time reporting the problem. I know that marking as "Not Repeatable" is not really a closing of a problem but
in reality it is (votes are returned; in the tracker they have the same color as closed items;
in practice you don't return to "Not Repeatable" issues unless there are new posts;
there are already a lot of old "Not Repeatable" issues).
3) I heard things almost like "It's your problem. You have to supply all needed information to repro, otherwise
we can't fix it". I can understand it and agree with it to some degree but you shouldn't forget that we're not paid as testers,
we have our work that we must do, moreover many of us are paying customers who are just willing to spend their free time
to report the problems and file requests,
that it's your (general-purpose) product, it's your job and in the end it's your responsibility to identify and fix problems.
"... purposefully rejecting bug reports", "Deliberately sweeping flaws ..."
I don't think you do this but where's the distinction between "purposefully rejecting bug reports"
and "ignoring the problem/question for a long time":
http://www.intellij.net/tracker/idea/viewSCR?publicId=2711
http://www.intellij.net/tracker/idea/viewSCR?publicId=4991
http://www.intellij.net/tracker/idea/viewSCR?publicId=3794
http://www.intellij.net/tracker/idea/viewSCR?publicId=2741
(though I understand that it's very difficult to please everyone and that at any given moment
you have other more important issues to work on or to discuss).
Having said all this, overall I'm happy with the current process (which I think improved lately)
and with the way JetBrains handle bugs and requests.
Timur Zambalayev
"Dmitry Lomov" <dsl@tepkom.ru> wrote in message news:3DA7DBEA.9000909@tepkom.ru...
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Timur,
please see my comments below.
Timur Zambalayev wrote:
> the problem initially or didn't read the messages carefully enough
> too much time to evaluate issues".
I am afraid you missed my point. Please re-read my post.
I do not question the facts. I only question the far-fetched
conclusions in corporate psychology that you draw from the facts.
> with the feeling that I'm just wasting my time reporting
> a closing of a problem but in reality it is (votes are returned;
> don't return to "Not Repeatable" issues unless there are new posts;
> needed information to repro, otherwise we can't fix it". I can
> understand it and agree with it to some degree but you shouldn't forget
> that we're not paid as testers, we have our work that we must do,
> free time to report the problems and file requests, that it's your
> (general-purpose) product, it's your job and in the end it's your
> responsibility to identify and fix problems.
Verily. I cannot but agree with you here.
However, we also want to be honest with you as a customers. This is
probably somewhat unexpected and causes confusion :)
So I would like to clarify once again our policy regarding bugs that we
cannot reproduce.
There could be three reasons why we cannot reproduce a particular bug
Either the bug have been fixed in course of other fixes, the bug report
is erroneous, or we are not clever enough to figure out the way to
reproduce it from the description user gives.
Previously, bug reports from all those categories have been given a "Not
repeatable" state, which, I agree, confused, and sometimes offended people.
Now we added one more state to the tracker - "More info needed". If the
report falls into the latter category, it is assigned this state. So,
this state is basically an "Open" state.
However, please bear in mind that if we cannot reproduce the bug, we
cannot fix it. In that case, instead of smiling a marketing smile and
saying, "We are working on this", we publicly advertise the fact - to
encourage people to help us. And, in many cases, this works - many times
(I would say even in most of the cases) people sent us very helpful
detailed bug reports.
We also take some steps to try to make problem more easily reproducible
(like adding more assertions to the code and likewise).
However, the corporate psychology conclusions of yours I mentioned
before amount precisely to this.
In all 4 cases you quote, we are not ignoring the issue. We doubt that
the issue should be fixed, we see that community opinion on the issue
diverses, and that community cannot arrive to a single conclusion. So,
we are just evaluating issues. We will form some decision in each of
those cases, but we just need some more time...
I hope you are satisfied with my explanations.
Yours,
Dmitry
--
Dmitry Lomov
JetBrains Inc. / IntelliJ Labs
http://www.intellij.com
"Develop with pleasure!"