JetBrains doesn't test its builds ?

Folks, why do I have an impression that JetBrains developers don't even run some minimal set of regression tests for a new build they make avilable for download ?
They don't even try to install it to notice the icon is wrong ? Are we supposed to be their testers reporting about most trivial bugs ? Honestly, I was expecting more attention and care to the crowd of EAP users. Sorry for the language, I really appreciate JetBrains and its product. Really. But the fact they don't test their builds surprises me a lot

55 comments
Comment actions Permalink

Btw, please, don't answer "It's an EAP, use IDEA release instead". I'm surely aware of it. But even EAP should be tested in-house somehow. The icon thing really shows that it was never even installed by the original developers.

0
Comment actions Permalink

What icon thing? I for one didn't notice it.

Oh... btw, it's EAP, use IDEA release instead.
;o)

Regards, Stefan.

0
Comment actions Permalink

What icon thing? I for one didn't notice it.


"So, the icon is a missile now?"
( http://www.intellij.net/forums/thread.jsp?forum=22&thread=39115&message=550476&q=69636f6e20696e7374616c6c416e7977687265#550476 )

Btw, just went over "908 again unusable"
( http://www.intellij.net/forums/thread.jsp?forum=22&thread=39177&tstart=0&trange=15 )
which just emphasizes my words ..

I would really like to hear JetBrains opinion about how do they test Aurora builds.

0
Comment actions Permalink

+10 thanks for letting us even have access to the EAP builds/releases

0
Comment actions Permalink

Btw, just went over "908 again unusable"


Here's what Hani Suleiman say:

What's depressing is that this EAP has been going on for quite a while, and doesn't look like it's even moving in the direction of a release yet. Even if it were, I'd imagine it's a few more months before it's stable enough to be called production release

Exactly ! Aurora seems moving in direction of adding nice-to-have features instead of stabilizing and shipping. A friend of mine gave up on using Aurora and moved to 3.0.5 IDEA release and it was a while ago. Things don't seem to change, though .. :(

0
Comment actions Permalink

Opps, sorry that was +10 for "Oh... btw, it's EAP, use IDEA release instead."

0
Comment actions Permalink

We work on our builds as well as on the versions compiled from the code taken from the VCS.

They don't even try to install it to notice the icon is wrong ?


I don't think that the wrong icon should be a reason to make a new build.

--

Best regards,
Eugene Zhuravlev
JetBrains, Inc, http://www.intellij.com
"Develop with pleasure!"


0
Comment actions Permalink

But do you test them ? It seems that the answer is "No, we have EAP people for it" :)

0
Comment actions Permalink

Well, I don't currently use #908 because of the one nasty bug that hits me every time
I type a key. On the other hand, several people reported that 908 works pretty well,
so I'd say this is a problem that may very well slip through EAP release tests.

Regarding the icon, this is a very minor cosmetic issue I couldn't care less about.
I really don't expect that the release of a new EAP build gets delayed because of a
desktop icon, which might annoy JetBrains themselves more than any user.

Sascha


0
Comment actions Permalink

We work on them isn't it the best testing? So I think the answer is "yes".

--

Best regards,
Eugene Zhuravlev
JetBrains, Inc, http://www.intellij.com
"Develop with pleasure!"

"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message news:28069360.1062070336307.JavaMail.itn@is.intellij.net...

But do you test them ? It seems that the answer is "No, we have EAP people for it" :)



0
Comment actions Permalink

About a wrong icon: it's only my fall. I'm not quite familiar with Install
Anywhere and I'm not quite attentive to such thing as an created icon.

Of course, we test every build before publish it as EAP.

"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message
news:28069360.1062070336307.JavaMail.itn@is.intellij.net...

But do you test them ? It seems that the answer is "No, we have EAP

people for it" :)


0
Comment actions Permalink


They don't even try to install it to notice the icon is wrong ?


I don't think that the wrong icon should be a reason to make a new build.


I agree, if they are anything like me - their "eap upgrade" process is
something similar to:

1. backup the idea directory from c:\whatever to c:\whatever.old
2. unzip the non-jre build zip (not the installer) into c:\whatever
3. run idea as before

Sometimes requries the cache directory in system clearing out - but leaves
me always to revert to the c:\whatever.old directory and continue with a
functional EAP version.

I know your aware youre using an EAP, but if the icon being correct is the
pollished finish your looking - then an EAP isnt for you. As there has been
positive and negative responses to this build (and Ive only found a couple
of bugs that werent in the previous, and some fixed that were in the
previous) its obvious that it is ready for wider scope usage.

Releasing an EAP build is saying "it works for our team in house, lets see
if it works for the rest of you". If it does - count yourself lucky, if it
doesnt - file a bug report.

Why people complain about a product that is unfinished, I will never
understand (as a product developer myself, it pisses me off that they will
happily critise the smallest of details missing the bigger picture) - and I
applaud JetBrains for having the balls to put public betas of a commerical
product out there. Well done.


0
Comment actions Permalink

I like your position. Seems like whatever the answer from our side it will
appear as whisper of a schoolboy in fault.

--

Best regards,
Maxim Shafirov
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"


"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message
news:28069360.1062070336307.JavaMail.itn@is.intellij.net...

But do you test them ? It seems that the answer is "No, we have EAP

people for it" :)


0
Comment actions Permalink

i'd say the ctrl-f and variants bug warrants a new build.

my humble opinion :)

0
Comment actions Permalink

No, Eugene, the way you describe it - the answer is still "No". "Testing" != "working on it". For me, testing means running automatically all regressions tests, unit tests, QA tests, performance tests. But as Stanislav say below - Aurora builds are tested which is much better answer.

0
Comment actions Permalink

It's a very tough position that jetbrains is in. When they go a couple of weeks or a month without an eap build everyone complains becuase they want a build.

0
Comment actions Permalink

Releasing an EAP build is saying "it works for our team in house,
lets see if it works for the rest of you". If it does - count
yourself lucky, if it doesnt - file a bug report.


You've hit the nail on the head here Dan, it's all down to luck. There is a
development cycle running concurrently with a release cycle, and the points
at when those coincide must be at best an educated guess for Jetbrains.
There's no way they can test to a commercial product standard every time,
especially as the product gets more and more complicated.

It's been said on here before, and sadly I'm sure it will have to be said
again:
If a new build doesn't work for you, bad luck. Keep the rattle in the pram,
fall back to a previous build that you're happy with, and wait quietly for
the next one. It's up to Jetbrains whether they concentrate on stabilizing
or new features, and they're plenty capable to make that decision
themselves.

Think about it, if they had tested it to the degree you're expecting then
they would have found the bugs and not released the build, so me and plenty
others would have been deprived, and you'd have been no better off.

N.


0
Comment actions Permalink

First, my point about the icon was misunderstood. Did you really think I care much about how does the EAP icon look like ? Of course no ! My point was the following "Did anybody at JetBrains install the new version and saw the wrong icon ?" Hell, it's not that tiny icon that was important to me. The fact that nobody seemed to notice it before the EAP crowd - that what surpised me. What made me think that may be nobody even installed the new build ... I hope my point is clear now.

Second, I don't complain. And I apologize if someone saw my words that way. I also (as many others) thankfull to JetBrains for allowing us to be in contact with latest EAP builds, CVS supports and JUnit integrations. All I tried to figure out is whether Aurora builds are tested in some way or not, as simple as that. So far, the answer we received is "We use it ourselves, that's our testing". Whether it's enough testing or not I think is a personal opinion of everyone. For me, it wouldn't be enough. But that's me.

Folks, really, I didn't mean to anger anybody. Beer ? :)

0
Comment actions Permalink

Please ;)

Evgeny Goldin wrote:

First, my point about the icon was misunderstood. Did you really
think I care much about how does the EAP icon look like ? Of course
no ! My point was the following "Did anybody at JetBrains install the
new version and saw the wrong icon ?" Hell, it's not that tiny icon
that was important to me. The fact that nobody seemed to notice it
before the EAP crowd - that what surpised me. What made me think that
may be nobody even installed the new build ... I hope my point
is clear now.

>

Second, I don't complain. And I apologize if someone saw my words
that way. I also (as many others) thankfull to JetBrains for allowing
us to be in contact with latest EAP builds, CVS supports and JUnit
integrations. All I tried to figure out is whether Aurora builds are
tested in some way or not, as simple as that. So far, the answer we
received is "We use it ourselves, that's our testing". Whether it's
enough testing or not I think is a personal opinion of everyone. For
me, it wouldn't be enough. But that's me.

>

Folks, really, I didn't mean to anger anybody. Beer ? :)



0
Comment actions Permalink

Releasing an EAP build is saying "it works for our team in house, lets see
if it works for the rest of you". If it does - count yourself lucky, if it
doesnt - file a bug report.


Good point.

Why people complain about a product that is unfinished, I will never
understand (as a product developer myself, it pisses me off that they will
happily critise the smallest of details missing the bigger picture) ...


What is the purpose of EAP? Jetbrains do not run it to make us happy.
They do it mostly to have a broad audience, that uses IDEA in a vary of
different environments and find (and hopefully report) bugs, they never
would find. Surely, it's a good side-effect, that we EAP-members are
able to steer the IDEA ship a little bit in the direction we like and
need it.

But this deal between Jetbrains and EAP users only works, if the
released EAP versions are stable enough, that we can use it in our daily
work.

Surely, I've started this thread too early, because wiping my cache
directory solved the major problems I had with the first actions in
build 908. Now I have a quite reasonable version, that only has a small
bug from time to time. But obviously there are other people, for those
this build is completely useless, because it crashes on each keystroke.
Fixing such severe bugs and releasing a new build soon (no new feature,
only the most severy bugs fixed) would make all happy: us, because we
get a more stable build to use and Jetbrains, because they get more bug
reports from daily work (more users are able to use the EAP versions).
Especially, because Jetbrains release new builds relatively seldom
compared to one or two years ago, I think, this is the way to go.

Just my 0.02 EUR.

Tom

0
Comment actions Permalink

Okay, okay... I've installed it twice before publishing in EAP. And I saw a
strange icon (I really didn't know that it's a Install Anywhere).
Our icons has been changed very often last time :)

"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message
news:3202268.1062073642003.JavaMail.itn@is.intellij.net...

First, my point about the icon was misunderstood. Did you really think I

care much about how does the EAP icon look like ? Of course no ! My point
was the following "Did anybody at JetBrains install the new version and saw
the wrong icon ?" Hell, it's not that tiny icon that was important to me.
The fact that nobody seemed to notice it before the EAP crowd - that what
surpised me. What made me think that may be nobody even installed the
new build ... I hope my point is clear now.
>

Second, I don't complain. And I apologize if someone saw my words that

way. I also (as many others) thankfull to JetBrains for allowing us to be in
contact with latest EAP builds, CVS supports and JUnit integrations. All I
tried to figure out is whether Aurora builds are tested in some way or not,
as simple as that. So far, the answer we received is "We use it ourselves,
that's our testing". Whether it's enough testing or not I think is a
personal opinion of everyone. For me, it wouldn't be enough. But that's me.
>

Folks, really, I didn't mean to anger anybody. Beer ? :)



0
Comment actions Permalink

I like yours, too. Seems like when lot's of people are screaming "We can't work !" your answer is "Whops, sorry, bad luck." Why not taking some more responsibility for something not working the way it should ?

0
Comment actions Permalink

Ok, sorry for making a wrong assumption.

0
Comment actions Permalink

Dude, you are kidding right? If you are depending on an EAP build for you "work", that's your fault.

Use the current release version for "work".

0
Comment actions Permalink

Evgeny Goldin wrote:

I like yours, too. Seems like when lot's of people are screaming "We
can't work !" your answer is "Whops, sorry, bad luck." Why not taking
some more responsibility for something not working the way it should?


But it does work the way it should.

-- Provides a preview of how the next major release of IDEA will look
and the features that will eventually be present, in order to satisfy
our curiosity: Check.

-- Provides an opportunity for us to guide the development so that
features are eventually implemented more closely to the way people want
them to work: Check.

-- Provides a way for us to help JetBrains report bugs that would not
affect their systems or their usage patterns, so that these bugs can be
fixed before the major release and not after: Check.

Not to mention that as an added bonus some of the builds can actually be
used for production work, despite not being actual releases

0
Comment actions Permalink

It wasn't me who started "Build XYZ is unusable" threads. Yes, I'm using the latest Aurora build for doing my work but, fortunately, didn't have something that stopped me completely till now. Please, I heard enough "use current release" to repeat it every time ...

0
Comment actions Permalink

The last thing I want to do is to start the same war again. Still

Seems like I'd to citate the words in bold you see each time you download a
new EAP build:

NOTE: THIS IS PREVIEW SOFTWARE AIMED AT PROVIDING YOU WITH AN OPPORTUNITY
TO TEST THE NEWEST FEATURES SCHEDULED FOR THE NEXT MAJOR RELEASE.
JetBrains, Inc. DOES NOT GUARANTEE THAT THIS SOFTWARE WILL RUN TROUBLE-FREE.
USING THIS SOFTWARE IS AT YOUR OWN DISCRETION AND DOES NOT ENTITLE YOU TO
ANY
CUSTOMER SUPPORT (EVEN IF YOU OWN A SUPPORT CONTRACT).

Seems like when lot's of people are screaming "We can't work !"

But they can. Nobody prevents them from installing previous build or using
released version.

Why not taking some more responsibility for something not working the way

it should ?
Don't shoot the pianist...

--

Best regards,
Maxim Shafirov
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"


"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message
news:6264075.1062074485140.JavaMail.itn@is.intellij.net...

I like yours, too. Seems like when lot's of people are screaming "We can't

work !" your answer is "Whops, sorry, bad luck." Why not taking some more
responsibility for something not working the way it should ?
>


0
Comment actions Permalink

Why do I always have to hear the same excuse for unstable and buggy builds - "It's an EAP". Because it is, you say ? As a developer - don't you feel responsible for allowing people to download something that works and not crashes every time they type a letter or trying to use a new feature ?

People, come on, I really don't complain, please, read my post below. All the noise is about whether JetBarins whishes their EAP builds to be more stable or not. It seems that when people are coming and saying "Hey, it's really unstable this time" the answer is "So what ? I told ya, it may not work, file a bug". Honestly, if it was me - I would be embarrassed to give such an answer with a "So what" attitude. I would say "Ok, we'll work to make the next build more stable. Probably instead of adding some more bells and whistles".

0
Comment actions Permalink

+1, if I could not work with eap (which is the case, because EJB support is down amongst other things), I don't use it and of course don't fill any bug report.
Don't have the time to install eap just to see, or to try if it works, because I'm not part of jetbrain's testing team. I don't think you save time with eap : you have new features but you don't have some old ones and you may have problems (stability and performance) with it that slow your work down. I consider, that EAP is helping jetbrains a lot more than users (other than some geeks), and it is therefore jetbrain's responsability to provide the most stable eap build possible. In order to let users give them feedback without having to spend a lot of time tackling with it.
As I see messages in forums, it will be clear for me that I will NOT install 908 : even if all the features I use were in the eap, it seems to be too instable. The result is : if people are thinking like me, 908 will not be much used and tested and it is jetbrains that will eventually suffer from this situation, not me.

0
Comment actions Permalink

Why do I always have to hear the same excuse

Wrong word excuse. Guess why.

As a developer - don't you feel responsible for allowing people to

download something that works and not crashes every time they type a letter
or trying to use a new feature ?
Yes I do.

I would say "Ok, we'll work to make the next build more stable"

We do. And there's no need to say that.

And we would did that faster and more effectively if I hadn't continue this
senseless quarrel with you. So I just shut up here.

Just one last note: if we do a full production testing for each EAP build
you'll get not more than one build in 1.5-2 month. This is not what EAP is
for.

Take the following statement as if it was written in bold:
The major point of EAP is not testing but letting us (JetBrains) know if the
product goes in right direction, what the community expects from the product
and how the product meets their needs.

I agree that EAP build that fails on each typing doesn't meet its goal so
least I should say: I'm sorry and try to take it a bit easier.

--

Best regards,
Maxim Shafirov
JetBrains, Inc / IntelliJ Software
http://www.intellij.com
"Develop with pleasure!"


"Evgeny Goldin" <no_mail@jetbrains.com> wrote in message
news:19072878.1062076720042.JavaMail.itn@is.intellij.net...

Why do I always have to hear the same excuse for unstable and buggy

builds - "It's an EAP". Because it is, you say ? As a developer - don't you
feel responsible for allowing people to download something that works and
not crashes every time they type a letter or trying to use a new feature ?
>

People, come on, I really don't complain, please, read my post below. All

the noise is about whether JetBarins whishes their EAP builds to be more
stable or not. It seems that when people are coming and saying "Hey, it's
really unstable this time" the answer is "So what ? I told ya, it may not
work, file a bug". Honestly, if it was me - I would be embarrassed to give
such an answer with a "So what" attitude. I would say "Ok, we'll work to
make the next build more stable. Probably instead of adding some more bells
and whistles".


0

Please sign in to leave a comment.