incremental patches

EAP builds get bigger and bigger, I have to think twice before downloading another 40Mb update. Why not to release incremental binary patches along with full versions?

For example Xdelta can be used: http://xdelta.org/
I've used idea3177.tar.gz and idea3193.tar.gz, both about 38Mb, to test it (don't have intermediate builds so actual results should be better). Xdelta made a 15Mb patch in several seconds.

16 comments
Comment actions Permalink

Konstantin Sobolev wrote:

EAP builds get bigger and bigger, I have to think twice before downloading another 40Mb update. Why not to release incremental binary patches along with full versions?

For example Xdelta can be used: http://xdelta.org/
I've used idea3177.tar.gz and idea3193.tar.gz, both about 38Mb, to test it (don't have intermediate builds so actual results should be better). Xdelta made a 15Mb patch in several seconds.


I've never really understood the concern over the size of the downloads.
Are there really a lot of professional developers without access to
broadband connections? And, at the current rate of EAP releases (less
than once per week, seems like) is it really such a burden? I'd rather
JetBrains work on getting features done and bugs fixed than getting
patches to work reliably.

Now if patching meant more frequent EAP releases, I'd be more
enthusiastic about the idea.

0
Comment actions Permalink

from the other side smaller files means that more people will download them and test fresh builds

BTW I have a very fast connection but have to pay for traffic

0
Comment actions Permalink

Maybe too much traffic costs a lot of money for Jetbrains?

Tom

0
Comment actions Permalink

Thomas Singer (MoTJ) wrote:

Maybe too much traffic costs a lot of money for Jetbrains?

Tom


Could be. Hence my comment about how I'd be for it if it meant more
frequent releases.

0
Comment actions Permalink

JB doesnt have a process in place for releasing patches from an obfuscated set of class files.

There is another thread discussing this as well. {:-)

Nick

0
Comment actions Permalink

why does it matter if classes are obfuscated? or if they are classes at all? Xdelta works with any binary files, be they .tar.gz, .exe or anything else.

0
Comment actions Permalink

why does it matter if classes are obfuscated? or if they are classes
at all? Xdelta works with any binary files, be they .tar.gz, .exe or
anything else.


Jar files from, say, build 2100, are not compatible with build 2101, because
the obfuscated names are generated differently (maybe randomly, I don't know)
at each build. So, this type of patch would leave parts of IDEA jars with
references to methods that don't exist anymore.

-Keith


0
Comment actions Permalink

Keith,

+Jar files from, say, build 2100, are not compatible with build 2101, because
the obfuscated names are generated differently (maybe randomly, I don't know)
at each build. So, this type of patch would leave parts of IDEA jars with
references to methods that don't exist anymore.+

I don't think this should make any difference, since the
resulting patch is used to bridge one binary file (v1) to
another (v2) and the result is the target binary file, 1:1
with respect to the original v2 file.
Although the change may have amounted to several class changes between two
successive EAP builds and would require only small patches
if someone had the info about the actual change, xdelta
cannot assume this, and instead creates the set of
instructions which would bring v1 to v2. So instead of
a <1M patch for example, if the info is known (ie. JB are
creating the patch) we get 15MB patch without needing to
know what the change actually was.

Much like any version control system does, really.

Cheers,

Bonny

0
Comment actions Permalink

exactly. Xdelta doesn't know anything about the nature of files it compares.

0
Comment actions Permalink

But the key thing here is that while the Java classes may or may not run, it will screw up the obfuscation, and Jetbrains will most likely not be able to understand any submitted stack traces.

I think this problem is solvable, but I dont think its as simple as it first seems.

0
Comment actions Permalink

Nick Pratt wrote:

>I think this problem is solvable, but I dont think its as simple as it first seems.

>

Delta files should work at the bit level, so
version_1 + delta_1_to_2
should be the exact copy of
version_2

What's complicated in that?

Alain

0
Comment actions Permalink

IMHO you're absolutely right.

The only drawback without incremental obfuscation is that there are much
more changes between two versions than what actually changed in code. So
the patch becomes bigger than necessary, but after applying the patch
you get an exact copy of the new version.

Alain Ravet wrote:

Nick Pratt wrote:

>> I think this problem is solvable, but I dont think its as simple as it
>> first seems.
>>
>>


Delta files should work at the bit level, so
version_1 + delta_1_to_2
should be the exact copy of
version_2

What's complicated in that?

Alain


--
Martin Fuhrer
Fuhrer Engineering AG
http://www.fuhrer.com

0
Comment actions Permalink

Hi,
4109 4 mb bigger then 4101. May be it's time to renew discussion on incremental patches?

0
Comment actions Permalink

And can library archives be detached from common archive? They are rather big and do they change in every release?

0
Comment actions Permalink

Actualy in most cases idea.jar changed and it compressed by 7zip takes 7 mb.

0
Comment actions Permalink

Hello Alexander,

AB> And can library archives be detached from common archive? They are
AB> rather big and do they change in every release?

I can very well imagine troubleshooting all kinds of problems that will inevitably
happen if we implement such a scheme and people start using mismatched versions
of libraries and common archives.

Binary patches are easier in this aspect - you need to get correct versions
of files, or the patch will not apply at all.

--
Dmitry Jemerov
Software Developer
JetBrains, Inc.
http://www.jetbrains.com
"Develop with pleasure!"


0

Please sign in to leave a comment.