View previous topic :: View next topic |
Author |
Message |
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Wed Jun 05, 2019 2:34 pm Post subject: "Xz format inadequate for long-term archiving" |
|
|
So apparently XZ is now BAD™ : http://lzip.nongnu.org/xz_inadequate.html
What are the thoughts on this, is over exaggeration or does it have some merit? (Like is the developer disliked or something) |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Wed Jun 05, 2019 2:39 pm Post subject: |
|
|
Ah, I think I can answer my own question:
Quote: |
In 2008 one of the designers of xz (Lasse Collin) warned me that lzip would become stuck with LZMA while others moved to LZMA2, LZMA3, LZMH, and other algorithms. Now xz-utils is usually unable to match the compression ratio of lzip because LZMA2 has more overhead than LZMA and, as expected, no new algorithms have been added to xz-utils.
|
So there is some rivalry involved, yet all arguments still seem fair. |
|
Back to top |
|
|
erm67 l33t
Joined: 01 Nov 2005 Posts: 653 Location: EU
|
Posted: Wed Jun 05, 2019 2:39 pm Post subject: |
|
|
Since that is what the developer of lzip data compressor thinks about xz it would be also interesting to hear what the developers of xz think about the lzip compression format _________________ Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia
My fediverse account: @erm67@erm67.dynu.net |
|
Back to top |
|
|
erm67 l33t
Joined: 01 Nov 2005 Posts: 653 Location: EU
|
Posted: Wed Jun 05, 2019 2:46 pm Post subject: |
|
|
BTW the standard for long-term document archiving is https://en.wikipedia.org/wiki/PDF/A and it only allow the (de)flate compression algorithm (used also by gzip) since it is considered to be patent free. _________________ Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia
My fediverse account: @erm67@erm67.dynu.net |
|
Back to top |
|
|
mike155 Advocate
Joined: 17 Sep 2010 Posts: 4438 Location: Frankfurt, Germany
|
Posted: Wed Jun 05, 2019 3:01 pm Post subject: Re: "Xz format inadequate for long-term archiving" |
|
|
Please stop spreading FUD. |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Wed Jun 05, 2019 3:57 pm Post subject: Re: "Xz format inadequate for long-term archiving" |
|
|
mike155 wrote: |
Please stop spreading FUD. |
I posted it here because I like others opinion on this, not to say this is my opinion? |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Wed Jun 05, 2019 3:59 pm Post subject: |
|
|
erm67 wrote: | Since that is what the developer of lzip data compressor thinks about xz it would be also interesting to hear what the developers of xz think about the lzip compression format |
Exactly!
Although we can read the (to me fair) reasoning, does not mean it's "THE TRUTH" |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Wed Jun 05, 2019 4:35 pm Post subject: |
|
|
Ah, this crank again. If you get into compression performance, ck's lrzip beats the crap out of both xz and lzip in speed and ratio anyway (even without -z), and has done for about 15 years. zstd outperforms them on all fronts simultaneously too.
lzip is still taking angry little pot-shots at a format that was obsoleted long ago, which shows you how relevant lzip is in today's world: it never was. |
|
Back to top |
|
|
Dr.Willy Guru
Joined: 15 Jul 2007 Posts: 547 Location: NRW, Germany
|
Posted: Thu Jun 06, 2019 8:57 am Post subject: |
|
|
Ant P. wrote: | If you get into compression performance, ck's lrzip beats the crap out of both xz and lzip in speed and ratio anyway (even without -z), and has done for about 15 years. zstd outperforms them on all fronts simultaneously too. |
According to http://www.mattmahoney.net/dc/text.html zstd has a worse compression-ratio than lzip and xz. |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Thu Jun 06, 2019 4:21 pm Post subject: |
|
|
I'll concede that, but I'd also prefer the 10 seconds of extra download time over 20 seconds of CPU time with possible OOM. |
|
Back to top |
|
|
Dr.Willy Guru
Joined: 15 Jul 2007 Posts: 547 Location: NRW, Germany
|
Posted: Fri Jun 07, 2019 7:12 am Post subject: |
|
|
Ant P. wrote: | I'll concede that, but I'd also prefer the 10 seconds of extra download time over 20 seconds of CPU time with possible OOM. |
Yeah afaik zstd's idea is to be used for on-the-fly compression to increase throughput more than archiving. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54212 Location: 56N 3W
|
Posted: Fri Jun 07, 2019 5:12 pm Post subject: |
|
|
Once you move away from parchment or clay tablets, nothing has lasted even 25 years.
We don't have a long-term archiving format. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Amity88 Apprentice
Joined: 03 Jul 2010 Posts: 260 Location: Third planet from the Sun
|
Posted: Sat Jul 27, 2019 4:15 pm Post subject: |
|
|
@Neddy,
Don't forget the cave paintings That was one good time tested long lasting data archive _________________
Ant P. wrote: | The enterprise distros sell their binaries. Canonical sells their users. |
Also... Be ignorant... Be happy! |
|
Back to top |
|
|
Goverp Veteran
Joined: 07 Mar 2007 Posts: 1993
|
Posted: Sun Jul 28, 2019 8:35 am Post subject: |
|
|
For archiving, surely the answer is "no compression". That way (a) you don't get problems if you lose the program, and (b) the archive contains redundant information that can help you detect and correct corruption, and (c) even if parts are corrupt, other parts will be salvageable.
An equally important part of archiving is the physical medium. Some of us will remember the batch of Ampex (?) tapes that went "read once" when the binder aged. Anyone still got important information on Zip disks? Or backups on floppy disks? (I've a backup on an 8" floppy, some punch cards, and some paper tape.) Good luck reading them.
I remember some researchers at Southampton university coming up with a very dense archive on glass that sounds ideal, providing enough people used it to ensure a perpetual supply of reading hardware. But like most such ideas, it hasn't caught on. _________________ Greybeard |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54212 Location: 56N 3W
|
Posted: Sun Jul 28, 2019 10:10 am Post subject: |
|
|
Goverp,
I still have an APPLE ][ that works with the original floppies. Its nearly 40 years old.
Paper tape is not an issue. I can read that by eye :) _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
|