Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
"Xz format inadequate for long-term archiving"
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
Maitreya
Guru
Guru


Joined: 11 Jan 2006
Posts: 426

PostPosted: Wed Jun 05, 2019 2:34 pm    Post subject: "Xz format inadequate for long-term archiving" Reply with quote

So apparently XZ is now BAD™ : http://lzip.nongnu.org/xz_inadequate.html



What are the thoughts on this, is over exaggeration or does it have some merit? (Like is the developer disliked or something)
Back to top
View user's profile Send private message
Maitreya
Guru
Guru


Joined: 11 Jan 2006
Posts: 426

PostPosted: Wed Jun 05, 2019 2:39 pm    Post subject: Reply with quote

Ah, I think I can answer my own question:
Quote:

In 2008 one of the designers of xz (Lasse Collin) warned me that lzip would become stuck with LZMA while others moved to LZMA2, LZMA3, LZMH, and other algorithms. Now xz-utils is usually unable to match the compression ratio of lzip because LZMA2 has more overhead than LZMA and, as expected, no new algorithms have been added to xz-utils.


So there is some rivalry involved, yet all arguments still seem fair.
Back to top
View user's profile Send private message
erm67
Guru
Guru


Joined: 01 Nov 2005
Posts: 438
Location: EU

PostPosted: Wed Jun 05, 2019 2:39 pm    Post subject: Reply with quote

Since that is what the developer of lzip data compressor thinks about xz it would be also interesting to hear what the developers of xz think about the lzip compression format :-)
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
erm67
Guru
Guru


Joined: 01 Nov 2005
Posts: 438
Location: EU

PostPosted: Wed Jun 05, 2019 2:46 pm    Post subject: Reply with quote

BTW the standard for long-term document archiving is https://en.wikipedia.org/wiki/PDF/A and it only allow the (de)flate compression algorithm (used also by gzip) since it is considered to be patent free.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
mike155
Veteran
Veteran


Joined: 17 Sep 2010
Posts: 1724
Location: Frankfurt, Germany

PostPosted: Wed Jun 05, 2019 3:01 pm    Post subject: Re: "Xz format inadequate for long-term archiving" Reply with quote

Maitreya wrote:
So apparently XZ is now BAD™ : http://lzip.nongnu.org/xz_inadequate.html

Please stop spreading FUD.
Back to top
View user's profile Send private message
Maitreya
Guru
Guru


Joined: 11 Jan 2006
Posts: 426

PostPosted: Wed Jun 05, 2019 3:57 pm    Post subject: Re: "Xz format inadequate for long-term archiving" Reply with quote

mike155 wrote:
Maitreya wrote:
So apparently XZ is now BAD™ : http://lzip.nongnu.org/xz_inadequate.html

Please stop spreading FUD.


I posted it here because I like others opinion on this, not to say this is my opinion?
Back to top
View user's profile Send private message
Maitreya
Guru
Guru


Joined: 11 Jan 2006
Posts: 426

PostPosted: Wed Jun 05, 2019 3:59 pm    Post subject: Reply with quote

erm67 wrote:
Since that is what the developer of lzip data compressor thinks about xz it would be also interesting to hear what the developers of xz think about the lzip compression format :-)


Exactly!

Although we can read the (to me fair) reasoning, does not mean it's "THE TRUTH"
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6017

PostPosted: Wed Jun 05, 2019 4:35 pm    Post subject: Reply with quote

Ah, this crank again. If you get into compression performance, ck's lrzip beats the crap out of both xz and lzip in speed and ratio anyway (even without -z), and has done for about 15 years. zstd outperforms them on all fronts simultaneously too.

lzip is still taking angry little pot-shots at a format that was obsoleted long ago, which shows you how relevant lzip is in today's world: it never was.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 502
Location: NRW, Germany

PostPosted: Thu Jun 06, 2019 8:57 am    Post subject: Reply with quote

Ant P. wrote:
If you get into compression performance, ck's lrzip beats the crap out of both xz and lzip in speed and ratio anyway (even without -z), and has done for about 15 years. zstd outperforms them on all fronts simultaneously too.

According to http://www.mattmahoney.net/dc/text.html zstd has a worse compression-ratio than lzip and xz.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6017

PostPosted: Thu Jun 06, 2019 4:21 pm    Post subject: Reply with quote

I'll concede that, but I'd also prefer the 10 seconds of extra download time over 20 seconds of CPU time with possible OOM.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 502
Location: NRW, Germany

PostPosted: Fri Jun 07, 2019 7:12 am    Post subject: Reply with quote

Ant P. wrote:
I'll concede that, but I'd also prefer the 10 seconds of extra download time over 20 seconds of CPU time with possible OOM.

Yeah afaik zstd's idea is to be used for on-the-fly compression to increase throughput more than archiving.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44191
Location: 56N 3W

PostPosted: Fri Jun 07, 2019 5:12 pm    Post subject: Reply with quote

Once you move away from parchment or clay tablets, nothing has lasted even 25 years.

We don't have a long-term archiving format.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Amity88
Apprentice
Apprentice


Joined: 03 Jul 2010
Posts: 221
Location: Third planet from the Sun

PostPosted: Sat Jul 27, 2019 4:15 pm    Post subject: Reply with quote

@Neddy,
Don't forget the cave paintings :P That was one good time tested long lasting data archive
_________________
Ant P. wrote:
The enterprise distros sell their binaries. Canonical sells their users.


Also... Be ignorant... Be happy! :)
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 729

PostPosted: Sun Jul 28, 2019 8:35 am    Post subject: Reply with quote

For archiving, surely the answer is "no compression". That way (a) you don't get problems if you lose the program, and (b) the archive contains redundant information that can help you detect and correct corruption, and (c) even if parts are corrupt, other parts will be salvageable.

An equally important part of archiving is the physical medium. Some of us will remember the batch of Ampex (?) tapes that went "read once" when the binder aged. Anyone still got important information on Zip disks? Or backups on floppy disks? (I've a backup on an 8" floppy, some punch cards, and some paper tape.) Good luck reading them.

I remember some researchers at Southampton university coming up with a very dense archive on glass that sounds ideal, providing enough people used it to ensure a perpetual supply of reading hardware. But like most such ideas, it hasn't caught on.
_________________
Greybeard
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44191
Location: 56N 3W

PostPosted: Sun Jul 28, 2019 10:10 am    Post subject: Reply with quote

Goverp,

I still have an APPLE ][ that works with the original floppies. Its nearly 40 years old.
Paper tape is not an issue. I can read that by eye :)
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum