Gentoo Forums
Gentoo Forums
Quick Search: in
[NEWS] Final step to smaller, better Portage tree
View unanswered posts
View posts from last 24 hours

rackathon
Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
dberkholz
Developer
Developer


Joined: 18 Mar 2003
Posts: 935
Location: Corvallis, OR, USA

PostPosted: Tue Feb 05, 2008 2:13 am    Post subject: [NEWS] Final step to smaller, better Portage tree Reply with quote

This forums thread is for discussion of the www.gentoo.org posting, "Final step taken in migration to smaller, better Portage tree." Post your comments and suggestions here.
Quote:
Each ebuild in the Portage tree used to come with its own digest file. When you emerged a package, this digest was used to verify that you had the same files the developer did.

A few days ago, all digest files were erased from the Portage tree in CVS as part of the final work to implement GLEP 44. This migrates that information to a single Manifest file per package instead of one digest for every ebuild of that package.

...
(more on gentoo.org)


Correction: This was just the final step in a long-term migration that actually reduced the size of your Portage tree last February. Thanks to Daniel Drake, Petteri Räty and Jan Kundrat for pointing this out.
_________________
Do unto others as you would have them do unto you.
What goes around comes around.


Last edited by dberkholz on Wed Feb 06, 2008 2:45 am; edited 2 times in total
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 625
Location: PL

PostPosted: Tue Feb 05, 2008 3:05 am    Post subject: Reply with quote

nice move.

i still prefer keeping it tucked together on squashfs + aufs mount - there's so many files that copying it over to another pc or just making a compressed backup takes an awful amount of time, though.
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
Naib
Veteran
Veteran


Joined: 21 May 2004
Posts: 2780
Location: UK - Birmingham

PostPosted: Tue Feb 05, 2008 3:15 am    Post subject: Reply with quote

niiiice
_________________
pkgcore, The way portage should be managed
Quote:
There is a forgotten, nay almost forbidden word, which means more to me than any other. That word is ENGLAND
Back to top
View user's profile Send private message
StifflerStealth
l33t
l33t


Joined: 03 Jul 2002
Posts: 886

PostPosted: Tue Feb 05, 2008 4:49 am    Post subject: Reply with quote

If we do our own ebuilds, how do we have it so there is not digest file? ebuild foo manifest creates a blank digest file. What's the official command?
_________________
Qt Copy ebuilds for Qt 3 and 4: Clicky <-- Updated 20080118!
Back to top
View user's profile Send private message
Chewi
Guru
Guru


Joined: 01 Sep 2003
Posts: 489
Location: Perth, Scotland

PostPosted: Tue Feb 05, 2008 5:00 am    Post subject: Reply with quote

Awesome! I've been thinking the tree is way too big for a couple of years now. This really helps.
Back to top
View user's profile Send private message
Voltago
Veteran
Veteran


Joined: 02 Sep 2003
Posts: 2122
Location: Hinter den Materiequellen

PostPosted: Tue Feb 05, 2008 5:10 am    Post subject: Reply with quote

StifflerStealth wrote:
What's the official command?

Code:
emerge =portage-2.1.4.1
Back to top
View user's profile Send private message
likewhoa
Guru
Guru


Joined: 04 Oct 2006
Posts: 432
Location: Brooklyn, New York

PostPosted: Tue Feb 05, 2008 5:12 am    Post subject: Reply with quote

good progress into GLEP features.
Back to top
View user's profile Send private message
V-Li
Developer
Developer


Joined: 02 Jan 2006
Posts: 258

PostPosted: Tue Feb 05, 2008 5:14 am    Post subject: Reply with quote

StifflerStealth wrote:
If we do our own ebuilds, how do we have it so there is not digest file? ebuild foo manifest creates a blank digest file. What's the official command?


Put the file manifest1_obsolete in the root of your own tree.
_________________
Christian Faulhammer, Gentoo Lisp project, GNU Emacs wrangler
http://www.gentoo.org/proj/en/lisp/ #gentoo-lisp on FreeNode

http://gentoo.faulhammer.org/
Back to top
View user's profile Send private message
slack---line
Guru
Guru


Joined: 01 Apr 2005
Posts: 458
Location: Sheffield, UK

PostPosted: Tue Feb 05, 2008 5:32 am    Post subject: Reply with quote

Sweet, good work to those who've implemented this, great, simple way of saving space.
Back to top
View user's profile Send private message
Xake
Guru
Guru


Joined: 11 Feb 2004
Posts: 454
Location: Good old and cold scandinavia

PostPosted: Tue Feb 05, 2008 5:46 am    Post subject: Reply with quote

StifflerStealth wrote:
If we do our own ebuilds, how do we have it so there is not digest file? ebuild foo manifest creates a blank digest file. What's the official command?


Code:
touch  manifest1_obsolete
Back to top
View user's profile Send private message
jomen
Guru
Guru


Joined: 04 Jan 2006
Posts: 415
Location: Leipzig / Sachsen / Germany

PostPosted: Tue Feb 05, 2008 6:11 am    Post subject: Reply with quote

I was using "porthole" for a graphical overview of available and installed packages.
Never used it to actually install stuff.
About 3 weeks back from now I was irritated to see that every single ebuild on my system was suddenly marked as:
[bad/blank digest]
I suppose the news discussed here is the reason for this behaviour?
What would I need to do to let porthole show valid ebuilds again instead of [bad/blank digest]?
this?
Code:
touch  manifest1_obsolete

where?

[edit]
Its nice to save some disc-space by this - I have solved this issue for me in the past by having a small partition for /usr/portage (excluding /usr/portage/distfiles)
which I created like so:
mke2fs -j -m 0 -b 1024 -i 1024 /dev/hdXx
to not waste so much space on lots of small files
_________________
Cheers
Back to top
View user's profile Send private message
V-Li
Developer
Developer


Joined: 02 Jan 2006
Posts: 258

PostPosted: Tue Feb 05, 2008 7:21 am    Post subject: Reply with quote

manifest1_obsolete is only for people preparing own ebuilds. The ebuild tool then does NOT create digest files anymore...for Porthole, I think you need an update.
_________________
Christian Faulhammer, Gentoo Lisp project, GNU Emacs wrangler
http://www.gentoo.org/proj/en/lisp/ #gentoo-lisp on FreeNode

http://gentoo.faulhammer.org/
Back to top
View user's profile Send private message
d2_racing
Veteran
Veteran


Joined: 25 Apr 2005
Posts: 3438
Location: Québec,Canada

PostPosted: Tue Feb 05, 2008 7:43 am    Post subject: Reply with quote

Nice :)
_________________

-P4 2.4Ghz FSB 800 Mhz
-Asus P4P800 S478 AGP 8x
-1024 Meg DDR PC3200 400 Mhz
-WD 160 Gig Serial ATA 150
-ATI 9600 PRO Ultra 128 Meg
-DVR-108
-DFE-538Tx
-KDE 3.5.8 on 2.6.24-Gentoo-r4
-Xorg 7.2
And a T60P 8744J2F
Back to top
View user's profile Send private message
kernelOfTruth
Veteran
Veteran


Joined: 20 Dec 2005
Posts: 2450
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Feb 05, 2008 8:03 am    Post subject: Reply with quote

likewhoa wrote:
good progress into GLEP features.


++

thanks a lot :)

this should also put less stress on the rsync mirrors 8)
_________________
Gentoo-based liveCD with JMicron support ,
gcc hardened/toolchain overlay
Back to top
View user's profile Send private message
d2_racing
Veteran
Veteran


Joined: 25 Apr 2005
Posts: 3438
Location: Québec,Canada

PostPosted: Tue Feb 05, 2008 8:06 am    Post subject: Reply with quote

And maybe the emerge --sync will be faster...
_________________

-P4 2.4Ghz FSB 800 Mhz
-Asus P4P800 S478 AGP 8x
-1024 Meg DDR PC3200 400 Mhz
-WD 160 Gig Serial ATA 150
-ATI 9600 PRO Ultra 128 Meg
-DVR-108
-DFE-538Tx
-KDE 3.5.8 on 2.6.24-Gentoo-r4
-Xorg 7.2
And a T60P 8744J2F
Back to top
View user's profile Send private message
jlh
Tux's lil' helper
Tux's lil' helper


Joined: 06 May 2007
Posts: 131
Location: Switzerland::Zürich

PostPosted: Tue Feb 05, 2008 8:25 am    Post subject: Reply with quote

Very nice! I keep my tree on a loop-mounted 1k-filesystem and even this way I saved over 20MB for a new total of about 240MB.

I hope to see even more improvements to the tree in the future. :)
Back to top
View user's profile Send private message
Mikachu
n00b
n00b


Joined: 03 Jan 2005
Posts: 2

PostPosted: Tue Feb 05, 2008 8:39 am    Post subject: Reply with quote

and i have mine on an xfs fs with 512 byte blocks :).
155M /usr/portage/
Back to top
View user's profile Send private message
jomen
Guru
Guru


Joined: 04 Jan 2006
Posts: 415
Location: Leipzig / Sachsen / Germany

PostPosted: Tue Feb 05, 2008 8:39 am    Post subject: Reply with quote

Quote:
...for Porthole, I think you need an update.

The latest version is 0.5 from December 2005 - no updates to reflect the changes to portage discussed here.
I already re-emerged it and nothing changed - it has just two USE-Flags ("nls" and "debug")
I think I'll go for "himerge" or "portato"...
_________________
Cheers
Back to top
View user's profile Send private message
Xake
Guru
Guru


Joined: 11 Feb 2004
Posts: 454
Location: Good old and cold scandinavia

PostPosted: Tue Feb 05, 2008 8:48 am    Post subject: Reply with quote

jomen wrote:
Quote:
...for Porthole, I think you need an update.

The latest version is 0.5 from December 2005 - no updates to reflect the changes to portage discussed here.
I already re-emerged it and nothing changed - it has just two USE-Flags ("nls" and "debug")
I think I'll go for "himerge" or "portato"...


https://sourceforge.net/tracker/index.php?func=detail&aid=1729279&group_id=96324&atid=614397
The bugreport for your problem in question, and as you can see people has had this problem since at least 2007-05, and as the maintainer have not had the time to make a new release since then (he said 2007-08 that he have had the fix in CVS for some time) I believe you're better off with another pkg-manager until he does have the time. Or someone fork.

EDIT: On the other hand you may not give up on porthole yet, 4 days ago the NEWS file had text added for an upcoming 0.6.0 release. So we may just have to hope it will soon be released. On the other hand himerge and portato is installable and should work side by side with porthole if you do not want to wait...;)
Back to top
View user's profile Send private message
Betelgeuse
Developer
Developer


Joined: 10 Aug 2004
Posts: 12
Location: Finland

PostPosted: Tue Feb 05, 2008 8:57 am    Post subject: Bit missleading Reply with quote

Portage has been filtering digests from --sync for quite a while so the only difference users are going to see is a potentially faster sync as the server side doesn't have the digests any more.
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 6297
Location: Córdoba (Spain)

PostPosted: Tue Feb 05, 2008 9:09 am    Post subject: Reply with quote

Not that I care too much... I never used any graphical frontend for any serious portage stuff, but I don't know what the current state of porthole is. In any case, if you are really interested you should checkout from cvs, the released versions are really really old and will not work ok with current portage versions.
_________________
Formerly known as 6thpink.
Gentoo Handbook | My home
Back to top
View user's profile Send private message
jomen
Guru
Guru


Joined: 04 Jan 2006
Posts: 415
Location: Leipzig / Sachsen / Germany

PostPosted: Tue Feb 05, 2008 9:18 am    Post subject: Reply with quote

Quote:
The bugreport for your problem in question, and as you can see people has had this problem since at least 2007-05

The [Bad/Blank digest] thing is not that long ago (for me)...and I was searching to find out why I had this "problem".
Did not find the bugreport you directed me to though - thanks!
If I'd found it I'd have known about the change to portage which until now slipped by my attention.
Since I use Porthole only to have a better overview of what is in the repositories and do everything else on command-line I'll just keep using it - I like the clean Interface...or am just used to it.
_________________
Cheers
Back to top
View user's profile Send private message
timeBandit
Veteran
Veteran


Joined: 30 Dec 2004
Posts: 1225
Location: here, there or in transit

PostPosted: Tue Feb 05, 2008 9:32 am    Post subject: Reply with quote

jomen wrote:
Since I use Porthole only to have a better overview of what is in the repositories and do everything else on command-line ....
Same here.

Kind of a non-event since the digest files have been gone for a while on the user side, but it's good to see the announcement in case of any problems. Either way, Commander Keen approves--he's all about saving space. :wink: (Or at least Earth.)
_________________
Plants are pithy, brooks tend to babble--I'm content to lie between them.
Super-short f.g.o checklist: Search first, strip comments, mark solved, help others.
Back to top
View user's profile Send private message
jancici
Tux's lil' helper
Tux's lil' helper


Joined: 27 Jan 2004
Posts: 140
Location: Slovakia

PostPosted: Tue Feb 05, 2008 10:23 am    Post subject: Reply with quote

I did run two commands
Code:
du -s /usr/portage/
2633079   /usr/portage/
du -s /usr/portage/distfiles/
2161991   /usr/portage/distfiles/


so my tree is 471088 big cca 460MB
I dont know how big it was beffore
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 625
Location: PL

PostPosted: Tue Feb 05, 2008 12:19 pm    Post subject: Reply with quote

my squashfs image went down from ~42-45 mb do 38mb. not bad.
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum