| View previous topic :: View next topic |
| Author |
Message |
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
 Posted: Tue Mar 14, 2006 8:10 am Post subject: |
 |
|
What version of portage are you currently using? Are you sure you did not downgrade portage? The working version is 2.1_pre1 or greater anything less is broken (although it is marked stable).
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
 |
_pF_
Apprentice
Joined: 22 Apr 2004
Posts: 195
Location: Central London
|
| Posted: Tue Mar 14, 2006 8:54 am Post subject: |
|
|
| drescherjm wrote: | | What version of portage are you currently using? Are you sure you did not downgrade portage? The working version is 2.1_pre1 or greater anything less is broken (although it is marked stable). |
2.0.54 - the clue was in the headline of my post. No, I didn't deliberately downgrade, but I might have missed it in that massive emerge.
2.0.54 is the version installed and that portage would still want to install. I'm using only defaults so I'd be surprised if portage had been upgraded before then downgraded, yet that seems like what happened.
Thanks, I'll investigate 2.1_pre1.
| Code: |
valli ~ # emerge -va portage
These are the packages that I would merge, in order:
Calculating dependencies ...done!
[ebuild R ] sys-apps/portage-2.0.54 -build -doc (-selinux) 0 kB
|
|
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
| Posted: Tue Mar 14, 2006 9:04 am Post subject: |
|
|
| Quote: | | 2.0.54 - the clue was in the headline of my post. |
Sorry, I missed that one. Anyways to get the fixed version you need to add sys-apps/portage ~arch to your /etc/portage/package keywords where ~arch is your architecture (x86, amd64 ...) and then update portage.
| Quote: | | I'm using only defaults so I'd be surprised if portage had been upgraded before then downgraded, yet that seems like what happened. |
Are you 100% certain that you did not use ACCEPT_KEYWORDS="~x86" emerge -u portage or something similar?
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
_pF_
Apprentice
Joined: 22 Apr 2004
Posts: 195
Location: Central London
|
| Posted: Tue Mar 14, 2006 9:25 am Post subject: |
|
|
Thanks John, i'll try that.
| drescherjm wrote: |
Are you 100% certain that you did not use ACCEPT_KEYWORDS="~x86" emerge -u portage or something similar? |
Yes, I definitely did not do that. It's baffling. |
|
| Back to top |
|
|
Chipper02
n00b
Joined: 09 Jan 2005
Posts: 3
|
| Posted: Tue Mar 21, 2006 9:24 pm Post subject: |
|
|
I did this and it worked great!
To noobies that don't know python, it is very important that tabs in the python file this tip asks you create are correct. Use kedit or gedit when pasting the code. My Vim install automatically (annoyomatically?) formats the pasted text incorrectly. Use something that does not auto-indent.
This is the error you will see if tabs don't match up. (Dont' ask me for help, I don't know python)
IndentationError: unindent does not match any outer indentation level
and here's what I got for times... (Includes the time to download the entire tree)
'time emerge --metadata'
real 0m10.298s
user 0m9.337s
sys 0m0.662s
10 seconds, thats a huge improvement from 10-12 minutes that it used to take.
If you aren't keen on changing the backend of portage to a database, you may want to try using Reiserfs. Many benchmarks have shown Reiserfs to be far superior in operations involving large numbers of small files (read: portage)
I am surprized that an enterprise class project like portage doesn't use a real database as the backend.
--
Chipper02
Insightful comments and intelligent discussion welcome. Flames will be routed to /dev/null. |
|
| Back to top |
|
|
BitJam
Advocate
Joined: 12 Aug 2003
Posts: 2248
Location: Silver City, NM
|
| Posted: Sat Apr 01, 2006 12:48 am Post subject: |
|
|
| Chipper02 wrote: | If you aren't keen on changing the backend of portage to a database, you may want to try using Reiserfs. Many benchmarks have shown Reiserfs to be far superior in operations involving large numbers of small files (read: portage)
|
This was the accepted wisdom here three years ago when I first installed Gentoo. Turns out that ReiserFS is a really *bad* choice for /usr/portage because all of those tiny little files get updated frequently causing ReiserFS to fragment badly.
I recently replaced the old Reiser partition that held /usr/portage. When backing up that partition I noticed that its speed had been reduced by a factor of 5 which I attribute to fragmentation. |
|
| Back to top |
|
|
_pF_
Apprentice
Joined: 22 Apr 2004
Posts: 195
Location: Central London
|
| Posted: Sat Apr 01, 2006 6:00 am Post subject: |
|
|
| BitJam wrote: |
I recently replaced the old Reiser partition that held /usr/portage. When backing up that partition I noticed that its speed had been reduced by a factor of 5 which I attribute to fragmentation. |
That five times is an interesting figure. As it happens, I do use Reiser. However I am still none the wiser as to why just one of my syncs over the past year or so completed in minutes rather than hours. |
|
| Back to top |
|
|
BitJam
Advocate
Joined: 12 Aug 2003
Posts: 2248
Location: Silver City, NM
|
| Posted: Sat Apr 01, 2006 7:12 am Post subject: |
|
|
| _pF_ wrote: | | That five times is an interesting figure. As it happens, I do use Reiser. However I am still none the wiser as to why just one of my syncs over the past year or so completed in minutes rather than hours. |
Probably just a bug in portage. 
Do your emerge syncs really take hours? If so, if you have some extra disk space, you could copy the entire /usr/portage tree somewhere else then erase /usr/portage/* and then move the copied files back in an attempt to do a temporary defrag.
Better still would be to copy all the files to a different partition and then make /usr/portage a symlink to the new location. These measures won't really fix the problem but if you do get a temporary speed-up (and nothing else has changed like a new version of portage) then it is a good indication that fragmentation is a problem for you.
It is really nuts that at the heart of the high-tech Gentoo system is a half a gig of little files that seem to cause no end of problems, slowness and inefficiency. For now I've relegated /usr/portage to its own partition so if it fragments, it won't fragment my entire system. This should also make it easier to do the el-cheepo copy defrag if needed. |
|
| Back to top |
|
|
_pF_
Apprentice
Joined: 22 Apr 2004
Posts: 195
Location: Central London
|
| Posted: Sat Apr 01, 2006 10:47 am Post subject: |
|
|
Yes, the syncs took hours, stuck around 50%.
Well, I've done the copy trick. All three quarters of a Gigabyte of it. We'll see how the next sync goes. I'll try Chipper02's idea if that doesn't work. Thanks, all. |
|
| Back to top |
|
|
starrbuck
Tux's lil' helper
Joined: 04 Apr 2005
Posts: 127
Location: North Texas
|
| Posted: Sun Apr 02, 2006 12:23 am Post subject: |
|
|
I confirmed this problem is gone when using Portage 2.1_pre7-r3.
_________________
Gentoo Linux is groovy, baby! Yeah! |
|
| Back to top |
|
|
Frapazoid
n00b
Joined: 03 Apr 2006
Posts: 18
|
| Posted: Tue Apr 04, 2006 2:48 pm Post subject: |
|
|
Yipes, I didn't know this was a common problem.
I've got the exact same issue (stalls for a very long time at %50) on my 400mhz MIPS R12k with 1.5gigs of RAM.
Happened the very first time I synced, in fact.
I'm using Reiserfs. ...Looks like a lot of people are, and having issues. I'll try ext2 over loopback, reiser over loopback, the du -s trick and everything else I can dig out of this thread and see what fixes it.
It actually fails at the moment - something about virtual/ghostscript-0 - but I made another thread about that.
_________________
My systems:
MIPS R12000 400mhz - 1536 Megs RAM - Impact SSE Video Board - 10k RPM 73gig SCSI - Gentoo Linux
Athlon K7-Thunderbird 1000mhz - 512 Megs RAM - GeForce 5.2 128meg Video - 5k RPM 40gig IDE - Fedora Core |
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
| Posted: Tue Apr 04, 2006 3:03 pm Post subject: |
|
|
| Quote: | I'm using Reiserfs. ...Looks like a lot of people are, and having issues. I'll try ext2 over loopback, reiser over loopback, the du -s trick and everything else I can dig out of this thread and see what fixes it.
|
Just upgrade to the fixed version 2.1_pre<whatever> and the problem goes away.
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
Frapazoid
n00b
Joined: 03 Apr 2006
Posts: 18
|
| Posted: Tue Apr 04, 2006 5:15 pm Post subject: |
|
|
Okay, I'll try that.
...Is it stable?
_________________
My systems:
MIPS R12000 400mhz - 1536 Megs RAM - Impact SSE Video Board - 10k RPM 73gig SCSI - Gentoo Linux
Athlon K7-Thunderbird 1000mhz - 512 Megs RAM - GeForce 5.2 128meg Video - 5k RPM 40gig IDE - Fedora Core |
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
| Posted: Tue Apr 04, 2006 7:07 pm Post subject: |
|
|
It is marked unstable, but it does not have the long delay bug that the stable version exhibits.
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
Frapazoid
n00b
Joined: 03 Apr 2006
Posts: 18
|
| Posted: Wed Apr 05, 2006 1:20 am Post subject: |
|
|
It works! It even fixed that problem I was having with ghostscript.
Thanks! I love you guys.
_________________
My systems:
MIPS R12000 400mhz - 1536 Megs RAM - Impact SSE Video Board - 10k RPM 73gig SCSI - Gentoo Linux
Athlon K7-Thunderbird 1000mhz - 512 Megs RAM - GeForce 5.2 128meg Video - 5k RPM 40gig IDE - Fedora Core |
|
| Back to top |
|
|
lyallp
Veteran
Joined: 15 Jul 2004
Posts: 1070
Location: Adelaide/Australia
|
| Posted: Thu Apr 06, 2006 11:48 pm Post subject: My problem went away! |
|
|
I found that when I blew the entire contents of PORTAGE_TMPDIR away, did an emerge --metadata, emerge sync screamed through in record time.
(be aware that I have previously set my PORTAGE_TMPDIR to something non-standard - I have portage in it's own filesystem, outside of /usr - so you might want to be slightly more careful than I was in your tidy up)
I also went through PKGDIR and did a rough cleanup of obsolete packages - ie, removed older versions.
No need to do all the fancy speedup things. My emerge is like new again
_________________
...Lyall |
|
| Back to top |
|
|
chrisstankevitz
Guru
Joined: 14 Dec 2003
Posts: 465
Location: Santa Barbara, CA, USA
|
| Posted: Sun Apr 09, 2006 7:23 pm Post subject: Re: My problem went away! |
|
|
| lyallp wrote: | | I found that when I blew the entire contents of PORTAGE_TMPDIR away, did an emerge --metadata |
If you just ran an emerge sync or emerge --metadata, the second run will go quickly unless you first:
rm -rf /var/cache/edb |
|
| Back to top |
|
|
lyallp
Veteran
Joined: 15 Jul 2004
Posts: 1070
Location: Adelaide/Australia
|
| Posted: Mon Apr 10, 2006 11:24 am Post subject: |
|
|
I stand corrected, a subsequent run of emerge --sync took just as long, stalling at 51% while cpu racked up to 98% for about 15 minutes.
...Lyall
_________________
...Lyall |
|
| Back to top |
|
|
marelven
n00b
Joined: 11 Jun 2005
Posts: 15
Location: Montreal, Canada
|
| Posted: Mon Apr 10, 2006 12:25 pm Post subject: |
|
|
Hi everyone,
I've tried the unstable version of portage (2.1_preX) and it does correct the update problem.
BUT it also breaks a lot of packages (KDE or Xorg, for example), especially if you try to do an
after the latest version of portage is built and installed. It is still experimental and it will break things.
So I would suggest everyone who is bugged by this cache update problem to use the cdb trick until the next version of portage is out. |
|
| Back to top |
|
|
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Mon Apr 10, 2006 1:00 pm Post subject: |
|
|
| marelven wrote: | Hi everyone,
I've tried the unstable version of portage (2.1_preX) and it does correct the update problem.
BUT it also breaks a lot of packages (KDE or Xorg, for example), especially if you try to do an
after the latest version of portage is built and installed. It is still experimental and it will break things.
So I would suggest everyone who is bugged by this cache update problem to use the cdb trick until the next version of portage is out. |
Err... actually it doesn't.
Bug reports would be wise if you're getting differences; most likely it's broken packages.
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
| Posted: Mon Apr 10, 2006 3:04 pm Post subject: |
|
|
| Quote: | I've tried the unstable version of portage (2.1_preX) and it does correct the update problem.
BUT it also breaks a lot of packages (KDE or Xorg, for example), especially if you try to do an
Code:
emerge -aDuN world
after the latest version of portage is built and installed. It is still experimental and it will break things. |
It has not broken a single thing on the 20+ computers I have upgraded. Did you properly use /etc/portage/package.keywords instead of ACCEPT_KEYWORDS="~x86" emerge portage?
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
marelven
n00b
Joined: 11 Jun 2005
Posts: 15
Location: Montreal, Canada
|
| Posted: Tue Apr 11, 2006 2:08 pm Post subject: |
|
|
Hello everyone,
| Quote: | Err... actually it doesn't.
Bug reports would be wise if you're getting differences; most likely it's broken packages. |
Actually, it did. I had to use backups to get back to a usable system. It could be broken packages but things were working perfectly well before I tried this latest version.
| Quote: | | Did you properly use /etc/portage/package.keywords instead of ACCEPT_KEYWORDS="~x86" emerge portage? |
Yes, I did. I also added dev-python/pycrypto to /etc/portage/package.keywords because the latest version of portage needed it.
After that, I used the usual commands to update the whole system : emerge aDu --newuse world, revdep-rebuild -p, etc.
Anyway, I can go through the whole process again to see where I've messed up... I'll come back later with more detailed informations. |
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2729
Location: Pittsburgh, PA, USA
|
| Posted: Tue Apr 11, 2006 2:21 pm Post subject: |
|
|
| Quote: | | emerge aDu --newuse world, revdep-rebuild -p |
Are you sure that a previous emerge did not cause the problem?
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
marelven
n00b
Joined: 11 Jun 2005
Posts: 15
Location: Montreal, Canada
|
| Posted: Tue Apr 11, 2006 4:02 pm Post subject: |
|
|
Hello again,
| Quote: | | Are you sure that a previous emerge did not cause the problem? |
Yes, I am. As I said in a previous post, my system was working well before I updated portage.
The more I think about it, the more I'm sure it's something I screwed. I mean, the 2.1 version of portage is working perfectly well for numerous people out there, so it's probably me who did something wrong.
Anyway, I will build the latest version of portage again, I'll play with it and let you know what I've found. |
|
| Back to top |
|
|
marelven
n00b
Joined: 11 Jun 2005
Posts: 15
Location: Montreal, Canada
|
| Posted: Mon Apr 17, 2006 10:36 pm Post subject: |
|
|
Hi everyone
I installed the 2.1 version of portage and I think I found where I messed things up.
The transition was going well and everything was smoot until I used the --depclean switch of emerge.
I use this option pretty often with the 2.0 version of portage to keep my system tidy, and it works very well. So I used this option, which is known to be dangerous, with the beta version of emerge without even checking the output. You know the rest... 
Sorry if I made anyone panic, it's all my fault. 
Now, I'll go hide myself in a very deep cave somewhere on a uncharted island. |
|
| Back to top |
|
|
|
Portage & Programming
