View previous topic :: View next topic |
Author |
Message |
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3267 Location: Canada
|
Posted: Thu Mar 01, 2012 7:34 am Post subject: |
|
|
xiber wrote: | Wow, thought it was only me having this ld issue. Been happening since firefox 9 for me. System grinds to a halt with 1GB of ram. After ram and swap are exausted, emerge fails. The only way I've been able to compile firefox is to shut everything down, including the xserver, and use a vt. I use tmux and let it compile while I'm at work. I've started to dread those little firefox point releases. |
Especially since they now come like once a week |
|
Back to top |
|
|
Chiitoo Administrator
Joined: 28 Feb 2010 Posts: 2572 Location: Here and Away Again
|
Posted: Thu Mar 01, 2012 3:37 pm Post subject: |
|
|
They obviously had to catch up with Opera's version numbering ! !! !
Which is what I actually use mainly.
I do have firefox around too, and an icecat... and so on... but Opera has been feeling the most suitable for me for a long time even before starting with Linux a bit over a year ago.
Now I'm wondering what it would be like to build Opera... hm... _________________ Kindest of regardses. |
|
Back to top |
|
|
Oyarsa n00b
Joined: 01 Jul 2002 Posts: 73 Location: Mars
|
Posted: Mon Mar 05, 2012 3:07 pm Post subject: |
|
|
Thanx for this thread everybody. I had the same problem on my old computer (1GHz Athlon, 768MB RAM, 2GB swap).
I recompiled overnight with "MAKEOPTS=-j1" and it worked. Still a lot of trouble for a backup browser, though.
Thanks to Gentoo this old (11 years now) crate is still running as an internet surfboard. _________________ Dew knot trussed yore spell chequer two fined awl ewer miss steaks.
Last edited by Oyarsa on Tue Mar 06, 2012 2:48 am; edited 1 time in total |
|
Back to top |
|
|
Anarchy Developer
Joined: 29 Jun 2005 Posts: 140
|
Posted: Tue Mar 06, 2012 1:36 am Post subject: Re: [firefox 10] difficult to build |
|
|
WvR wrote: | Laptop computer, Compaq, 2GB mem, 4GB swap, 2 CPUs, AMD64.
While compiling Firefox-10, I noticed that my laptop became very slow, up to the point of being almost unresponsive. I was able to go into a text terminal from Gnome (Ctrl-Alt-F1) and log in as root. I used "top" to diagnose the problem and noticed that the linker ("ld") was using a very large amount of memory: 1.4 GB (!). As a result the RAM was overflowing and my computer was swapping a lot and became very slow. I killed ld and let the emerge die.
I rebooted the computer to have a clean RAM, and turned off XDM to reduce memory use. I compiled firefox-10 from a text terminal. At the end of the compilation phase, ld used up to 1500 MB of memory (!). I think this is a very large amount. If you run anything else at the same time, the amount of swapping becomes quite large. Any other users with similar issues? Should this be filed as a bug and if so, where? It is very inconvenient that this ebuild requires so much memory. |
The ebuild suggest such standards as the build takes 1G alone to link libxul. In order to understand the process you would have to be deeply involved with the development process upstream. The reason that this was the choosen method, was the fact that startup cache generation was add which helped to increase the startup time, along with reducing initial overhead as far as ram when running for the first time. |
|
Back to top |
|
|
keet Guru
Joined: 09 Sep 2008 Posts: 568
|
Posted: Tue Mar 06, 2012 2:24 am Post subject: |
|
|
Another option that worked for me was to plug in an external hard drive (on a computer that had only 512 MB RAM) and use that temporarily as swap space -- just for Firefox. I wish that they hadn't adopted the rapid release/version number system. What do they get now, a full version number for fixing spelling errors in documentation and code comments? |
|
Back to top |
|
|
Clad in Sky l33t
Joined: 04 May 2007 Posts: 887 Location: Germany
|
Posted: Tue Mar 06, 2012 7:03 am Post subject: |
|
|
Didn't happen here, I think.
I'm mostly compiling stuff and doing other things on my computer at the same time. Didn't feel any difference to less memory intensive packages.
My machine is now nearly exactly 4 years old (bought it 8th March 2008) - AMD Phenom with 4GB of RAM, so nothing fancy by today's standards.
Strange. _________________ Kali Ma
Now it's autumn of the aeons
Dance with your sword
Now it's time for the harvest |
|
Back to top |
|
|
HomeUser n00b
Joined: 22 Jan 2006 Posts: 51
|
Posted: Wed Mar 14, 2012 8:06 pm Post subject: ld stuck |
|
|
Also by me ld was stuck with firefox 10.0.1-r1 and with thunderbird 10.0.1-r1. Did a few try's the past days.
It ended either in a kernel crash or, the times I forgot to mount a somewhat bigger disk with "device full".
(I got Thunderbird in the end compiled yesterday.)
But that was only the second emerge, suggested by revdep-rebuild, while at least firefox seems to function normally without recompile.
My emerge log shows normal compilation just a few days ago, after the same --sync for both the same programs.
(Old machine with AMD Athlon(tm) 3000+ 1536MiB)
wo mrt 7 21:30:35 CET 2012 >>> emerge (53 of 207) www-client/firefox-10.0.1-r1 to /
wo mrt 7 21:30:36 CET 2012 === (53 of 207) Cleaning (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
wo mrt 7 21:30:39 CET 2012 === (53 of 207) Compiling/Merging (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
wo mrt 7 23:42:33 CET 2012 === (53 of 207) Merging (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
wo mrt 7 23:42:57 CET 2012 >>> AUTOCLEAN: www-client/firefox:0
wo mrt 7 23:42:57 CET 2012 === Unmerging... (www-client/firefox-8.0)
wo mrt 7 23:43:13 CET 2012 >>> unmerge success: www-client/firefox-8.0
do mrt 8 19:33:36 CET 2012 >>> emerge (12 of 150) mail-client/thunderbird-10.0.1-r1 to /
do mrt 8 19:33:36 CET 2012 === (12 of 150) Cleaning (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
do mrt 8 19:33:40 CET 2012 === (12 of 150) Compiling/Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
do mrt 8 20:12:00 CET 2012 Started emerge on: Mar 08, 2012 21:11:59
do mrt 8 20:12:00 CET 2012 (garbage)
do mrt 8 20:12:38 CET 2012 >>> emerge (1 of 1) media-video/mplayer-1.0_rc4_p20110322-r1
(Did have some tries taking hours with ld like taking 80% of memory, computer slowing down and dying, But I see today that it did succeeded yesterday.)
di mrt 13 18:24:53 CET 2012 >>> emerge (1 of 14) mail-client/thunderbird-10.0.1-r1 to /
di mrt 13 18:24:53 CET 2012 === (1 of 14) Cleaning (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
di mrt 13 18:25:02 CET 2012 === (1 of 14) Compiling/Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
di mrt 13 21:02:07 CET 2012 === (1 of 14) Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
di mrt 13 21:02:25 CET 2012 >>> AUTOCLEAN: mail-client/thunderbird:0
di mrt 13 21:02:25 CET 2012 === Unmerging... (mail-client/thunderbird-10.0.1-r1)
di mrt 13 21:02:35 CET 2012 >>> unmerge success: mail-client/thunderbird-10.0.1-r1
(But a new emerge and forced stop of firefox is in the logs.)
Could that be the same problem? I see now garbage in the emerge.log, perhaps disk structure problem or an other problem with the linker/emerge. (No garbage in ..messages or summary log
I often hope that some packets like LibreOffice can someday be offered in a more modular form,
this could perhaps still be a special problem with building those Mozilla packets |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Wed Mar 14, 2012 8:14 pm Post subject: |
|
|
@HomeUser, do you have swap enabled? If so, how big is it? These days, Firefox and Thunderbird take upwards of 8GiB of /var/tmp space and 2GiB of virtual memory to build properly.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
HomeUser n00b
Joined: 22 Jan 2006 Posts: 51
|
Posted: Wed Mar 14, 2012 9:06 pm Post subject: |
|
|
John R. Graham wrote: | @HomeUser, do you have swap enabled? If so, how big is it? These days, Firefox and Thunderbird take upwards of 8GiB of /var/tmp space and 2GiB of virtual memory to build properly.
- John |
My swap is 1574332k. Use to play some television on the other monitor, but Mplayer only takes 5-6% CPU. I suppose the graphic cart does the biggest job. |
|
Back to top |
|
|
tld Veteran
Joined: 09 Dec 2003 Posts: 1816
|
Posted: Tue Mar 20, 2012 12:48 am Post subject: |
|
|
haha...glad I found this. Pretty much explains why I had issues on my old 2.3 Ghz P4 with 768MB of ram
No way am I attempting this....guess it's time to move to firefox-bin for me.
I hate to say it, but the list of ways that Mozilla team has been pissing me off grows more every day. If it wasn't for the extensions I depend on (especially for web development) I'm not sure I'd put up with this crap for another second. Speaking of which, you gotta love how the Mozilla team added a "Web Developer" tool menu, when the same name had already been in use by the Web Developer extension for ages...now there are TWO of them with completely different contents...I could go on for hours...so I won't get started...
Tom |
|
Back to top |
|
|
navilein n00b
Joined: 07 Feb 2007 Posts: 7
|
Posted: Mon May 07, 2012 9:02 pm Post subject: |
|
|
Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).
I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...
I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.
But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!
I don't know if it's me, or there is some lack of common-sense in this linking...
(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft) |
|
Back to top |
|
|
Anarchy Developer
Joined: 29 Jun 2005 Posts: 140
|
Posted: Thu May 10, 2012 2:19 am Post subject: |
|
|
navilein wrote: | Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).
I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...
I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.
But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!
I don't know if it's me, or there is some lack of common-sense in this linking...
(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft) |
Common sense tells ya that you can only take software so far with outdated hardware. This was a decision that was not made lightly. As far as your rude and uncalled for remarks you can keep them to yourself. If you have a problem due to hardware limitations this is not the fault of the software, You could always make a new xulrunner/firefox-3.6 ebuild for the last release if you really wanted to throw into a local overlay. |
|
Back to top |
|
|
PhaTaL-Err0r n00b
Joined: 18 Apr 2012 Posts: 6
|
Posted: Thu May 10, 2012 1:55 pm Post subject: |
|
|
navilein wrote: | Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).
I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...
I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.
But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!
I don't know if it's me, or there is some lack of common-sense in this linking...
(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft) |
This is out of the hands of the Gentoo developers, Mozilla has pretty much stopped supporting old hardware since Firefox 3.6, and as stated by Anarchy, you can only take modern day software so far with outdated hardware. If you already have Firefox 3.6 emerged then mask all the newer generations of Firefox in /etc/portage/package.mask; or if you have the know-how write up an ebuild and stick it in your local overlay -- and I know you don't wanna hear this, but yes, you DO need to seriously upgrade your PC to a more modern system. You can only go so far with it with modern software, YES, older hardware such as yours is touted as funtional throughout the Linux communities, however their usefulness is more for servers, firewalls, etc... and not modern day productivity such as gaming, internet based stuff; web browsing, VoIP, etc.
There are specialized Linux distros for older hardware too, Puppy Linux: Wary is one of the best for older systems. Maybe see what versions they are using of apps you use regularly and emerge them or make ebuilds for them. However even as specialized as Puppy Linux: Wary is, on CPU's such as what you have it runs only "reasonably well". You're gonna have to face it sooner or later to upgrade to something better and faster. |
|
Back to top |
|
|
sphakka Tux's lil' helper
Joined: 24 Jun 2003 Posts: 79
|
Posted: Mon May 21, 2012 11:27 am Post subject: |
|
|
Hi all,
I have the same issue with SeaMonkey . As far as I can understand this is a general issue with linking huge binaries. I'm not an ld specialist, though from its man page I see some interesting options to trade off memory vs speed:
Code: |
--no-keep-memory
--reduce-memory-overheads
|
Does anybody know how to inject those options into the emerge process?
-- ^s |
|
Back to top |
|
|
sphakka Tux's lil' helper
Joined: 24 Jun 2003 Posts: 79
|
Posted: Mon May 21, 2012 9:12 pm Post subject: |
|
|
In the meanwhile I sorted out some emerge stats with a script of mine. Results for SeaMonkey and script available here. Anybody interested in firefox build stats, please, grab it, run it like this (needs genlop installed)...
Code: | $ ./emergetstat firefox |
...and post the results, along with CPU/RAM details. |
|
Back to top |
|
|
laudaka n00b
Joined: 23 Mar 2012 Posts: 3 Location: The Hague, Netherlands
|
Posted: Wed Sep 19, 2012 2:04 am Post subject: |
|
|
Same issue here. But I won't complain. On the contrary. Most Firefox users are Windows users and there's also a part that's Ubuntu/openSUSE/whatever Linux user. And none of those users builds Firefox themselves. So I perfectly understand why Mozilla doesn't make it a priority to cater the needs of the small group of users who do build Firefox themselves.
I considered it a challenge to build Firefox 10.0.7 . And yes I also saw the memory usage going up to nearly 2 GiB! And indeed it was a single process of the ld utility hogging all my memory. My hardware specs:
* Intel Core 2 Duo CPU E6550 @ 2.33GHz
* 1 GiB of RAM
* 372 GiB SATA hard drive
And the 1 GiB of RAM is the bottle neck of course here. I had a 1 GiB RAM swap partition but that's really too small for building Firefox. The chance of running out of memory is obviously too big. I growed the swap partition to 2 GiB and Firefox compiles flawlessly now. Even if it takes about two hours.
Three tips if you like to play with your system:
* If your system becomes irresponsive: Remember you can suspend a job in the Bash shell with Ctrl-Z. Of course the execution of the emerge command will completely stop, but at least you can work without any trouble on the pc. When you walk away from the pc use the fg command to resume the execution of the emerge command. It's not ideal, but for me it works.
* Your system may become so irresponsive that you have to wait minutes to get to the terminal window where emerge is running. So you have to wait minutes before you can even press Ctrl-Z to suspend emerge. It helps a bit to nice emerge. You will have to nice emerge from the start for that to work well. It appears to me nicing a process works very well if the irresponsiveness is because of that process using the CPU excessively. But in this case the problem is excessive reading and writing to the hard drive, but even here nicing helps a bit.
* If you want to have an idea what you will be facing next time, how many hours of compile time, use time emerge firefox. At least then next time you'll know how long you will have to wait. It will also tell you how long the processor has been crunching numbers, and how long the total process has taken. Because the processor has to wait a lot here for reads and writes to the hard drive the time the processor has been in use is rather low here.
Don't complain! See it as an opportunity to learn more about Linux while you're trying to deal with limited hardware capacity. I had quite some fun.
Paulus/laudaka
P.S. Building Thunderbird 10.0.7 also hogs your memory when running ld |
|
Back to top |
|
|
schorsch_76 Guru
Joined: 19 Jun 2012 Posts: 450
|
Posted: Wed Sep 19, 2012 5:52 am Post subject: |
|
|
One possible solution for users of old hardware.
Build that fat software packages in a VM on a todays machine. Limit the CHOST to i486-pc-gnu-linux. So did i with my very old and very limited Via Eden Board (bought maybe 2001). I made a package of the compiled software and moved that thing to the old machine. You could even make a fsarchive after upgrading/building in the vm. It will save you time and possibly money (you can use that old hardware with current software). But on the other hand you have an additional vm to manage and the hassle with creating the archive and deploying it.
Bye
schorsch |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9678 Location: almost Mile High in the USA
|
Posted: Wed Sep 19, 2012 6:40 am Post subject: |
|
|
There was another/several threads about firefox. This is both a problem in Windows and Linux where the link phase eats up a lot of memory.
That being said I was able to compile on these machines:
1GB RAM + 1GB swap (using Fvwm W/M no gnome, etc.) This is my server and I don't use the console much so I don't really notoice how slow it gets. I don't have much else running at the time which helps.
384MB RAM + 1.5GB swap <- talk about s.l.o.w. 1GB swap was insufficient to compile. This was a temp machine. It's up to 640MB RAM now but still needs over 1GB swap to finish.
2GB (and 4GB RAM) RAM, portage_tmpdir on hard disk, these should work fine with limited or no swap
8GB RAM (portage_tmpdir on tmpfs)
One 4GB(512MB swap, 64-bit) machine I have with portage_tmpdir on tmpfs failed compiling firefox10.
It is specifically the link phase (ld) that eats the machine.
Remember, if all else fails, there is firefox-bin. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
laudaka n00b
Joined: 23 Mar 2012 Posts: 3 Location: The Hague, Netherlands
|
Posted: Sun Sep 23, 2012 11:41 pm Post subject: |
|
|
Thanks for the additional tips They may come in handy some time. |
|
Back to top |
|
|
pigeon768 l33t
Joined: 02 Jan 2006 Posts: 683
|
Posted: Tue Sep 25, 2012 4:40 pm Post subject: |
|
|
You can control options sent to ld with the LDFLAGS option in /etc/make.conf. Read up on `man ld` for the options you'll want to use. Useful ones to reduce memory requirements include --reduce-memory-overheads and --no-keep-memory. Each option from `man ld` will need to be prefixed with '-Wl,' (capital W, lowercase L. The comma is important.) You do not want to overwrite the default LDFLAGS options, so start with `LDFLAGS="${LDFLAGS}"` and then add more options.
To put it all together, try adding a line like this to /etc/make.conf: Code: | LDFLAGS="${LDFLAGS} -Wl,--reduce-memory-overheads -Wl,--no-keep-memory" | YMMV. This will make ld run more slowly except in the specific circumstances when the low memory modes do not swap and the normal mode does swap. Consider adding those lines to /etc/portage/env/www-client/firefox (or whatever. Chromium is another offender.) rather than your system wide /etc/make.conf.
Also, if you have -ftracer, -O3, -ggdb, graphite etc in your CFLAGS, disable them for potentially dramatic reductions in memory usage. Disable pgo if you have it enabled, that uses tons of RAM. If you have only 2 gigs of RAM and you're running Gentoo, you would be better off sticking to x86 instead of amd64. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|