Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[firefox 10] difficult to build
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Desktop Environments
View previous topic :: View next topic  
Author Message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2289
Location: Canada

PostPosted: Thu Mar 01, 2012 7:34 am    Post subject: Reply with quote

xiber wrote:
Wow, thought it was only me having this ld issue. Been happening since firefox 9 for me. System grinds to a halt with 1GB of ram. After ram and swap are exausted, emerge fails. The only way I've been able to compile firefox is to shut everything down, including the xserver, and use a vt. I use tmux and let it compile while I'm at work. I've started to dread those little firefox point releases.



Especially since they now come like once a week :)
Back to top
View user's profile Send private message
Chiitoo
l33t
l33t


Joined: 28 Feb 2010
Posts: 856
Location: Here and Away Again

PostPosted: Thu Mar 01, 2012 3:37 pm    Post subject: Reply with quote

They obviously had to catch up with Opera's version numbering ! !! !

Which is what I actually use mainly.

I do have firefox around too, and an icecat... and so on... but Opera has been feeling the most suitable for me for a long time even before starting with Linux a bit over a year ago.

Now I'm wondering what it would be like to build Opera... hm...
_________________
Kind Regards,
~ The Noob Unlimited ~

Sore wa sore, kore wa kore.
Back to top
View user's profile Send private message
Oyarsa
n00b
n00b


Joined: 01 Jul 2002
Posts: 73
Location: Mars

PostPosted: Mon Mar 05, 2012 3:07 pm    Post subject: Reply with quote

Thanx for this thread everybody. I had the same problem on my old computer (1GHz Athlon, 768MB RAM, 2GB swap).
I recompiled overnight with "MAKEOPTS=-j1" and it worked. Still a lot of trouble for a backup browser, though.

Thanks to Gentoo this old (11 years now) crate is still running as an internet surfboard. :)
_________________
Dew knot trussed yore spell chequer two fined awl ewer miss steaks.


Last edited by Oyarsa on Tue Mar 06, 2012 2:48 am; edited 1 time in total
Back to top
View user's profile Send private message
Anarchy
Developer
Developer


Joined: 29 Jun 2005
Posts: 108

PostPosted: Tue Mar 06, 2012 1:36 am    Post subject: Re: [firefox 10] difficult to build Reply with quote

WvR wrote:
Laptop computer, Compaq, 2GB mem, 4GB swap, 2 CPUs, AMD64.

While compiling Firefox-10, I noticed that my laptop became very slow, up to the point of being almost unresponsive. I was able to go into a text terminal from Gnome (Ctrl-Alt-F1) and log in as root. I used "top" to diagnose the problem and noticed that the linker ("ld") was using a very large amount of memory: 1.4 GB (!). As a result the RAM was overflowing and my computer was swapping a lot and became very slow. I killed ld and let the emerge die.

I rebooted the computer to have a clean RAM, and turned off XDM to reduce memory use. I compiled firefox-10 from a text terminal. At the end of the compilation phase, ld used up to 1500 MB of memory (!). I think this is a very large amount. If you run anything else at the same time, the amount of swapping becomes quite large. Any other users with similar issues? Should this be filed as a bug and if so, where? It is very inconvenient that this ebuild requires so much memory.


The ebuild suggest such standards as the build takes 1G alone to link libxul. In order to understand the process you would have to be deeply involved with the development process upstream. The reason that this was the choosen method, was the fact that startup cache generation was add which helped to increase the startup time, along with reducing initial overhead as far as ram when running for the first time.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 392

PostPosted: Tue Mar 06, 2012 2:24 am    Post subject: Reply with quote

Another option that worked for me was to plug in an external hard drive (on a computer that had only 512 MB RAM) and use that temporarily as swap space -- just for Firefox. I wish that they hadn't adopted the rapid release/version number system. What do they get now, a full version number for fixing spelling errors in documentation and code comments?
Back to top
View user's profile Send private message
Clad in Sky
l33t
l33t


Joined: 04 May 2007
Posts: 778
Location: Germany

PostPosted: Tue Mar 06, 2012 7:03 am    Post subject: Reply with quote

Didn't happen here, I think.
I'm mostly compiling stuff and doing other things on my computer at the same time. Didn't feel any difference to less memory intensive packages.
My machine is now nearly exactly 4 years old (bought it 8th March 2008) - AMD Phenom with 4GB of RAM, so nothing fancy by today's standards.
Strange.
_________________
Kali Ma
Now it's autumn of the aeons
Dance with your sword
Now it's time for the harvest
Back to top
View user's profile Send private message
HomeUser
n00b
n00b


Joined: 22 Jan 2006
Posts: 51

PostPosted: Wed Mar 14, 2012 8:06 pm    Post subject: ld stuck Reply with quote

Also by me ld was stuck with firefox 10.0.1-r1 and with thunderbird 10.0.1-r1. Did a few try's the past days.
It ended either in a kernel crash or, the times I forgot to mount a somewhat bigger disk with "device full".
(I got Thunderbird in the end compiled yesterday.)
But that was only the second emerge, suggested by revdep-rebuild, while at least firefox seems to function normally without recompile.

My emerge log shows normal compilation just a few days ago, after the same --sync for both the same programs.
(Old machine with AMD Athlon(tm) 3000+ 1536MiB)

    wo mrt 7 21:30:35 CET 2012 >>> emerge (53 of 207) www-client/firefox-10.0.1-r1 to /
    wo mrt 7 21:30:36 CET 2012 === (53 of 207) Cleaning (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
    wo mrt 7 21:30:39 CET 2012 === (53 of 207) Compiling/Merging (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
    wo mrt 7 23:42:33 CET 2012 === (53 of 207) Merging (www-client/firefox-10.0.1-r1::/usr/portage/www-client/firefox/firefox-10.0.1-r1.ebuild)
    wo mrt 7 23:42:57 CET 2012 >>> AUTOCLEAN: www-client/firefox:0
    wo mrt 7 23:42:57 CET 2012 === Unmerging... (www-client/firefox-8.0)
    wo mrt 7 23:43:13 CET 2012 >>> unmerge success: www-client/firefox-8.0


    do mrt 8 19:33:36 CET 2012 >>> emerge (12 of 150) mail-client/thunderbird-10.0.1-r1 to /
    do mrt 8 19:33:36 CET 2012 === (12 of 150) Cleaning (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
    do mrt 8 19:33:40 CET 2012 === (12 of 150) Compiling/Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
    do mrt 8 20:12:00 CET 2012 Started emerge on: Mar 08, 2012 21:11:59
    do mrt 8 20:12:00 CET 2012 (garbage)
    do mrt 8 20:12:38 CET 2012 >>> emerge (1 of 1) media-video/mplayer-1.0_rc4_p20110322-r1


(Did have some tries taking hours with ld like taking 80% of memory, computer slowing down and dying, But I see today that it did succeeded yesterday.)
    di mrt 13 18:24:53 CET 2012 >>> emerge (1 of 14) mail-client/thunderbird-10.0.1-r1 to /
    di mrt 13 18:24:53 CET 2012 === (1 of 14) Cleaning (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
    di mrt 13 18:25:02 CET 2012 === (1 of 14) Compiling/Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
    di mrt 13 21:02:07 CET 2012 === (1 of 14) Merging (mail-client/thunderbird-10.0.1-r1::/usr/portage/mail-client/thunderbird/thunderbird-10.0.1-r1.ebuild)
    di mrt 13 21:02:25 CET 2012 >>> AUTOCLEAN: mail-client/thunderbird:0
    di mrt 13 21:02:25 CET 2012 === Unmerging... (mail-client/thunderbird-10.0.1-r1)
    di mrt 13 21:02:35 CET 2012 >>> unmerge success: mail-client/thunderbird-10.0.1-r1

(But a new emerge and forced stop of firefox is in the logs.)

Could that be the same problem? I see now garbage in the emerge.log, perhaps disk structure problem or an other problem with the linker/emerge. (No garbage in ..messages or summary log
I often hope that some packets like LibreOffice can someday be offered in a more modular form,
this could perhaps still be a special problem with building those Mozilla packets
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 7769
Location: Somewhere over Atlanta, Georgia

PostPosted: Wed Mar 14, 2012 8:14 pm    Post subject: Reply with quote

@HomeUser, do you have swap enabled? If so, how big is it? These days, Firefox and Thunderbird take upwards of 8GiB of /var/tmp space and 2GiB of virtual memory to build properly.

- John
_________________
This space intentionally left blank.
Back to top
View user's profile Send private message
HomeUser
n00b
n00b


Joined: 22 Jan 2006
Posts: 51

PostPosted: Wed Mar 14, 2012 9:06 pm    Post subject: Reply with quote

John R. Graham wrote:
@HomeUser, do you have swap enabled? If so, how big is it? These days, Firefox and Thunderbird take upwards of 8GiB of /var/tmp space and 2GiB of virtual memory to build properly.

- John


My swap is 1574332k. Use to play some television on the other monitor, but Mplayer only takes 5-6% CPU. I suppose the graphic cart does the biggest job.
Back to top
View user's profile Send private message
tld
l33t
l33t


Joined: 09 Dec 2003
Posts: 830

PostPosted: Tue Mar 20, 2012 12:48 am    Post subject: Reply with quote

haha...glad I found this. Pretty much explains why I had issues on my old 2.3 Ghz P4 with 768MB of ram :D

No way am I attempting this....guess it's time to move to firefox-bin for me.

I hate to say it, but the list of ways that Mozilla team has been pissing me off grows more every day. If it wasn't for the extensions I depend on (especially for web development) I'm not sure I'd put up with this crap for another second. Speaking of which, you gotta love how the Mozilla team added a "Web Developer" tool menu, when the same name had already been in use by the Web Developer extension for ages...now there are TWO of them with completely different contents...I could go on for hours...so I won't get started...

Tom
Back to top
View user's profile Send private message
navilein
n00b
n00b


Joined: 07 Feb 2007
Posts: 7

PostPosted: Mon May 07, 2012 9:02 pm    Post subject: Reply with quote

Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).

I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...

I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.

But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!

I don't know if it's me, or there is some lack of common-sense in this linking...

(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft)
Back to top
View user's profile Send private message
Anarchy
Developer
Developer


Joined: 29 Jun 2005
Posts: 108

PostPosted: Thu May 10, 2012 2:19 am    Post subject: Reply with quote

navilein wrote:
Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).

I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...

I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.

But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!

I don't know if it's me, or there is some lack of common-sense in this linking...

(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft)


Common sense tells ya that you can only take software so far with outdated hardware. This was a decision that was not made lightly. As far as your rude and uncalled for remarks you can keep them to yourself. If you have a problem due to hardware limitations this is not the fault of the software, You could always make a new xulrunner/firefox-3.6 ebuild for the last release if you really wanted to throw into a local overlay.
Back to top
View user's profile Send private message
PhaTaL-Err0r
n00b
n00b


Joined: 18 Apr 2012
Posts: 6

PostPosted: Thu May 10, 2012 1:55 pm    Post subject: Reply with quote

navilein wrote:
Dear developers,
I have an AMD-k6 @ 476MHz laptop with 128MB Ram. To my regular swap partition, I've added another two swap files on harddisk of 1G and 2G, precisely because it got stuck at firefox compile time (linking precisely). And even so, this is the second time I've left it run for over 1 week trying to link the bloody thing... Today it failed for the second time (kernel panic).

I've also tried the precompiled binary versions of firefox-bin and opera, but helas!, their binaries have a problem running on i586 AMD-k6 processors (I've found about it googling, something about the 'mov' operation not properly implemented on i586 for these kind of processors, or something like that). They fail miserably at start up with "Illegal instruction"...

I know my system is old and slow, but I've been using Gentoo on this system for over 6 years now, compiling everything from scratch and updating only once in a while. Slow, it's true, but steady and sure thing, once compilation finished.

But now I'm stuck! Really stuck big time! I can easily get by with 128MB ram and 350MB swap, for all my user needs! Why should your (poor) software development impose throwing a perfectly working system, that would otherwise fully satisfy one's computing needs ?
Isn't it a tad bit excessive to require 1.5GB memory ONLY for linking ONE single bloody program???... Especially for such commonly used program as a web browser, literally present in every single computer, and probably the most used program on any computer!

I don't know if it's me, or there is some lack of common-sense in this linking...

(Please don't tell me to buy myself a newer system or I'll tell you go work for micro$oft)


This is out of the hands of the Gentoo developers, Mozilla has pretty much stopped supporting old hardware since Firefox 3.6, and as stated by Anarchy, you can only take modern day software so far with outdated hardware. If you already have Firefox 3.6 emerged then mask all the newer generations of Firefox in /etc/portage/package.mask; or if you have the know-how write up an ebuild and stick it in your local overlay -- and I know you don't wanna hear this, but yes, you DO need to seriously upgrade your PC to a more modern system. You can only go so far with it with modern software, YES, older hardware such as yours is touted as funtional throughout the Linux communities, however their usefulness is more for servers, firewalls, etc... and not modern day productivity such as gaming, internet based stuff; web browsing, VoIP, etc.

There are specialized Linux distros for older hardware too, Puppy Linux: Wary is one of the best for older systems. Maybe see what versions they are using of apps you use regularly and emerge them or make ebuilds for them. However even as specialized as Puppy Linux: Wary is, on CPU's such as what you have it runs only "reasonably well". You're gonna have to face it sooner or later to upgrade to something better and faster.
Back to top
View user's profile Send private message
sphakka
n00b
n00b


Joined: 24 Jun 2003
Posts: 64

PostPosted: Mon May 21, 2012 11:27 am    Post subject: Reply with quote

Hi all,

I have the same issue with SeaMonkey . As far as I can understand this is a general issue with linking huge binaries. I'm not an ld specialist, though from its man page I see some interesting options to trade off memory vs speed:
Code:

--no-keep-memory
--reduce-memory-overheads

Does anybody know how to inject those options into the emerge process?

-- ^s
Back to top
View user's profile Send private message
sphakka
n00b
n00b


Joined: 24 Jun 2003
Posts: 64

PostPosted: Mon May 21, 2012 9:12 pm    Post subject: Reply with quote

In the meanwhile I sorted out some emerge stats with a script of mine. Results for SeaMonkey and script available here. Anybody interested in firefox build stats, please, grab it, run it like this (needs genlop installed)...
Code:
$ ./emergetstat firefox

...and post the results, along with CPU/RAM details.
Back to top
View user's profile Send private message
laudaka
n00b
n00b


Joined: 23 Mar 2012
Posts: 3
Location: The Hague, Netherlands

PostPosted: Wed Sep 19, 2012 2:04 am    Post subject: Reply with quote

Same issue here. But I won't complain. On the contrary. Most Firefox users are Windows users and there's also a part that's Ubuntu/openSUSE/whatever Linux user. And none of those users builds Firefox themselves. So I perfectly understand why Mozilla doesn't make it a priority to cater the needs of the small group of users who do build Firefox themselves.

I considered it a challenge to build Firefox 10.0.7 . And yes I also saw the memory usage going up to nearly 2 GiB! And indeed it was a single process of the ld utility hogging all my memory. My hardware specs:
* Intel Core 2 Duo CPU E6550 @ 2.33GHz
* 1 GiB of RAM
* 372 GiB SATA hard drive

And the 1 GiB of RAM is the bottle neck of course here. I had a 1 GiB RAM swap partition but that's really too small for building Firefox. The chance of running out of memory is obviously too big. I growed the swap partition to 2 GiB and Firefox compiles flawlessly now. Even if it takes about two hours.

Three tips if you like to play with your system:
* If your system becomes irresponsive: Remember you can suspend a job in the Bash shell with Ctrl-Z. Of course the execution of the emerge command will completely stop, but at least you can work without any trouble on the pc. When you walk away from the pc use the fg command to resume the execution of the emerge command. It's not ideal, but for me it works.
* Your system may become so irresponsive that you have to wait minutes to get to the terminal window where emerge is running. So you have to wait minutes before you can even press Ctrl-Z to suspend emerge. It helps a bit to nice emerge. You will have to nice emerge from the start for that to work well. It appears to me nicing a process works very well if the irresponsiveness is because of that process using the CPU excessively. But in this case the problem is excessive reading and writing to the hard drive, but even here nicing helps a bit.
* If you want to have an idea what you will be facing next time, how many hours of compile time, use time emerge firefox. At least then next time you'll know how long you will have to wait. It will also tell you how long the processor has been crunching numbers, and how long the total process has taken. Because the processor has to wait a lot here for reads and writes to the hard drive the time the processor has been in use is rather low here.

Don't complain! See it as an opportunity to learn more about Linux while you're trying to deal with limited hardware capacity. I had quite some fun. :D
Paulus/laudaka

P.S. Building Thunderbird 10.0.7 also hogs your memory when running ld :o :D
Back to top
View user's profile Send private message
schorsch_76
Apprentice
Apprentice


Joined: 19 Jun 2012
Posts: 210

PostPosted: Wed Sep 19, 2012 5:52 am    Post subject: Reply with quote

One possible solution for users of old hardware.

Build that fat software packages in a VM on a todays machine. Limit the CHOST to i486-pc-gnu-linux. So did i with my very old and very limited Via Eden Board (bought maybe 2001). I made a package of the compiled software and moved that thing to the old machine. You could even make a fsarchive after upgrading/building in the vm. It will save you time and possibly money (you can use that old hardware with current software). But on the other hand you have an additional vm to manage and the hassle with creating the archive and deploying it.

Bye
schorsch
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3939
Location: USA

PostPosted: Wed Sep 19, 2012 6:40 am    Post subject: Reply with quote

There was another/several threads about firefox. This is both a problem in Windows and Linux where the link phase eats up a lot of memory.

That being said I was able to compile on these machines:
1GB RAM + 1GB swap (using Fvwm W/M no gnome, etc.) This is my server and I don't use the console much so I don't really notoice how slow it gets. I don't have much else running at the time which helps.
384MB RAM + 1.5GB swap <- talk about s.l.o.w. 1GB swap was insufficient to compile. This was a temp machine. It's up to 640MB RAM now but still needs over 1GB swap to finish.
2GB (and 4GB RAM) RAM, portage_tmpdir on hard disk, these should work fine with limited or no swap
8GB RAM (portage_tmpdir on tmpfs)

One 4GB(512MB swap, 64-bit) machine I have with portage_tmpdir on tmpfs failed compiling firefox10.

It is specifically the link phase (ld) that eats the machine.

Remember, if all else fails, there is firefox-bin.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
laudaka
n00b
n00b


Joined: 23 Mar 2012
Posts: 3
Location: The Hague, Netherlands

PostPosted: Sun Sep 23, 2012 11:41 pm    Post subject: Reply with quote

Thanks for the additional tips :) They may come in handy some time.
Back to top
View user's profile Send private message
pigeon768
l33t
l33t


Joined: 02 Jan 2006
Posts: 669

PostPosted: Tue Sep 25, 2012 4:40 pm    Post subject: Reply with quote

You can control options sent to ld with the LDFLAGS option in /etc/make.conf. Read up on `man ld` for the options you'll want to use. Useful ones to reduce memory requirements include --reduce-memory-overheads and --no-keep-memory. Each option from `man ld` will need to be prefixed with '-Wl,' (capital W, lowercase L. The comma is important.) You do not want to overwrite the default LDFLAGS options, so start with `LDFLAGS="${LDFLAGS}"` and then add more options.

To put it all together, try adding a line like this to /etc/make.conf:
Code:
LDFLAGS="${LDFLAGS} -Wl,--reduce-memory-overheads -Wl,--no-keep-memory"
YMMV. This will make ld run more slowly except in the specific circumstances when the low memory modes do not swap and the normal mode does swap. Consider adding those lines to /etc/portage/env/www-client/firefox (or whatever. Chromium is another offender.) rather than your system wide /etc/make.conf.

Also, if you have -ftracer, -O3, -ggdb, graphite etc in your CFLAGS, disable them for potentially dramatic reductions in memory usage. Disable pgo if you have it enabled, that uses tons of RAM. If you have only 2 gigs of RAM and you're running Gentoo, you would be better off sticking to x86 instead of amd64.
_________________
My political bias.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Desktop Environments All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum