| View previous topic :: View next topic |
| Author |
Message |
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
 Posted: Fri Jul 20, 2012 9:42 am Post subject: [Solved] webkit-gtk, ld out of memory, 32-bit system |
 |
|
Hello,
I've been trying to update my system over the past week, dealing with all kinds of issues but after couple of days of Googling, I brought down the 700 odd packages to about 25 more left to update. My issue right now is with webkit-gtk. At first my tmpfs which I used to build packages was too small to accommodate the build. Fair, I increased the size of it, remounted and tried again. Same thing. I decided to do it on the disk instead seeing as I only have 3GB of RAM. I quickly put it in notmpfs.conf and fired away. Ran out of RAM during the build. OK. I decided to make a 4GB swap (versus my normal 1GB) and point my system to that. After getting a lot further than I normally did, it came to the linking part of the installation. Surprise surprise, Memory exhausted error. More Google and it seems that I am running out of memory addresses for a single process as I'm on 32-bit system. More Google and I can't find a solution. Time to finally register on these forums that helped me so much in the past.
My question is, is there anything I can do to decrease the amount of memory needed during linking? I'd like to mention that a couple of things seem to depend on the new webkit-gtk and I'd very much like to have it, if at all possible. The logs follow:
build.log
http://pastebin.com/R8RERUrx
make.conf
| Code: |
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
CFLAGS="-O2 -march=i686 -pipe -ggdb"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j3"
EMERGE_DEFAULT_OPTS="--jobs=3 --load-average=2 --with-bdeps y"
ACCEPT_LICENSE="*"
ACCEPT_KEYWORDS="~x86"
FEATURES="splitdebug ccache"
CCACHE_DIR="/var/tmp/ccache"
CCACHE_SIZE="500M"
#PORTAGE_NICENESS=5
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST="i686-pc-linux-gnu"
#GENTOO_MIRRORS="http://mirror.bytemark.co.uk/gentoo/ http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/"
#SYNC="rsync://rsync.uk.gentoo.org/gentoo-portage"
USE="gtk gnome -qt4 -kde alsa -minimal input_devices_mouse bzip2 python X dbus jpeg sqlite png aac acpi bash-completion bluetooth canna cups curl cxx emacs exif ffnpeg flac ftp gif gstreamer hscolour matroska mp4 mpeg mplayer ncurses networkmanager ogg opengl pdf perl socks5 sockets ssl zsh-completion tk truetype unicode3 mousewheel unicode nsplugin cjk scim cls anthy m17n-lib immqt-bc gd latex pulseaudio pulse sasl smpeg mp3 threads openssl -nss dhcpcd gnutls"
VIDEO_CARDS="intel vesa fbdev"
LINGUAS="ja ko zh_CN"
CURL_SSL="openssl"
USE_PYTHON="2.7"
source /var/lib/layman/make.conf
|
emerge --info
| Code: |
Portage 2.1.11.7 (default/linux/x86/10.0, gcc-4.6.3, glibc-2.15-r2, 3.0.6-gentoo i686)
=================================================================
System Settings
=================================================================
System uname: Linux-3.0.6-gentoo-i686-Intel-R-_Core-TM-2_Duo_CPU_L7700_@_1.80GHz-with-gentoo-2.1
Timestamp of tree: Mon, 16 Jul 2012 13:30:01 +0000
ccache version 3.1.7 [enabled]
app-shells/bash: 4.2_p36
dev-java/java-config: 2.1.12
dev-lang/python: 2.7.3-r2, 3.1.4-r3, 3.2.3-r1
dev-util/cmake: 2.8.8-r3
dev-util/pkgconfig: 0.27
sys-apps/baselayout: 2.1-r1
sys-apps/openrc: 0.10.5
sys-apps/sandbox: 2.6
sys-devel/autoconf: 2.13, 2.69
sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6, 1.12.2
sys-devel/binutils: 2.22-r1
sys-devel/gcc: 4.5.3-r2, 4.6.3
sys-devel/gcc-config: 1.7.3
sys-devel/libtool: 2.4.2
sys-devel/make: 3.82-r3
sys-kernel/linux-headers: 3.4-r1 (virtual/os-headers)
sys-libs/glibc: 2.15-r2
Repositories: gentoo gentoo-haskell
ACCEPT_KEYWORDS="x86 ~x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=i686 -pipe -ggdb"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=i686 -pipe -ggdb"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=3 --load-average=2 --with-bdeps y"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -march=i686 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_GB.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="ja ko zh_CN"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/haskell"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X aac acl acpi alsa anthy bash-completion berkdb bluetooth bzip2 canna cjk cli cls cracklib crypt cups curl cxx dbus dhcpcd dri emacs exif ffnpeg flac fortran ftp gd gdbm gif gnome gnutls gpm gstreamer gtk hscolour iconv immqt-bc ipv6 jpeg latex m17n-lib matroska modules mousewheel mp3 mp4 mpeg mplayer mudflap ncurses networkmanager nls nptl nsplugin ogg opengl openmp openssl pam pcre pdf perl png pppd pulse pulseaudio python readline sasl scim session smpeg sockets socks5 sqlite ssl tcpd threads tk truetype unicode unicode3 x86 xorg zlib zsh-completion" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CURL_SSL="openssl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ja ko zh_CN" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="intel vesa fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7"
Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
|
Thanks in advance!
Last edited by Fuuzetsu on Sat Jul 21, 2012 9:35 pm; edited 3 times in total |
|
| Back to top |
|
 |
VoidMage
Advocate
Joined: 14 Oct 2006
Posts: 4809
|
| Posted: Fri Jul 20, 2012 10:43 am Post subject: |
|
|
| Fuuzetsu wrote: | MAKEOPTS="-j3"
EMERGE_DEFAULT_OPTS="--jobs=3 --load-average=2 --with-bdeps y" |
Try using lower values for these.
There's something odd here - 2GB of RAM + 1GB of swap should be enough for webkit-gtk. |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Fri Jul 20, 2012 2:33 pm Post subject: |
|
|
| VoidMage wrote: |
Try using lower values for these.
There's something odd here - 2GB of RAM + 1GB of swap should be enough for webkit-gtk. |
4 hours of waiting later, same result:
http://pastebin.com/jbd8EKjG
EDIT:
ld -v shows
| Code: | | GNU gold (GNU Binutils 2.22) 1.11 |
so I guess it was using gold after all...
Last edited by Fuuzetsu on Fri Jul 20, 2012 11:23 pm; edited 2 times in total |
|
| Back to top |
|
|
padoor
Advocate
Joined: 30 Dec 2005
Posts: 3911
Location: india
|
| Posted: Fri Jul 20, 2012 5:06 pm Post subject: |
|
|
my desktop is p4 2.4 Ghz 512mb ram. 1.6 GB swap
i had tried 5 times to merge.
finally it merged ok in 4 hrs.
this time i used nox option in kernel line for booting.
i did the merge in vt1
check if any use flag missing or extra
you have more than enough mem. it should not run short of memory.
the X may be using any files the webkit wants to link or something.
try nox option.
_________________
reach out a little bit more to catch it (DON'T BELIEVE the advocate part under my user name) |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Fri Jul 20, 2012 5:35 pm Post subject: |
|
|
| padoor wrote: | my desktop is p4 2.4 Ghz 512mb ram. 1.6 GB swap
i had tried 5 times to merge.
finally it merged ok in 4 hrs.
this time i used nox option in kernel line for booting.
i did the merge in vt1
check if any use flag missing or extra
you have more than enough mem. it should not run short of memory.
the X may be using any files the webkit wants to link or something.
try nox option. |
Even though I don't believe that's the issue, I'll try without X.
I thought that I should check some info out about the old build and something strange came up. It would seem that I have both versions installed? Am I reading this wrong?
Output of eix:
| Code: |
[U] net-libs/webkit-gtk
Available versions:
(2) 1.6.1-r201 1.6.3-r200 (~)1.8.0-r200 (~)1.8.1-r200 (~)1.8.1-r201
(3) 1.6.1-r301 1.6.3-r300 (~)1.8.0-r300 (~)1.8.1-r300 (~)1.8.1-r301
{{aqua coverage debug doc +geoloc +gstreamer +introspection +jit spell test (+)webgl webkit2}}
Installed versions: 1.4.3-r200(2)(05:37:42 06/01/12)(gstreamer introspection jit -aqua -coverage -debug -doc -spell -test) 1.8.1-r301(3)(21:17:09 18/07/12)(geoloc gstreamer introspection jit webgl -aqua -coverage -debug -doc -spell -test -webkit2)
Homepage: http://www.webkitgtk.org/
Description: Open source web browser engine
|
Output of emerge --info 'webkit-gtk'
| Code: |
net-libs/webkit-gtk-1.4.3-r200 was built with the following:
USE="gstreamer introspection jit (-aqua) -coverage -debug -doc -spell -test"
CFLAGS="-O2 -march=i686 -pipe"
CXXFLAGS="-O2 -march=i686 -pipe"
net-libs/webkit-gtk-1.8.1-r301 was built with the following:
USE="geoloc gstreamer introspection jit webgl (-aqua) -coverage -debug -doc -spell -test -webkit2"
|
Not sure what I should think.
EDIT:
... so I just did this
| Code: |
ShanaX61s% emerge -pav webkit-gtk
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] net-libs/webkit-gtk-1.8.1-r301 USE="geoloc gstreamer introspection jit webgl (-aqua) -coverage -debug -doc -spell -test -webkit2" 0 kB
Total: 1 package (1 reinstall), Size of downloads: 0 kB
|
It would seem that somewhere along the way amongst my tries, the package actually DID build. I just hope it isn't half broken. Sorry for bothering you all. I do wonder why it kept running out of memory though, in case the issue comes back with a future update.
EDIT2:
Intrigued, I went back and checked why would I have two versions. Then I remembered slots, and surely enough, quick Google search resulted in me finding out that the (2) and (3) in eix output are two different slots. In fact, revdep-rebuild wants to update the one in slot 2. I better start building without X then...
Last edited by Fuuzetsu on Fri Jul 20, 2012 6:59 pm; edited 2 times in total |
|
| Back to top |
|
|
Ant P.
Veteran
Joined: 18 Apr 2009
Posts: 1916
Location: UK
|
| Posted: Fri Jul 20, 2012 5:48 pm Post subject: |
|
|
| You could try using gold to link it instead of the default linker (use `binutils-config`). It's pretty risky though. |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Fri Jul 20, 2012 7:11 pm Post subject: |
|
|
| Ant P. wrote: | | You could try using gold to link it instead of the default linker (use `binutils-config`). It's pretty risky though. |
Emerging right now with ld.gold set using binutils-config; seems like a far better bet than the suggestion to do it without X loaded. |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Fri Jul 20, 2012 11:18 pm Post subject: |
|
|
| Fuuzetsu wrote: |
Emerging right now with ld.gold set using binutils-config; seems like a far better bet than the suggestion to do it without X loaded. |
...it seems that it still used the same linker so the config probably didn't affect the emerge? Or does it show up as ld anyway?
Here is the log of another failed build; this time the message is different:
http://pastebin.com/vyktXtML
EDIT:
| Code: |
ShanaX61s% ld -v
GNU gold (GNU Binutils 2.22) 1.11
|
So it was running gold after all… |
|
| Back to top |
|
|
Ant P.
Veteran
Joined: 18 Apr 2009
Posts: 1916
Location: UK
|
| Posted: Sat Jul 21, 2012 1:22 am Post subject: |
|
|
It's using mmap, so that's progress at least - if it runs out of RAM it'll swap to the mmap'ed file. It's still hitting the VM limit doing that though.
Try using a 64-bit kernel (export ARCH=x86_64; make oldconfig; make). It shouldn't have any negative effect on the system but the ld process will get 4GB of VM to itself. If that doesn't work then it looks like you're stuck. |
|
| Back to top |
|
|
verty
n00b
Joined: 21 Jul 2012
Posts: 1
|
| Posted: Sat Jul 21, 2012 11:28 am Post subject: |
|
|
| Try it without -ggdb. You may find that building it with debug symbols like that cannot be done on 32-bit. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 6423
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sat Jul 21, 2012 3:34 pm Post subject: |
|
|
I suspect it's just a lack of swap space. It builds here just fine on a 32-bit system with only 1GiB of RAM, but with 4GiB of swap. Trying again now with -ggdb set in CFLAGS and FEATURES="splitdebug".
- John
_________________
This space intentionally left blank. |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Sat Jul 21, 2012 5:38 pm Post subject: |
|
|
| John R. Graham wrote: | I suspect it's just a lack of swap space. It builds here just fine on a 32-bit system with only 1GiB of RAM, but with 4GiB of swap. Trying again now with -ggdb set in CFLAGS and FEATURES="splitdebug".
- John |
I made a 4GB swap file when the problem first occurred following Gentoo Wiki.
I'll be trying with -ggdb removed in a while, I need to sort my system out after upgrading hundreds of packages and a new kernel at once. Boy that reboot wasn't fun. I failed to build 64-bit kernel because apparently the type of CPU I selected is not compatible with x86-64 instruction set... Weird but I didn't have that much time to look into it and compiled 32-bit first in hopes of getting something working. I'll report back Later™. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 6423
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sat Jul 21, 2012 7:27 pm Post subject: |
|
|
Built just fine here with -ggdb in the CFLAGS.
- John
_________________
This space intentionally left blank. |
|
| Back to top |
|
|
Fuuzetsu
n00b
Joined: 20 Jul 2012
Posts: 14
Location: Bath, UK
|
| Posted: Sat Jul 21, 2012 9:35 pm Post subject: |
|
|
I have removed the -ggdb flag, using freshly compiled 3.4.4 kernel and ld.bdf and I just found that webkit-gtk:2 has successfully built and installed, without any issues.
I don't know which of the above did it (I'm betting on -ggdb) but I now have what I wanted! Thanks everyone for help and bearing with me. Editing the thread to [Solved]. |
|
| Back to top |
|
|
|
Portage & Programming
