Forums

Skip to content

Advanced search
  • Quick links
    • Unanswered topics
    • Active topics
    • Search
  • FAQ
  • Login
  • Register
  • Board index Assistance Portage & Programming
  • Search

qtwebengine taking more than a day to emerge

Problems with emerge or ebuilds? Have a basic programming question about C, PHP, Perl, BASH or something else?
Post Reply
Advanced search
89 posts
  • 1
  • 2
  • 3
  • 4
  • Next
Author
Message
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

qtwebengine taking more than a day to emerge

  • Quote

Post by urcindalo » Mon Jan 14, 2019 2:38 pm

Hi!

Whenever portage decides to emerge qtwebengine, it takes almost forever 8O :

Code: Select all

$ sudo splat qtwebengine
 * dev-qt/qtwebengine-5.11.1

        Emerged at: dom oct 21 00:01:10 2018
        Build time: 1 day, 15 hours, 15 minutes, and 20 seconds

        Emerged at: lun dic 10 13:02:54 2018
        Build time: 18 hours, 30 minutes, and 8 seconds

        Emerged at: lun ene  7 06:06:01 2019
        Build time: 1 day, 6 hours, 24 minutes, and 44 seconds

 * dev-qt/qtwebengine-5.11.3

        Emerged at: vie ene 11 09:41:04 2019
        Build time: 1 day, 9 hours, 54 minutes, and 29 seconds
The particular box where this happens is old, I know, but equivalent packages take only a few hours:

Code: Select all

$ sudo splat webkit-gtk
 * net-libs/webkit-gtk-2.22.2

        Emerged at: dom oct 14 12:46:40 2018
        Build time: 7 hours, 9 minutes, and 29 seconds

        Emerged at: vie oct 26 17:38:55 2018
        Build time: 9 hours, 3 minutes, and 1 second

        Emerged at: lun nov 26 03:13:26 2018
        Build time: 6 hours, 43 minutes, and 48 seconds

        Emerged at: lun ene  7 00:51:34 2019
        Build time: 5 hours, 14 minutes, and 8 seconds

 * net-libs/webkit-gtk-2.22.5

        Emerged at: jue ene 10 23:16:22 2019
        Build time: 9 hours, 5 minutes, and 5 seconds

.....

$ sudo splat libreoffice
 * app-office/libreoffice-6.1.4.2

        Emerged at: mar ene  8 12:46:00 2019
        Build time: 4 hours, 31 minutes, and 41 seconds

Any ideas why? I have no clue.
Here is the hardware info:

Code: Select all

$ sudo lshw
lab                         
    description: Desktop Computer
    product: System Product Name (To Be Filled By O.E.M.)
    vendor: System manufacturer
    version: System Version
    serial: System Serial Number
    width: 4294967295 bits
    capabilities: smbios-2.5 dmi-2.5 smp vsyscall32
    configuration: boot=normal chassis=desktop family=To Be Filled By O.E.M. sku=To Be Filled By O.E.M. uuid=609A0D5B-3CE4-DE11-8276-E0CB4E39193A
  *-core
       description: Motherboard
       product: P5QL/EPU
       vendor: ASUSTeK Computer INC.
       physical id: 0
       version: Rev X.0x
       serial: 102963730001615
       slot: To Be Filled By O.E.M.
     *-firmware
          description: BIOS
          vendor: American Megatrends Inc.
          physical id: 0
          version: 0408
          date: 07/20/2009
          size: 64KiB
          capacity: 960KiB
          capabilities: isa pci pnp apm upgrade shadowing escd cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer int10video acpi usb ls120boot zipboot biosbootspecification
     *-cpu
          description: CPU
          product: Intel(R) Core(TM)2 Quad CPU    Q8300  @ 2.50GHz
          vendor: Intel Corp.
          physical id: 4
          bus info: cpu@0
          version: Intel(R) Core(TM)2 Quad CPU Q8300 @ 2.50GHz
          serial: To Be Filled By O.E.M.
          slot: LGA775
          size: 2500MHz
          capacity: 3800MHz
          width: 64 bits
          clock: 333MHz
          capabilities: x86-64 fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm pti tpr_shadow vnmi flexpriority dtherm
          configuration: cores=4 enabledcores=4 threads=4
        *-cache:0
             description: L1 cache
             physical id: 5
             slot: L1-Cache
             size: 128KiB
             capacity: 128KiB
             capabilities: internal write-back data
             configuration: level=1
        *-cache:1
             description: L2 cache
             physical id: 6
             slot: L2-Cache
             size: 4MiB
             capacity: 4MiB
             capabilities: internal write-back unified
             configuration: level=2
     *-memory
          description: System Memory
          physical id: 33
          slot: System board or motherboard
          size: 4GiB
        *-bank:0
             description: DIMM DDR2 Synchronous 800 MHz (1,2 ns)
             product: PartNum0
             vendor: Manufacturer0
             physical id: 0
             serial: SerNum0
             slot: DIMM0
             size: 2GiB
             width: 64 bits
             clock: 800MHz (1.2ns)
        *-bank:1
             description: DIMM [empty]
             product: PartNum1
             vendor: Manufacturer1
             physical id: 1
             serial: SerNum1
             slot: DIMM1
        *-bank:2
             description: DIMM DDR2 Synchronous 800 MHz (1,2 ns)
             product: PartNum2
             vendor: Manufacturer2
             physical id: 2
             serial: SerNum2
             slot: DIMM2
             size: 2GiB
             width: 64 bits
             clock: 800MHz (1.2ns)
        *-bank:3
             description: DIMM [empty]
             product: PartNum3
             vendor: Manufacturer3
             physical id: 3
             serial: SerNum3
             slot: DIMM3
     *-pci
          description: Host bridge
          product: 4 Series Chipset DRAM Controller
          vendor: Intel Corporation
          physical id: 100
          bus info: pci@0000:00:00.0
          version: 03
          width: 32 bits
          clock: 33MHz
        *-pci:0
             description: PCI bridge
             product: 4 Series Chipset PCI Express Root Port
             vendor: Intel Corporation
             physical id: 1
             bus info: pci@0000:00:01.0
             version: 03
             width: 32 bits
             clock: 33MHz
             capabilities: pci pm msi pciexpress normal_decode bus_master cap_list
             configuration: driver=pcieport
             resources: irq:16 ioport:c000(size=4096) memory:fa000000-fe9fffff ioport:d0000000(size=268435456)
           *-display
                description: VGA compatible controller
                product: G98 [GeForce 8400 GS Rev. 2]
                vendor: NVIDIA Corporation
                physical id: 0
                bus info: pci@0000:01:00.0
                version: a1
                width: 64 bits
                clock: 33MHz
                capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
                configuration: driver=nouveau latency=0
                resources: irq:24 memory:fd000000-fdffffff memory:d0000000-dfffffff memory:fa000000-fbffffff ioport:cc00(size=128) memory:c0000-dffff
        *-usb:0
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #4
             vendor: Intel Corporation
             physical id: 1a
             bus info: pci@0000:00:1a.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:16 ioport:b800(size=32)
        *-usb:1
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #5
             vendor: Intel Corporation
             physical id: 1a.1
             bus info: pci@0000:00:1a.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:21 ioport:b880(size=32)
        *-usb:2
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #6
             vendor: Intel Corporation
             physical id: 1a.2
             bus info: pci@0000:00:1a.2
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:18 ioport:bc00(size=32)
        *-usb:3
             description: USB controller
             product: 82801JI (ICH10 Family) USB2 EHCI Controller #2
             vendor: Intel Corporation
             physical id: 1a.7
             bus info: pci@0000:00:1a.7
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pm debug ehci bus_master cap_list
             configuration: driver=ehci-pci latency=0
             resources: irq:18 memory:f9fffc00-f9ffffff
        *-multimedia
             description: Audio device
             product: 82801JI (ICH10 Family) HD Audio Controller
             vendor: Intel Corporation
             physical id: 1b
             bus info: pci@0000:00:1b.0
             version: 00
             width: 64 bits
             clock: 33MHz
             capabilities: pm msi pciexpress bus_master cap_list
             configuration: driver=snd_hda_intel latency=0
             resources: irq:25 memory:f9ff8000-f9ffbfff
        *-pci:1
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Root Port 1
             vendor: Intel Corporation
             physical id: 1c
             bus info: pci@0000:00:1c.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode bus_master cap_list
             configuration: driver=pcieport
             resources: irq:17 ioport:1000(size=4096) memory:f0000000-f03fffff ioport:f8f00000(size=1048576)
        *-pci:2
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Port 2
             vendor: Intel Corporation
             physical id: 1c.1
             bus info: pci@0000:00:1c.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode bus_master cap_list
             configuration: driver=pcieport
             resources: irq:16 ioport:e000(size=4096) memory:feb00000-febfffff ioport:f0400000(size=2097152)
           *-ide UNCLAIMED
                description: IDE interface
                product: JMB368 IDE controller
                vendor: JMicron Technology Corp.
                physical id: 0
                bus info: pci@0000:03:00.0
                version: 00
                width: 32 bits
                clock: 33MHz
                capabilities: ide pm pciexpress msi bus_master cap_list
                configuration: latency=0
                resources: ioport:ec00(size=8) ioport:e880(size=4) ioport:e800(size=8) ioport:e480(size=4) ioport:e400(size=16) memory:febf0000-febfffff
        *-pci:3
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Root Port 3
             vendor: Intel Corporation
             physical id: 1c.2
             bus info: pci@0000:00:1c.2
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode bus_master cap_list
             configuration: driver=pcieport
             resources: irq:18 ioport:d000(size=4096) memory:fea00000-feafffff ioport:f8e00000(size=1048576)
           *-network
                description: Ethernet interface
                product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
                vendor: Realtek Semiconductor Co., Ltd.
                physical id: 0
                bus info: pci@0000:02:00.0
                logical name: eth0
                version: 02
                serial: e0:cb:4e:39:19:3a
                size: 100Mbit/s
                capacity: 1Gbit/s
                width: 64 bits
                clock: 33MHz
                capabilities: pm msi pciexpress msix vpd bus_master cap_list rom ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd autonegotiation
                configuration: autonegotiation=on broadcast=yes driver=r8169 driverversion=2.3LK-NAPI duplex=full ip=192.168.212.11 latency=0 link=yes multicast=yes port=MII speed=100Mbit/s
                resources: irq:26 ioport:d800(size=256) memory:f8eff000-f8efffff memory:f8ee0000-f8eeffff memory:feaf0000-feafffff
        *-usb:4
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #1
             vendor: Intel Corporation
             physical id: 1d
             bus info: pci@0000:00:1d.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:23 ioport:b080(size=32)
        *-usb:5
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #2
             vendor: Intel Corporation
             physical id: 1d.1
             bus info: pci@0000:00:1d.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:19 ioport:b400(size=32)
        *-usb:6
             description: USB controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #3
             vendor: Intel Corporation
             physical id: 1d.2
             bus info: pci@0000:00:1d.2
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:18 ioport:b480(size=32)
        *-usb:7
             description: USB controller
             product: 82801JI (ICH10 Family) USB2 EHCI Controller #1
             vendor: Intel Corporation
             physical id: 1d.7
             bus info: pci@0000:00:1d.7
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pm debug ehci bus_master cap_list
             configuration: driver=ehci-pci latency=0
             resources: irq:23 memory:f9fff800-f9fffbff
        *-pci:4
             description: PCI bridge
             product: 82801 PCI Bridge
             vendor: Intel Corporation
             physical id: 1e
             bus info: pci@0000:00:1e.0
             version: 90
             width: 32 bits
             clock: 33MHz
             capabilities: pci subtractive_decode bus_master cap_list
        *-isa
             description: ISA bridge
             product: 82801JIB (ICH10) LPC Interface Controller
             vendor: Intel Corporation
             physical id: 1f
             bus info: pci@0000:00:1f.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: isa bus_master cap_list
             configuration: driver=lpc_ich latency=0
             resources: irq:0
        *-ide:0
             description: IDE interface
             product: 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
             vendor: Intel Corporation
             physical id: 1f.2
             bus info: pci@0000:00:1f.2
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: ide pm bus_master cap_list
             configuration: driver=ata_piix latency=0
             resources: irq:19 ioport:a000(size=8) ioport:9c00(size=4) ioport:9880(size=8) ioport:9800(size=4) ioport:9480(size=16) ioport:9400(size=16)
        *-serial UNCLAIMED
             description: SMBus
             product: 82801JI (ICH10 Family) SMBus Controller
             vendor: Intel Corporation
             physical id: 1f.3
             bus info: pci@0000:00:1f.3
             version: 00
             width: 64 bits
             clock: 33MHz
             configuration: latency=0
             resources: memory:f9fff400-f9fff4ff ioport:400(size=32)
        *-ide:1
             description: IDE interface
             product: 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
             vendor: Intel Corporation
             physical id: 1f.5
             bus info: pci@0000:00:1f.5
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: ide pm bus_master cap_list
             configuration: driver=ata_piix latency=0
             resources: irq:19 ioport:b000(size=8) ioport:ac00(size=4) ioport:a880(size=8) ioport:a800(size=4) ioport:a480(size=16) ioport:a400(size=16)
These are the HD partitions:

Code: Select all

$ df -h
S.ficheros     Tamaño Usados  Disp Uso% Montado en
/dev/sda2         40G    28G  9,9G  74% /
devtmpfs         2,0G      0  2,0G   0% /dev
tmpfs            2,0G      0  2,0G   0% /dev/shm
tmpfs            2,0G   1,2M  2,0G   1% /run
tmpfs            2,0G      0  2,0G   0% /sys/fs/cgroup
tmpfs            2,0G    12K  2,0G   1% /tmp
/dev/sda6         22G    42M   22G   1% /var/tmp
/dev/sda3        168G   140G   20G  88% /home
tmpfs            395M   8,0K  395M   1% /run/user/1000
/dev/sda7        208G   183G   26G  88% /run/media/ramiro/Trabajo
Top
fedeliallalinea
Administrator
Administrator
User avatar
Posts: 31985
Joined: Sat Mar 08, 2003 11:15 pm
Location: here
Contact:
Contact fedeliallalinea
Website

  • Quote

Post by fedeliallalinea » Mon Jan 14, 2019 2:55 pm

qtwebengine include chromium code, and no, webkit-gtk isn't equivalent to qtwebengine in term of compilation time.
In my i7-4930K_CPU_@_3.40GHz

Code: Select all

     Thu Jan  3 10:36:09 2019 >>> dev-qt/qtwebengine-5.11.3
       merge time: 1 hour, 4 minutes and 5 seconds.

     Thu Sep  8 08:05:35 2016 >>> net-libs/webkit-gtk-2.4.11-r200
       merge time: 25 minutes and 15 seconds.
Questions are guaranteed in life; Answers aren't.

"Those who would give up essential liberty to purchase a little temporary safety,
deserve neither liberty nor safety."
- Ben Franklin
https://www.news.admin.ch/it/nsb?id=103968
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

  • Quote

Post by urcindalo » Mon Jan 14, 2019 3:11 pm

Thanks for the info.

By "equivalent" I mean source code size: I also show libreoffice compilation time.

In your case, qtwebengine compilation time doubles that of webkit-gtk, but in mine it is up to six or seven times larger because, as a side note, I must say the compilation times showed by splat are not real: last one took a little bit more than two days, not 1.4 as reported.
So, I find it strange it takes that long.

For instance, in a more powerful box I have the compilation times for qtwebengine and webkit-gtk are:

Code: Select all

$ sudo splat qtwebengine
 * dev-qt/qtwebengine-5.11.3

        Emerged at: jue ene 10 10:11:42 2019
        Build time: 3 hours, 33 minutes, and 42 seconds

.....

$ sudo splat webkit-gtk
net-libs/webkit-gtk-2.22.5

        Emerged at: jue ene 10 19:42:00 2019
        Build time: 1 hour, 28 minutes, and 6 seconds
As you can see, about two times more, as in your case, not six or seven.
Last edited by urcindalo on Tue Jan 15, 2019 7:55 am, edited 1 time in total.
Top
Chiitoo
Ninja Apprentice
Ninja Apprentice
User avatar
Posts: 3076
Joined: Sun Feb 28, 2010 5:36 pm
Location: Sore wa sore, kore wa kore... nanoda.

><)))°€

  • Quote

Post by Chiitoo » Mon Jan 14, 2019 9:50 pm

For a bit of a comparison, I have a log of 6 hours, 36 minutes, and 6 seconds on qtwebengine-5.11.1, on an AMD Athlon 7750 dual core processor with around 3 GiBs of RAM. I do forget if that build made use of distcc or not, but I'd still probably expect yours to be able to do better.

The Qt WebEngine build really likes RAM. A lot. Which you seem to have around 4 GiB, and my first suspect therefore would be major swapping during the build.

If you have been building with USE="jumbo-build", I might try without it, and/or lowering the MAKEOPTS value.

Or/and add more RAM, if possible. :]
Kindest of regardses.
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

Re: ><)))°€

  • Quote

Post by urcindalo » Tue Jan 15, 2019 7:59 am

Chiitoo wrote:The Qt WebEngine build really likes RAM. A lot. Which you seem to have around 4 GiB, and my first suspect therefore would be major swapping during the build.
Maybe that's the reason. My fast box has 8Gb RAM. I wonder if 8Gb is the bare minimum for qtwebengine to compile "normally" (as other packages do).
Top
376212
n00b
n00b
Posts: 37
Joined: Wed Feb 28, 2018 1:43 pm

  • Quote

Post by 376212 » Tue Jan 15, 2019 9:28 am

Code: Select all


dev-qt/qtwebengine-5.11.3

Emerged at: Mi Jan  2 13:37:06 2019
Build time: 1 hour, 29 minutes, and 49 seconds

compiled with ggdb debug symbols, -j16 makeopts and in my ram disk. it eats up sometimes 22 gb of my systems memory (just only the build files in ram disk) but also there where some other heavy tasks running so it could be a lot faster next time.

i concur with my colleagues here: swapping is a b***h that extends SERIOUSLY compile time and please dont forget: qtwebengine is one of the biggest packages in gentoos repository so it takes its time
Top
asturm
Developer
Developer
Posts: 9496
Joined: Thu Apr 05, 2007 4:07 pm

Re: ><)))°€

  • Quote

Post by asturm » Tue Jan 15, 2019 11:42 am

urcindalo wrote:Maybe that's the reason. My fast box has 8Gb RAM. I wonder if 8Gb is the bare minimum for qtwebengine to compile "normally" (as other packages do).
No, it builds fine on my boxes with only 4 GB. But when I do that, I let it build overnight or while doing nothing else on the system.
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

Re: ><)))°€

  • Quote

Post by urcindalo » Tue Jan 15, 2019 11:57 am

asturm wrote:
urcindalo wrote:Maybe that's the reason. My fast box has 8Gb RAM. I wonder if 8Gb is the bare minimum for qtwebengine to compile "normally" (as other packages do).
No, it builds fine on my boxes with only 4 GB. But when I do that, I let it build overnight or while doing nothing else on the system.
This is what I do whenever I see qtwebengine is to be compiled:
1) I reboot the computer
2) I ssh to it from another box
3) I run a screen session to upgrade
4) I quit and, every now and then, I ssh again to look at the progress.

I noticed that, if I don't proceed this way, qtwebengine even fails to compile (emerging is interrupted).

The thing that you can do it overnight with just 4Gb RAM makes me wonder what's the issue in my case.
Top
376212
n00b
n00b
Posts: 37
Joined: Wed Feb 28, 2018 1:43 pm

  • Quote

Post by 376212 » Tue Jan 15, 2019 2:21 pm

you should also check if your cpu or memory is temporary clocked down because longtime compilings like qtwebengine can stress out bad cooling systems in some boxes or notebooks and therefore activate some slowing safety measures inside your online cpus
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

  • Quote

Post by urcindalo » Wed Jan 16, 2019 8:02 am

mael wrote:you should also check if your cpu or memory is temporary clocked down because longtime compilings like qtwebengine can stress out bad cooling systems in some boxes or notebooks and therefore activate some slowing safety measures inside your online cpus
Good catch! It seems quite logical. I'll take a look.
Top
brokendreams
n00b
n00b
Posts: 25
Joined: Sun Nov 11, 2018 1:06 am
Location: clinton,ia US

  • Quote

Post by brokendreams » Wed Jan 16, 2019 5:37 pm

Adding --quiet to the emerge command will speed this up some. As well as adding -w to cflags and cxxflags (disables printing of most warning messages). Shaved 30mins off my build time (Intel 7700hq cpu). Started at 2 hrs...dropped it down to just under 90mins and thats with all use flags enabled. And portage tmpdir on tmpfs.
Top
curmudgeon
Veteran
Veteran
Posts: 1746
Joined: Fri Aug 08, 2003 1:39 pm

  • Quote

Post by curmudgeon » Wed Jan 16, 2019 6:04 pm

I hate having to emerge qtwebengine (and recently spent the day emerging it because of an dev-libs/icu update, and then had to emerge it a couple of days later when a newer version of qtwebengine itself became stable (GRRRRR!).

It takes between five and fifteen hours on my machines, and used to crash more often than not. The biggest problem (which has only appeared in the most recent versions) is that each thread takes up about 2.5 GB of RAM, so if you have lots of cores, but not a corresponding amount of RAM, you will thrash the hard drive by swapping everything at best, or exhaust memory and crash.

I can at least get it to compile by limiting threads to one on machines with 4 GB of RAM, and two on machines with 8 GB of RAM (instead of the usual -j5). That actually seems faster (haven't done any empirical tests), because the processes aren't always stalled waiting for RAM to become available).

The easiest way to do that is:

Code: Select all

# echo dev-qt/qtwebengine makeopts-j1 >> /etc/portage/package.env
# mkdir /etc/portage/env
# echo "MAKEOPTS='-j1'" >> /etc/portage/env/makeopts-j1
Top
mike155
Advocate
Advocate
Posts: 4438
Joined: Fri Sep 17, 2010 11:33 pm
Location: Frankfurt, Germany

  • Quote

Post by mike155 » Wed Jan 16, 2019 6:22 pm

I hate having to emerge qtwebengine
I fully agree. But I found a solution: I changed a few USE Flags and qtwebengine is not getting pulled in anymore.

Really, look at the packages that pull in qtwebengine

Code: Select all

equery d qtwebengine
If you're lucky, you can get rid of qtwebengine by changing a few USE flags. Maybe you will also have to drop a few packages that pull in qtwebengine unconditionally. But it's better to drop a few packages than having to compile qtwebengine!
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

  • Quote

Post by urcindalo » Thu Jan 17, 2019 9:54 am

curmudgeon wrote:I hate having to emerge qtwebengine (and recently spent the day emerging it because of an dev-libs/icu update, and then had to emerge it a couple of days later when a newer version of qtwebengine itself became stable (GRRRRR!).

It takes between five and fifteen hours on my machines, and used to crash more often than not. The biggest problem (which has only appeared in the most recent versions) is that each thread takes up about 2.5 GB of RAM, so if you have lots of cores, but not a corresponding amount of RAM, you will thrash the hard drive by swapping everything at best, or exhaust memory and crash.

I can at least get it to compile by limiting threads to one on machines with 4 GB of RAM, and two on machines with 8 GB of RAM (instead of the usual -j5). That actually seems faster (haven't done any empirical tests), because the processes aren't always stalled waiting for RAM to become available).

The easiest way to do that is:

Code: Select all

# echo dev-qt/qtwebengine makeopts-j1 >> /etc/portage/package.env
# mkdir /etc/portage/env
# echo "MAKEOPTS='-j1'" >> /etc/portage/env/makeopts-j1
You have described my situation to the letter and your suggestion seems to me like the easiest solution. Thanks!!
I just configured my system as per your instructions.
Top
sao98021
Tux's lil' helper
Tux's lil' helper
User avatar
Posts: 145
Joined: Tue Apr 03, 2012 2:59 pm
Location: Michigan

  • Quote

Post by sao98021 » Thu Jan 17, 2019 11:22 am

Code: Select all

     Sat Jun  9 12:13:43 2018 >>> dev-qt/qtwebengine-5.9.4
       merge time: 2 hours, 33 minutes and 39 seconds.

     Sun Sep 23 04:57:08 2018 >>> dev-qt/qtwebengine-5.9.6-r1
       merge time: 2 hours, 50 minutes.

    Fri Jan  4 18:53:01 2019 >>> dev-qt/qtwebengine-5.11.1
       merge time: 2 hours, 30 minutes and 47 seconds.

     Thu Jan 17 02:47:47 2019 >>> dev-qt/qtwebengine-5.11.3
       merge time: 2 hours, 33 minutes and 18 seconds.

Code: Select all

[?] dev-qt/qtwebengine
     Available versions:  (5) 5.9.6-r1(5/5.9)^t 5.11.1(5/5.11)^t ~5.11.2(5/5.11)^t
       {alsa bindist debug designer geolocation +jumbo-build pax_kernel pulseaudio +system-ffmpeg +system-icu test widgets}
     Installed versions:  5.11.3(5/5.11)^t(02:35:03 AM 01/17/2019)(alsa jumbo-build system-ffmpeg system-icu widgets -bindist -debug -designer -geolocation -pax_kernel -pulseaudio -test)
     Homepage:            https://www.qt.io/
     Description:         Library for rendering dynamic web content in Qt5 C++ and QML applications
jumbo-build and 8gb ram, makeopts -j7
Top
urcindalo
l33t
l33t
Posts: 623
Joined: Tue Feb 08, 2005 2:58 pm
Location: Almeria, Spain

  • Quote

Post by urcindalo » Thu Jan 17, 2019 3:06 pm

mike155 wrote:
I hate having to emerge qtwebengine
I fully agree. But I found a solution: I changed a few USE Flags and qtwebengine is not getting pulled in anymore.

Really, look at the packages that pull in qtwebengine

Code: Select all

equery d qtwebengine
If you're lucky, you can get rid of qtwebengine by changing a few USE flags. Maybe you will also have to drop a few packages that pull in qtwebengine unconditionally. But it's better to drop a few packages than having to compile qtwebengine!
Apart from following curmudgeon's directions, just in case, I tried also yours... and it was easier than expected. It turned out the only packages depending on qtwebengine were: knotes -which I don't use-; luminance-hdr -which I can replace with its appimage- and mendeleydesktop, which I can install through flatpak. I also had to remove the webengine USE flag for kdecore-meta and konqueror and the gui USE flag for gpsbabel. No big deal, after all, to get rid of this monster :lol:
Top
dmpogo
Advocate
Advocate
Posts: 3716
Joined: Thu Sep 02, 2004 9:21 pm
Location: Canada

  • Quote

Post by dmpogo » Thu Jan 17, 2019 8:01 pm

mike155 wrote:
I hate having to emerge qtwebengine
I fully agree. But I found a solution: I changed a few USE Flags and qtwebengine is not getting pulled in anymore.

Really, look at the packages that pull in qtwebengine

Code: Select all

equery d qtwebengine
If you're lucky, you can get rid of qtwebengine by changing a few USE flags. Maybe you will also have to drop a few packages that pull in qtwebengine unconditionally. But it's better to drop a few packages than having to compile qtwebengine!
The point is that one probably needs at least one of the huge packages for web browsing - firefox, chromium or qtwebengine (webkit is a lighter option - but is there good browser based on it left ?) However, you perhapds do not need more than one. So make your pick judicially and try to get rid of the rest.
Top
sao98021
Tux's lil' helper
Tux's lil' helper
User avatar
Posts: 145
Joined: Tue Apr 03, 2012 2:59 pm
Location: Michigan

  • Quote

Post by sao98021 » Thu Jan 17, 2019 8:09 pm

dmpogo wrote:
mike155 wrote:
I hate having to emerge qtwebengine
I fully agree. But I found a solution: I changed a few USE Flags and qtwebengine is not getting pulled in anymore.

Really, look at the packages that pull in qtwebengine

Code: Select all

equery d qtwebengine
If you're lucky, you can get rid of qtwebengine by changing a few USE flags. Maybe you will also have to drop a few packages that pull in qtwebengine unconditionally. But it's better to drop a few packages than having to compile qtwebengine!
The point is that one probably needs at least one of the huge packages for web browsing - firefox, chromium or qtwebengine (webkit is a lighter option - but is there good browser based on it left ?) However, you perhapds do not need more than one. So make your pick judicially and try to get rid of the rest.
https://linux.palemoon.org/ forked from firefox, like a faster firefox. its deps https://github.com/gentoo-mirror/palemo ... 3.0.ebuild
Top
376212
n00b
n00b
Posts: 37
Joined: Wed Feb 28, 2018 1:43 pm

  • Quote

Post by 376212 » Fri Jan 18, 2019 3:25 pm

he is like me probably a qt-5/kde-5 user, so migrating to palemoon and other gtk+ based stuff is not what he wants, right? are you using falkon by any chance?
Top
dmpogo
Advocate
Advocate
Posts: 3716
Joined: Thu Sep 02, 2004 9:21 pm
Location: Canada

  • Quote

Post by dmpogo » Fri Jan 18, 2019 5:39 pm

mael wrote:he is like me probably a qt-5/kde-5 user, so migrating to palemoon and other gtk+ based stuff is not what he wants, right? are you using falkon by any chance?
I am using falkon, indeed :) gave up on firefox when it went to 60+ version with rust/clang requirements. Now suffer with qtwebengine recompilations :)
Top
dmpogo
Advocate
Advocate
Posts: 3716
Joined: Thu Sep 02, 2004 9:21 pm
Location: Canada

  • Quote

Post by dmpogo » Fri Jan 18, 2019 11:10 pm

Honestly speaking, it would go long way in aleviating problems with long compilations, if there was some coordination between releases of large packages and their dependencies which cause recompilation. icu is one of the main culprits. All the time there is update to icu, recompilation, and then couple of days later there is update to qtwebengine or libreoffice, and you start all over again. Or in reverse order.
Top
xor_
n00b
n00b
Posts: 10
Joined: Sun Aug 09, 2009 8:51 am

  • Quote

Post by xor_ » Sat Jan 19, 2019 11:01 pm

The point is that one probably needs at least one of the huge packages for web browsing - firefox, chromium or qtwebengine (webkit is a lighter option - but is there good browser based on it left ?) However, you perhapds do not need more than one. So make your pick judicially and try to get rid of the rest.
I've resolved to locally updating the current firefox-binaries outside of portage.

No doubt even qtwekbit has become a monster. It brought my mom's perfectly good i5 t430 with 4GB of RAM to its knees, and it overheats my t440p with 8 GB. (Which got me looking into micro-managing the intel p-states driver).

I've also been trying to eliminate it, but so far my need for teamviewer to remote-administrate my mom keeps me keeping it around.
Top
dmpogo
Advocate
Advocate
Posts: 3716
Joined: Thu Sep 02, 2004 9:21 pm
Location: Canada

  • Quote

Post by dmpogo » Wed Jan 23, 2019 12:20 am

dmpogo wrote:Honestly speaking, it would go long way in aleviating problems with long compilations, if there was some coordination between releases of large packages and their dependencies which cause recompilation. icu is one of the main culprits. All the time there is update to icu, recompilation, and then couple of days later there is update to qtwebengine or libreoffice, and you start all over again. Or in reverse order.
And just few days as I wrote my post, the icu is again updated and qtwebengine and libreoffice need to be recompiled :(
Top
Jaglover
Watchman
Watchman
User avatar
Posts: 8291
Joined: Sun May 29, 2005 1:57 am
Location: Saint Amant, Acadiana

  • Quote

Post by Jaglover » Wed Jan 23, 2019 12:34 am

ccache may be evil, but it helps in such cases.
My Gentoo installation notes.
Please learn how to denote units correctly!
Top
mike155
Advocate
Advocate
Posts: 4438
Joined: Fri Sep 17, 2010 11:33 pm
Location: Frankfurt, Germany

  • Quote

Post by mike155 » Wed Jan 23, 2019 12:45 am

And just few days as I wrote my post, the icu is again updated and qtwebengine and libreoffice need to be recompiled
Come on, mask the new version in /etc/portage/package.mask. Keep the mask until you really want to update qtwebengine or Libreoffice.
Top
Post Reply

89 posts
  • 1
  • 2
  • 3
  • 4
  • Next

Return to “Portage & Programming”

Jump to
  • Assistance
  • ↳   News & Announcements
  • ↳   Frequently Asked Questions
  • ↳   Installing Gentoo
  • ↳   Multimedia
  • ↳   Desktop Environments
  • ↳   Networking & Security
  • ↳   Kernel & Hardware
  • ↳   Portage & Programming
  • ↳   Gamers & Players
  • ↳   Other Things Gentoo
  • ↳   Unsupported Software
  • Discussion & Documentation
  • ↳   Documentation, Tips & Tricks
  • ↳   Gentoo Chat
  • ↳   Gentoo Forums Feedback
  • ↳   Duplicate Threads
  • International Gentoo Users
  • ↳   中文 (Chinese)
  • ↳   Dutch
  • ↳   Finnish
  • ↳   French
  • ↳   Deutsches Forum (German)
  • ↳   Diskussionsforum
  • ↳   Deutsche Dokumentation
  • ↳   Greek
  • ↳   Forum italiano (Italian)
  • ↳   Forum di discussione italiano
  • ↳   Risorse italiane (documentazione e tools)
  • ↳   Polskie forum (Polish)
  • ↳   Instalacja i sprzęt
  • ↳   Polish OTW
  • ↳   Portuguese
  • ↳   Documentação, Ferramentas e Dicas
  • ↳   Russian
  • ↳   Scandinavian
  • ↳   Spanish
  • ↳   Other Languages
  • Architectures & Platforms
  • ↳   Gentoo on ARM
  • ↳   Gentoo on PPC
  • ↳   Gentoo on Sparc
  • ↳   Gentoo on Alternative Architectures
  • ↳   Gentoo on AMD64
  • ↳   Gentoo for Mac OS X (Portage for Mac OS X)
  • Board index
  • All times are UTC
  • Delete cookies

© 2001–2026 Gentoo Foundation, Inc.

Powered by phpBB® Forum Software © phpBB Limited

Privacy Policy

 

 

magic