qtwebengine build error (SOLVED)

[andrewwalker27]

I'm having trouble upgrading qtwebengine to qtwebengine-5.12.1 as the build fails an hour and a half into the building. The last few lines are as follows

Code: Select all

[20771/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/shared_worker_connector_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_S$
[20772/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/ssl_private_key_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROW$
[20773/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/ssl_client_auth_handler.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_B$
[20774/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/shared_worker_host.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSI$
[20775/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/startup_task_runner.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWS$
[20776/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/stream.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BR$
[20777/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/stream_context.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -$
[20778/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/stream_handle_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSI$
[20779/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/stream_registry.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING $
[20780/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/stream_url_request_job.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BR$
[20781/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/background_memory_tracing_observer.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -D$
[20782/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/shared_worker_service_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAF$
FAILED: obj/content/browser/browser/shared_worker_service_impl.o
/usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/shared_worker_service_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DS$
x86_64-pc-linux-gnu-g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[20783/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/background_tracing_config_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL$
[20784/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/site_instance_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSI$
[20785/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/ssl_error_handler.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSIN$
[20786/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/ssl_manager.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSA$
[20787/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/storage_partition_impl_map.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAF$
[20788/22843] /usr/bin/x86_64-pc-linux-gnu-g++ -MMD -MF obj/content/browser/browser/storage_partition_impl.o.d -DENABLE_SCREEN_CAPTURE=1 -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BR$
ninja: build stopped: subcommand failed.
make[3]: *** [Makefile.gn_run:595: run_ninja] Error 1
make[3]: Leaving directory '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/work/qtwebengine-everywhere-src-5.12.1/src/core'
make[2]: *** [Makefile:82: sub-gn_run-pro-make_first] Error 2
make[2]: Leaving directory '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/work/qtwebengine-everywhere-src-5.12.1/src/core'
make[1]: *** [Makefile:81: sub-core-make_first] Error 2
make[1]: Leaving directory '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/work/qtwebengine-everywhere-src-5.12.1/src'
make: *** [Makefile:49: sub-src-make_first] Error 2
 ^[[31;01m*^[[0m ERROR: dev-qt/qtwebengine-5.12.1::gentoo failed (compile phase):
 ^[[31;01m*^[[0m   emake failed
 ^[[31;01m*^[[0m
 ^[[31;01m*^[[0m If you need support, post the output of `emerge --info '=dev-qt/qtwebengine-5.12.1::gentoo'`,
 ^[[31;01m*^[[0m the complete build log and the output of `emerge -pqv '=dev-qt/qtwebengine-5.12.1::gentoo'`.
 ^[[31;01m*^[[0m The complete build log is located at '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/temp/build.log'.
 ^[[31;01m*^[[0m The ebuild environment file is located at '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/temp/environment'.
 ^[[31;01m*^[[0m Working directory: '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/work/qtwebengine-everywhere-src-5.12.1'
 ^[[31;01m*^[[0m S: '/var/tmp/portage/dev-qt/qtwebengine-5.12.1/work/qtwebengine-everywhere-src-5.12.1'

There is a bug listed https://bugs.gentoo.org/678332 that says it may be due to an older installed version but I have uninstalled it and the problem still occurs. Has anyone got any suggestions before I start another two hour attempt to install it?
Is this likely a gcc internal error or have ninjas sneaked in and murdered it? Is it worth changing my gcc version?

[fedeliallalinea]

Code: Select all

x86_64-pc-linux-gnu-g++: fatal error: Killed signal terminated program cc1plus

This is an out of memory error, try to disable jumbo-build or lower MAKEOPTS value (or both)

[andrewwalker27]

Thanks, I'll try with -jumbo-build use flag. Just wondering how you can tell it's a low memory issue though? I don't see anything specifically stating that and shouldn't my swap partition sort that out?
I'm also curious about what the jumbo-build flag does, the only info I could find is

Combine source files to speed up build process.

[fedeliallalinea]

Thanks, I'll try with -jumbo-build use flag. Just wondering how you can tell it's a low memory issue though? I don't see anything specifically stating that and shouldn't my swap partition sort that out?

Usaually "Killed signal terminated program cc1plus" means a out of memory error. To be sure, you can also check dmesg

I'm also curious about what the jumbo-build flag does, the only info I could find is

https://chromium.googlesource.com/chrom ... s/jumbo.md

[NeddySeagoon]

andrewwalker27,

Swap cannot be used for everything. There are degrees of swapping too.

1. The kernel can drop read buffers. There is nothing to do before hand. The data will be reread if its needed again.
2. The kernel can flush 'dirty' buffers to permanent storage. This was RAM waiting to be written and the write must complete before the content can be dropped.
3. The kernel can drop code pages. As code is mmaped to RAM, next time its needed, you get a page fault and the code is reloaded.
4. The kernel can move some dynamically allocated RAM to swap. Its only dynamically allocated RAM that is ever written to swap.

Swap does not make up for the lack of real RAM. Notice that not having any swap does not prevent swapping. It only robs the kernel of the option to page dynamically allocated RAM.

You mentioned arm7a. That gives you a 4GB address space. Big packages on small machines often run out of RAM.
USE=jumbo-build makes things worse. Parallel make adds to the pressure on RAM.

[andrewwalker27]

Well I've finally succeeded with the build but it appears the problem was not necessarily the jumbo-build use flag. I tried -jumbo-build with the same result but what actually cured the problem was MAKEOPTS="-j2" and -jumbo-build.
I should mention my machine is a six core PhenomII and the problem is, as you suggested, lack of ram but in case anyone else has similar issues, I had 8 Gig of RAM onboard and it still wasn't enough!
This doesn't bode well for lower spec machines, I assumed mine was reasonably powerful, just though I'd leave this info here if anyone else is struggling.
Thanks for all the help, I've learnt quite a bit in the process, time to buy some more RAM I think!

[asturm]

8GB RAM is quite meagre for a 6-core machine. I can build qtwebengine fine with a rather old 3-core/4GB RAM, although it takes a while.

[NeddySeagoon]

andrewwalker27,

C++ can want 2G RAM per parallel make.
A PhenomII is unlikely to make a wise investment choice. Mine is 10 years old and I maxed out the RAM at 16G some years ago because my son egged me on so he could have the 8G it was originally fitted with :)

There is another solution.
You can set per package environment variables, so you don't have to remember.
Builds won't be any faster but they won't break either.

[AstroFloyd]

Interestingly, I came to this page with this problem for the same package. I have two cores/four hyperthreads and 8Gb RAM, and use MAKEOPTS="-j4", which seems to much for dev-qt/qtwebengine. However, if the issue is the number of parallel make threads vs. RAM, why does this issue mostly (only?) seem to occur for qtwebengine? I would expect the same problem to arise for many (large) packages, since they are all compiled on the same system with the same settings (though I'll experiment with a different -j for qtwebengine now).

BTW, my build crashed after ~4h (out of an expected ~10.5h). I found that I can restart the merge with -j3 after replacing -j4 with -j3 in /var/tmp/portage/portage/dev-qt/qtwebengine-<version>/temp/environment and running ebuild /usr/portage/dev-qt/qtwebengine/qtwebengine-5.15.2_p20210625.ebuild merge, which may save me a couple of hours of waiting for my laptop to become usable again.

[pietinger]

[...] However, if the issue is the number of parallel make threads vs. RAM, why does this issue [...] occur [...] ?

Dont forget it makes a difference if one of your compile-jobs has to deal with a 200-lines source-file or a bigger one ...

The most packages we have dont use more than 0,4 - 0,5 GB RAM per thread. If you have 8 cores and 8 GB RAM you can use for these -j8 ... "rust" needs up to 4 GB RAM for one thread, so even -j3 will be too much for 8 GB RAM.

I would expect the same problem to arise for many (large) packages, since they are all compiled on the same system with the same settings (though I'll experiment with a different -j for qtwebengine now).

You will get also problems with: rust, libreoffice, sometimes with gcc and the most times with other browser-monsters.