| View previous topic :: View next topic |
| Author |
Message |
Budoka
l33t
Joined: 03 Jun 2012
Posts: 777
Location: Tokyo, Japan
|
 Posted: Sun Feb 16, 2020 5:52 am Post subject: Why does rust require so much disk space???[SOLVED] |
 |
|
Getting a little tired of this. First, it required 8GB. Now it has crept up to 10 GBs. Why does RUST require so much and what exactly is using it? I rather get rid of it if it requires that much space to compile.
| Quote: | Running pre-merge checks for dev-lang/rust-1.41.0
* Checking for at least 10 GiB disk space at "/var/tmp/portage/dev-lang/rust-1.41.0/temp" ... [ !! ]
* There is NOT at least 10 GiB disk space at "/var/tmp/portage/dev-lang/rust-1.41.0/temp"
*
* Space constraints set in the ebuild were not met!
* The build will most probably fail, you should enhance the space
* as per failed tests.
*
* ERROR: dev-lang/rust-1.41.0::gentoo failed (pretend phase):
* Build requirements not met!
|
Last edited by Budoka on Thu Mar 05, 2020 11:43 am; edited 1 time in total |
|
| Back to top |
|
 |
C5ace
Guru
Joined: 23 Dec 2013
Posts: 493
Location: Brisbane, Australia
|
| Posted: Sun Feb 16, 2020 6:15 am Post subject: |
|
|
Because Rust is a pile of useless, resource zapping wasteful shit.
_________________
Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes. |
|
| Back to top |
|
|
Juippisi
Developer
Joined: 30 Sep 2005
Posts: 762
Location: /home
|
| Posted: Sun Feb 16, 2020 6:19 am Post subject: |
|
|
It's explained here,
https://bugs.gentoo.org/709362
There's also rust-bin for those who don't want to compile it. |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Sun Feb 16, 2020 6:39 pm Post subject: |
|
|
| Rust's too special to use mainline LLVM, so it bundles an entire copy of it. |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2918
|
| Posted: Sun Feb 16, 2020 7:03 pm Post subject: |
|
|
At least the new system-bootstrap now saves us from also extracting what is essentially rust-bin to build rust every time.
But yeah, the space requirement takes about the worst case scenario into consideration which most people aren't using (like say a debug build with included llvm and all features). It's explained in the aforementioned bug, but there's talk to introduce variable requirements given how out of hand it's getting. |
|
| Back to top |
|
|
CaptainBlood
Advocate
Joined: 24 Jan 2010
Posts: 4004
|
| Posted: Sun Feb 16, 2020 7:12 pm Post subject: |
|
|
| Ant P. wrote: | | Rust's too special to use mainline LLVM, so it bundles an entire copy of it. |
Really? 
Thks 4 ur attention. |
|
| Back to top |
|
|
grumblebear
Apprentice
Joined: 26 Feb 2008
Posts: 204
|
| Posted: Sun Feb 16, 2020 8:48 pm Post subject: |
|
|
| I have installed rust with USE=system-llvm and it's always working fine for compiling firefox. |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2918
|
| Posted: Sun Feb 16, 2020 8:58 pm Post subject: |
|
|
| grumblebear wrote: | | I have installed rust with USE=system-llvm and it's always working fine for compiling firefox. |
It did have issues in the past with some rust+llvm versions that gentoo always work to fix given it wants to keep that flag usable, which I too am happy to leave selected.
Although even if you use the flag it'll still extract llvm/clang/lld/etc... taking up 483MB for no reason  (this is about half of the entire rustc-1.41.0-src.tar.xz) |
|
| Back to top |
|
|
Budoka
l33t
Joined: 03 Jun 2012
Posts: 777
Location: Tokyo, Japan
|
| Posted: Mon Feb 17, 2020 5:03 am Post subject: |
|
|
Thanks for the info everyone.
So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.
I'm embarrassed because after all this time I should know this but what am I doing wrong? |
|
| Back to top |
|
|
CaptainBlood
Advocate
Joined: 24 Jan 2010
Posts: 4004
|
| Posted: Mon Feb 17, 2020 8:00 am Post subject: |
|
|
| Ionen wrote: | | Although even if you use the flag it'll still extract llvm/clang/lld/etc... taking up 483MB for no reason (this is about half of the entire rustc-1.41.0-src.tar.xz) |
Is the embedded llvm version known?
Thks 4 ur attention, interest & support. |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2918
|
| Posted: Mon Feb 17, 2020 8:52 am Post subject: |
|
|
| Budoka wrote: | | So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now. |
Do you really need rust in your world file? Usually things depend on virtual/rust which should be happy if you got either installed, then you should be able to "emerge -c dev-lang/rust" (they can coexist using eselect), not that I've ever had rust-bin installed so I'm not sure if there's things of note. "emerge --deselect category/package" can also be used to remove things from world file without uninstalling.
I'm likely just stating things you already know though, if still asking to install then not sure what's going on (maybe an overlay depending on dev-lang/rust directly? In which case it should stop you from depcleaning it unless you've been using -C/--unmerge that ignores this)
| CaptainBlood wrote: | | Is the embedded llvm version known? |
Version shown in the source is technically 9.0.0. It uses a "fork" of llvm as its submodule which could be based on non-release versions and/or modified, didn't look deeper into it (1.41.0 is using this specifically). Gentoo is on 9.0.1 with the pre-release 10.0.0 unkeyworded for now, so it's older. |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3686
|
| Posted: Mon Feb 17, 2020 10:32 am Post subject: |
|
|
| Ionen wrote: | | grumblebear wrote: | | I have installed rust with USE=system-llvm and it's always working fine for compiling firefox. |
It did have issues in the past with some rust+llvm versions that gentoo always work to fix given it wants to keep that flag usable, which I too am happy to leave selected. |
Maybe it's time to ask for removing the mask?
| Code: | $ emerge -pvc virtual/rust
Calculating dependencies... done!
virtual/rust-1.41.0 pulled in by:
dev-util/cbindgen-0.12.2 requires >=virtual/rust-1.37.0, =virtual/rust-1.41.0
mail-client/thunderbird-68.5.0 requires =virtual/rust-1.41.0, >=virtual/rust-1.34.0
www-client/firefox-68.5.0 requires >=virtual/rust-1.34.0, =virtual/rust-1.41.0
>>> No packages selected for removal by depclean
Packages installed: 900
Packages in world: 98
Packages in system: 43
Required packages: 900
Number to remove: 0 |
All these are working well with dev-lang/rust-1.41.0[system-llvm] |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2918
|
| Posted: Mon Feb 17, 2020 10:55 am Post subject: |
|
|
| charles17 wrote: | | Maybe it's time to ask for removing the mask? |
I'd agree, both 9.0.1 and 1.41.0 been stabilized already and haven't heard of anything wrong with that combo (at most could make it a <1.41.0 mask if there's doubt for older versions) |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23356
|
| Posted: Tue Feb 18, 2020 2:49 am Post subject: |
|
|
| C5ace wrote: | | Because Rust is a ... |
Rust has well known faults, some of which may be to blame for the reported issue here. However, your characterization is unfair (and I say this as someone who is quite frustrated with the Rust community's general antipathy toward Linux distributions) and not constructive. Budoka asked a legitimate question and your post is, at best, confirmation of the reported problem: that Rust requires vast resources. Budoka sought a justification for the requirements, and arguably sought advice on how to avoid dealing with the problem. Your post neither explained its requirements (not even in a critical manner, such as alleging that its requirements could be lower if its developers did X instead of Y), nor aided Budoka in dealing with the burden Rust imposes. Please try to focus more on the question. Ant P.'s response, while openly critical of Rust, is still informative and on-topic by observing that Rust could be more efficient if it didn't bundle a private copy of LLVM. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23356
|
| Posted: Tue Feb 18, 2020 2:56 am Post subject: |
|
|
| Budoka wrote: | So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.
I'm embarrassed because after all this time I should know this but what am I doing wrong? |
The advice typically offered in this situation is:- Add --tree to your emerge command to see which package causes the unwanted dependency. When you find that package, inspect its ebuild to see whether you can eliminate the dependency with a USE change.
- If, as in this case, you are very sure you don't want the unwanted dependency installed (as opposed to being merely curious why a seemingly inappropriate package is needed), add the unwanted dependency to /etc/portage/package.mask. This will deny Portage the ability to solve the depgraph in that way. If an alternate solution can be found, it will be used and the unwanted dependency avoided. If no alternate solution can be found, because the unwanted dependency is a hard dependency, then the resulting error message should help you understand why the dependency is sought, and what choices you can make to eliminate it.
|
|
| Back to top |
|
|
NTU
Apprentice
Joined: 17 Jul 2015
Posts: 187
|
| Posted: Fri Feb 21, 2020 9:30 am Post subject: |
|
|
| C5ace wrote: | | Because Rust is a pile of useless, resource zapping wasteful shit. |
All I ever see are Rust fanboys talk about how "secure it is, UNDER ALL CIRCUMSTANCES." So, Rust _isn't_ perfect?  |
|
| Back to top |
|
|
Budoka
l33t
Joined: 03 Jun 2012
Posts: 777
Location: Tokyo, Japan
|
| Posted: Wed Mar 04, 2020 8:48 am Post subject: |
|
|
| Hu wrote: | | Budoka wrote: | So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.
I'm embarrassed because after all this time I should know this but what am I doing wrong? |
The advice typically offered in this situation is:- Add --tree to your emerge command to see which package causes the unwanted dependency. When you find that package, inspect its ebuild to see whether you can eliminate the dependency with a USE change.
- If, as in this case, you are very sure you don't want the unwanted dependency installed (as opposed to being merely curious why a seemingly inappropriate package is needed), add the unwanted dependency to /etc/portage/package.mask. This will deny Portage the ability to solve the depgraph in that way. If an alternate solution can be found, it will be used and the unwanted dependency avoided. If no alternate solution can be found, because the unwanted dependency is a hard dependency, then the resulting error message should help you understand why the dependency is sought, and what choices you can make to eliminate it.
|
Thank you very much, Hu and everyone else as well. So after spending some time on this 1) Trying to fully understand the issue and 2) Doing some tweaking to try to address it. it seems that I am stuck. As best I can tell both Firefox and Thunderbird require rust and at version lower than the current one.
| Code: | # emerge --depclean -pv --tree virtual/rust dev-lang/rust
Calculating dependencies... done!
dev-lang/rust-1.37.0 pulled in by:
virtual/rust-1.37.0 requires =dev-lang/rust-1.37.0*
virtual/rust-1.37.0 pulled in by:
dev-util/cbindgen-0.12.2 requires =virtual/rust-1.37.0, >=virtual/rust-1.37.0
mail-client/thunderbird-68.5.0 requires >=virtual/rust-1.34.0, =virtual/rust-1.37.0
www-client/firefox-68.5.0 requires =virtual/rust-1.37.0, >=virtual/rust-1.34.0
|
| Code: | # emerge -uvND --quiet-build y @world
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild U ] dev-lang/rust-1.41.1:stable/1.41::gentoo [1.37.0:stable/1.37::gentoo] USE="-clippy -debug -doc -libressl (-nightly) (-parallel-compiler) -rls -rustfmt (-system-bootstrap) (-system-llvm) -wasm" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="sse2" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV% -Sparc -SystemZ -WebAssembly -XCore" 199,161 KiB
[ebuild U ] virtual/rust-1.41.1::gentoo [1.37.0::gentoo] ABI_X86="(64%*) -32% (-x32)" 0 KiB
|
I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal. Especially since I am on a laptop. Is there any other hack or option that I can use to try to deal with this problem that I am missing? Or am I really at 1) Dump Firefox/Thunderbird (Not an option) 2) Don't ever upgrade rust and hope it doesn't break something down the line? 3)Find a creative way to add more space to root.
There is no way to tell emerge to ignore space requirements? |
|
| Back to top |
|
|
ct85711
Veteran
Joined: 27 Sep 2005
Posts: 1791
|
| Posted: Wed Mar 04, 2020 3:53 pm Post subject: |
|
|
| Well, another alternate path that is always available; is to just go with the -bin version of firefox and thunderbird; thus getting rid of rust altogether. (note, you may need to move your profile from firefox to firefox-bin, and same for thunderbird; but just simply moving the entire profile folder and telling the -bin versions to use the existing profile isn't difficult so no loss in any settings). |
|
| Back to top |
|
|
sebB
l33t
Joined: 02 Mar 2011
Posts: 806
Location: S.O. France
|
| Posted: Wed Mar 04, 2020 6:01 pm Post subject: |
|
|
| Quote: | | There is no way to tell emerge to ignore space requirements? |
It depend on how many space you have in your tmp.
For me rust need 6.2GB space.
To ignore the warning I use
| Code: | | I_KNOW_WHAT_I_AM_DOING="yes" emerge -1 rust |
Another solution is to mount your tmp in TMPFS when compiling rust |
|
| Back to top |
|
|
molletts
Tux's lil' helper
Joined: 16 Feb 2013
Posts: 133
|
| Posted: Wed Mar 04, 2020 6:30 pm Post subject: |
|
|
| Budoka wrote: | | I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal... |
Do you have 10GB free on another filesystem, either on the laptop's HDD/SSD or on an external drive? If so, you could point PORTAGE_TMPDIR at it instead of having the build take place on your root FS.
I use settings in /etc/portage/env/ and /etc/portage/package.env/ to build big packages like Rust in temporary storage on a hard drive instead of in a RAM-based tmpfs. |
|
| Back to top |
|
|
Budoka
l33t
Joined: 03 Jun 2012
Posts: 777
Location: Tokyo, Japan
|
| Posted: Thu Mar 05, 2020 11:40 am Post subject: |
|
|
| sebB wrote: | | Quote: | | There is no way to tell emerge to ignore space requirements? |
It depend on how many space you have in your tmp.
For me rust need 6.2GB space.
To ignore the warning I use
| Code: | | I_KNOW_WHAT_I_AM_DOING="yes" emerge -1 rust |
Another solution is to mount your tmp in TMPFS when compiling rust |
Ignoring the warning got the job done! Thanks. But I do realize the danger in doing so, so in the future, I will mount my tmp.
Thanks |
|
| Back to top |
|
|
Budoka
l33t
Joined: 03 Jun 2012
Posts: 777
Location: Tokyo, Japan
|
| Posted: Thu Mar 05, 2020 11:43 am Post subject: |
|
|
| molletts wrote: | | Budoka wrote: | | I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal... |
Do you have 10GB free on another filesystem, either on the laptop's HDD/SSD or on an external drive? If so, you could point PORTAGE_TMPDIR at it instead of having the build take place on your root FS.
I use settings in /etc/portage/env/ and /etc/portage/package.env/ to build big packages like Rust in temporary storage on a hard drive instead of in a RAM-based tmpfs. |
I just had a "duh" moment and smacked myself in the head. Another example of me getting stuck in the proverbial weeds. Of course, this would be the ideal solution for the issue I was having. Thanks all. |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2918
|
| Posted: Thu Mar 05, 2020 1:01 pm Post subject: |
|
|
Building rust on tmpfs got a bit easier with the parallel-compiler flag (experimental, so masked in stable), assuming got enough space on tmpfs itself but then not enough to deal with multiple rustc processes that can sometime use 4GB ram each during compilation.
While -jX cause multiple rustc processes to spawn, -Zthreads=X (RUSTFLAGS) will make a single rustc use X threads when possible (with roughly same ram usage). Makes -j1-2 more viable, although still better to have it a bit higher.
Sadly, I haven't got that to work while building firefox (rust seems to stick to a single thread despite -Zthreads= being present, didn't investigate further), works great when building rust itself though (Edit: will need system-bootstrap USE to use your own parallel-compiler with future builds though, I don't think the pre-built rust used to bootstrap it supports this). |
|
| Back to top |
|
|
tld
Veteran
Joined: 09 Dec 2003
Posts: 1859
|
| Posted: Thu Mar 05, 2020 1:57 pm Post subject: |
|
|
| ct85711 wrote: | | Well, another alternate path that is always available; is to just go with the -bin version of firefox and thunderbird; thus getting rid of rust altogether. (note, you may need to move your profile from firefox to firefox-bin, and same for thunderbird; but just simply moving the entire profile folder and telling the -bin versions to use the existing profile isn't difficult so no loss in any settings). |
Currently I've replaced Firefox with palemoon from the palemoon overlay (https://github.com/deu/palemoon-overlay) and that's been great...no rust of other BS involved. As far as Thunderbird, at some point I want to look into replacing that with Claws mail of the ilke. For now I'm running an old pre-rust version out of my local overlay (52.9.1). Mozilla can keep all their bloated BS frankly. |
|
| Back to top |
|
|
|
Portage & Programming
