Why does rust require so much disk space???[SOLVED]

[Budoka]

Getting a little tired of this. First, it required 8GB. Now it has crept up to 10 GBs. Why does RUST require so much and what exactly is using it? I rather get rid of it if it requires that much space to compile.

Running pre-merge checks for dev-lang/rust-1.41.0
* Checking for at least 10 GiB disk space at "/var/tmp/portage/dev-lang/rust-1.41.0/temp" ... [ !! ]
* There is NOT at least 10 GiB disk space at "/var/tmp/portage/dev-lang/rust-1.41.0/temp"
*
* Space constraints set in the ebuild were not met!
* The build will most probably fail, you should enhance the space
* as per failed tests.
*
* ERROR: dev-lang/rust-1.41.0::gentoo failed (pretend phase):
* Build requirements not met!

[C5ace]

Because Rust is a pile of useless, resource zapping wasteful shit.

[Juippisi]

It's explained here,
https://bugs.gentoo.org/709362

There's also rust-bin for those who don't want to compile it.

[Ant P.]

Rust's too special to use mainline LLVM, so it bundles an entire copy of it.

[Ionen]

At least the new system-bootstrap now saves us from also extracting what is essentially rust-bin to build rust every time.

But yeah, the space requirement takes about the worst case scenario into consideration which most people aren't using (like say a debug build with included llvm and all features). It's explained in the aforementioned bug, but there's talk to introduce variable requirements given how out of hand it's getting.

[CaptainBlood]

Rust's too special to use mainline LLVM, so it bundles an entire copy of it.

Really?
Thks 4 ur attention.

[grumblebear]

I have installed rust with USE=system-llvm and it's always working fine for compiling firefox.

[Ionen]

I have installed rust with USE=system-llvm and it's always working fine for compiling firefox.

It did have issues in the past with some rust+llvm versions that gentoo always work to fix given it wants to keep that flag usable, which I too am happy to leave selected.

Although even if you use the flag it'll still extract llvm/clang/lld/etc... taking up 483MB for no reason (this is about half of the entire rustc-1.41.0-src.tar.xz)

[Budoka]

Thanks for the info everyone.

So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.

I'm embarrassed because after all this time I should know this but what am I doing wrong?

[CaptainBlood]

Although even if you use the flag it'll still extract llvm/clang/lld/etc... taking up 483MB for no reason (this is about half of the entire rustc-1.41.0-src.tar.xz)

Is the embedded llvm version known?

Thks 4 ur attention, interest & support.

[Ionen]

So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.

Do you really need rust in your world file? Usually things depend on virtual/rust which should be happy if you got either installed, then you should be able to "emerge -c dev-lang/rust" (they can coexist using eselect), not that I've ever had rust-bin installed so I'm not sure if there's things of note. "emerge --deselect category/package" can also be used to remove things from world file without uninstalling.

I'm likely just stating things you already know though, if still asking to install then not sure what's going on (maybe an overlay depending on dev-lang/rust directly? In which case it should stop you from depcleaning it unless you've been using -C/--unmerge that ignores this)

Is the embedded llvm version known?

Version shown in the source is technically 9.0.0. It uses a "fork" of llvm as its submodule which could be based on non-release versions and/or modified, didn't look deeper into it (1.41.0 is using this specifically). Gentoo is on 9.0.1 with the pre-release 10.0.0 unkeyworded for now, so it's older.

[charles17]

I have installed rust with USE=system-llvm and it's always working fine for compiling firefox.

It did have issues in the past with some rust+llvm versions that gentoo always work to fix given it wants to keep that flag usable, which I too am happy to leave selected.

Maybe it's time to ask for removing the mask?

Code: Select all

$ emerge -pvc virtual/rust

Calculating dependencies... done!
  virtual/rust-1.41.0 pulled in by:
    dev-util/cbindgen-0.12.2 requires >=virtual/rust-1.37.0, =virtual/rust-1.41.0
    mail-client/thunderbird-68.5.0 requires =virtual/rust-1.41.0, >=virtual/rust-1.34.0
    www-client/firefox-68.5.0 requires >=virtual/rust-1.34.0, =virtual/rust-1.41.0

>>> No packages selected for removal by depclean
Packages installed:   900
Packages in world:    98
Packages in system:   43
Required packages:    900
Number to remove:     0

All these are working well with dev-lang/rust-1.41.0[system-llvm]

[Ionen]

Maybe it's time to ask for removing the mask?

I'd agree, both 9.0.1 and 1.41.0 been stabilized already and haven't heard of anything wrong with that combo (at most could make it a <1.41.0 mask if there's doubt for older versions)

[Hu]

Because Rust is a ...

Rust has well known faults, some of which may be to blame for the reported issue here. However, your characterization is unfair (and I say this as someone who is quite frustrated with the Rust community's general antipathy toward Linux distributions) and not constructive. Budoka asked a legitimate question and your post is, at best, confirmation of the reported problem: that Rust requires vast resources. Budoka sought a justification for the requirements, and arguably sought advice on how to avoid dealing with the problem. Your post neither explained its requirements (not even in a critical manner, such as alleging that its requirements could be lower if its developers did X instead of Y), nor aided Budoka in dealing with the burden Rust imposes. Please try to focus more on the question. Ant P.'s response, while openly critical of Rust, is still informative and on-topic by observing that Rust could be more efficient if it didn't bundle a private copy of LLVM.

[Hu]

So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.

I'm embarrassed because after all this time I should know this but what am I doing wrong?

The advice typically offered in this situation is:

• Add --tree to your emerge command to see which package causes the unwanted dependency. When you find that package, inspect its ebuild to see whether you can eliminate the dependency with a USE change.
• If, as in this case, you are very sure you don't want the unwanted dependency installed (as opposed to being merely curious why a seemingly inappropriate package is needed), add the unwanted dependency to /etc/portage/package.mask. This will deny Portage the ability to solve the depgraph in that way. If an alternate solution can be found, it will be used and the unwanted dependency avoided. If no alternate solution can be found, because the unwanted dependency is a hard dependency, then the resulting error message should help you understand why the dependency is sought, and what choices you can make to eliminate it.

[NTU]

Because Rust is a pile of useless, resource zapping wasteful shit.

All I ever see are Rust fanboys talk about how "secure it is, UNDER ALL CIRCUMSTANCES." So, Rust _isn't_ perfect?

[Budoka]

So I can't be bothered with compiling this anymore so emerge(d) rust-bin. However, when I do a world update it still wants to pull in and compile dev-lang/rust-1.41.0. rust-bin is in my World file now.

I'm embarrassed because after all this time I should know this but what am I doing wrong?

The advice typically offered in this situation is:

• Add --tree to your emerge command to see which package causes the unwanted dependency. When you find that package, inspect its ebuild to see whether you can eliminate the dependency with a USE change.
• If, as in this case, you are very sure you don't want the unwanted dependency installed (as opposed to being merely curious why a seemingly inappropriate package is needed), add the unwanted dependency to /etc/portage/package.mask. This will deny Portage the ability to solve the depgraph in that way. If an alternate solution can be found, it will be used and the unwanted dependency avoided. If no alternate solution can be found, because the unwanted dependency is a hard dependency, then the resulting error message should help you understand why the dependency is sought, and what choices you can make to eliminate it.

Thank you very much, Hu and everyone else as well. So after spending some time on this 1) Trying to fully understand the issue and 2) Doing some tweaking to try to address it. it seems that I am stuck. As best I can tell both Firefox and Thunderbird require rust and at version lower than the current one.

Code: Select all

# emerge --depclean -pv --tree virtual/rust dev-lang/rust

Calculating dependencies... done!
  dev-lang/rust-1.37.0 pulled in by:
    virtual/rust-1.37.0 requires =dev-lang/rust-1.37.0*

  virtual/rust-1.37.0 pulled in by:
    dev-util/cbindgen-0.12.2 requires =virtual/rust-1.37.0, >=virtual/rust-1.37.0
    mail-client/thunderbird-68.5.0 requires >=virtual/rust-1.34.0, =virtual/rust-1.37.0
    www-client/firefox-68.5.0 requires =virtual/rust-1.37.0, >=virtual/rust-1.34.0

Code: Select all

# emerge -uvND --quiet-build y @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] dev-lang/rust-1.41.1:stable/1.41::gentoo [1.37.0:stable/1.37::gentoo] USE="-clippy -debug -doc -libressl (-nightly) (-parallel-compiler) -rls -rustfmt (-system-bootstrap) (-system-llvm) -wasm" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="sse2" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV% -Sparc -SystemZ -WebAssembly -XCore" 199,161 KiB
[ebuild     U  ] virtual/rust-1.41.1::gentoo [1.37.0::gentoo] ABI_X86="(64%*) -32% (-x32)" 0 KiB

I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal. Especially since I am on a laptop. Is there any other hack or option that I can use to try to deal with this problem that I am missing? Or am I really at 1) Dump Firefox/Thunderbird (Not an option) 2) Don't ever upgrade rust and hope it doesn't break something down the line? 3)Find a creative way to add more space to root.

There is no way to tell emerge to ignore space requirements?

[ct85711]

Well, another alternate path that is always available; is to just go with the -bin version of firefox and thunderbird; thus getting rid of rust altogether. (note, you may need to move your profile from firefox to firefox-bin, and same for thunderbird; but just simply moving the entire profile folder and telling the -bin versions to use the existing profile isn't difficult so no loss in any settings).

[sebB]

There is no way to tell emerge to ignore space requirements?

It depend on how many space you have in your tmp.
For me rust need 6.2GB space.
To ignore the warning I use

Code: Select all

I_KNOW_WHAT_I_AM_DOING="yes" emerge -1 rust

Another solution is to mount your tmp in TMPFS when compiling rust

[molletts]

I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal...

Do you have 10GB free on another filesystem, either on the laptop's HDD/SSD or on an external drive? If so, you could point PORTAGE_TMPDIR at it instead of having the build take place on your root FS.

I use settings in /etc/portage/env/ and /etc/portage/package.env/ to build big packages like Rust in temporary storage on a hard drive instead of in a RAM-based tmpfs.

[Budoka]

There is no way to tell emerge to ignore space requirements?

It depend on how many space you have in your tmp.
For me rust need 6.2GB space.
To ignore the warning I use

Code: Select all

I_KNOW_WHAT_I_AM_DOING="yes" emerge -1 rust

Another solution is to mount your tmp in TMPFS when compiling rust

Ignoring the warning got the job done! Thanks. But I do realize the danger in doing so, so in the future, I will mount my tmp.

Thanks

[Budoka]

I just have no way to make 10GB available to rust to compile it. I am running LUKS/LVM so I guess in theory I could extend my root(?) but this probably not ideal...

Do you have 10GB free on another filesystem, either on the laptop's HDD/SSD or on an external drive? If so, you could point PORTAGE_TMPDIR at it instead of having the build take place on your root FS.

I use settings in /etc/portage/env/ and /etc/portage/package.env/ to build big packages like Rust in temporary storage on a hard drive instead of in a RAM-based tmpfs.

I just had a "duh" moment and smacked myself in the head. Another example of me getting stuck in the proverbial weeds. Of course, this would be the ideal solution for the issue I was having. Thanks all.

[Ionen]

Building rust on tmpfs got a bit easier with the parallel-compiler flag (experimental, so masked in stable), assuming got enough space on tmpfs itself but then not enough to deal with multiple rustc processes that can sometime use 4GB ram each during compilation.

While -jX cause multiple rustc processes to spawn, -Zthreads=X (RUSTFLAGS) will make a single rustc use X threads when possible (with roughly same ram usage). Makes -j1-2 more viable, although still better to have it a bit higher.

Sadly, I haven't got that to work while building firefox (rust seems to stick to a single thread despite -Zthreads= being present, didn't investigate further), works great when building rust itself though (Edit: will need system-bootstrap USE to use your own parallel-compiler with future builds though, I don't think the pre-built rust used to bootstrap it supports this).

[tld]

Well, another alternate path that is always available; is to just go with the -bin version of firefox and thunderbird; thus getting rid of rust altogether. (note, you may need to move your profile from firefox to firefox-bin, and same for thunderbird; but just simply moving the entire profile folder and telling the -bin versions to use the existing profile isn't difficult so no loss in any settings).

Currently I've replaced Firefox with palemoon from the palemoon overlay (https://github.com/deu/palemoon-overlay) and that's been great...no rust of other BS involved. As far as Thunderbird, at some point I want to look into replacing that with Claws mail of the ilke. For now I'm running an old pre-rust version out of my local overlay (52.9.1). Mozilla can keep all their bloated BS frankly.