High RAM usage with shells on musl

[cntrmx]

Hello,

When I set /bin/bash ( using chsh ) as my default shell, after a reboot I get 77 MB of RAM usage which is way too high for my system.
On the other hand when I use /bin/ash ( busybox shell ) it gets me to the expected 13 MB of RAM.

I'm having trouble believing that bash ( including ksh and statically compiled dash ) would consume more than 60 MB of RAM and need help finding the issue.

It seems those shells are triggering something that causes my RAM to go up.

Please ask for any info about system or files that you might need.

Thank you.

[RumpletonBongworth]

Firstly, it's stated in the BUGS section of the bash(1) manual that "it's too big and too slow."

Secondly, interactive instances of bash always source /etc/bash/bashrc. Given an unmodified Gentoo system, it will source files in turn that increase the environmental footprint, though probably not by that much. For example, the declaration of the LS_COLORS environment variable entails storing about 2 KiB of extra data (how that's arranged under the hood being another matter).

Thirdly, you could try building with the (confusingly named) "mem-scramble" USE flag, which will result in bash using its own, bundled, malloc implementation. Should it make any appreciable difference, I would be interested to know about it.

EDIT: Also, make sure that the history files of both shells are empty upon launch, so that the in-memory history buffers are initially unpopulated. Otherwise, you might not be conducting a fair comparison.

[cntrmx]

How do I recompile with mem-scramble? Adding to package.use doesn't work, I suppose that's because it is in parenthesis?

Code: Select all

app-shells/bash-5.2_p37::gentoo  USE="(readline) -afs -bashlogger -examples (-mem-scramble) -net -nls -pgo -plugins -verify-sig"

EDIT: Could it be masked because I am using 32 bit musl?

[RumpletonBongworth]

How do I recompile with mem-scramble?

Ah. You are either using musl libc or the HPPA architecture. In that case, though it is possible to unmask the USE flag, I would hastily retract my suggestion to experiment with the internal malloc. It would seem that the flag is masked for good reason; see the applicable comments in both profiles/arch/hppa/package.use.mask and profiles/features/musl/package.use.mask, relative to your gentoo repo directory.

[cntrmx]

Yeah, it's not worth messing with that

Code: Select all

# bash-malloc relies on sbrk which is implemented
# as a fail-only stub in musl. breaks horribly if enabled.

[RumpletonBongworth]

It might also be worth trying app-shells/loksh, which is a direct port of the unified sh/ksh implementation from OpenBSD. It is quite comfortable to use as an interactive shell and has a good feature set, including arrays. It would be interesting to see how that stacks up against busybox ash.

[cntrmx]

Very interesting, I tried installing the shell you mentioned and setting it.

The result is the same as with bash. It seems something is being triggered that causes RAM to increase.

EDIT: It is possible that the reason is that my busybox is statically linked.

EDIT: Just compiled the dash shell with static USE Flag but the result is the same as with bash or ksh.

[cntrmx]

So, I created a new user and assigned a bash shell, and when I su into the user the RAM jumps.

It is indeed something related to bash sourcing I think.

[RumpletonBongworth]

If you think it has something to do with the initialisation process, compare against sudo bash --norc. Why sudo? Because, invoked that way, it will execute bash directly, rather than invoke an interim shell for the purpose of parsing the given command (which can't be accomplished with su). As for the --norc option, it will suppress the sourcing of /etc/bash/bashrc and ~/.bashrc. The other files are not a concern because the aforementioned command will not invoke bash as a login shell. You might also compare against HISTFILE=/dev/null sudo -E bash --norc.

[cntrmx]

Well, I tried running sudo bash --norc and HISTFILE=/dev/null sudo -E bash --norc, but, alas, the RAM still jumps.

[cntrmx]

I also tried running env -i with bash, but seems that environment variables are not the cause.