Primacy of /etc/bashrc.d vs /etc/profile.d

[jesnow]

Hi folks.

My system has worked just fine for a long long time, but its behavior just changed. Sure I can "just fix it" but it seems like there might be an unintended effect of something that changed in gentoo.

I am used to setting system aliases that affect every interactive shell in the system. Things like "alias dir = ls -al". These go into a drop-in file in /etc/profile.d/ since it's something I changed for my system.

But now since my most recent update, the dropins in /etc/profile.d are run but then overwritten with other aliases sourced in the directory /etc/bashrc.d/

So, I question the wisdom of turning every single config file into a directory. But there's nothing I can do about it, that's just modern life. I guess it's better than the windows registry or whatever voodoo they do in macos. At least these are human readable files. But I really miss easily remembered and consistent text files that configure your system.

But so I should be able to "source /etc/profile" and reasonably expect that the aliases in the dropin directory /etc/profile.d/ are executed and not stepped on. But that's not what happens.

Instead, the alias commands in /etc/bashrc.d/ are executed, as if /etc/profile.d/ didn't exist.

So the result is not at all what I want:

Code: Select all

bartali /home/jesnow # alias
alias df='df -h'
alias diff='diff --color=auto'
alias dir='dir --color=auto'
alias du='du -sh'
alias grep='grep --color=auto'
alias ls='ls --color=auto'
alias vdir='vdir --color=auto'

Where in /etc/profile.d/aliases.sh I had written

Code: Select all

#  /etc/profile.d/aliases.sh system aliases

echo "/etc/profile.d/aliases.sh Aliases sourced"
alias dir='ls -alh'
alias du='du -sh'
alias df='df -h'

Notice that I added an echo statement so I can see what's going on. I did that for /etc/bashrc.d/10-gentoo-color.bash too. The end state is that /etc/profile.d is run, but overridden by /etc/bashrc.d/10-gentoo-color.bash.

So now when I "source /etc/profile", I get

Code: Select all

bartali /home/jesnow # source /etc/profile
/etc/profile.d/aliases.sh Aliases sourced
/etc/bash/bashrc.d/10-Gentoo-Color.bash aliases sourced

Which is kinda what I want and expect, but the profile.d aliases are overridden.

What is confusing me is that /etc/bash/bashrc.d/10-gentoo-color.bash seems to be going out if its way *not* to do this:

Code: Select all

if (( gentoo_color > 0 )); then
        # Colorize the output of diff(1), grep(1) and a few coreutils utilities.
        for _ in diff dir grep ls vdir; do
                alias "$_=$_ --color=auto"
        done
.
.
.
fi

Instead of overriding my aliases it really should be just adding "--color=auto" to them. This is what I want, but it's not happening. Also confusingly, when I su to root

Code: Select all

jesnow@bartali ~ $ su
Password: 
/etc/bash/bashrc.d/10-Gentoo-Color.bash aliases sourced
/etc/profile.d/aliases.sh Aliases sourced
/etc/bash/bashrc.d/10-Gentoo-Color.bash aliases sourced
bartali /home/jesnow # 

/etc/bash/bashrc.d/* are sourced *twice!*.
This is really not what should be happening.
And when I open a new shell:

Code: Select all

/etc/bash/bashrc.d/10-Gentoo-Color.bash aliases sourced
jesnow@bartali ~ $ 

/etc/profile.d/* aren't sourced at all.

There is an ancient bug where this is all discussed:
https://bugs.gentoo.org/4854

So what went wrong with the basic bash startup scripts? I've had it working the way I want for at least a decade, and something changed.


Cheers,
Jon.

[jesnow]

To be clear: I know how to "git 'er done" as they say in Texas, but I would like the result to be correct. I'm clearly missing something.

[pjp]

When did you notice the change?

I thought there was a related news item, but I'm not seeing it. Maybe it could be related to a change intorduced by RumpletonBongworth where some issues are discussed in this post (and in other posts in that thread and possibly in other threads):

viewtopic-p-8830535.html#8830535

Of course it might not be related to that.

The part that tries to keep the old alias an only appending color seems like a major oops. By the declaration in the loop, _ is only the command, not what it may have been aliased to.

[jesnow]

Thanks!

I only noticed it yesterday, and the files in /etc/bashrc.d/ all have yesterday's filedate. So it sounds like it's related as you say to the introduction of bashrc scripts to include the no-readline functionality, I have no idea about that. I am running a very trailing-edge system in order not to have this exact kind of issue.

I'm going to let my emerge -DNua @world finish before I think about it any more. It hasn't even started on chromium or rust yet, so it could be a while.

Any further insight gratefully accepted.

Cheers,
Jon

When did you notice the change?

I thought there was a related news item, but I'm not seeing it. Maybe it could be related to a change intorduced by RumpletonBongworth where some issues are discussed in this post (and in other posts in that thread and possibly in other threads):

viewtopic-p-8830535.html#8830535

Of course it might not be related to that.

The part that tries to keep the old alias an only appending color seems like a major oops. By the declaration in the loop, _ is only the command, not what it may have been aliased to.

[Hu]

As I understand it, /etc/profile.d is only evaluated by login shells. /etc/bash/bashrc.d is evaluated by both login and non-login interactive shells. Therefore:

I am used to setting system aliases that affect every interactive shell in the system. Things like "alias dir = ls -al". These go into a drop-in file in /etc/profile.d/ since it's something I changed for my system.

I expect that running /bin/bash -i without -l will not read /etc/profile.d, so I think it is not correct to say it will "affect every interactive shell."

But so I should be able to "source /etc/profile" and reasonably expect that the aliases in the dropin directory /etc/profile.d/ are executed and not stepped on. But that's not what happens.

They are executed, and later stepped on.

What is confusing me is that /etc/bash/bashrc.d/10-gentoo-color.bash seems to be going out if its way *not* to do this:

Code: Select all

if (( gentoo_color > 0 )); then
        # Colorize the output of diff(1), grep(1) and a few coreutils utilities.
        for _ in diff dir grep ls vdir; do
                alias "$_=$_ --color=auto"
        done

No, this is not trying to preserve your aliases. It is merely a convenient shortcut for setting several very similar aliases.

Instead of overriding my aliases it really should be just adding "--color=auto" to them. This is what I want, but it's not happening.

I can see why you would want it to do that, but that is not presently what it attempts to do.

Also confusingly, when I su to root

Code: Select all

jesnow@bartali ~ $ su

You started a non-login shell for root, which we generally discourage. This may (or may not) explain the odd result you show.

And when I open a new shell:

How was this new shell opened?

[jesnow]

OK, now I'm doubly confused. Why is it not the intention to preserve aliases I set systemwide in /etc/profile.d?
Why even have /etc/profile.d if it is simply overridden?

Cheers,
Jon.

[Hu]

I don't know why bashrc.d forcibly sets those aliases. Perhaps the author expected no one would have set those aliases in profile.d, and so did not include a check whether they were defined.

As I said above, this is not being overridden because it's in profile.d. It's being overridden because profile.d runs first, and bashrc.d has no explicit check to avoid overriding things that were already done. If you had put something in profile.d that the bashrc.d fragment never touched, then your setting would not be overridden. I think you would have the same problem if you defined these in a bashrc.d that is evaluated earlier than the Gentoo bashrc.d fragment. This also suggests a solution: define these in bashrc.d, and in a file which sorts later than the Gentoo bashrc.d fragment. This solves two problems:

• First, by evaluating yours later, you overwrite Gentoo, rather than the other way around.
• Second, by defining yours in bashrc.d, they actually do apply to all interactive bash uses. Your current setup only affects login shells.

[mv]

Code with bashisms (like "alias") does not belong to /etc/profile.d anyway as it would break other shells which also source this directory at startup. Moreover, IIRC, alias works only in interactive shells (which a startup shell in general is not).

[pjp]

Alias isn't exclusive to bash, but your point still stands.


Interestingly, 'man bash' has no mention of /etc/bash (only /etc/bash_completion.d/). Section "Invocation" only mention /etc/profile and dot files in a user's home directory. Nor does app-shells/bash have any USE flags indicating the Gentoo files can be turned off. I'm guessing it is customary for distributions to provide such convenience enhancements.

Having scripts do things then requiring users to create scripts to undo them seems an interesting mechanism. Although it may partially explain some of the performance of modern computing.

[mv]

Interestingly, 'man bash' has no mention of /etc/bash

By default, SYS_BASHRC is not defined, and the commented (i.e. “recommended”) value is /etc/bash.bashrc.

[Zucca]

Code: Select all

NBLK-WAX9X ~ # tree /etc/bash
/etc/bash
├── bash_logout
├── bashrc
└── bashrc.d
    ├── 10-gentoo-color.bash
    ├── 10-gentoo-title.bash
    ├── 15-gentoo-bashrc-check.bash
    ├── alias.bash
    └── blesh_init.bash

2 directories, 7 files

Yeah. Default gentoo setup has non-default bashrc setup.

[eschwartz]

Having scripts do things then requiring users to create scripts to undo them seems an interesting mechanism. Although it may partially explain some of the performance of modern computing.

Well, you are also more than free to simply delete it from the existing script, given that it is a conffile and will simply trigger the usage of dispatch-conf on bash package updates.

It was in fact specifically moved out of /etc/bash/bashrc to give people this freedom, to more easily modify / suppress the color handling if they don't like the Gentoo defaults without forcing to run dispatch-conf for unrelated changes elsewhere in a giant config file.

[pjp]

And that's a very good reason, and given how config files work in general, I have no better solution.

I like some color, but often the degree to which it can make something unreadable is astonishing. For example I just used less on 10-gentoo-color.bash and it was hard enough to read what the variable "NO_COLOR" was, so I had to cat the file to less. Which leads me to the point I was making above. That script checks to see if NO_COLOR is set. So the script needs to run to see that it doesn't need to run. And if the script was removed and someone decided they wanted to enable color, then they'd have to figure out how to get / re-enable the script.

For me at least the underlying problem is that color doesn't have (or I'm not aware of) a way to manage it effectively. Especially for remote shells where less color is available.

[Zucca]

Well, you are also more than free to simply delete it from the existing script, given that it is a conffile and will simply trigger the usage of dispatch-conf on bash package updates.

... or set INSTALL_MASK in make.conf to block those files from installing in the first place? ;)

[Hu]

You could do that, but then any subsequent updates you want to know about would be hidden. Overriding them in the live filesystem and watching for dispatch-conf to warn of changes is the safer option.

[Zucca]

watching for dispatch-conf to warn of changes is the safer option.

That is definitely true.

[pjp]

But if you simply delete the file, then it gets reinstalled. If you leave a file with a one line comment "# Empty to prevent reinstall" then you still have to get red of the .cfg files that return.

The first alternatives that come to mind would be an rc-config like enable / disable, or a scripts.enable/disable file (similar to the various allow / deny files that are or were under /etc).

[Hu]

The options are:

• Edit the file to remove the undesirable behavior, possibly replacing the entire file with a one-line comment. Advantage: you get notified if a future change tries to modify this file, and you can decide whether the updated version still deserves to be disabled on this system. Disadvantage: you must deal with the notification even when the update is one you will ultimately decide to reject.
• Delete the file. I am unsure whether it is automatically restored as pjp says, but assuming it is, you may have a worse trade-off than above, depending on whether Portage silently restores the file or whether it writes a ._cfg_ and wants you to make the replacement live.
• Use INSTALL_MASK to inhibit installing the file. As a result, Portage never writes the ._cfg_ file, which could be seen as an advantage if you have decided you will always reject updates.

I don't want to prejudge the value of updates that have not been written yet, so I want to see that an update is proposed and make a decision when that happens. Therefore, in my opinion, the best option is one that instructs Portage to notify me of future updates. For an administrator who has decided to summarily reject all future updates, INSTALL_MASK is probably a better choice.

[flexibeast]

it was hard enough to read what the variable "NO_COLOR" was, so I had to cat the file to less

As as aside, one can pass the `-L` option to less(1) to prevent it doing LESSOPEN preprocessing - by default, via /usr/bin/lesspipe - such as colorisation. If you can't do this because you want/need non-colour-related preprocessing, you could set LESSCOLOR=0.

[pjp]

Delete the file. I am unsure whether it is automatically restored as pjp says, but assuming it is, you may have a worse trade-off than above, depending on whether Portage silently restores the file or whether it writes a ._cfg_ and wants you to make the replacement live.

I am guessing that it is replaced based on the behavior of CONFIG_PROTECT. Since there would be no file to protect, creating a .cfg would seem odd. And worse would be not creating the actual file, because if you did want it back, it would seem very difficult. Given the way it works, "notify me" via .cfg file seems like the best available option.

As as aside, one can pass the `-L` option to less(1) to prevent it doing LESSOPEN preprocessing - by default, via /usr/bin/lesspipe - such as colorisation. If you can't do this because you want/need non-colour-related preprocessing, you could set LESSCOLOR=0.

I know there are options, it's usually easier to just do the cat / less thing. I can also set aliases and maybe some other methods I'm forgetting. And of course there's always trying (again) to figure out colors, which seems worse than audio, which I haven't had to mess with once I got it to work eons ago.

[jesnow]

Code with bashisms (like "alias") does not belong to /etc/profile.d anyway as it would break other shells which also source this directory at startup. Moreover, IIRC, alias works only in interactive shells (which a startup shell in general is not).

OK, now I'm triple confused. I simply need my system aliases to be set how I like them and I thought /etc/profile or /etc/profile.d would be the place to do that. Perhaps they should be in /etc/bash/bashrc.d/999-my-system-aliases.sh? That way I have the last word.



Cheers,
Jon.

[Zucca]

Perhaps they should be in /etc/bash/bashrc.d/999-my-system-aliases.sh? That way I have the last word.



Cheers,
Jon.

That is the "correct" place, if you use bash.

[RumpletonBongworth]

When did you notice the change?

I thought there was a related news item, but I'm not seeing it. Maybe it could be related to a change intorduced by RumpletonBongworth where some issues are discussed in this post (and in other posts in that thread and possibly in other threads):

viewtopic-p-8830535.html#8830535

Of course it might not be related to that.

Indeed, it is not. The only difference now is that the code responsible has been relocated to 10-gentoo-color.bash.

The problem is that the prior existence of any alias - or, indeed, function - bearing the name of the intended alias is not taken into account. This would be fairly easy to address. I shall mull this over and perhaps submit a patch to that end.

EDIT: It is worth noting that the way in which colour support is detected is considerably more effective than it used to be. Without a doubt, there will be cases in which colour support is detected where it previously was not. In turn, that means that the probability of the aliases being defined is somewhat greater than it used to be.

[RumpletonBongworth]

Code with bashisms (like "alias") does not belong to /etc/profile.d anyway as it would break other shells which also source this directory at startup. Moreover, IIRC, alias works only in interactive shells (which a startup shell in general is not).

All implementations of the Shell Command Language must support alias. As long as the file is given a suffix of ".sh", it is perfectly acceptable to define aliases in this way, provided that the values of the aliases constitute portable code. Further, if by "startup shell", you mean login shell, you are mistaken. A login shell can very well be an interactive shell at the same time. Regardless, it is easy to determine whether the shell is in an interactive shell in pure sh.

Code: Select all

# Is the shell interactive?
case $- in *i*) ;; *) return ;; esac

# Upon reaching here, the answer is yes.

Further, bash can be detected in the following manner.

Code: Select all

if [ "$BASH" ]; then
	: engage in bash-specific things here
fi

That Gentoo's system-wide bashrc clobbers aliases bearing certain names remains a distinct matter, of course.

[flexibeast]

All implementations of the Shell Command Language must support alias.

Indeed. Here's POSIX on the matter.