View previous topic :: View next topic |
Author |
Message |
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Wed Mar 28, 2018 4:16 pm Post subject: [TOPIC] What does the shebang really do? |
|
|
Mod edit: Split and judiciously cherry picked from [solved] dash as /bin/sh as the thread had veered decidedly off topic. — JRG
A script begins with a shebang so there will be no problem. Nonetheless, your system must have a bash interpreter (installed by default).
Last edited by Mr. T. on Sat Mar 31, 2018 5:49 am; edited 1 time in total |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21489
|
Posted: Thu Mar 29, 2018 1:58 am Post subject: |
|
|
helecho wrote: | A script begins with a shebang so there will be no problem. Nonetheless, your system must have a bash interpreter (installed by default). | This completely ignores the problem that the OP wanted to address. Bash enables some bashisms even when run as /bin/sh. If a script says it runs in /bin/sh, but actually uses certain Bash extensions, it will run fine on systems where /bin/sh is bash, but fail when /bin/sh points to dash. Dash provides all features that /bin/sh is required to provide, so it should be safe to use for /bin/sh, provided that all scripts that request /bin/sh really do work without Bash extensions. OP wanted to know whether any such bashisms were present. |
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Thu Mar 29, 2018 6:24 am Post subject: |
|
|
@Hu: the original post is not precise enough to support your statement because scripts can be developed by anyone.
You probably assume that scripts developed by Gentoo are reliable. Moreover, an administrator is responsible for
scripts added. Finally, it is reasonable to assume that scripts developed have a correct shebang. Therefore,
I correctly considered the problem, at least partially. |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3310 Location: Rasi, Finland
|
Posted: Thu Mar 29, 2018 9:54 am Post subject: |
|
|
helecho wrote: | A script begins with a shebang so there will be no problem. | About shebangs...
Does anyone know if OpenRC is "flexible enough" to support different interpreters via shebang? For example if someone crazy anough would want to write a init script in lua? (I'm not.) This, of course, creates a whole new set of problems because functions and variables aren't passed to the lua environment (well, variables maybe).
Moreover, does OpenRC even respect the shebang or does it automatically run the init script using /sbin/openrc-run? _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Thu Mar 29, 2018 11:36 am Post subject: |
|
|
Lua can interact with C code.
Off Topic
I would like to design a package manager implemented in Lua / C to remove Bash (ebuilds, eclasses). Lua is higher level than Bash and provides various facilities.
I like their prototype based programming.
Code: | mt = {__index = function () local z = "non nil"; return z end }
table = {}
setmetatable(table, mt)
print(table.var) |
helecho
Last edited by Mr. T. on Thu Mar 29, 2018 12:30 pm; edited 4 times in total |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3310 Location: Rasi, Finland
|
Posted: Thu Mar 29, 2018 12:06 pm Post subject: |
|
|
helecho wrote: | implemented in Lua / C to remove Bash (ebuilds, eclasses). | Then you'd need to translate all the ebuilds and eclasses to lua.
It would be easier just to start your own distribution. I'm serious here. _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Thu Mar 29, 2018 12:35 pm Post subject: |
|
|
@Zucca: I would like to fundamentally change the package manager but Gentoo is a malleable meta distribution. |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Thu Mar 29, 2018 12:49 pm Post subject: |
|
|
helecho wrote: | @Zucca: I would like to fundamentally change the package manager but Gentoo is a malleable meta distribution. | A few years ago a few of us contemplated a plugin approach to portage and ebuilds... Such ebuilds could be written in the supported languages as long as they adhered to an api. Some means to determine the language of the ebuilds early would be needed as at the moment they are just bash-sourced _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Thu Mar 29, 2018 1:12 pm Post subject: |
|
|
@Naib: I have not started my personal work yet but my main motivation is to simplify the interface of the package manager.
In other word, I hope "my" package manager will be easier to use than current Gentoo package managers. |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Thu Mar 29, 2018 1:33 pm Post subject: |
|
|
helecho wrote: | @Naib: I have not started my personal work yet but my main motivation is to simplify the interface of the package manager.
In other word, I hope "my" package manager will be easier to use than current Gentoo package managers. | my advice... start with the resolver, this is the bit that is spagetti in portage. The generation of the install scripts (ie basically sourcing the ebuild) is relatively simple _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Thu Mar 29, 2018 2:51 pm Post subject: |
|
|
@Naib: I will start from scratch but I will eventually reuse algorithms that come from pkgcore. Portage code is not very attractive.
I need to evaluate how to exploit ebuilds (more precisely data included in ebuilds). The package manager could be redefined
dynamically like emacs (operation, configuration for packages). I do not know the other features.
I believe that package management in a dynamic way would be funny and give flexibility. |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21489
|
Posted: Fri Mar 30, 2018 2:18 am Post subject: |
|
|
helecho wrote: | @Hu: the original post is not precise enough to support your statement because scripts can be developed by anyone.
You probably assume that scripts developed by Gentoo are reliable. Moreover, an administrator is responsible for
scripts added. Finally, it is reasonable to assume that scripts developed have a correct shebang. Therefore,
I correctly considered the problem, at least partially. | OP doesn't believe it reasonable to assume the scripts have a correct shebang, and I agree with him. To recap:- A shell script with shebang may specify /bin/sh or /bin/bash (or other choices irrelevant here).
- GNU Bash recognizes certain non-POSIX-sh extensions, regardless of the name from which GNU Bash was invoked.
- Dash does not recognize some non-POSIX-sh extensions that GNU Bash accepts.
- A shell script with a shebang of /bin/sh may or may not contain "bashisms" (non-POSIX-sh extensions).
- If /bin/sh is an alias for GNU Bash, then a shell script that contains bashisms will work.
- If /bin/sh is not GNU Bash, then a shell script that contains bashisms will not work.
- Many systems use GNU Bash for /bin/sh. Testing a script that contains bashisms on such a system will succeed, even though it should not succeed. This pattern has enabled many scripts to develop bashisms that only troubled people who elected to use non-GNU Bash shells for their /bin/sh.
- OP wants to use non-GNU Bash for his /bin/sh.
- OP worries that such scripts still exist, and will break when he switches.
If I understand your post correctly, you posit that everyone who wrote a shebang picked the correct interpreter for their script. Specifically, that if the script uses any bashisms, then the shebang definitely points to /bin/bash, even though GNU Bash itself does not enforce that the author write this correctly. Additionally, you posit that any subsequent maintenance that introduced a bashism also included a fix to the shebang to require GNU Bash instead of requiring any POSIX-sh shell. History provides counterexamples to these "for all" propositions. |
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Fri Mar 30, 2018 5:28 am Post subject: |
|
|
@Hu: The environment allows to conform to reality so the shebang should not be incorrect.
Code: | larry $ ls -l hello
-rwxr--r-- 1 larry larry 18 march 30 06:12 hello
larry $ cat hello
# print hello world on screen
echo hello world!
larry $ bash hello
hello world
larry $ nano hello
...
larry $ cat hello
#!/bin/bash
# print hello world on screen
echo hello world!
larry $ ./hello
hello world! |
A shebang allows users to run a shell script without specifying the interpreter on the command line.
GNU Bash runs in POSIX mode if the shebang is /bin/sh and specific features are explicitly specified in the documentation. |
|
Back to top |
|
|
i4dnf Apprentice
Joined: 18 Sep 2005 Posts: 271 Location: Bucharest, Romania
|
Posted: Fri Mar 30, 2018 8:52 am Post subject: |
|
|
@helecho, I'm terribly sorry to have to ask you this, but have you actually read the post you're replying to?
Hint, relevant part quoted below:
Hu wrote: | To recap:- A shell script with shebang may specify /bin/sh or /bin/bash (or other choices irrelevant here).
- GNU Bash recognizes certain non-POSIX-sh extensions, regardless of the name from which GNU Bash was invoked.
- Dash does not recognize some non-POSIX-sh extensions that GNU Bash accepts.
- A shell script with a shebang of /bin/sh may or may not contain "bashisms" (non-POSIX-sh extensions).
- If /bin/sh is an alias for GNU Bash, then a shell script that contains bashisms will work.
- If /bin/sh is not GNU Bash, then a shell script that contains bashisms will not work.
- Many systems use GNU Bash for /bin/sh. Testing a script that contains bashisms on such a system will succeed, even though it should not succeed. This pattern has enabled many scripts to develop bashisms that only troubled people who elected to use non-GNU Bash shells for their /bin/sh.
- OP wants to use non-GNU Bash for his /bin/sh.
- OP worries that such scripts still exist, and will break when he switches.
|
Another hint: https://packages.gentoo.org/packages/dev-util/checkbashisms _________________ "The only difference between me and a madman is that I am not MAD" (SALVATOR DALI) |
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Fri Mar 30, 2018 9:54 am Post subject: |
|
|
@i4dnf: Hu assumed that the scripts were not properly developed. I partially agree with him because the environment allows to create compliant scripts.
The mentioned problems would only be due to an incorrect shebang (cf. remark below), therefore, we come back to what I said:
helecho wrote: | @Hu: the original post is not precise enough to support your statement because scripts can be developed by anyone.
You probably assume that scripts developed by Gentoo are reliable. Moreover, an administrator is responsible for
scripts added. Finally, it is reasonable to assume that scripts developed have a correct shebang. Therefore,
I correctly considered the problem, at least partially. |
Note: A lack of knowledge or incompetence of contributor(s).
Edit: The symbolic link allows to execute a POSIX compliant script (#!/bin/sh) with a POSIX capable interpreter.
helecho |
|
Back to top |
|
|
i4dnf Apprentice
Joined: 18 Sep 2005 Posts: 271 Location: Bucharest, Romania
|
Posted: Fri Mar 30, 2018 10:29 am Post subject: |
|
|
Let's try this again:
Hu wrote: | - A shell script with a shebang of /bin/sh may or may not contain "bashisms" (non-POSIX-sh extensions).
- If /bin/sh is an alias for GNU Bash, then a shell script that contains bashisms will work.
- If /bin/sh is not GNU Bash, then a shell script that contains bashisms will not work.
|
And: Hu wrote: | - Many systems use GNU Bash for /bin/sh. Testing a script that contains bashisms on such a system will succeed, even though it should not succeed. This pattern has enabled many scripts to develop bashisms that only troubled people who elected to use non-GNU Bash shells for their /bin/sh
|
So, no, in fact logic (and prior experience) dictates that: helecho wrote: | it is NOT reasonable to assume that scripts developed have a correct shebang. | (correction and emphasis added) _________________ "The only difference between me and a madman is that I am not MAD" (SALVATOR DALI) |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Fri Mar 30, 2018 10:53 am Post subject: |
|
|
helecho wrote: | GNU Bash runs in POSIX mode if the shebang is /bin/sh [...] |
helecho ... then all that's needed is a definition of "POSIX mode"!!! ... because for bash '--posix' doesn't effect the point Hu was making: non-POSIX "bashisms" are accepted by /bin/sh => /bin/bash
Code: | % /bin/bash --posix
bash-4.4$ if [[ -n $HOME ]] ; then echo "non posix" ; fi
non posix
bash-4.4$ echo -e "non posix"
non posix
bash-4.4$ echo {non,posix}
non posix
bash-4.4$ bar="non posix" ; declare -u bar="$bar" ; echo $bar
NON POSIX
bash-4.4$ foo=1 ; let foo=${foo}+1 ; echo $foo
2
bash-4.4$ eselect sh list
Available POSIX shell implementations:
[1] bash
[2] dash *
[3] mksh
bash-4.4$ su -c 'eselect sh set 1'
bash-4.4$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2018-03-30 12:52 /bin/sh -> bash*
bash-4.4$ echo 'foo=1 ; let foo=${foo}+1 ; echo $foo' > test.sh
bash-4.4$ /bin/sh ./test.sh
2 |
helecho wrote: | [...] and specific features are explicitly specified in the documentation. |
The docs (man bash) may state that "[b]ash can be configured to be POSIX-conformant by default" and "--posix ... [c]hange the behavior of bash where the default operation differs from the POSIX standard to match the standard" but this means absolutely nothing when it comes to execution (as the above code illustrates clearly).
best ... khay |
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Fri Mar 30, 2018 11:01 am Post subject: |
|
|
@4dfn: Please, do not replace words in a quotation!! It's a good way to create confusion.
needMoreFirmware wrote: | How viable is it?, the wiki states that some scripts on Gentoo have bashisms on them, I checked the init scripts at /etc/init.d/ with checkbashisms for a quick check and they all seemed to be fine, where else should I look for bashisms before trying to replace /bin/sh with dash?, or should I even try it at all? |
The original poster asks whether it is safe to set Dash as the system interpreter.
needMoreFirmware wrote: | [...] the wiki states that some scripts on Gentoo [...] |
needMoreFirmware wrote: | [...] where else should I look for bashisms [...] |
The original poster does not know who developed scripts on his system.
needMoreFirmware wrote: | [...] I checked the init scripts at /etc/init.d/ with checkbashisms for a quick check [...] |
The original poster wonders (asks) whether Gentoo scripts are safe.
helecho wrote: | @Hu: the original post is not precise enough to support your statement because scripts can be developed by anyone. [...] |
Edit: fixed a "incorrect" word.
helecho
Last edited by Mr. T. on Fri Mar 30, 2018 12:04 pm; edited 1 time in total |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Fri Mar 30, 2018 11:07 am Post subject: |
|
|
*sigh* this discussion should have ended by post #3. _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Fri Mar 30, 2018 11:14 am Post subject: |
|
|
@khayyam: GNU Bash properly considers the shebang because it adapts its operations: #!/bin/sh unlike a supposed contributor (shebang incorrectly used).
Edit: @Hu @i4dfn: I partially agree with you; it is a matter of perception. I do not want to talk about that for a decade. |
|
Back to top |
|
|
needMoreFirmware n00b
Joined: 21 Mar 2018 Posts: 21
|
Posted: Fri Mar 30, 2018 12:48 pm Post subject: |
|
|
I'm genuinely confused at what is being discussed at this point, my question was simple, I wanted to know whether gentoo's scripts (and only gentoo's) contain an important amount of bashisms, important as in enough to break stability, since this wiki entry seems to indicate that it does contain some, there is no deeper analysis needed on what I did or did not mean. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Fri Mar 30, 2018 1:12 pm Post subject: |
|
|
helecho wrote: | @khayyam: GNU Bash properly considers the shebang because it adapts its operations: #!/bin/sh unlike a supposed contributor (shebang incorrectly used). |
helecho ... no, it clearly doesn't, unless you think that bashisms are somehow "posix". Anyhow, this discussion seems to be going nowhere fast, you're not actually engaging with what's presented but expecting that if you throw enough words, or statements, one after another, someone might make some sense of it.
To make that criticism absolutely concrete please answer this question: in what way can it be said that "bash properly considers the shebang because it adapts its operations" when the following (and what was presented previously) shows that this clearly isn't the case?
Code: | % cat test.sh
#!/bin/sh
foo=1 ; let foo=${foo}+1 ; echo $foo
% checkbashisms test.sh
possible bashism in test.sh line 2 (let ...):
foo=1 ; let foo=${foo}+1 ; echo $foo
% su -c 'eselect sh set 1'
% ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2018-03-30 15:02 /bin/sh -> bash
% /bin/sh ./test.sh
2
% su -c 'eselect sh set 2'
% ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2018-03-30 15:04 /bin/sh -> dash
% /bin/sh ./test.sh
test.sh: 2: test.sh: let: not found
1 |
best ... khay |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Fri Mar 30, 2018 1:22 pm Post subject: |
|
|
needMoreFirmware wrote: | I'm genuinely confused at what is being discussed at this point, my question was simple, I wanted to know whether gentoo's scripts (and only gentoo's) contain an important amount of bashisms, important as in enough to break stability, since this wiki entry seems to indicate that it does contain some, there is no deeper analysis needed on what I did or did not mean. | Ignore those scratching their ego's here.
What the wiki is saying is a word of warning because the norm is to have bash act as the sh interpreter and thus it is all too easy for someone to accidentally add bashism to a sh script. With sh -> bash this would work but sh -> dash this would not work.
If we look at gentoo and pretty much just @system... Portage needs bash, the ebuilds and portage makes extensive use of bashism's and this is why the associated scripts have #!/bin/bash and the ebuilds are sourced. OpenRC was developed by a BSD advodate and he wanted full POSIX compliance and thus mandated sh, true sh. All init script will have a #!/bin/sh shebang. They work with bash and should work with dash (if any don't please raise a bug as a bashism may have accidentally crept in)
so in setting sh -> dash your system should boot. HOWEVER... your system isn't just portage and openRC... it is what you want installed. A 3rd party app my have put #!/bin/sh but it was developed on a bash system and thus this might not work and this is is what hte wiki is warning about.
The best bet is to try with dash. Your system will boot and you MIGHT run into a script from another package that fails _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Fri Mar 30, 2018 1:23 pm Post subject: |
|
|
needMoreFirmware wrote: | I'm genuinely confused at what is being discussed at this point, [...] |
needMoreFirmware ... and you have a right to, there isn't much of a discussion, helecho is simply trying to bamboozle his/her way into your conciousness.
needMoreFirmware wrote: | [...] my question was simple, I wanted to know whether gentoo's scripts (and only gentoo's) contain an important amount of bashisms, important as in enough to break stability, since this wiki entry seems to indicate that it does contain some, there is no deeper analysis needed on what I did or did not mean. |
Any bashism in an initscript would be considered a bug, and should be reported. That said, I've been using /bin/dash as posix shell for some time without issue, and though I have encountered bashisms in the past I don't remember the last time I did so. Having checked with checkbashisms I think you'll probably survive the experience :)
best ... khay |
|
Back to top |
|
|
Mr. T. Guru
Joined: 26 Dec 2016 Posts: 477
|
Posted: Fri Mar 30, 2018 3:04 pm Post subject: |
|
|
@Naib @khayyam: your declarations are dissapointing and disrespectful: it seems to you equal that the interlocutor has a good faith; you advocate an offensive approach.
@khayyam: GNU Bash supports more features than a POSIX shell but is a POSIX capable shell. The documentation explicitly states what differs concerning POSIX features.
My statement you have previously quoted is not a criticism but a comment supporting my previous posts. My words as you interpret them are meaningless because you distort them.
I would even say your statement is meaningless: "non POSIX "bashisms" are accepted by Bash" (see below). Stop fantasizing: I will not certify anything and certainly not POSIX!
khayyam wrote: | helecho ... then all that's needed is a definition of "POSIX mode"!!! ... because for bash '--posix' doesn't effect the point Hu was making: non-POSIX "bashisms" are accepted by /bin/sh => /bin/bash |
helecho |
|
Back to top |
|
|
|