View previous topic :: View next topic |
Author |
Message |
midnite Guru


Joined: 09 Apr 2006 Posts: 471 Location: Hong Kong
|
Posted: Thu Apr 14, 2022 3:35 pm Post subject: Basic about "shell" and shell script |
|
|
As far as I understand (please correct me if wrong), different shells are different programs. They can be find in packages.
In most systems (including mine), /bin/sh -> bash. AFAIU sh and bash (as well as zsh) are different shells. I have chosen to use zsh as my daily user (not root). zsh is the shell used in the Gentoo version of SysRescueCD. It is more convenient than bash IMO. I installed app-shells/zsh from portage. How can I install /bin/sh? As it is not available in packages. /bin/sh is just a symlink pointing to /bin/bash. So /bin/sh is actually not present in my system.
AFAIU different shells are different [Wikipedia: Comparison of command shells]. And /bin/sh is kind of the ancestor of most shells. Most later shells are developed base on /bin/sh.
To write shell scripts as most portable as possible, we should use syntax of the /bin/sh. Even if I use #!/bin/sh but my system is symlinking it to /bin/bash, I cannot run (try and debug) my scripts in /bin/sh, is it?
As many people use the terms Bourne shell (the basic shell) and POSIX shell interchangeably, they are actually two entries (two rows in the table) in Wikipedia. Are they the same or actually different? Which one has the maximum compatibility?
https://www.geeksforgeeks.org/difference-between-sh-and-bash/
Quote: | sh is also called Bourne Shell. sh is a command programming language described by POSIX standard. |
_________________ - midnite. |
|
Back to top |
|
 |
Goverp Advocate


Joined: 07 Mar 2007 Posts: 2295
|
Posted: Thu Apr 14, 2022 5:34 pm Post subject: |
|
|
On Gentoo (and I think most Linuxes) /bin/sh is a symbolic link to whichever shell you choose. On Gentoo you can set the link with "eselect sh".
That's sort of the "system" shell - anything shell script that starts "#!/bin/sh" will use it. If you want to code a script for a particular shell, you'd write "#!/bin/rsh" as the first line.
I'm too young to know the full history of Unix shells. If you want to code for "most" shells you should stick to POSIX compliant shells. Bash is extended way past the POSIX definitions; it has a setting somewhere to make it properly compliant. A good POSIX compliant shell for Gentoo is dash. It has the advantage of being very fast and reliable. A number of Linux distributions use dash as their "system" shell, and if you install dash and then "eselect sh set dash" it will be the system shell for your Gentoo system.
What you get at the command line is usually bash. It's specified in the /etc/passwd entry for each user, and the default entry is /bin/bash. It's more friendly than dash - its history commands are better, and dash doesn't do tab-completion. _________________ Greybeard |
|
Back to top |
|
 |
eccerr0r Watchman

Joined: 01 Jul 2004 Posts: 10078 Location: almost Mile High in the USA
|
Posted: Thu Apr 14, 2022 6:00 pm Post subject: |
|
|
The bourne shell describes a shell language. POSIX tried to standardize this language.
/bin/sh just needs to be a program that can execute bourne shell scripts, ideally understands the POSIX standard bourne shell. It does not matter what it is, just needs to execute it. Any shell that understands bourne shell will do as long as the consumers (all the scripts with #!/bin/sh as its first line) does not use extended feaures -- and this isn't always true.)
Bash does fine, along with ksh, ash, dash, etc., etc. as long as it's not csh, tcsh, or zsh. Technically csh also has a POSIX standard which tcsh tries to adhere to, but it sure won't run bourne shell scripts.
As for bourne shell compatibility, you just have to know your specific target. For the most part I don't see many people using strict bourne shell anymore as it's slow due to a lot of optimizations added in later shells that were not design issues in the original bourne shell.
Note that for so called "embedded" designs that end up with 1GB of memory, just use bash and all its bashisms. Unless your "embedded unix" is less than say 64MB or so, don't bother trying to not use bash. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
 |
midnite Guru


Joined: 09 Apr 2006 Posts: 471 Location: Hong Kong
|
|
Back to top |
|
 |
mv Watchman


Joined: 20 Apr 2005 Posts: 6781
|
Posted: Thu Apr 14, 2022 8:40 pm Post subject: |
|
|
eccerr0r wrote: | /bin/sh just needs to be a program that can execute bourne shell scripts, ideally understands the POSIX standard bourne shell. |
On general unix systems (e.g. on most sun machines), /bin/sh is not POSIX compliant. Neither is /usr/bin/env sh. In general unix systems, you can expect only a Bourne shell which is full of bugs and miles away from being POSIX. There is a POSIX way to get the path to a (rather) POSIX compatible shell, but I forgot how this was. Something with getconf, probably.
On Linux, /bin/sh is usually either bash or dash, both being somewhat POSIX compatible (at least, by far more compatible than the classical Bourne shell ever was).
AFAIK, there exists only one free fully POSIX compatible shell: Bosh from Jörg Schilling. You can find it e.g. in app-shells/schily-tools in the mv overlay.
Quote: | Bash does fine, along with ksh, ash, dash, etc., etc. as long as it's not csh, tcsh, or zsh. |
zsh should be completely fine as /bin/sh: It is rather similar to ksh but has much greater interactive features.
Quote: | Unless your "embedded unix" is less than say 64MB or so, don't bother trying to not use bash. |
Using dash instead of bash can rather speed up the boot process (if you do not use systemd). But some init-scripts might still have bashisms (which are usually not a problem if you use bash or zsh [and probably also ksh - I never tried]) as /bin/sh.
Last edited by mv on Thu Apr 14, 2022 8:43 pm; edited 1 time in total |
|
Back to top |
|
 |
eccerr0r Watchman

Joined: 01 Jul 2004 Posts: 10078 Location: almost Mile High in the USA
|
Posted: Thu Apr 14, 2022 8:43 pm Post subject: |
|
|
Again there is no base bourne shell, nobody would use it.
- hell no, there is definitely no tab completion.
- It doesn't even have command line editing. Typo? Type it again from the beginning unless it was at the end of the line. If you already pressed enter...
- I believe the "! style history" was specified in the original bourne shell (which still exists in bash) but don't bother trying using your arrow keys to recall
- It has no job control. Can't suspend jobs. Forgot to make it asynchronous? Either abort it or wait till it's done before you can get your prompt back.
- more things would fork (as it was defined) so you pay your fork() penalty much more often.
There are some shells like ash and dash that sort of tries to at least add readline(or at least a homemade subset) and job control features in and then some, to make it somewhat "functional" instead of "braindead".
Anyway, shellcheck is simply keyworded ~ unstable, not masked - you can just accept that, and it should merge fine in Gentoo just fine.
---
And hmm... looks like zsh will support csh-like syntax and bourne-like syntax. What is it trying to do... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Last edited by eccerr0r on Thu Apr 14, 2022 8:50 pm; edited 1 time in total |
|
Back to top |
|
 |
mv Watchman


Joined: 20 Apr 2005 Posts: 6781
|
Posted: Thu Apr 14, 2022 8:50 pm Post subject: |
|
|
eccerr0r wrote: | Again there is no base bourne shell, nobody would use it.
- hell no, there is definitely no tab completion.
- It doesn't even have command line editing.
... |
You mentioned only the interactive features. What is worse is that things like ${var#...} and ${var%...} are missing which are used quite often in scripts. I think very early versions did not even allow you to define functions. That's why autoconf generates so horrible configure scripts: If you need to call a function several times you have to define it in m4 to copy the function code several times into the configure script. |
|
Back to top |
|
 |
midnite Guru


Joined: 09 Apr 2006 Posts: 471 Location: Hong Kong
|
Posted: Thu Apr 14, 2022 9:36 pm Post subject: |
|
|
eccerr0r wrote: | - hell no, there is definitely no tab completion.
- It doesn't even have command line editing. Typo? Type it again from the beginning unless it was at the end of the line. If you already pressed enter... | Sounds like the adb shell in my old Android.
I don't actually want to use bourne shell. If it is the ancestor of most shells, if my scripts run correctly in bourne shell, then they are compatible enough. I am trying to find a baseline standard, the largest common factor.
Quote: | What is worse is that things like ${var#...} and ${var%...} are missing which are used quite often in scripts. |
Parameter Expansion is POSIX standard, I think.
Hmm....I think I was misled by the Wikipedia table. There is no such thing as "POSIX shell". POSIX is a standard (a family of standards). People build shell programs, some try to follow the POSIX standard, while some opt for more features and ignoring the POSIX standard. _________________ - midnite. |
|
Back to top |
|
 |
figueroa Advocate


Joined: 14 Aug 2005 Posts: 3026 Location: Edge of marsh USA
|
Posted: Fri Apr 15, 2022 3:20 am Post subject: |
|
|
My opinion, it would be a favor to you and your system to re-emerge bash. It is widely assumed across the Linuxverse that bash exists. On most Debian variants, /bin/sh is a symlink to dash, though bash is usually the main shell. I think dash is a bit of a red herring. Having bash on board would give you a less breakable and more normal system. _________________ Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi -wayland |
|
Back to top |
|
 |
mv Watchman


Joined: 20 Apr 2005 Posts: 6781
|
Posted: Fri Apr 15, 2022 6:05 am Post subject: |
|
|
midnite wrote: | Parameter Expansion is POSIX standard, I think. |
Yes, it is POSIX standard. But as I have written, the original Bourne shell is miles away from the POSIX standard.
As mentioned on another place, also dash (even less bash) are not POSIX. For instance, they lack a lot of i18n support required by POSIX. That's why dash is so fast: Supporting locales as required by POSIX would require much more sophisticated parsing (note that there are locales like Big5 and many more which are much less handy than utf8). |
|
Back to top |
|
 |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4275 Location: Rasi, Finland
|
Posted: Fri Apr 15, 2022 12:48 pm Post subject: |
|
|
If I write a shell script to other than bash, I'll make it run on busybox sh. I also tend to test them on mksh (ksh -like shell).
Busybox sh has just enough features but is also enough limited, so that all scripts I've written work on most shells (those which feel like POSIX, so excludes fish for example) then.
If you require arrays, then busybox sh (and POSIX too) wouldn't cut it, unless you use push from fellow member mv. ;)
But then again, there's awk in almost every *NIX system. I write inline awk code for, more compllex things, inside my shell scripts and make them into shell functions.
There's also another topic about which package owns /bin/sh. _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
 |
xanderal Tux's lil' helper


Joined: 06 Mar 2019 Posts: 133 Location: Germany
|
Posted: Tue Apr 19, 2022 3:08 pm Post subject: |
|
|
Ok, I'm running both Linux and BSD systems, so portability is very important for me and something along those lines seems to be your use case as well, so here are my two cents:
When scripting on e.g. OpenBSD stuff is really easy because /bin/sh is actually a proper POSIX-compliant executable, so that's close to foolproof to work with (sadly I'm not aware of any Linux distro that makes it that easy for us).
But in general I use #!/bin/sh and test it with shellcheck(1). That has support for sh/bash/dash/ksh but is not perfect. Nevertheless it should flag most blunders.
I don't know how far you want to take your portability. If it's only about your stuff running in different shells on a Linux system with GNU core utils, then this should suffice.
But if you're looking into making it run with busybox or other core utils (like e.g. the different BSD ones), then you might have to consider some more stuff:
Not every flag you like is supported, so read manpages (well written ones include a STANDARDS section that among other things tells you which flags are extensions) and other proper reference material (stack overflow is not a reference).
Just an example: Say you want to replace some string, what do you use? 'sed -i'? Nope, you don't, at least not always. It works with GNU and OpenBSD, but e.g. on FreeBSD this might not work.
Or to quote from the OpenBSD manpage on sed(1):
https://man.openbsd.org/sed wrote: |
The sed utility is compliant with the IEEE Std 1003.1-2008 (“POSIX.1”) specification.
The flags [-aEiru] are extensions to that specification.
|
So in summary: If you want your
midnite wrote: |
shell scripts as most portable as possible
|
you might have to dig further than just what shebang you use. But using #!/bin/sh and making sure shellcheck is happy should get you at least 90% of the way there.
ETA:
Zucca wrote: |
But then again, there's awk in almost every *NIX system. I write inline awk code for, more compllex things, inside my shell scripts and make them into shell functions.
|
Same thing here: Sure there might be an awk on the target system but which one? GNU awk? The original "one true awk" by Aho, Kernighan and Weinberger? Something else entirely? |
|
Back to top |
|
 |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4275 Location: Rasi, Finland
|
Posted: Tue Apr 19, 2022 4:32 pm Post subject: |
|
|
xanderal wrote: | Sure there might be an awk on the target system but which one? GNU awk? The original "one true awk" by Aho, Kernighan and Weinberger? Something else entirely? | I use Busybox there too. It really is the easiest way to "guarantee" a some level of compability, not bullet proof. _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
 |
xanderal Tux's lil' helper


Joined: 06 Mar 2019 Posts: 133 Location: Germany
|
Posted: Wed Apr 20, 2022 12:24 am Post subject: |
|
|
Zucca wrote: | I use Busybox there too. It really is the easiest way to "guarantee" a some level of compability, not bullet proof. | But that doesn't get you far when you don't know the system your tool is running on. Some of my stuff is done not just for myself but for others as well, so I don't know whether the scripts will be run on some Linux, BSD, MacOS, whatever...
I would agree though that (provided the feature set of Busybox suffices) this could be a useful starting point for midnite - very much depends on how portable this stuff needs to be/how much control they have over the OS it's supposed to be running on.
If I might ask a tiny question for my own benefit (but hopefully OP's as well), @Zucca:
Have you in real world application ever run into a problem where Busybox was too restricted in the implemented features? Or have I misunderstood you earlier and the limitations are more of a hypothetical worry one can mainly forget about? |
|
Back to top |
|
 |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4275 Location: Rasi, Finland
|
Posted: Wed Apr 20, 2022 7:59 am Post subject: |
|
|
@xanderal, so far I only miss arrays in busybox shell, but there are ways to overcome that limitation if one really needs arrays.
busybox awk works just fine. I just need to avoid some functions like gensub. I've written my custom initramfs scripts to be run with with busybox.
I also looked at toybox to replace busybox, but it doesn't implement sh and awk yet (or they reside in 'pending' directory).
One quite frustrating, but manageable, incompability between GNU, busybox and toybox was mktemp. With careful choosing of arguments it "beacame" portable too. Then there's date. It's a same story, but you can also use awk for most date realted things.
So it's not like busybox is ever too limited, but the portability is the tricky part in my opinion. Since shell scripting isn't just the syntax, but the command line tools too. _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
 |
Genone Retired Dev


Joined: 14 Mar 2003 Posts: 9648 Location: beyond the rim
|
Posted: Wed Apr 20, 2022 8:25 am Post subject: |
|
|
midnite wrote: | So I cannot literally emerge (install) /bin/sh? |
Nope, as that "project" kinda died decades ago. Even if you get the source, you'll have a hard time to get it running on a modern OS. See https://www.in-ulm.de/~mascheck/bourne/ for some background information. |
|
Back to top |
|
 |
mv Watchman


Joined: 20 Apr 2005 Posts: 6781
|
Posted: Thu Apr 21, 2022 6:57 pm Post subject: |
|
|
Genone wrote: | Nope, as that "project" kinda died decades ago. Even if you get the source, you'll have a hard time to get it running on a modern OS |
bosh is based on the original source code. Schily has also maintained obosh which has no features added compared to the SVR4 variant. You can install obosh from schily-tools from the mv overlay. |
|
Back to top |
|
 |
xanderal Tux's lil' helper


Joined: 06 Mar 2019 Posts: 133 Location: Germany
|
Posted: Sat Apr 23, 2022 11:31 pm Post subject: |
|
|
Zucca wrote: | @xanderal, so far I only miss arrays in busybox shell, but there are ways to overcome that limitation if one really needs arrays.
busybox awk works just fine. I just need to avoid some functions like gensub. I've written my custom initramfs scripts to be run with with busybox.
I also looked at toybox to replace busybox, but it doesn't implement sh and awk yet (or they reside in 'pending' directory).
One quite frustrating, but manageable, incompability between GNU, busybox and toybox was mktemp. With careful choosing of arguments it "beacame" portable too. Then there's date. It's a same story, but you can also use awk for most date realted things.
So it's not like busybox is ever too limited, but the portability is the tricky part in my opinion. Since shell scripting isn't just the syntax, but the command line tools too. | Thanks for that, @Zucca! Trying busybox was in the back of my mind for a while now but I never got around to it. Good to know what to watch out for  |
|
Back to top |
|
 |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4275 Location: Rasi, Finland
|
Posted: Tue Apr 26, 2022 7:17 pm Post subject: |
|
|
Zucca wrote: | Then there's date. It's a same story, but you can also use awk for most date realted things. | I'll add a clarification here. There are no date functions in nawk/bwk/one true awk. However the busybox awk seems to have them... along with gawk.
Off-topic: If you need to process large datasets where you need to convert dates, an awk implementation which has these date functions is really fast compared to executing date for each conversion. But you don't want to implement your own date conversion functions in raw awk either. Timezones (changing ones), leap years and leap seconds will guarantee you a place in some kind of asylum if you try. ;) _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
 |
palak231 n00b

Joined: 04 Jun 2022 Posts: 1
|
|
Back to top |
|
 |
xanderal Tux's lil' helper


Joined: 06 Mar 2019 Posts: 133 Location: Germany
|
Posted: Sat Aug 06, 2022 9:55 am Post subject: Re: Basic about shell and shell script |
|
|
Hi Palak,
palak231 wrote: | Well while writing any shell ( https://www.techgeekbuzz.com/blog/shell-commands-in-linux/ ) programs we use /bin/sh and /bin/bash. | No, there is a number of reasons why one might use a different shell, for example since there are things (e.g. in the many different versions of ksh) that don't work (or work differently) in sh or bash so you specify /bin/ksh or /bin/oksh or whatever. So no, we don't just use /bin/sh or /bin/bash.
For one, there is significant difference between them and for two: Just because it's written on stackoverflow, doesn't mean it's true.
In the majority of cases people have /bin/sh only as a symlink to /bin/bash, /bin/dash or something similar. So in a sense, there isn't much of a meaningful difference in most cases.
But if you had read and understood the discussion we had before you might have realised that the majority is different from everyone.
So in summary:
- /bin/sh can be an executable in its own right and not just a symlink.
- When /bin/sh is in fact a symlink people symlink it to a wide variety of posix-compliant shells, i.e. not just /bin/bash (Sidenote: bash is also not always living in /bin/bash but sometimes in e.g. /usr/bin/bash. Figuring out on which systems that is the case, is left as an exercise to the reader ).
- That stackoverflow answer is incomplete and should not be referenced without mentioning that caviat.
- It is useful to read a thread before commenting on it.
Last comment: It's cool that you are here. Even if my rant might not look like it: I'm happy you're here ... but please do your research before commenting somewhere. That way you get a learning experience, OP get's a more useful answer and everyone else will likely appreciate it as well. |
|
Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|