portable sed in place editing

[Dominique_71]

Something that was not obvious to me to find. It is 3 possibilities to make an in place file sed editing:

• 1) On linux, we can use:

  Code: Select all

  sed -i 'sed command' file

  If file don't exist, no empty file will be created. But that will not work on some systems, as example freebsd.

• 2) On both linux and freebsd, we can add a .suffix to -i:

  Code: Select all

  sed -i.bak 'sed command' file

  If file don't exist, no empty file will be created.

• 3) It is not in place editing but the result is the same, and it is the safest way to do it - on both linux and freebsd, we can use redirection and a temporary file:

  Code: Select all

  sed 'sed command' file > file.tmp
  mv -f file.tmp file

  or

  Code: Select all

  sed 'sed command 1' file > file.tmp
  sed 'sed command 2' file.tmp > file

  If file don't exist, that will create an empty file, which can cause nasty bugs. That imply it is best and necessary to test if file exist and run these 2 commands only if it exist.

[Hu]

If you really don't want to rely on GNU sed, you could rely on redirection ordering instead.

Code: Select all

sed -e p < file > file.tmp && mv file.tmp file

I prefer expecting -i to work, but at least the form I show has the advantage that if the input does not exist, you get a shell error redirecting the input before it ever creates the output, so you don't get the junk temporary file. It also solves a minor TOCTOU issue, by not trying to access the source filename twice (once in the test, again inside sed).

[Dominique_71]

Thanks for the tips Hu. As fvwm-crystal's main contributor, I was asked if it can support freebsd and is looking at it. Almost all of the needed fixes was a few path changes with the Makefile, and some of the sed calls.

[Hu]

I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.

[Dominique_71]

I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.

Sure. I don't know either for the other distributions. Which imply the safest way is to not use gsed explicitly and change the few calls that are not already compatible with the freebsd version. That will also insure the POSIX.2 compatibility according to sed's man page on freebsd. Which in turn should imply no maintenance on the long run.

[eschwartz]

I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.

Various distros install g* symlinks for some or all GNU utilities, various other distros don't. There's no particular guarantee of anything.

What you could do is check if "command -v gsed" and then use that where possible, but fall back to "sed" if it isn't possible and simply hope that gsed is installed on all systems where the default sed isn't the GNU one.

Note: BSD sed supports -i just fine. That is the entire problem. Some sed implementations support "sed -i pattern file" to write in-place, some support "sed -i .backupname pattern file", and it is impossible to support both approaches, so as a result it's effectively impossible for POSIX to standardize the -i flag.

[RumpletonBongworth]

If you really don't want to rely on GNU sed, you could rely on redirection ordering instead.

Code: Select all

sed -e p < file > file.tmp && mv file.tmp file

I prefer expecting -i to work, but at least the form I show has the advantage that if the input does not exist, you get a shell error redirecting the input before it ever creates the output, so you don't get the junk temporary file. It also solves a minor TOCTOU issue, by not trying to access the source filename twice (once in the test, again inside sed).

This is an appropriate way of addressing the problem and is generally how so-called "in-place" editing works to begin with.

A better choice still can be ed(1). Alas, the Linux world accords it no respect, despite its status as a standard utility. Consequently, GNU/Linux distros often don't bother to include it as part of the base system. Of course, that doesn't stop anyone from requiring for it to be installed for a given program to function.