Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
/etc/fstab.d for flexible configuration
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Sun Jul 25, 2010 10:19 am    Post subject: /etc/fstab.d for flexible configuration Reply with quote

Get the current version from GitHub: https://github.com/Atha/update-conf.d
Code:
git clone git://github.com/Atha/update-conf.d.git

There is also an ebuild included in the git repo. To use it, simply add it to your local portage repository, e.g. /usr/local/portage or similar. I suggest putting it into app-admin. Don't forget to run ebuild update-conf-9999.ebuild digest on it. You should then be able to use it by running emerge app-admin/update-conf.


------

Hi!

In January 2008 I wrote a very simple script to manage fstab entries in the way like environment variables in /etc/env.d are managed. For this I created separated fstab files in /etc/fstab.d and wrote the following update-fstab script (that I would recommend going into /usr/local/bin).
  • If you find this script useful – use it.
  • If you want to comment on my script (and my deriative idea for /etc/fstab.d) – I welcome it!
  • If you want to enhance my script – go ahead.
  • I'd like to hereby release it in the terms of the GPL-2 or later. But I don't really know it this is actually possible for such a simple thing like this shell script. Anyway, this is what I want.



Current version: here.

The following is the old version:

Code:
#!/bin/bash

## SETUP
# fstabdpath sets the path where the individual fstab file are located, this should normaly be /etc/fstab.d.
fstabdpath="/etc/fstab.d"
# fstabdverbose sets the verbosity level. Default is 1 (no messages). If you want verbose output set this to 2.
fstabdverbose="1"


## FUNCTIONS
function message () {
  [ ${fstabdverbose} -gt "1" ] && echo $1
}


## MAIN PROGRAM
if [ ! -d ${fstabdpath} ] ; then
  echo "update-fstab: ${fstabdpath} is not present!"
  exit 1
fi

if [ -e ${fstabdpath}/fstab ] ; then
  rm -f ${fstabdpath}/fstab 2>&1
  message "Deleted existing ${fstabdpath}/fstab."
fi

#pushd ${fstabdpath} 2&>1
touch ${fstabdpath}/fstab
echo "# /etc/fstab" >> ${fstabdpath}/fstab
echo "# automatically generated by update-fstab" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
echo "# change values in /etc/fstab.d/* only!" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
fstab_files=`ls -1 ${fstabdpath} | grep -E "^[0-9]{2}.*"`
message "Found valid files: ${fstab_files}"
for fstab_file in $fstab_files ; do
  (cat ${fstabdpath}/${fstab_file} | grep -E "^[^#].*") >> ${fstabdpath}/fstab
  message "Added contents of ${fstabdpath}/${fstab_file} to ${fstabdpath}/fstab."
done
#popd 2&>1

if [ -e /etc/fstab ] ; then
  mv -f /etc/fstab /etc/fstab.d.bak 2>&1
  message "Existing /etc/fstab renamed to /etc/fstab.d.bak."
fi
if [ -e ${fstabdpath}/fstab ] ; then
  mv -f ${fstabdpath}/fstab /etc/fstab 2>&1
  message "New file system table ${fstabdpath}/fstab moved to /etc/fstab."
else
  echo "update-fstab: ${fstabdpath}/fstab not found!"
  exit 1
fi

exit 0

My /etc/fstab.d directory contains files like:
01base
02pseudo
05windows
10media

IMPORTAINT: Only files starting with a two-digit number are processed. This way you can easily deactivate just one file, or activate it again – by simply renaming it.

Code:
# /etc/fstab.d/01base
# AMD Phenom II X6 1090T + MSI 890FXA-GD70
#
# Revision: 2010-07-02


# /boot
# /dev/sda3
UUID=001122334455-6677-8899-AABBCCDDEEFF       /boot                           ext2            noauto,noatime,nodiratime,errors=remount-ro             1 2

# /
# /dev/sda6
UUID=001122334455-6677-8899-AABBCCDDEEFF       /                               ext4            noatime,nodiratime,errors=remount-ro                    0 1

# /var
# /dev/sda9
UUID=001122334455-6677-8899-AABBCCDDEEFF       /var                            ext4            noatime,nodiratime,errors=remount-ro                    0 1

# /usr/local
# /dev/sda7
UUID=001122334455-6677-8899-AABBCCDDEEFF       /usr/local                      ext4            noatime,nodiratime,errors=remount-ro,acl,user_xattr     0 1

# /var/cache/portage-trees
# /dev/sda8
UUID=001122334455-6677-8899-AABBCCDDEEFF       /var/cache/portage-trees        ext4            noatime,nodiratime,errors=remount-ro                    0 1

# portage-trees bindings
/var/cache/portage-trees/portage                /usr/portage                    none            bind                                                    0 0
/var/cache/portage-trees/layman                 /usr/local/layman               none            bind                                                    0 0

# /home
# /dev/sda10
UUID=001122334455-6677-8899-AABBCCDDEEFF       /home                           ext4            noatime,nodiratime,errors=remount-ro,acl,user_xattr     0 2


# swap
# /dev/sda5
/dev/sda5                                       none                            swap            sw                                                      0 0


# Use the RAM!
tmpfs                                           /tmp                            tmpfs           size=500m,mode=1777                                     0 0
tmpfs                                           /var/tmp/portage                tmpfs           size=2g,uid=250,gid=250,mode=755                        0 0


Code:
# /etc/fstab.d/02pseudo
# AMD Phenom II X6 1090T + MSI 890FXA-GD70
#
# Revision: 2010-07-02

# pseudo filesystems
#proc           /proc                   proc            defaults                        0 0
#sys            /sys                    sysfs           defaults                        0 0
shm             /dev/shm                tmpfs           nodev,nosuid,noexec             0 0
none    /proc/sys/fs/binfmt_misc        binfmt_misc     noauto,defaults                 0 0


Code:
# /etc/fstab.d/10media
# AMD Phenom II X6 1090T + MSI 890FXA-GD70
#
# Revision: 2010-07-02

# <fs>                  <mountpoint>    <type>          <opts>                                          <dump/pass>
#/dev/fd0               /mnt/floppy     auto                    noauto,user                                     0 0
#/dev/cdrom             /mnt/cdrom      auto                    noauto,user,ro                                  0 0
/dev/cdrom              /mnt/cdrom      udf,hfsplus,iso9660     noauto,user,ro,exec                             0 0
/dev/cdrom              /mnt/dvdram     udf                     noauto,user,rw,exec,noatime,nodiratime,unhide   0 0
#/dev/pktcdvd/pktcdvd0  /mnt/dvdrw      udf                     noauto,user,rw,unhide,iocharset=utf8            0 0


But for these mostly static enties I didn't create the script. I used to have temporary entries like changing NFS shares or USB media that needed special options, so I had to change my /etc/fstab file very often.
I found this script and the /etc/fstab.d directory very handy and useful for this type of usage.

Also, now I can try experimental options and temporary fstab entries in my /etc/fstab that are deleted (reverted, reset) to the /etc/fstab.d entries once I re-run update-fstab.

It is intended that the output includes only lines that don't start with # (comments) and empty lines. The previous /etc/fstab is backed up to /etc/fstab.d.bak.

I RECOMMEND BACKING UP YOUR ORIGINAL /etc/fstab! DON'T USE /etc/fstab.d.bak FOR YOUR BACKUP!

If this is useful for anyone, feel free to use it or enhance it. I'd like it to be GPL-2 or later licensed.

Thanks.


Last edited by Atha on Sun Sep 15, 2013 10:40 pm; edited 5 times in total
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Sun Jul 25, 2010 2:24 pm    Post subject: Reply with quote

hello Atha,

I think you should try to make your script POSIX compliant, and run it with /bin/sh.

Code:
fstab_files=`ls -1 ${fstabdpath} | grep -E "^[0-9]{2}.*"`
message "Found valid files: ${fstab_files}"
for fstab_file in $fstab_files ; do

  • no need for the -1 switch for 'ls' since it will automatically detects it is outputting to a pipe and won't use several columns
  • I'm not sure the message "Found valid files: ${fstab_files}" is really usefull other than for debugging
  • So, why no do
    Code:
    for fstab_file in $fstabdpath}/[0-9][0-9]* ; do
       echo "debug: $fstab_file"
    instead?

_________________
The End of the Internet!
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Sun Jul 25, 2010 4:11 pm    Post subject: Reply with quote

truc wrote:
I think you should try to make your script POSIX compliant, and run it with /bin/sh.

Thanks, I changed that first line. Is the rest of the script okay with /bin/sh and POSIX?

truc wrote:
Code:
fstab_files=`ls -1 ${fstabdpath} | grep -E "^[0-9]{2}.*"`
message "Found valid files: ${fstab_files}"
for fstab_file in $fstab_files ; do

  • no need for the -1 switch for 'ls' since it will automatically detects it is outputting to a pipe and won't use several columns
  • I'm not sure the message "Found valid files: ${fstab_files}" is really usefull other than for debugging
  • So, why no do
    Code:
    for fstab_file in $fstabdpath}/[0-9][0-9]* ; do
       echo "debug: $fstab_file"
    instead?


I changed the following lines:
FROM:
Code:
fstab_files=`ls -1 ${fstabdpath} | grep -E "^[0-9]{2}.*"`
message "Found valid files: ${fstab_files}"
for fstab_file in $fstab_files ; do
  (cat ${fstabdpath}/${fstab_file} | grep -E "^[^#].*") >> ${fstabdpath}/fstab
  message "Added contents of ${fstabdpath}/${fstab_file} to ${fstabdpath}/fstab."
done

TO:
Code:
for fstab_file in $fstab_files/[0-9][0-9]* ; do
  # echo "debug: $fstab_file"
  (cat ${fstabdpath}/${fstab_file} | grep -E "^[^#].*") >> ${fstabdpath}/fstab
  message "Added contents of ${fstabdpath}/${fstab_file} to ${fstabdpath}/fstab."
done


But now, when fstabdverbose="2", I get the following:
Code:
# update-fstab
Added contents of /etc/fstab.d//01base /etc/fstab.d//02pseudo /etc/fstab.d//05windows /etc/fstab.d//10media /etc/fstab.d//31user /etc/fstab.d//81wine to /etc/fstab.d/fstab.
Existing /etc/fstab renamed to /etc/fstab.d.bak.
New file system table /etc/fstab.d/fstab moved to /etc/fstab.


So, this changes the behavious of the for ; do ; done function to only one run with all files, instead of just one file for each for ; do iteration.

That was not intended, but works too.

Well, thanks for you comment.
A.
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Sun Jul 25, 2010 8:05 pm    Post subject: Reply with quote

Atha wrote:
truc wrote:
I think you should try to make your script POSIX compliant, and run it with /bin/sh.

Thanks, I changed that first line. Is the rest of the script okay with /bin/sh and POSIX?

It doesn't look like yet, IIRC 'function' is not a valid 'word?' in POSIX shell, therefore you should change things like
Code:
function message () {
  [ ${fstabdverbose} -gt "1" ] && echo $1
}

to
Code:
message () {
  [ ${fstabdverbose} -gt "1" ] && echo $1
}



If /bin/sh is a symlink to /bin/bash, try it with /bin/dash instead.


Quote:
I changed the following lines:
FROM:
(snip)
TO:
Code:
for fstab_file in $fstab_files/[0-9][0-9]* ; do
  # echo "debug: $fstab_file"
  (cat ${fstabdpath}/${fstab_file} | grep -E "^[^#].*") >> ${fstabdpath}/fstab
  message "Added contents of ${fstabdpath}/${fstab_file} to ${fstabdpath}/fstab."
done


But now, when fstabdverbose="2", I get the following:
Code:
# update-fstab
Added contents of /etc/fstab.d//01base /etc/fstab.d//02pseudo /etc/fstab.d//05windows /etc/fstab.d//10media /etc/fstab.d//31user /etc/fstab.d//81wine to /etc/fstab.d/fstab.
Existing /etc/fstab renamed to /etc/fstab.d.bak.
New file system table /etc/fstab.d/fstab moved to /etc/fstab.


So, this changes the behavious of the for ; do ; done function to only one run with all files, instead of just one file for each for ; do iteration.


it's for fstab_file in $fstabdpath/[0-9][0-9]* ; do not for fstab_file in $fstab_files/[0-9][0-9]* ; do

Also,
Code:
(cat ${fstabdpath}/${fstab_file} | grep -E "^[^#].*") >> ${fstabdpath}/fstab

Why are you
  • using cat here?
  • spawning a subshell (with the ())?
  • using the -E grep switch although your RE is 'just' a hum... a Regular Expression! ;)

Try this instead:
Code:
grep '^[^#]' $fstab_file >> $fstabdpath/fstab



last thing I see is
Code:
if [ -e /etc/fstab ] ; then
  mv -f /etc/fstab /etc/fstab.d.bak 2>&1
  message "Existing /etc/fstab renamed to /etc/fstab.d.bak."
fi

which could probably be changed to something like:
Code:
mv -f /etc/fstab /etc/fstab.d.bak && message "Existing /etc/fstab renamed to /etc/fstab.d.bak."

(and why are you redirecting stderr to stdout?)
_________________
The End of the Internet!
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Sun Jul 25, 2010 9:38 pm    Post subject: Reply with quote

Okay, so I reworked it.

  • @truc: I incorporated all your fixes. Thanks again.
  • It now refuses to work if /etc/fstab.d/fstab is present (since this file may have been added by a user, and we don't want to overwrite it without pointing it out).
  • Errors are now on stderr (>&2)
  • The check for /etc/fstab.d/fstab isn't really required, since the script just created this file (that's its main purpose!) – so why bother looking for it? We know it's there!


So this is the new script:
Code:
#!/bin/sh

## SETUP
# fstabdpath sets the path where the individual fstab file are located, this should normaly be /etc/fstab.d.
fstabdpath="/etc/fstab.d"
# fstabdverbose sets the verbosity level. Default is 1 (no messages). If you want verbose output (i.e. to debug) set this to 2.
fstabdverbose="2"


## FUNCTIONS
message () {
  [ ${fstabdverbose} -gt "1" ] && echo $1
}


## MAIN PROGRAM
if [ ! -d ${fstabdpath} ] ; then
  echo "update-fstab: ${fstabdpath} is not present!" >&2
  exit 1
fi

if [ -e ${fstabdpath}/fstab ] ; then
  echo "update-fstab: please remove ${fstabdpath}/fstab before you run this script.
update-fstab: NOTE: It may have been left by a previously run update-fstab, but you should check anyway." >&2
  exit 1
fi

touch ${fstabdpath}/fstab && message "${fstabdpath}/fstab created"
echo "# /etc/fstab" >> ${fstabdpath}/fstab
echo "# automatically generated by the update-fstab script" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
echo "# Please change the according lines in /etc/fstab.d/* if you want them to be permanent," >> ${fstabdpath}/fstab
echo "# otherwise they will not survive the next invocation of update-fstab!" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
message "Header added"
for fstab_file in ${fstabdpath}/[0-9][0-9]* ; do
  grep '^[^#].*' ${fstab_file} >> ${fstabdpath}/fstab
  message "Added: ${fstab_file}"
done

mv -f /etc/fstab /etc/fstab.d.bak && message "Existing /etc/fstab renamed to /etc/fstab.d.bak"
mv -f ${fstabdpath}/fstab /etc/fstab && message "New file system table ${fstabdpath}/fstab moved to /etc/fstab"

exit 0


BTW, I found this while searching for how to make a shell script POSIX compliant:
http://www.pixelbeat.org/programming/shell_script_mistakes.html
It's nice, only I'm too much of a nooby, so I don't understand all of it.

Cheers,
A.
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Sun Jul 25, 2010 10:49 pm    Post subject: Reply with quote

One other thing while I'm at it
Code:
touch ${fstabdpath}/fstab && message "${fstabdpath}/fstab created"
echo "# /etc/fstab" >> ${fstabdpath}/fstab
echo "# automatically generated by the update-fstab script" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
echo "# Please change the according lines in /etc/fstab.d/* if you want them to be permanent," >> ${fstabdpath}/fstab
echo "# otherwise they will not survive the next invocation of update-fstab!" >> ${fstabdpath}/fstab
echo "#" >> ${fstabdpath}/fstab
message "Header added"


You might want to use the here-document feature of the shell instead:
Code:
cat<<'EOT' > ${fstabdpath}/fstab && message "Header added"
# /etc/fstab
# automatically generated by the update-fstab script
#
# Please change the according lines in /etc/fstab.d/* if you want them to be permanent,
# otherwise they will not survive the next invocation of update-fstab!
#
EOT



EDIT: and...
Code:
fstabdverbose="2"
(...)
message () {
  [ ${fstabdverbose} -gt "1" ] && echo $1


2 comments:
  • since fstabdverbose is supposed to be a number, I wouldn't use double quotes
  • why not echoing every argument instead of just $1?

Code:
fstabdverbose=2
(...)
message () {
  [ ${fstabdverbose} -gt 1 ] && echo $@

_________________
The End of the Internet!
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Sat Aug 14, 2010 6:15 pm    Post subject: Reply with quote

Thanks!

This should be it then. I added some useful information and made all the improvements that truc came up with. I hope it now is conform to standards so it will run on a lot of Unix-based systems. (But who would want to manage their fstab entries manually except for Gentoo Linux people?) It sure is faster than it was before – so again, big big thanks to truc.

I learned a lot about shell scripts any maybe my next script (if any) will be better right away.

So, here the (up to this point) final version:

Code:
#!/bin/sh
# Script for flexible /etc/fstab.d configuration
# From Atha, with a lot of improvements from truc - thanks!
#
# You can find information on how to use this script at
# http://forums.gentoo.org/viewtopic.php?p=6364143
#
# This script idealy goes into /usr/local/bin and is called update-fstab -
# you need to configure your fstab entries in separate files in /etc/fstab.d;
# only filenames starting with two digits are included!
# Examples: /etc/fstab.d/01base or /etc/fstab.d/61nfs-dm8000, to name just two.
#
# Copyright 2008 Atha
# Distributed under the terms of the GNU General Public License v2 or later


## SETUP
# fstabdpath sets the path where the individual fstab file are located, this should normaly be /etc/fstab.d.
fstabdpath="/etc/fstab.d"
# fstabdverbose sets the verbosity level. Default is 0 (no messages). If you want verbose output (i.e. to debug) set this to 1.
fstabdverbose="1"


## FUNCTIONS
message () {
  [ ${fstabdverbose} -gt 0 ] && echo $@
}


## MAIN PROGRAM
if [ ! -d ${fstabdpath} ] ; then
  echo "update-fstab: ${fstabdpath} is not present!" >&2
  exit 1
fi

if [ -e ${fstabdpath}/fstab ] ; then
  echo "update-fstab: please remove ${fstabdpath}/fstab before you run this script.
update-fstab: NOTE: It may have been left by a previously run update-fstab, but you should check anyway." >&2
  exit 1
fi

cat << 'EOT' > ${fstabdpath}/fstab && message "${fstabdpath}/fstab created, header added"
# /etc/fstab
# automatically generated by the update-fstab script
#
# Please change the according lines in /etc/fstab.d/* if you want them to be permanent,
# otherwise they will not survive the next invocation of update-fstab!
#
EOT

for fstab_file in ${fstabdpath}/[0-9][0-9]* ; do
  grep '^[^#].*' ${fstab_file} >> ${fstabdpath}/fstab
  message "Added: ${fstab_file}"
done

mv -f /etc/fstab /etc/fstab.d.bak && message "Existing /etc/fstab renamed to /etc/fstab.d.bak"
mv -f ${fstabdpath}/fstab /etc/fstab && message "New file system table ${fstabdpath}/fstab moved to /etc/fstab"

exit 0



I added some useful comments on how to use it, but basically you should look here for help.

I hope this is useful for Gentoo Linux und *nix people.
Cheers,
A.
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Tue Feb 01, 2011 3:09 pm    Post subject: About automatically updates of /etc/fstab.d/01base Reply with quote

Hi,
I wonder if you have any elegant solution to overcome modifications made by the system to /etc/fstab (might be rare, but still possible).

As far as I understand, one might move original /etc/fstab into /etc/fstab.d/01base (and edit it to remove non-system/base mounting points to place them in other /etc/fstab.d/* files) before using update-fstab script.
Hence, it'll be possible to add extra functionality to apply any modifications made to /etc/fstab by the system, by patching /etc/fstab.d/01base.
It would require to store a copy of the latest generated fstab somewhere in /etc/fstab.d to perform diff & patch, etc.

Maybe I can implement this.
Any remark on this idea before I go forward?

If I bring modifications to this script, I surely will create a git repository... or maybe a repo already exists?
A public repo is often better to review modifications and send patches.
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Tue Feb 01, 2011 4:55 pm    Post subject: Re: About automatically updates of /etc/fstab.d/01base Reply with quote

NicoLarve wrote:
Hi,
I wonder if you have any elegant solution to overcome modifications made by the system to /etc/fstab (might be rare, but still possible).

As far as I understand, one might move original /etc/fstab into /etc/fstab.d/01base (and edit it to remove non-system/base mounting points to place them in other /etc/fstab.d/* files) before using update-fstab script.
Hence, it'll be possible to add extra functionality to apply any modifications made to /etc/fstab by the system, by patching /etc/fstab.d/01base.
It would require to store a copy of the latest generated fstab somewhere in /etc/fstab.d to perform diff & patch, etc.


I like this idea. Only, I've originally seperated Gentoo's “vanilla” /etc/fstab into [/etc/fstab.d/]01base and 02pseudo (and additionally 10media for the /dev/cdrom and /dev/fd0 examples, if they're still available on a fresh Gentoo installation). We could of course reunite them into, say, 01gentoo or so. 01base is also okay.

NicoLarve wrote:
Maybe I can implement this.
Any remark on this idea before I go forward?


I'm not really an expert, but does the system actually update /etc/fstab at all? I'm a Gentoo user since release 1.2 (was it 2002?) and I don't remember it ever happening. I've always had the impression that /etc/fstab is purely user controlled. (The Gentoo way.)

NicoLarve wrote:
If I bring modifications to this script, I surely will create a git repository... or maybe a repo already exists?
A public repo is often better to review modifications and send patches.


Please, go ahead. I'd be happy if you improve my script.
No repo currently exists, as I didn't think it would be necessary for such a tiny script. But, if you think this is the best way: go ahead.


To be honest, I'm quite surprised that someone other than me actually uses this script. It sure does make he happy though. So I can only say: Thanks for using it! Thanks for improving it! Thanks for participating!

Cheers,
A.
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Tue Feb 01, 2011 6:02 pm    Post subject: Re: About automatically updates of /etc/fstab.d/01base Reply with quote

Atha wrote:
NicoLarve wrote:
Maybe I can implement this.
Any remark on this idea before I go forward?


I'm not really an expert, but does the system actually update /etc/fstab at all? I'm a Gentoo user since release 1.2 (was it 2002?) and I don't remember it ever happening. I've always had the impression that /etc/fstab is purely user controlled. (The Gentoo way.)

Actually, it rarely happens but, for example under Debian, when the system wants to ensure it will boot the good device (e.g. on some systems SAS & SATA disks might have different device files after a reboot) it modifies fstab to use UUIDs instead of a device path such as /dev/sda1.

Atha wrote:
To be honest, I'm quite surprised that someone other than me actually uses this script. It sure does make he happy though. So I can only say: Thanks for using it! Thanks for improving it! Thanks for participating!

It's been a couple of month (years?) that I regularly google for "/etc/hosts.d" (I don't use LDAP but I like to automate things) or "/etc/fstab.d".
Your script seems good and have been polished thanks to users ont his thread!
Moreover, maybe I will consider to port it to /etc/hosts and a few other config files that aren't '.ded' yet.

Your script is already installed on two of my machines! ;-)
And will be on 3 others really soon!
Indeed, I use svn to version some system config files and I share them, as appropriate, on multiple machines. I already created an fstab.d file to define standard mount points for my main usb mass storage device (using LABEL=...), then I will share this file on multiple machines where I often plug these devices. This is powerful for many reason incliding the fact that any device will be mounted to the same path on any machine, giving you opportunity to run scripts on them, etc.

It'll may take some time, but yes, I'll go ahead!

And, finally: a repo (even private) is always a good thing when people comes to give you feedback.
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Tue Feb 01, 2011 7:22 pm    Post subject: Re: About automatically updates of /etc/fstab.d/01base Reply with quote

NicoLarve wrote:
Actually, it rarely happens but, for example under Debian, when the system wants to ensure it will boot the good device (e.g. on some systems SAS & SATA disks might have different device files after a reboot) it modifies fstab to use UUIDs instead of a device path such as /dev/sda1.


This would happen in /etc/fstab, so you would have to check what acutally was changed and “back-adpot” the corresponding entries in /etc/fstab.d/* – which will have to go between the comments and empty lines. If you could really modify the script to make this possible – wow! I couldn't think of a way of doing this…

NicoLarve wrote:
It's been a couple of month (years?) that I regularly google for "/etc/hosts.d" (I don't use LDAP but I like to automate things) or "/etc/fstab.d".
Your script seems good and have been polished thanks to users ont his thread!
Moreover, maybe I will consider to port it to /etc/hosts and a few other config files that aren't '.ded' yet.

Your script is already installed on two of my machines! ;-)
And will be on 3 others really soon!


I'm very happy this idea of mine was of good use. This is more than I had ever hoped. Simply because I figured: if it had been a good idea, someone else would have made such a script already. At least so was my thinking…

NicoLarve wrote:
It'll may take some time, but yes, I'll go ahead!

And, finally: a repo (even private) is always a good thing when people comes to give you feedback.


Yes, please do. Please link your enhancements here, be it the modified script entirely or a cvs repo or … any other possibility.

Thanks!
A.
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Sat Jun 11, 2011 10:44 pm    Post subject: Support for /etc/*.d/ Reply with quote

Hi,

Like I mentioned above I was interested in extending your script to support for /etc/*.d/ and I did it a couple of month ago. I tested it a lot (on about 12 machines on 3 different networks). Because it is generic, it is called update-conf.d.

So, one can consider spliting its /etc/lambda entries into /etc/lambda.d/*.

In order to reduce risks, the script first looks for a config file (/etc/update-conf.d.conf) to check if it is authorized to rebuild a particular /etc/lambda file on the basis of /etc/lambda.d/* files. The config file just consist in one confg file name per line (e.g. fstab, hosts, etc.).

Note that in /etc/lambda.d/*, as in the original script, '*' must be a file whose name begins with [0-9][0-9].

This version adds a little more verbosity both on stdout (what is done) and in the resulting file (where config lines come from, really helpful).

For the moment, this code is a part of a really big repo of mine, so it hasn't its own repo, maybe I'll do it one day (with its full history). I also have other nice ideas such as protecting /etc/lamba against manual edition and if that still happen, to automatically move such "illegal config" from /etc/lamba to a "lost+found" file (e.g. in /etc/lambda.d/lost+found) to finally warn the user (admin).

Here it is:
(view the patch related to the current version (2011-06-12) on this thread)
Code:
#!/bin/sh
#
# Script for flexible /etc/*.d configuration
# From Atha, with a lot of improvements from truc - thanks!
# Generalized for /etc/*.d by Nicolas Bercher nbercher@yahoo.fr
#
# This script ideally goes into /usr/local/sbin and is called update-conf.d -
# you need to configure your <conf> entries in separate files in /etc/<conf>.d;
# only filenames starting with two digits are included!

# Examples: /etc/fstab.d/01base or /etc/hosts.d/61nfs-dm8000, to name just two.
#
# Copyright 2011 Nicolas Bercher
# Copyright 2008 Atha
# Distributed under the terms of the GNU General Public License v2 or later
#
# You can find information on how to use this script at
# http://forums.gentoo.org/viewtopic.php?p=6364143
#


## SETUP
# script configuration path:
scriptconf="/etc/update-conf.d.conf"
# root of configuration files:
root="/etc"
# name of the configuration file to process:
conf="${1}"
# configuration file path:
confpath="${root}/${conf}"
# path to the backup of the current/previous configuration file:
bkpconfpath="${confpath}.d.bak"
# <conf.d> directory path:
dpath="${root}/${conf}.d"
# path to the intermediate location of the new configuration file based on <conf>.d/[0-9][0-9]* files:
dconfpath="${dpath}/${conf}"
# flag_verbose sets the verbosity level.
# Default is 0 (no messages). If you want verbose output (i.e. to debug) set this to 1.
flag_verbose="1"


## FUNCTIONS
message () {
  [ ${flag_verbose} -gt 0 ] && echo $@
}


## MAIN PROGRAM
if [ ! -f "${scriptconf}" ]; then
        echo "Script configuration file '${scriptconf}' not found!" >&2
        exit 1
fi

if ! grep "^${conf}$" "${scriptconf}"; then
        echo "Not allowed to process file ${confpath}, see ${scriptconf}." >&2
        exit 1
fi

if [ ! -d "${dpath}" ] ; then
  echo "${0}: ${dpath} is not present!" >&2
  exit 1
fi

if [ -e "${dconfpath}" ] ; then
  echo "${0}: please remove ${dconfpath} before you run this script.
${0}: NOTE: It may have been left by a previously run ${0}, but you should check anyway." >&2
  exit 1
fi

cat << 'EOT' > "${dconfpath}" && message "${dconfpath} created, header added"
# Configuration file automatically generated by the update-conf.d
# script.
#
# Please change the according lines in /etc/<conf.d>/* if you want
# them to be permanent, otherwise they will not survive the next
# invocation of update-conf.d!
#
EOT

for dconf_file in "${dpath}"/[0-9][0-9]* ; do
  echo "" >> "${dconfpath}"
  echo "# ${dconf_file}:" >> "${dconfpath}"
  grep '^[^#].*' "${dconf_file}" >> "${dconfpath}"
  message "Added: ${dconf_file}"
done

mv -f "${confpath}" "${bkpconfpath}" && message "Existing ${confpath} renamed to ${bkpconfpath}"
mv -f "${dconfpath}" "${confpath}" && message "New file system table ${dconfpath} moved to ${confpath}"

exit 0
Back to top
View user's profile Send private message
GuyPaddock
n00b
n00b


Joined: 18 Aug 2010
Posts: 2
Location: Rochester, NY

PostPosted: Wed Sep 14, 2011 7:50 pm    Post subject: Reply with quote

Excellent work, guys! I was just Googling for a solution to updating fstab dynamically, and this looks like exactly what I need.

One thing: In the generic script, I'm guessing that the line that reads:
Code:
mv -f "${dconfpath}" "${confpath}" && message "New file system table ${dconfpath} moved to ${confpath}"


Should read:
Code:
mv -f "${dconfpath}" "${confpath}" && message "New configuration file ${dconfpath} moved to ${confpath}"


... since it shouldn't be specific to fstab.
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Wed Oct 26, 2011 6:12 pm    Post subject: GitHub Reply with quote

I've created a GitHub repository for my update-fstab script.

https://github.com/Atha/fstab.d

You should be able to clone it with git like this:
git clone git://github.com/Atha/fstab.d.git

@NicoLarve: should I add your update-conf.d as well?
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Wed Oct 26, 2011 8:59 pm    Post subject: Makefile Reply with quote

Okay, I've added update-conf.d to the repository at GitHub. I also added a Makefile for easier installation.

@NicoLarve: I hope it was okay to upload your script to the repository.

Everybody, you're welcome to test it and give feedback:
Edit Makefile and set PREFIX=/tmp or any other directory you have write access to as a regular user. Copy your system fstab (as a test subject) to the PREFIX directory, like so: mkdir /tmp/etc ; cp /etc/fstab /tmp/etc/fstab – it will be used by the Makefile to install an initial /tmp/etc/fstab.d/00fstab. Then install it with make build ; make install and run the script (with the full path, because of the PREFIX=/tmp: /tmp/usr/local/sbin/update-conf.d) as you would normally do as root. You can also test the make uninstall on that PREFIX.

One thing may be a problematic approach: the automatic creation of /etc/fstab.d/00fstab and /etc/update-conf.d.conf. Because it will overwrite (without asking) any existing files. As I said: feedback please!
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Wed Oct 26, 2011 10:05 pm    Post subject: GitHub: update-conf.d Reply with quote

I renamed it to update-conf.d:

URL: https://github.com/Atha/update-conf.d
GIT: git clone git://github.com/Atha/update-conf.d.git
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Mon Jan 23, 2012 1:53 pm    Post subject: Re: Makefile Reply with quote

Atha wrote:
@NicoLarve: I hope it was okay to upload your script to the repository.

Yes, I'm 1000% ok with that.

PS: sorry for being long to respond, it seems the notifier failed to keep me informed of new posts in this forum.
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Mon Jan 23, 2012 1:55 pm    Post subject: Reply with quote

GuyPaddock wrote:
Excellent work, guys! I was just Googling for a solution to updating fstab dynamically, and this looks like exactly what I need.

One thing: In the generic script, I'm guessing that the line that reads:
Code:
mv -f "${dconfpath}" "${confpath}" && message "New file system table ${dconfpath} moved to ${confpath}"


Should read:
Code:
mv -f "${dconfpath}" "${confpath}" && message "New configuration file ${dconfpath} moved to ${confpath}"


... since it shouldn't be specific to fstab.

I think you'r right!
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Mon Jan 23, 2012 8:47 pm    Post subject: Re: Makefile Reply with quote

NicoLarve wrote:
Yes, I'm 1000% ok with that.

PS: sorry for being long to respond, it seems the notifier failed to keep me informed of new posts in this forum.

Nico, do you want write access to the repository? I can then send you the required GitHub key.

p.s. Glad you agree with it.

EDIT: http://help.github.com/linux-set-up-git/
Send me your public SSH key so I can add it to the GitHub repository for you to have access.


Last edited by Atha on Mon Jan 23, 2012 9:29 pm; edited 1 time in total
Back to top
View user's profile Send private message
NicoLarve
n00b
n00b


Joined: 01 Feb 2011
Posts: 6

PostPosted: Mon Jan 23, 2012 9:16 pm    Post subject: Re: Makefile Reply with quote

Atha wrote:
Nico, do you want write access to the repository? I can then send you the required GitHub key.
Yes, this could be really useful.
I think I found a bug in the section that writes the /etc/<conf> header.
Could you send it via pm or something else?
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Mon Jan 23, 2012 9:35 pm    Post subject: Re: Makefile Reply with quote

NicoLarve wrote:
Atha wrote:
Nico, do you want write access to the repository? I can then send you the required GitHub key.
Yes, this could be really useful.
I think I found a bug in the section that writes the /etc/<conf> header.
Could you send it via pm or something else?


http://help.github.com/linux-set-up-git/

Create your SSH key pair (password is really importaint to remember/write down somewhere!). Send me your public SSH key so I can add you to this GitHub repository. Then you’ll have full access and can correct/modify/improve/change/add whatever you see fit.

http://help.github.com/create-a-repo/ – under 2. commit your README you can see how to commit changes.

A PM should be enough, just be sure the SSH key is exactly as you have it in your id_rsa.pub.
Back to top
View user's profile Send private message
javeree
Guru
Guru


Joined: 29 Jan 2006
Posts: 453

PostPosted: Tue Mar 12, 2013 9:49 am    Post subject: Reply with quote

Would there be an ebuild for this that we can copy into an overlay ?

It is easy enough to execute the makes manually, but an ebuild -9999 would automatically get the latest version from the git.
I could not get en ebuild for git working.
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Thu Mar 14, 2013 9:50 pm    Post subject: Reply with quote

javeree wrote:
I could not get en ebuild for git working.

Me neither. But I will keep on trying. Time is limited at the moment.
javeree wrote:
Would there be an ebuild for this that we can copy into an overlay ?

I’m not very much into the overlay thing, although I do use layman.
javeree wrote:
It is easy enough to execute the makes manually, but an ebuild -9999 would automatically get the latest version from the git.

How do -9999 ebuilds keep themselves updated? (automatically)

Anyway, a good idea.
Back to top
View user's profile Send private message
javeree
Guru
Guru


Joined: 29 Jan 2006
Posts: 453

PostPosted: Fri Mar 22, 2013 3:35 pm    Post subject: Reply with quote

Quote:
How do -9999 ebuilds keep themselves updated? (automatically)

They don't. Portage only looks at the -9999 and always sees an unchanged ebuild, so it thinks there is no ebuild. You have to manually execute emerge -1 ebuild to rebuild with the latest git.

Below an ebuild I wrote that seems to work for me

Code:
# $Header$

EAPI=4
inherit git-2

DESCRIPTION="script for flexible /etc/<conf>.d configuration"
HOMEPAGE="http://forums.gentoo.org/viewtopic.php?p=6364143"
EGIT_REPO_URI="git://github.com/Atha/${PN}.d"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="x86"
IUSE=""

RDEPEND=""
DEPEND="${RDEPEND}"

DOCS="README USAGE"

src_prepare() {
        echo "patching the makefile with s:^PREFIX=:PREFIX=${D}: (required to prevent make to write outside the sandbox)"
        sed -i "s:^PREFIX=:PREFIX=${D}:" Makefile
        echo "patching the configuration to ensure it looks in /etc instead of in the sandbox"
        sed -i 's%@CONFIGDIR@%/etc%' update-conf.d.in

        echo "patching the executable to allow files in subdirectories (e.g. /etc/portage/make.conf.d)"
        sed -i 's:^dconfpath=.*:dconfpath="${dpath}/$(basename ${conf})":' update-conf.d
}
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 228

PostPosted: Fri Mar 22, 2013 7:00 pm    Post subject: ebuild Reply with quote

Looks great! Thanks!
(Though I have to confess that I didn’t find the time yet to test it…)

Is it okay if I add your ebuild to my GitHub repository?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum