Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How do you log changes of /etc files? My solution.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
teika
n00b
n00b


Joined: 19 Feb 2011
Posts: 68
Location: japan

PostPosted: Mon Jun 27, 2011 4:35 am    Post subject: How do you log changes of /etc files? My solution. Reply with quote

Hi. How do you log changes of /etc files? Or you don't keep logs?

I edit config files as a normal user, and manage them under git. Obviously you have to sync and set the mode correctly between /etc and git-repo. It's cumbersome to be the super user each time.

I recently wrote a python-3.2 script to automate. Its wrapper is in bash and named "SUC.sh", and it uses sudo to facilitate things. Usage:

Code:
$ SUC.sh add <args>     # Registers a file. Its filenames as normal user (called "local") and root ("remote"), and optionally the mode get recorded. Authentication is required.
$ SUC.sh rm  <file>     # De-registers a file. Authentication is required.
$ SUC.sh put <files>    # Installs local files to remote. Authentication is required.
$ SUC.sh get <files>    # The opposite of 'put'. Fetch the remote files. If any of them can't be read as a normal user, authentication is required.
$ SUC.sh status <files> # Reports if local and remote don't coincide, or the mode is not correct. Authentication is NOT required.
$ SUC.sh diff <files>   # Prints diff between local and remote. If necessary, authentication is required.


It's supplied with bash-completion. Obviously command names mimic git.

Now, are there better replacements?

Even if mine has some good points, I don't think it's good to publish my code which is personal and probably not secure enough. It'd be better if someone with skill designs one from scratch.

But FYI, codes are found at: https://gitorious.org/cheesy-etc-manager/pages/Home

## If you edit them as the super user, than "etckeeper" package may be a good solution, which is not in the portage tree. It saves the entire /etc under git or other vcs, with file mode and other care. One problem might be (actually I've never tried etckeeper though), however, that things get mixed; you probably want to be informed only on files of your concern, but when you upgrade many packages, you can't filter the info in git status / log.

Regards.


Last edited by teika on Tue Jun 28, 2011 12:39 am; edited 1 time in total
Back to top
View user's profile Send private message
avx
Advocate
Advocate


Joined: 21 Jun 2004
Posts: 2108

PostPosted: Mon Jun 27, 2011 2:50 pm    Post subject: Reply with quote

I basically have a wrapper around $EDITOR, which checks via `realpath` if file in question is located somewhere in /etc and if that's the case, creates a copy of the file in /etc/.etc and adds the date/time to it. Once the editor quits, it diffs both files and removes the copy if I didn't make changes.

It's not perfect, but it's quite easy to do.
_________________
++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.
Back to top
View user's profile Send private message
teika
n00b
n00b


Joined: 19 Feb 2011
Posts: 68
Location: japan

PostPosted: Tue Jun 28, 2011 12:53 am    Post subject: Reply with quote

If you know some vcs, I strongly recommend one. Since I started using svn, (of which use I forgot, fortunately, since switching to git.:) I can edit /etc files without fear of screwing my system. And when you're unsure how to fix a file, you often have to edit many times to get the right config. In that case, you only want to keep "correct" versions, and throw away intermediate ones. Vcs is also good in this aspect, too. (I know some people tend to forget to check in.)

Yep, git is difficult to learn, so I don't recommend it for a sole use of keeping /etc . But mercurial or bazaar may be easy...

# In my first post, I've added the link to my codes. I don't explain it, but running it without any arg will print the basic usage. Have fun guessing how to use. :P
Back to top
View user's profile Send private message
rogerx
n00b
n00b


Joined: 06 Apr 2004
Posts: 70

PostPosted: Sat Sep 10, 2011 8:00 am    Post subject: Reply with quote

I simply just do it the old fashion way:

# cp /etc/configfile /etc/configfile.20110909

or at the very least,

# cp /etc/configfile /etc/configfile.old


RSync backups are routinely done here on about a monthly basis, so if I really screw-up something (unlikely these days), I've usually got something I can look at if I do forget to backup a config file. During etc-update, I'm usually within a terminal multiplexer (GNU Screen), so I can easily toggle terminals while viewing etc-update stdout, compare diffs & copy files.
_________________
Roger
http://rogerx.freeshell.org/
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1569
Location: U.S.A.

PostPosted: Wed Sep 14, 2011 8:12 am    Post subject: Reply with quote

I just use dispatch-conf, which is like etc-update, except it also archives changed config files.
_________________
"The accumulation of all power, legislative, executive, and judiciary in the same hands...may justly be pronounced the very definition of tyranny."
--James Madison, Federalist 46
Back to top
View user's profile Send private message
rogerx
n00b
n00b


Joined: 06 Apr 2004
Posts: 70

PostPosted: Wed Sep 14, 2011 8:40 pm    Post subject: Reply with quote

Oh really???

dispatch-conf archives old config files!

(Putting this on my TODO list to learn!)

Many thanks for the info.
_________________
Roger
http://rogerx.freeshell.org/
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 10118
Location: Córdoba (Spain)

PostPosted: Wed Sep 14, 2011 9:38 pm    Post subject: Reply with quote

I just can't bother with anything like an RCS for /etc. Not at all. I just do a full backup via cron every day. The full backup takes not even one megabyte. You can argue it's a waste of space, I could also argue that RCSs do store a lot of crap just to track the lesser change you make in a file. Not that they don't have a use, but it's not for /etc in my opinion, unless you have an extremely complicated system and very special needs.

Those needing to save every file state could just use inotifywait to backup any modified file in /etc with a time stamp on it. It just takes a oneliner to do so.
_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
hujuice
Apprentice
Apprentice


Joined: 16 Oct 2007
Posts: 199
Location: Rome, Italy

PostPosted: Wed Nov 09, 2011 6:38 pm    Post subject: Reply with quote

I feel very interesting this reading on kerneltrap.org: http://kerneltrap.org/mailarchive/git/2008/8/9/2877724
(The mentioned "etckeeper" seems to be not integrated with portage.)

I'm in the "backup party" (my choice is for a daily rdiff-backup run, easy and effective).
IMO, git is a real gem for many tasks, but not for administrative operations.

Regards,
HUjuice
_________________
Who haven't a spine, should have a method.
Chi non ha carattere, deve pur avere un metodo.
Back to top
View user's profile Send private message
rogerx
n00b
n00b


Joined: 06 Apr 2004
Posts: 70

PostPosted: Thu Nov 10, 2011 1:37 am    Post subject: Reply with quote

I continue to use rsync to backup the entire system to an external firewire hard drive.

File: $HOME/bin/seagate-host3.sh
Code:

$ rsync --archive --one-file-system --hard-links --human-readable --inplace \
     --numeric-ids --delete --delete-excluded --progress \
     --exclude-from=/home/roger/etc/rsync-seagate-fa-exclude.txt \
     --include-from=/home/roger/etc/rsync-seagate-fa-include.txt \
     / /mnt/seagate-freeagent/localhost3-gentoo/


For /etc, I continue to use etc-update. If I edit the files with an editor (VIM), I then make a backup before writing to the original with a naming scheme of /etc/file_name.date. The following clipping from my .vimrc does this automatically:

File: $HOME/.vimrc
Code:

set backup
" Default backups go here and fallback is /tmp.
set backupdir=${HOME}/.vim/backup,${HOME}/tmp,${HOME}/tmp
" Using something like the following when root modifies an /etc file or
" when a user modifies their dot config files.
autocmd BufWritePre /etc/*,~/.* let &bex = '.' . strftime("%Y%m%d") | set backupdir=.
" The following adds a '~' to the filename.
"autocmd BufWritePre /etc/*,~/.* let &bex = '.' . strftime("%Y%m%d") . '~'


The above will create a /etc/fstab.20111109 after writing using VIM. (But as I think of this, VIM creates/saves the file just before writing, which then VIM is saving an already modified version -- so probably should somehow create/save a file using the original buffer before writing the changes?)


The git/cvs method for backing up /etc just seems labor intensive and tedious when compared to using rsync for a full monthly backup and routine file renaming using date, as editing /etc file happens only every other week. Unless your managing 10+ hosts, as I've managed 3 hosts here with the above.[/code]
_________________
Roger
http://rogerx.freeshell.org/
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1569
Location: U.S.A.

PostPosted: Thu Nov 10, 2011 3:33 am    Post subject: Reply with quote

The rsnapshot package is an easy way to do that and will maintain the file rotation for you at multiple intervals (you can have hourly, daily, weekly, and monthly backups concurrently). You can script such a thing yourself using rsync, of course, but they've already done it for you, and it's pretty efficient and featureful (I think it's written in Perl). All you have to do is create a config file.

I have used this to backup my /etc/ directory before, and it works well and is extremely efficient, only backing up what has changed and keeping all of the above in tens of megabytes of space. Since it uses hard links, restoring from any period of time is easy (each backup appears to be its own complete copy of /etc, despite there being no actual, physical redundancy). It will back up over the network too.

I continue to use this to backup various parts of my systems, including /etc, but I find that dispatch-conf is better for managing config file changes.
_________________
"The accumulation of all power, legislative, executive, and judiciary in the same hands...may justly be pronounced the very definition of tyranny."
--James Madison, Federalist 46
Back to top
View user's profile Send private message
smparkes
n00b
n00b


Joined: 21 Jul 2011
Posts: 8
Location: United States

PostPosted: Tue Jan 10, 2012 5:16 pm    Post subject: Reply with quote

I don't find tracking changes to be the biggest problem. For that, virtually any copy mechanism works great.

What I don't have any solution for (and I wonder if anyone does) is for merging upstream changes. For lots of flies, this is pretty straightforward. For a few, it's a hideous nightmare. And the few that it is a nightmare for, tend to be critical ones: bind, postfix, mail scanners, databases ...

With git, I can imagine a much better way to do this (and I wonder if anyone knows of anything along this line): in git, you can keep a branch that tracks upstream and then incrementally merge the changes between upstream updates into your master. This is different than etc-update, subversion, etc. in that because of the way git works, it knows, often, which upstream updates can be included without conflict and which can't. It does the ones is can and only asks about the ones it can't.

This would, for example, update the doc links in postfix, but leave my customizations unchanged.

In my dream world, a gentoo tool would automatically maintain that upstream branch and stage the changes between upstream revs and my master, allowing me to review them before committing them. And stage this outside of /etc so I could do all the reviews offline, commit them, and then just 'git pull' the changes into /etc.

I try to do this manually but it's error prone. I have to checkout the upstream branch, use etc-update to commit the upstream change, and then checkout master and merge in the changes. It's too easy to do the etc-update against master, which blows the whole model. And as mentioned above, doing this all on a live /etc is a bit unstable.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1569
Location: U.S.A.

PostPosted: Tue Jan 10, 2012 6:29 pm    Post subject: Reply with quote

dispatch-conf
_________________
"The accumulation of all power, legislative, executive, and judiciary in the same hands...may justly be pronounced the very definition of tyranny."
--James Madison, Federalist 46
Back to top
View user's profile Send private message
smparkes
n00b
n00b


Joined: 21 Jul 2011
Posts: 8
Location: United States

PostPosted: Tue Jan 10, 2012 6:40 pm    Post subject: Reply with quote

Yeah, I know about dispatch-conf. I used to use it but abondoned it a while ago (and I don't remember why ...)

git's branches and merges are a lot more powerful, if they could be harnessed ...
Back to top
View user's profile Send private message
someoneG
n00b
n00b


Joined: 26 Nov 2011
Posts: 13

PostPosted: Tue Jan 17, 2012 1:02 am    Post subject: git. Reply with quote

I use this script to store owner/permissions (as git only stores the exec bits) and run the git comands.

Code:
cd /etc
find /etc -not -type l -not -iwholename '*/.git/*' -exec stat {} -c "chmod %a \"%n\"; chown %U:%G \"%n\"" \; > /etc/.gitSetPerm.sh
chmod 700 /etc/.gitSetPerm.sh
git add .
git commit -a
git gc
/etc/.gitSetPerm.sh
chmod 700 /etc/.gitSetPerm.sh
chmod -R 700 /etc/.git


im sure it could be safer, userfriendlier and so on, but it does what i want and if i want to abort the commit, hitting CTRL + C numerous times after quitting the commit dialog does the trick.
So when i change stuff and think i am done, i simply run that script and write a commit message thats all - and if i forget then ill notice it the next time, when committing.

there were some moments that etcGit saved my ass.
also there were moments where etcGit mess'd up my system: my first run of git gc in /etc without having stored file owners/permissions and i ended up with anything being 664 (or 775 - as exec bits are stored in git) and owned by root:root ...

but nowadays i pretty much like that approach.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum