Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Planning Mercurial for /etc
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Thu Dec 10, 2009 9:54 am    Post subject: Planning Mercurial for /etc Reply with quote

Hello.

Could someone give me an advice which Mercurial-model of branches to use for this scenario:

I'd like to manage some /etc folders of different Gentoo work stations. Most of the periodical file changes by etc-update I need to perform equally on all machines, some files like /etc/make.conf or /etc/portage/packages.keywords are individual.

I think of having a separate branch for each machine. But how would I only pull /etc/conf.d/ntp-client or even whole /etc/conf.d/ from one branch to another? Or how would I merge only /etc/mask.conf using vimdiff because I like to have a new FEATURE or KEYWORD from any other make.conf?
Or would I better go for individual branches beside a common branch holding files common to all machines?

Other topics like Versioning /etc with subversion is safe? have already covered similar topics, but I rather would like to know the Mercurial way, since I begin to like that decentraliced version control system. Before I used to have SVN, then for dezentralizing that I used to set SVK on top.

By the way with regard to security what is to be said against private hosting on the free hoster bitbucket.org as long as no sensitive data like clear passwords of wpa_supplicant are tracked? I would need a kind of central main repository always available, since I would need changes of a host who is eventually down at the moment.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770


Last edited by Massimo B. on Tue Jan 05, 2010 11:46 am; edited 1 time in total
Back to top
View user's profile Send private message
tomk
Bodhisattva
Bodhisattva


Joined: 23 Sep 2003
Posts: 7221
Location: Sat in front of my computer

PostPosted: Fri Dec 11, 2009 10:22 am    Post subject: Reply with quote

Moved from Gentoo Chat to Other Things Gentoo.
_________________
Search | Read | Answer | Report | Strip
Back to top
View user's profile Send private message
c00l.wave
Apprentice
Apprentice


Joined: 24 Aug 2003
Posts: 264

PostPosted: Fri Dec 11, 2009 10:27 pm    Post subject: Reply with quote

I don't know if sharing configs across multiple hosts is practical. emerge will still generate ._cfg... files on upgrades and you would still need to remove them. If you are using different versions on different systems you may have configs that one version understands but an older one doesn't. You would have to keep track of that.

Concerning security I think it's a bad idea to push your configs to a public hoster (even if your repository is marked private it still could be accessed without your knowledge). Mercurial keeps a copy of ALL branches in ALL revisions around on EVERY host. If one host gets compromised, all of them are (taking the worst-case scenario) because all configs are known. Publishing passwords from wpa_supplicant & co. on external servers may also be an issue in terms of data privacy. Again, consider one host (or your bitbucket repository) hacked and all that data is known.

The big advantage of a DVCS is that it doesn't matter where you put your changes. You won't really need bitbucket in your case; you could also pull all changes to a flash drive or similar. As usual, you should pull on / push to the other hosts often.


Beside that, I've been working with Mercurial for about half a year now. Real named branches (hg branch) are being kept around forever unless you apply tools to your repositories to remove one, which inevitably causes a loss of synchronisation across the old repositories since it contains other changesets and cannot be simply pulled any longer (or it will fork up into another branch creating some mess in your repo). That's because Mercurial always appends metadata, it never changes or deletes old revisions. So it's usually better to have multiple repositories if these branch names won't be persistent (like feature branches in development). Unfortunately, you wouldn't be able to tell what branch belongs to which host if you use them this way. Also, pushing your changes creates multiple remote heads which are all unnamed:

Code:

... creating one repo, cloning 3 instances from it, making changes in those and (force) pushing back to the first one. Standing in the instance we pushed to:

$ hg merge
abort: branch 'default' has 3 heads - please merge with an explicit rev
$ hg branch
default
$ hg branches
default                        3:06bbc6ccfd32
$ hg heads
changeset:   3:06bbc6ccfd32
tag:         tip
parent:      0:b1dbb4b1b894
user:        ...
date:        Fri Dec 11 23:07:34 2009 +0100
summary:     added a line for host 2

changeset:   2:6f196f06003f
parent:      0:b1dbb4b1b894
user:        ...
date:        Fri Dec 11 23:07:22 2009 +0100
summary:     changed a line on host 3

changeset:   1:94dfa7a53352
user:        ...
date:        Fri Dec 11 23:07:51 2009 +0100
summary:     host 4


So in your case you would be better using real named branches, but if your hostnames will change at some time, you will still have the old ones around (or you have to edit one repository to have it renamed/removed, then cloning all hosts from that one again).

Code:

$ hg branches
otherhost                      8:2bedfcd329f3
host2                          7:3c13b0ba9749
yetanotherbox.com              5:7e82882e9665
mainline                       0:a98e2398c4aa (inactive)
$ hg branch
host2
$ hg heads
changeset:   8:2bedfcd329f3
branch:      otherhost
tag:         tip
parent:      6:d1232b856b0d
user:        ...
date:        Fri Dec 11 23:21:18 2009 +0100
summary:     some other one

changeset:   7:3c13b0ba9749
branch:      host2
parent:      4:5e6c80fc6b5d
user:        ...
date:        Fri Dec 11 23:21:29 2009 +0100
summary:     maybe now?

changeset:   5:7e82882e9665
branch:      yetanotherbox.com
parent:      2:e2c9427bd2d3
user:        ...
date:        Fri Dec 11 23:19:23 2009 +0100
summary:     yet another change

_________________
nohup nice -n -20 cp /dev/urandom /dev/null &
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Tue Aug 12, 2014 9:25 am    Post subject: Migrating hg to etckeeper Reply with quote

Hi, I'm tracking /etc and others for years now in hg. That means I have /.hg with some ignores to track /etc /usr/local/ and stuff in /home/mo/.local

Now even commit does change the permissions and owner of files, especially in /home/mo which belong to root afterwards. I guess sys-apps/etckeeper can track permissions and owners.

So how do I migrate an existing hg repository to etckeeper while keeping the configuration and history?
I did etckeeper init -d / which only appended stuff to my /.hgignore.

I checked /usr/share/doc/etckeeper-1.12/examples/bashrc:
I don't like to have automatic commits after emerge because I like to track changes after emerges using hg status.
So what is the exact workflow to keep tracking with etckeeper? What etckeeper commands are required around hg commit?
I guess this is
Code:
etckeeper pre-commit
hg commit
..which does store metadata in .etckeeper. What would be the other way round to restore metadata such as permissions?
And then how can the following hg commit be forced to not change permissions (I guess this happens at Keyword replacement).
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Fri Aug 22, 2014 6:36 am    Post subject: Reply with quote

I figured out that this is exactly what etckeeper has added automatically to my hgrc:
Code:
[hooks]
# pre-commit hook for etckeeper, to store metadata and do sanity checks
pre-commit = etckeeper pre-commit -d /
/etc/etckeeper/pre-commit.d/20warn-problem-files:

elif [ "$VCS" = hg ]; then
   special=$(find . ! -type d ! -type f ! -type l | exclude_internal) || true
   hardlinks=$(find . -type f ! -links 1 -exec hg status {} \; | exclude_internal ) || true

Too bad, this is done on / here and takes far too long. I guess I'm going to disable that or rather move the [ -z "$AVOID_SPECIAL_FILE_WARNING" ] earlier to avoid the find. But then 30store-metadata has the same problem and runs find on /.
The resulting /.etckeeper now includes all files in / eventhouth ignored.
My setup is based on / with an hgignore file like
Code:
syntax: glob
**
*.bac*
This allows to add files only explicitly without caring about untracked files.

I filed a bug to the bugtracker at Debian, I guess this is the only place where etckeeper development is tracked: #758854 - etckeeper pre-commit ignores .hgignore.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Mon Feb 23, 2015 2:36 pm    Post subject: Reply with quote

I started patching etckeper for my needs and for the mercurial branch only:
http://etckeeper.branchable.com/todo/Adding_support_for_.hgignore/

Remember, I was adding / to the repo, ignoring everything but some special folders, but etckeeper was never returning with find on / to find special files and permissions. So I patched etckeeper for only recording special files and permissions of files relevant for Mercurial, so considering the .hgignore.

BTW. is this some job that could be easily done with Salt as well? That is:
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Tue May 30, 2017 12:25 pm    Post subject: Reply with quote

Having the root filesystem tracked by /.hg makes some ebuilds breaking like dev-python/pygame-1.9.3:
Code:
 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-12232.log"
 *
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: open_wr
S: deny
P: /.hg/cache/.tags2-visible-wjYQbI
A: /.hg/cache/.tags2-visible-wjYQbI
R: /.hg/cache/.tags2-visible-wjYQbI
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i

F: symlink
S: deny
P: /.hg/wlock
A: /.hg/wlock
R: /.hg/wlock
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i

F: open_wr
S: deny
P: /.hg/wlock
A: /.hg/wlock
R: /.hg/wlock
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i

F: open_wr
S: deny
P: /.hg/cache/.tags2-visible-YYq7Rt
A: /.hg/cache/.tags2-visible-YYq7Rt
R: /.hg/cache/.tags2-visible-YYq7Rt
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i

F: symlink
S: deny
P: /.hg/wlock
A: /.hg/wlock
R: /.hg/wlock
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i

F: open_wr
S: deny
P: /.hg/wlock
A: /.hg/wlock
R: /.hg/wlock
C: /usr/bin/python2.7 /usr/lib/python-exec/python2.7/hg identify -i
 * --------------------------------------------------------------------------------

>>> Failed to emerge dev-python/pygame-1.9.3, Log file:

>>>  '/var/log/portage/dev-python:pygame-1.9.3:20170530-104536.log.gz'
Any way to hide the /.hg for subdirectories or make emerge compile in a chroot sandbox?
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Tue Sep 24, 2019 8:02 am    Post subject: Reply with quote

The solution for this issue would be https://bugs.gentoo.org/686468#c4.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum