Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Cloning A Gentoo Installation
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
fredbear5150
n00b
n00b


Joined: 11 Oct 2003
Posts: 51

PostPosted: Sun Sep 24, 2017 7:24 am    Post subject: Cloning A Gentoo Installation Reply with quote

I've cloned Gentoo installations on a number of occasions over the years and never really encountered any major problems until recently.

Usually, I use rsync - I start the source PC on the Gentoo Installation CD, I mount the partitions exactly as I would for a new installation (on /mnt/gentoo) and then I mount the backup hard disk on, say, /mnt/backup.

I then do:

Code:
cd /mnt/gentoo
rsync -avr --exclude="mnt" exclude="sys" exclude="proc" exclude="tmp" * /mnt/backup/


I then boot the Gentoo Installation CD, mount the partitions as above, and then do an rsync restore. I then recreate the /sys, /proc, and /mnt directories on the target PC, change the hostname etc. and boot it up. Oh, and install grub also.

On the target PC, I then notice that some files appearing to be related to glibc are missing. For example, when I do a kernel recompile:

Code:
tejat /usr/src/linux # make
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory
compilation terminated.
make[2]: *** [scripts/Makefile.host:107: scripts/basic/fixdep] Error 1
make[1]: *** [Makefile:450: scripts_basic] Error 2
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:105:23: fatal error: sys/types.h: No such file or directory
compilation terminated.
make[1]: *** [scripts/Makefile.host:107: scripts/basic/fixdep] Error 1
make: *** [Makefile:450: scripts_basic] Error 2


The kernel recompile on the source PC works fine, the implications are therefore that I've missed out backing up some important files.

This has happened at least twice to me now, there are no errors when I run the rsync backup and re-install.

I'm clearly missing something obvious here, but when I look up other pages about cloning Gentoo with rsync, they do exclude /proc, /sys, /mnt and /tmp.
Back to top
View user's profile Send private message
nokilli
Apprentice
Apprentice


Joined: 25 Feb 2004
Posts: 170

PostPosted: Sun Sep 24, 2017 7:42 am    Post subject: Reply with quote

You probably want to add the -A and -X switches to your rsync command unless you're not using ACL's or SELinux or whatever. How that would account for your errors I do not know but I do know I've been burned when copying my Gentoo installation in the past and that was the culprit in my case.
_________________
Today is the first day of the rest of your Gentoo installation.
Back to top
View user's profile Send private message
fredbear5150
n00b
n00b


Joined: 11 Oct 2003
Posts: 51

PostPosted: Sun Sep 24, 2017 7:58 am    Post subject: Reply with quote

I don't think that's the problem here.

I've just done a comparison where I have done a find for files named "types.h" on both systems, dumped the outputs to files and then done a diff on both - there is one difference:

/usr/include/sys/types.h

which is missing on the destination PC.

Investigating further, the entire /usr/include/sys directory is missing on the destination PC.

I am beginning to wonder if the problem is my use of rsync, and it is using "exclude=sys" as a pattern match across multiple directories, not just specifically for the "/mnt/gentoo/sys" that I want to exclude.

I will have to try the backup again.
Back to top
View user's profile Send private message
nokilli
Apprentice
Apprentice


Joined: 25 Feb 2004
Posts: 170

PostPosted: Sun Sep 24, 2017 8:25 am    Post subject: Reply with quote

I think you're right. I'm looking at my similar script and I've got the leading slash char...

Code:
rsync -aAXxv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/var/tmp/*","/lost+found/*"} $src $dst

_________________
Today is the first day of the rest of your Gentoo installation.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2252
Location: $TERM

PostPosted: Fri Sep 29, 2017 10:39 am    Post subject: Reply with quote

Code:
--exclude="mnt" exclude="sys" exclude="proc" exclude="tmp" *


Wrong

Wrong

Wrong

Wrong.

Everything named mnt, sys, proc, tmp will be excluded. You must use /tmp, /proc, /sys etc... see man rsync for details.

Using * will excluded hidden files.
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Pioneer, Kingston, WD, Yahoo, MSI, XFX, Huawei
My blog
Back to top
View user's profile Send private message
Sakarias
n00b
n00b


Joined: 12 May 2007
Posts: 46

PostPosted: Sun Oct 08, 2017 11:11 am    Post subject: Reply with quote

nokilli wrote:
I think you're right. I'm looking at my similar script and I've got the leading slash char...

Code:
rsync -aAXxv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/var/tmp/*","/lost+found/*"} $src $dst


Code:
 cp -a

Is very simple. I use Ubuntu install media. Then edit fstab and install grub.
_________________
---
Sakarias
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 11434

PostPosted: Sun Oct 08, 2017 4:59 pm    Post subject: Reply with quote

Using cp -a is simple, and also wrong because it will copy without regard to filesystem traversal, so unless you take care to exclude the various virtual filesystems, you copy things you should not. Once you take that care, it's no longer simple.
Back to top
View user's profile Send private message
Section_8
Guru
Guru


Joined: 22 May 2004
Posts: 547
Location: Arlington, TX, US

PostPosted: Sun Oct 08, 2017 10:19 pm    Post subject: Reply with quote

Maybe I'm misunderstanding this thread, but the OP said:
Quote:
Usually, I use rsync - I start the source PC on the Gentoo Installation CD, I mount the partitions exactly as I would for a new installation (on /mnt/gentoo) and then I mount the backup hard disk on, say, /mnt/backup.

If you've booted a live CD and are cloning your file systems on an HD, you don't need to worry about excluding /proc, /dev, /sys, etc, do you? I've migrated my system from drive to drive many times over the years doing this, and I've never worried about these directories.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 11434

PostPosted: Mon Oct 09, 2017 12:50 am    Post subject: Reply with quote

Sorry, I may have been a bit hasty. I read Sakarias' suggestion to use cp -a in isolation, forgetting the context higher up that assumes use of careful mounting to ensure that virtual filesystems are not an issue. I should have stated that my caveat applies in the general case, but not with the specific assumptions the original poster described. Far too often, I see threads of this form where someone recommends cp -a or similar when the source is a running system with the virtual filesystems mounted. In that case, it is wrong. In the specific case described by the original poster, it is safe.

On the other hand, if you assume there are no special filesystems mounted, then there is no reason to exclude mnt, sys, etc. because they will be empty. They only have undesirable content if the virtual filesystems are present.
Back to top
View user's profile Send private message
nativemad
Developer
Developer


Joined: 30 Aug 2004
Posts: 892
Location: Switzerland

PostPosted: Mon Oct 09, 2017 12:11 pm    Post subject: Reply with quote

dE_logics wrote:

Everything named mnt, sys, proc, tmp will be excluded. You must use /tmp, /proc, /sys etc... see man rsync for details.

Using * will excluded hidden files.

I vote for using even --exclude=/sys/* --exclude=/proc/* etc. That way it creates the empty dirs and /* is shorter than mkdir! :wink:
Also /run/* could be omitted nowadays.
_________________
Power to the people!
Back to top
View user's profile Send private message
Roman_Gruber
Advocate
Advocate


Joined: 03 Oct 2006
Posts: 3806
Location: Austro Bavaria

PostPosted: Mon Oct 09, 2017 1:50 pm    Post subject: Reply with quote

Well your question just reflects on how I do my backups.

bootup sysrescue-livecd

First time I use a new disc, else just format the filesystem: do the handbook work, partitions, bootloader, filesystems

cp -avr

Constraints => laptop -> external usb 3.0 SATA 2.5" case

--

becasue of uefi constraints, I have to redo te bootloader after swapping my discs.

I do not see any issues using cp -avr with sysrescue-livecd when you want to clone your gentoo installation.

--

cp is far simpler as those awesome utilies like rysnc, rdiffbackup and anything else the geeks suggests. cp just works and does it job
Back to top
View user's profile Send private message
nativemad
Developer
Developer


Joined: 30 Aug 2004
Posts: 892
Location: Switzerland

PostPosted: Mon Oct 09, 2017 1:56 pm    Post subject: Reply with quote

Roman_Gruber wrote:

cp is far simpler as those awesome utilies like rysnc, rdiffbackup and anything else the geeks suggests. cp just works and does it job

CP might be simpler, but needs the get the source machine offline! I occasionally clone running servers with rsync just fine! :wink:
_________________
Power to the people!
Back to top
View user's profile Send private message
nokilli
Apprentice
Apprentice


Joined: 25 Feb 2004
Posts: 170

PostPosted: Mon Oct 09, 2017 8:34 pm    Post subject: Reply with quote

nativemad wrote:
Roman_Gruber wrote:

cp is far simpler as those awesome utilies like rysnc, rdiffbackup and anything else the geeks suggests. cp just works and does it job

CP might be simpler, but needs the get the source machine offline! I occasionally clone running servers with rsync just fine! :wink:

Yes, one of the big reasons I run Linux is so that I don't have to reboot everytime I do something like changing my desktop background or setting the time. Unless I just recompiled my kernel or am using nvidia-drivers, rebooting should never be necessary.

We should probably mention though that backing up a running system implies root on lvm so than you can snapshot, and that implies your using an initramfs. So I think cp -a is totally defensible in the case where you're doing backups only every now and then. But is doing backups only every now and then defensible?

On my rig I can do a backup at any time, without rebooting... hell I could put it in a cron job and probably never notice. There was a time with Gentoo where that was mandatory. Can't really recall the last time I had to restore a backup to fix an emerge issue though. Still, how is it not a good idea to have your system set up to allow for easily performing backups?
_________________
Today is the first day of the rest of your Gentoo installation.
Back to top
View user's profile Send private message
nativemad
Developer
Developer


Joined: 30 Aug 2004
Posts: 892
Location: Switzerland

PostPosted: Tue Oct 10, 2017 5:32 am    Post subject: Reply with quote

nokilli,
sure, snapshots are the best option for fast backups. But that could also be a problem for databases and such, just as rsync can be troublesome with a running system.
But if you leave out the vivid data, then rsync is just fine. It is also nice if you do it multiple times as it just takes the changes.
The best way for properly backing up would probably be dumping the db, taking a snapshot, rsync the snapshot (they can get discarded) and tar the result.
Without LVM a backup can be just dumping, rsync & tar. The only difference is that the source-system needs to stand still a bit longer (at least on the first run this can take a while...).

But the question was pure (one time) cloning. Rsync got the advantage that you can use it through ssh and change the partition layout on the target host even with lvm in the mix. This is not easily possible with lvm snapshots! :wink:
_________________
Power to the people!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum