Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Compiling by Proxy: Rsync Edition.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Wed Apr 27, 2005 6:32 am    Post subject: Compiling by Proxy: Rsync Edition. Reply with quote

Compiling By Proxy: Building Packages for a Slow Computer on a Fast One. Rsync Version.

Greetings,

Do you have a paper weight, a.k.a. a dinosaur, a.k.a. an old computer that is slower than dirt... and you want to use Gentoo on in (e.g. maybe for a router, or a web server for developement purposes)? But, you don't want to wait literally days to compile updates at a time? If you already have Gentoo installed on a newer (and hopefully, considerably faster!) computer, I have a trick that just might help. :)

The purpose of this Howto is to explain how you can configure a new computer to safely compile binary packages for a legacy (old) computer, and copy them over to the legacy computer over your network. This will save a HUGE amount of time, as the vast majority of your time doing emerges is sucked up by the compile time. Installing packages which could otherwise take overnight could be compiled and installed in an hour or two, depending on your setup.

What makes this Howto different from the last one is that, instead of using NFS, we will be using Rsync. Now, you might be familiar with rsync from using Portage; however, rsync is actually it's own, standalone app which can be used to syncronize any directory with another one. This is especially handy with backups (e.g., where there isn't a great deal of difference between the existing backup folder and the folder being backed up), or in our case making incremental changes to an entire system.

Of course, there are strengths and weaknesses to using this versus using NFS with prebuilt packages (like what we did here: https://forums.gentoo.org/viewtopic-t-319841.html). For instance, NFS is set up to continually serve out files between computers: while our setup was pretty secure, there is always that potential that someone finds a vulnerability in NFS. Rsync, however, when used between separate computers, sets up a temporary connection which is disconnected after it completes. And indeed, it can use ssh and thus adds an additional level of security. Also, we now need to maintain only one portage tree, which cuts down on administrative overhead. Unlike NFS, rsync comes with Gentoo, and does not need any special kernel options. This will make life easier if you are doing a fresh install of Gentoo on the old computer. However, rsync brings with it some overhead, so in theory the NFS version of this howto should always be faster. My recommendation is that you decide which would better suit your needs, based on these points.

Moving along, this howto assumes:
1) You have a faster computer, with plenty of RAM, a good processor speed, and at least a few Gigs of free space.
2) You have a legacy computer, which at the very least meets the minimum requirements to actually run Gentoo.
3) They have compatible architectures. Both are x86? you are good to go. amd64 and x86? that's my setup. :) x86 and something else... err you will want to check the docs for the fruity arch.
4) They are networked together... I figure this one was obvious. I'm using IP/ethernet.

Caveats:
A) If the computers are actually quite similar in terms of performance, doing distributed compiles would better suit your needs. See the Gentoo doc here: distcc (link)
B) Since we will be using only one portage tree, this Howto does not require a local rsync mirror for Portage. However, in case you still want this for your setup: local rsync mirror (link).
C) There are actually many ways to skin this cat. This HowTo is based on how i set this up myself... however if you see any steps you think you can skip, or else you can do something different that would be better for your setup, go ahead and deviate from this Howto!

1. Preparing the new, fast computer. We need to set aside some disk space on the newer computer. We will be using this to set up a parody environment... I'll explain the meaning of that in a bit. If you have free, unpartitioned space on your hard drive, set up a new partition and format it. 2-5GB seems to be a decent. My recommendation 5GB, or the size of the disk on the old computer plus 1GB for portage plus the ccache size you intend to use, whichever is smaller.

1a. Create your mount point. I use /mnt/g32. Use whatever is preferable for you:
Code:
# mkdir /mnt/g32


1b. Mount it and put it in your fstab:
Code:
# mount /dev/hda5 /mnt/g32


Code:
# nano -w fstab

...
/dev/hda5   reiserfs   noatime,notail      0 1


Shortly, we will be using this to create another install of Gentoo. This may seem strange, but let me explain. We will be rsyncing between your fast computer and your slow computer. However, you don't want to cripple your nice new rig with code that was compiled an old processor architecture, and the legacy computer will segfault like a fiend if you use code compiled for a newer arch. This is a bit of a pickle. To solve this problem, we will create a chroot environment on the new computer that is parallel to the Gentoo installation on the old computer. This nice thing about x86 (and amd64), the processors were made to be backwards compatible with software made for an earlier x86 processor, as far back as 386. That is why we can, say, compile for a Pentium 1 on a Pentium 4 system and not break your compiler. Note, this installation (hereafter, I'll call it the "parody" install) can also serve as a backup, in case Gentoo on the old computer gets b0rked or compromised.

Alternative to 1. If instead you don't have unpartitioned space or else don't care to use it for this purpose, just set up a folder on your system. Preferably, put it in a place you can remember.

2. Getting Started Before we go on, there is a fundamental question to answer: Do you arleady have Gentoo installed on the legacy computer, or are you doing a fresh install? The initial set up process will be different depending in how you answer.

2a. Fresh Install. Skip down to 2b if you are NOT doing a fresh install on the legacy computer. We won't go into detail about the install process, as most of it is the same as in the Gentoo Documentation (link).

2a(i). Boot the legacy computer using the LiveCD. Per the Gentoo instructions, configure your network and prep your disk(s). Mount your partitions as you want them set.

Alternative to 2a(i). If you cannot boot to CD, you can use the Slackware bootdisk method (https://forums.gentoo.org/viewtopic-t-33121-highlight-slackware.html). Use the stage3 tarball. Install through to the point where you chroot into the environment. Of course, chroot in.

2a(ii). Set the root password on the legacy computer. Note that we will lose this password, we just need it for ssh later on.

2a(iii). While you are doing the above, you can start installing Gentoo on the partition we set up on the faster computer. Since you already are running Gentoo (or at least some other form of Linux) on the new computer, you don't have to use a Live CD. Whatever installation process you choose, follow those instructions except instead of using the Live CD, set it up through the Linux which you are currently using on that box. (If I may make a suggestion, I recommend using the Stage1/3 NPTL install method {link}). By doing this on the fast computer, we will be saving many hours (days!?) over compiling the kernel and system files on the old computer.

Note, make sure you set the "-march" (or -mcpu) CFLAG to the proper archtecture for the legacy computer, and try not to get your make.conf file for the chroot environment confused with the make.conf for your regular Gentoo system! :oops: If you don't know what processor it is, try to find out, and if all else fails, i suggest defaulting back to 486. (And don't change your CHOST, no matter what.)

While you are at it, you might as well start installing whatever apps you want, like shorewall or apache or whatever. If you haven't, don't forget to use rc-update to add everything you need to default runlevel, and also set up your root password (just in case) and fstab file.

2a(iv). It turns out that you need rsync installed on both machines. The easiest way to get the legacy computer prepared for the process is to download and unpack a stage3 tarball onto it (we'll overwrite it soon, but we need it because the LiveCD currently doesn't have rsync). Then chroot in.

2a(v). On the legacy computer, start up sshd:
Code:
# /etc/init.d/sshd start
Also set up a root password:
Code:
# passwd


2a(vi). Are we ready to do our first rysnc? I think so. First, exit the chroot on the new computer. Second, this is the recommended command to rsync from the new computer to the old:
Code:
# rsync -azv --stats --progress --exclude=/proc/* --exclude=/usr/portage/* --exclude=/root/.ccache/* -e ssh [source location] root@[destination computer]:/mnt/gentoo/


Later I will explain all the options. For source location, enter the folder location parody install, and the destination should be either the IP of the legacy computer, or its computer name (if you defined one for it in your hosts file). Thus, this is what it would look like for on my configuration:

Code:
# rsync -azv --stats --progress --exclude=/proc/* --exclude=/usr/portage/* --exclude=/root/.ccache/* -e ssh /mnt/g32/ root@192.168.0.16:/mnt/gentoo/


It will ask for a password, and for this you use the root password you put in earlier. Important Note!!! make sure you don't forget the trailing slash on the source location! (i.e. /mnt/g32/ instead of /mnt/g32). Otherwise, you will copy the folder itself, which will mess things up a bit.

You will get a password prompt. Put in the password you set up earlier on the legacy computer.

2a(vi). You can now exit the chroot and reboot the legacy computer. If you set everything up right, it will boot without problems, and you can log in (the password it will use, however, will be the one you set up on the parody). If it doesn't boot right, use to LiveCD to chroot in and fix any problems.

2b. This step is for syncing the partition we created earlier with an existing install of Gentoo on the legacy computer. If you did the fresh install, Skip to step 3.

2b(i). Start sshd on the legacy computer:
Code:
# /etc/init.d/sshd start


2b(ii). Guess what, we are already set to rsync the parody install:

Code:
# rsync -azv --stats --progress --exclude=/proc/* --exclude=/sys/* -e ssh root@[source computer]:/ [destination location]


Again, we will shortly explain the options. Note that the source and destinations are different from the instructions in 2a. This is because we are going in the opposite direction: instead of copying files from the parody install to the legacy computer, we are copying files from the legacy computer to the parody install. Source computer is the IP or name of the legacy computer. With my configuration, that looks like this:

Code:
# rsync -azv --stats --progress --exclude=/proc/* --exclude=/sys/* -e ssh root@192.168.0.16:/ /mnt/g32/


Again, let's not make a mess: don't forget that trailing slash in the source location. You will get prompted for a password: enter the root password for your legacy computer.

3. Set up portage. As noted, we need to use only one portage tree. That means we can set it up so your regular Gentoo install on the fast computer, the parody install, and the old computer all share the same portage tree. While this is optional, it will save time in the long run.

3a. On your fast computer, in the regular environment, remove the portage folder. If you are nervous about this, you can back it up for now, and then delete it later. E.g.:
Code:
# mv /usr/portage /usr/portage.back


3b. Create a symbolic link to the portage folder in the parody install. If, for instance, you mounted the parody partition at /mnt/g32:
Code:
# ln -s /mnt/g32/usr/portage /usr/portage


And that is it. Regardless of whether you are in your regular environment or chrooted into the parody, you can emerge --sync once. On your regular install of Gentoo, the system will think it is using /usr/portage (even though it is actually using /mnt/g32/usr/portage). I have used this for a while, and I have not noticed any problems with this.

Since we rely will on the parody install to emerge applications for the old computer, technically the old computer won't even need portage. We will see how we can exploit that fact to save a bit of time ...

Finalle. You are done with the initial setup. Now, in case you are a bit confused, let me quickly reiterate what you just did. You created a parody installation on your newer computer, which is a replication whats on your older computer. Now, you are set to leverage the horsepower on your newer computer (which is a lot faster than the old one) to compile applications, libraries, the kernel, and system files for the old computer. Then, you can use rsync to "install" everythin you compiled and configured on the parody over to the old computer. Very cool? I think so. 8)

Now let's talk about rysnc some more. For reference, the manpage for rsync can be found here: http://www.hmug.org/man/1/rsync.html

For our purposes, here are some notable options:

-a --archive
This option will preserve almost all file attributes, and perform recursion through directories.

-z --compress
This will use compression for transport, saving bandwidth.

-v --verbose
Reports in the console every file that is transfered. Optional, but handy if you're easily amused by scrolling text ;)

--progress
Shows progress for each file. Optional.

--stats
Shows more statistics when rsync is complete. Optional.

-u --update
(Short for update only.) If rsync runs into a file that is NEWER in the destination location, it will not sync that file. If you have files that will change on the legacy computer (for instance, if you are using mysql or another database system), use this option so they don't get smashed.

--delete
If a file exists in the destination location that doesn't exist in the source location, this gives rsync the permission to delete it. Good if you want an exact replica between the parody and the old computer, but there are good reasons not to use this option.

--exclude=[file/folder]
rsync will not syncronize the specified file, or folder and its contents. Some folders, like portage, don't need to be synced and thus we'll use this flag to save time. In other cases, like /proc, we don't want to risk smashing up the system and thus we exclude these just in case.

-e --rsh=[remote shell]
Specifies remote shell to use, if needed. We use ssh.

Using some of these options, you can syncronize your legacy computer after everytime you do an emerge --update world on your parody install. This is a suggested rsync command (again, using my config):
Code:
# rsync -vazu --stats --progress --exclude=/proc/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/root/.ccache/* --exclude=/usr/portage/* -e ssh /mnt/g32/ root@192.168.0.16:/


Since we don't want to type that in everytime, Let's make it into a script:

# nano -w /usr/bin/sync-leg-comp
Code:
#!/bin/bash
rsync -vazu --stats --progress --exclude=/proc/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/root/.ccache/* --exclude=/usr/portage/* -e ssh /mnt/g32/ root@192.168.0.16:/


You can name it just about anything you want, but make sure the command is all one line. Set the script as an executable:
Code:
# chmod +x /usr/bin/sync-leg-comp


And now you can just type the script name and hit enter.
Code:
# sync-leg-comp


You can also rsync just a specific folder. For instance, let's say you changes some config files in /etc. But, you don't want to sync the whole system since it would take too much time. We-e-e-e-ell...
Code:
rsync -vazu --stats -e ssh /mnt/g32/etc/ root@192.168.0.16:/etc/

This doesn't take much time at all.

There you go. :)

Appendix

-- Please be sure to let me know in this thread if this works for you, or if you have any suggestions, or if you need help on anything.

-- Updates

5/14/05: updated instructions for the fresh install, because both machine apparently have to have rsync installed
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
davens
Tux's lil' helper
Tux's lil' helper


Joined: 07 Jul 2003
Posts: 107
Location: waterdown, ONT

PostPosted: Thu Oct 05, 2006 2:10 am    Post subject: success Reply with quote

FYI

this worked great for me! thanks so much.

installed gentoo 2006.0 on a compaq lte5000
pentium 75, 40M ram

only comment:rsyncing to legacy i used
Code:

# rsync -azv --stats --progress --exclude=/proc/* --exclude=/usr/portage/* --exclude=/root/.ccache/* -e ssh [source location] root@[destination computer]:/

(no /mnt/gentoo/ since im already chrooted in)

ive been able to boot into the system, just need to optimize and fix a few little things but looks promising,

thanks
_________________
you haven't really burned until you've burned from a CLI
Back to top
View user's profile Send private message
Motormouz
n00b
n00b


Joined: 05 Nov 2003
Posts: 5

PostPosted: Fri Oct 06, 2006 7:56 am    Post subject: Reply with quote

What about distcc? Wouldn't that be much simpler?!
Back to top
View user's profile Send private message
Mad Merlin
Veteran
Veteran


Joined: 09 May 2005
Posts: 1149

PostPosted: Thu Oct 12, 2006 2:36 am    Post subject: Reply with quote

Very clever idea. I like it even though I don't have a use for it at the moment. I must say I knew where you were going almost immediately though.
_________________
Game! - Where the stick is mightier than the sword!
Back to top
View user's profile Send private message
pdl
n00b
n00b


Joined: 06 Apr 2006
Posts: 42

PostPosted: Tue Mar 13, 2007 12:09 am    Post subject: An installation with no portage and no compiler? Reply with quote

I like this idea of proxy compiling. Can I save disk space on the slow box by doing ALL the compiling on the fast machine? Really all of it, so the slow box doesn't need to have a compiler installed at all? The fast machine could also be given the jobs of managing the portage tree [both storing and updating], downloading and storing the distribution files, and maybe more. I think I can arrange all this. My real question is, ''How much more can the slow box do without?'' In more detail, suppose I want to use my full-featured fast Gentoo rig to produce a system image for the slower box that will start out as a simple bash-executer, ogg-player, and X-terminal. How much of a standard Gentoo installation can I delete without breaking these few basic functions? Thanks!

Philosophically-minded readers may wish to ponder, ''What is the meaning of 'Gentoo System'?'' Can a compiler-less box deserve this name?
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Wed Mar 14, 2007 1:27 am    Post subject: Reply with quote

That is a really good question.

i know GCC is not essential if you aren't compiling anything on that box. probably the same thing for things GCC depend on. i just don't know enough to say much beyond that.

the biggest space saver from what i've seen is to share portage from the new machine rather than maintaining two portage trees. portage can take up around 1 to 3 gigs, and is easy to separate out like that.
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
spigot
n00b
n00b


Joined: 10 Jun 2004
Posts: 12

PostPosted: Wed Mar 14, 2007 9:51 pm    Post subject: Reply with quote

This is great. I've actually used this to compile for a laptop that's faster than my desktop, just because I don't want to leave my laptop on all the time, and I do most of my work on my desktop. It's very handy to explore a new package and just add it to the laptop proxy install so I get it on the next rsync.

I ran into an issue. I mount /proc and /dev in my proxy install before I chroot. (If I remember correctly it was to prevent some compiling problems.) I was installing a package in the proxy that wanted to update /etc/modprobe.conf, and it turns out it uses uname to do that. I'm assuming uname pulls it's info from /proc because it had the wrong kernel listed. (laptop version was newer, and it listed desktop version). I eventually solved the issue by running modules-update on the laptop, but will there be another situation where this comes up? Maybe it won't be uname, but something else?

Would it be worth setting up a fake /proc by using rsync from the laptop to the desktop? Would that cause other issues?

Thanks to Sheepdogj15 for putting all this together. It's quite handy.
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Mar 15, 2007 4:56 am    Post subject: Reply with quote

I don't recall ever needing proc.

Yea, i definitely don't recommend using your host system's /proc or /dev in the chroot. I don't know about setting up a fake proc. (maybe you can, i just don't know how offhand) one option, if you have NFS set up, is to mount the /proc from your legacy PC into the chroot directory as an nfs share. I don't know how well that'll work though, as not everything plays nice with NFS.

maybe you could copy over /proc, but i've found that things get flaky when you try to do something like that.
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
Stevendefeij
n00b
n00b


Joined: 07 Aug 2006
Posts: 10
Location: Koudekerk aan den Rijn

PostPosted: Mon Mar 19, 2007 9:39 pm    Post subject: Nice Guide I read you're other NFS one too. Reply with quote

Just one thing, I'm trying to run the Gentoo clone for mij laptop complete separate from the rest of my system (and making it easily transferable) by running it inside vmware.

However, I need to setup portforwarding from my host.

What port does rsync use by default or how do you force using a certain port on rsync?
Back to top
View user's profile Send private message
trofos
n00b
n00b


Joined: 07 Jun 2007
Posts: 1

PostPosted: Thu Jun 07, 2007 11:58 am    Post subject: Reply with quote

What a great idea. I installed on a Intel Pentium 133MHz, 32 RAM and it works perfectly. I, usually, use this old PC as an "MP3 player" and now it get back to life again.
PS- unfortunately old fans make too much noise.
_________________
online casino
Back to top
View user's profile Send private message
Stevendefeij
n00b
n00b


Joined: 07 Aug 2006
Posts: 10
Location: Koudekerk aan den Rijn

PostPosted: Thu Jun 07, 2007 7:15 pm    Post subject: Reply with quote

trofos wrote:
What a great idea. I installed on a Intel Pentium 133MHz, 32 RAM and it works perfectly. I, usually, use this old PC as an "MP3 player" and now it get back to life again.
PS- unfortunately old fans make too much noise.


Why don't you use an new heatsink from a pentium3 or athlon XP or something like that as a passive heatsink? It should be big enough to keep it cool without a fan.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum