Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Cloning whole installations on more than one similar machine
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 8:47 pm    Post subject: Cloning whole installations on more than one similar machine Reply with quote

Hi everybody,

I have read a nice thread about making a backup of the root partition to another partition and more like it. Now I am curious how to solve a situation like this:

Imagine an office with 30 exactly identical machines and a server running linux with a very big harddrive.

Now how to install 30 exactly identical machines without having to do the installation process more than once?

I want to install one machine from stage1, thus optimising everything according to the hardware present. Then I can make the necessary changes to config files, install additional applications and so on. Now I need to clone not just one partition but all partitions.

I know that Symantecs Norton Ghost can do that but I am looking for a way doing this without using commercial Windows software.

The process would be ideal like this:

    install one template machine from stage1 with everything that is needed
    clone that harddrive and save it to the server
    get the cloned harddrive image via the network and copy it to every machine


That last step would require the server to be running a server software that supplies the image to a client program which can be booted on every machine from a disk or a CD and then writes that image mercilessly on every disk.

Is this possible? I would like to make a discussion out of this thread. Perhaps this can evolve from a brainstorming to an actual walkthrough. Solving this problem would mean a major breakthrough of installing Gentoo (or any other Linux) on any number of similar machines, ideal for hardware pools in schools, universities and other organisations with the motivation to run Linux.

I am eager for your replies :-)

Tobias
Back to top
View user's profile Send private message
To
Veteran
Veteran


Joined: 12 Apr 2003
Posts: 1145
Location: Coimbra, Portugal

PostPosted: Wed Aug 20, 2003 9:11 pm    Post subject: Reply with quote

Hummm, if I had to do it I'll do it this way:
1. make a boot floopy disk from the first install
2. set up the particions and mount them
3. cp -n ( or cp -a can't remeber for sure the options to preserv the owners of files/dirs ) from each particion to the target partions
4. place the hard disk on the new computer, and boot from floopy
5. set up the boot loader

Problably there are easy ways, but that's Tó's way, ewps... the lame way :lol:


_________________

------------------------------------------------
Linux Gandalf 3.2.35-grsec
Gentoo Base System version 2.2
------------------------------------------------
Back to top
View user's profile Send private message
chardros
Tux's lil' helper
Tux's lil' helper


Joined: 18 Apr 2002
Posts: 136
Location: Andromeda

PostPosted: Wed Aug 20, 2003 9:16 pm    Post subject: Reply with quote

Simple option would be to just use dd. Once you get your first machine installed, just image the entire drive w/ dd, and upload the image to the server. Then, you could boot the other systems via the live cd and either ftp the image down, or mount the image's source via nfs or samba or something... then explode the image onto the new machine's hard disk. Something as simple as:

On the template machine, after complete, do:
dd if=/dev/hda of=/some/where/template_system.img

Then, on each identical system do:
dd if=/path/to/template_system.img of=/dev/hda

I've never actually done this, but it should work I'd think. Thoughts?
Back to top
View user's profile Send private message
bmichaelsen
Veteran
Veteran


Joined: 17 Nov 2002
Posts: 1276
Location: Hamburg, Germany

PostPosted: Wed Aug 20, 2003 9:23 pm    Post subject: Reply with quote

First of all, did you concider using just one installation image on the server and mount this via NFS from the clients - would be easier to update 8O :D etc... this only works for exactly the same hardware
For copying the whole drive you could use
Code:
dd if=/dev/hda of=somefileonserver

see the man-page of dd for more.
_________________
etc-proposals, your friendly config file updater!
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 9:24 pm    Post subject: not quite yet :-) Reply with quote

Hi Tó,

Not quite yet. Place yourself into an administrator who has to install a VERY big number of identical machines. You don't want to come near the machines with anything like a screwdriver to open every single machine. That takes too much time.

I am really looking for something like a Norton Ghost solution where you boot from a CD which starts a program that writes an diskimage to the target machine which it gets from a network machine. So if you want to install 50 PCs at the same time you only need one server with the server software and 50 CDs with the client software from which you boot. Then you can "install" 50 machines at a time.

The server doesn't even have to be running a special server software. If it has a nfs share where the image resides the bootable CD only needs to connect to that share and get the image from there.

Now the important question? Is there a tool which can make and write back images from whole harddisks which runs under Linux and is under an open license? I thought I have seen something like it on a Knoppix CD recently but I'm not sure anymore.

cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
bmichaelsen
Veteran
Veteran


Joined: 17 Nov 2002
Posts: 1276
Location: Hamburg, Germany

PostPosted: Wed Aug 20, 2003 9:27 pm    Post subject: Reply with quote

The problem with the dd approch is:
Where do you store the image? its hddd-sized and you cant store in on the one disc available (because you are imaging it) - so you need some kind of network filesystem to store the image.
Alternative: tar the whole setup and do the partitioning manually.
_________________
etc-proposals, your friendly config file updater!
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 9:30 pm    Post subject: we are getting close :-) Reply with quote

Hi bmichaelsen,

That already sounds good. But the trick is to save as much time as possible. So the ideal solution would be to boot from a CD on a fresh PC and just point some installer routine to a location and then it goes on automatically. That way you can run from machine to machine and you're done in a jiffy. You don't want to spend more then just a minute at one PC.

Maybe anybody else has thought about that and already started a GPL project, offering such an "installer" script. Otherwise it shouldn't be that hard to burn a modified live CD from some distribution which mounts a nfs share and starts a script automatically. Or should it?

cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
madchaz
l33t
l33t


Joined: 01 Jul 2003
Posts: 993
Location: Quebec, Canada

PostPosted: Wed Aug 20, 2003 9:32 pm    Post subject: Reply with quote

like was said before, dd will do that.

it's ment for that exact use.

norton ghost would also be something you can use if you want. The latest versions understand linux partitions, and if they don't you can use "exact image" that does a bite per bite copy.
_________________
Someone asked me once if I suffered from mental illness. I told him I enjoyed every second of it.
www.madchaz.com A small candle of a website. As my lab specs on it.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 9:35 pm    Post subject: cool signature :-) Reply with quote

Quote:
We are Pentium of Borg. Division is futile. You will be approximated.


btw, I love your signature :-)
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 9:39 pm    Post subject: dd it is then :-) Reply with quote

madchaz wrote:
like was said before, dd will do that.

it's ment for that exact use.

norton ghost would also be something you can use if you want. The latest versions understand linux partitions, and if they don't you can use "exact image" that does a bite per bite copy.


Does dd do an "exact image" bit per bit? And does it make an image of the whole disk or just a single partition?

Well, let's summarise until here:


    install that one template machine
    run dd over it and save the image onto the nfs share
    boot any fresh PC with a live CD and mount the nfs share
    write back the image(s?) from the server (with dd also?)


cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Wed Aug 20, 2003 9:59 pm    Post subject: dd manpage Reply with quote

I just ran the dd manpage and I really have to admit most of it is chinese to me. :roll:

Is there a comprendable introduction to dd?

regards,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
cyrillic
Watchman
Watchman


Joined: 19 Feb 2003
Posts: 7313
Location: Groton, Massachusetts USA

PostPosted: Wed Aug 20, 2003 11:23 pm    Post subject: Reply with quote

TobiWan wrote:
Now the important question? Is there a tool which can make and write back images from whole harddisks which runs under Linux and is under an open license? I thought I have seen something like it on a Knoppix CD recently but I'm not sure anymore.

It sounds like you are talking about partimage http://www.partimage.org
This is included on recent versions of KNOPPIX http://www.knopper.net/knoppix
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Thu Aug 21, 2003 8:41 am    Post subject: that's what I've been looking for :-) Reply with quote

Hi cyrillic,

that's what I've been looking for. Did you ever use it? I have just noted an entry in the KDE menu on the Knoppix CD that looked interesting by its name.

If it can produce an image and store it on an nfs share and recover it from there later, then what I described above shouldn't be a problem, should it?

cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Thu Aug 21, 2003 8:57 am    Post subject: partimage Reply with quote

In fact, that is exactly what I've been looking for:

Quote:
This utility can be used to install many identical PCs. For example, if you buy 50 PCs, with the same hardware, and you want to install the same linux systems on all 50 PCs, you will save a lot of time. Indeed, you just have to install on the first PC and create an image from it. For the 49 others, you can use the image file and Partition Image's restore function.
from the partimage website

Mmh. Has anybody here ever used it that way? Any experiences?

I guess I'll be playing with it a little at work.

cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Thu Aug 21, 2003 9:02 am    Post subject: bootable CD Reply with quote

:D They even offer a bootable, ready to use CD :lol:
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
blubber
Retired Dev
Retired Dev


Joined: 26 Apr 2002
Posts: 152
Location: Enschede, Netherlands

PostPosted: Thu Aug 21, 2003 9:14 am    Post subject: Reply with quote

Personally, I have a tar/bz2'ed image of all my computers on my server. When I want to reinstall one, for whatever reason, I just boot the livecd, reformat, and unzip the image. Works perfect, only thing I have to do is run grub, and maybe compile a new kernel. After that I just run emerge --deep -Uu world, at night to update the machine...
Back to top
View user's profile Send private message
gravis
Apprentice
Apprentice


Joined: 18 Apr 2002
Posts: 176
Location: Compiègne, France

PostPosted: Thu Aug 21, 2003 9:22 am    Post subject: Reply with quote

Hey all.

the GLIS team is working on the new version of GLIS (Gentoo Linux Install System). This new version will allow separation of gui and installation process.
The Gui's will create a config file needed for installation. Once you have this file, you will be able to reproduce on several machines.

more info on #glis on irc.freenode.net
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Thu Aug 21, 2003 9:30 am    Post subject: image versus emerging Reply with quote

gravis wrote:
Hey all.

the GLIS team is working on the new version of GLIS (Gentoo Linux Install System). This new version will allow separation of gui and installation process.
The Gui's will create a config file needed for installation. Once you have this file, you will be able to reproduce on several machines.

more info on #glis on irc.freenode.net


Thanks for the info. Though I guess an identical installation like you described would still mean, the machine would have to go through the proces of compiling everything. And that can take hours. The idea is to do the compilation just once and then transfer that installation to identical machines. Saves a lot of time ;-)

cheers,
Tobias
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
Stalione
Guru
Guru


Joined: 21 Apr 2002
Posts: 335

PostPosted: Thu Aug 21, 2003 4:26 pm    Post subject: Reply with quote

Gravice wrote
Quote:

the GLIS team is working on the new version of GLIS (Gentoo Linux Install System). This new version will allow separation of gui and installation process.
The Gui's will create a config file needed for installation. Once you have this file, you will be able to reproduce on several machines.


Sounds like Red Hat kickstart file idea. I am a big fan of that approach. At our datacenter I was able to setup a nice setup of PXE boot server to mass install RH linux on similar hardware. A customized install took less than 4 mins for each server. Of couse that was using binary packages. Also another software that might be interest to some of you is pfars (http://pfars.sourceforge.net). I was able to use this to replicate debian systems. The GLIS team might want to concider studying both (kickstart and pfars) approach and coming up with something even better.

I am a big fan of the PXE boot approach since most new nic cards are capable of doing PXE boot (network boot).

I myself am looking for a solution to clone my laptop gentoo install because I don't relaly like to go home and plug in the monitor and stuff to it. I was searchign for a way to be able to take my laptop, plug a network cable and start some sort of a sync process (probably rsync like pfars). and sync up my home desktop and work laptop. I might look into trying to make pfars do it for me.
Back to top
View user's profile Send private message
zeromaster
n00b
n00b


Joined: 02 Jun 2003
Posts: 7
Location: vienna,austria

PostPosted: Thu Aug 21, 2003 7:30 pm    Post subject: Reply with quote

i was thinking to buy a via epia as soe sort of all-in-one server with my configs(mozilla,..),dhcp,firewall,mp3,divx(both with database),router,game-server(cs,ut) and also remote access.i want to do this caus i have my pc as somewhat of a testmachine and i´m getting my friends old laptop and when my friends come to let them go in the internet and play multiplayer.for you it would be good to have mozilla and your configs and mails on a remotly accessable server.and these epias are small fanless and cheap.
Back to top
View user's profile Send private message
pmjdebruijn
Guru
Guru


Joined: 24 Jul 2003
Posts: 506
Location: Sittard, The Netherlands

PostPosted: Thu Aug 21, 2003 8:10 pm    Post subject: Reply with quote

I just installed my system.
Then booted from the livecd, mounted the partitions, did a chroot...
tar cvjpf backup.tar.bz2 .

I uploaded backup.tar.bz2 to a ftp server

then manually partitioned my new harddisks, formatted and mounted them.

I downloaded backup.tar.bz2 from a ftp server

tar xvjpf backup.tar.bz2

than reran grub

and I was done...

Bye,
DrZ
Back to top
View user's profile Send private message
pmjdebruijn
Guru
Guru


Joined: 24 Jul 2003
Posts: 506
Location: Sittard, The Netherlands

PostPosted: Thu Aug 21, 2003 8:11 pm    Post subject: Reply with quote

There is also this available:

Mondo Rescue
http://www.microwerks.net/~hugo/

Bye,
DrZ
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Thu Aug 21, 2003 9:51 pm    Post subject: Re: Cloning whole installations on more than one similar mac Reply with quote

TobiWan wrote:
Hi everybody,


I know that Symantecs Norton Ghost can do that but I am looking for a way doing this without using commercial Windows software.

Tobias


Quote:
Phrealon is a bootable linux CD based onSlackware Linux 8.0 designed to allow the easy imaging of multiple workstations. It utilizes the updcast set of Linux tools to accomplish this.


http://www.halfling.org/phrealon/

I have copied a Gentoo installation from one machine to serveral others with identical disks using it. It is very fast and simple to use.
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
CopterGuy85
n00b
n00b


Joined: 15 Aug 2003
Posts: 27

PostPosted: Thu Aug 21, 2003 10:09 pm    Post subject: Reply with quote

I'm working on a solution similar to this: I want to make a bootable system restore CD for my home computer. I mean just something I can stick in drive, boot the computer off of, run a script, and my system is back how I need it.

I did a complete reinstall of my system over the past couple of days, and after each major compile step (bootstrap, system install, and kernel creation) I used partimage to make an image of my system (and after the last step, also my boot) partition and save it to my other hard drive.

Hopefully today and tomorrow will be spent rigging up a basic root filesystem (that will fit in a 4MB initrd image) and using it to create a bootable CD with isolinux. Then I just need to attach partimage and the image files, maybe write a couple scripts, and I'll have a CD that can restore my system to whatever stage I want.

TobiWan, as soon as I get all this figured out, I'll post it on these boards somewhere. It seems to be the easiest way for me, though I'm not sure if it's exactly what you need. I wouldn't recommend dd though; even though your hardware is identical, it copies the entire drive/partition regardless of how much space is actually used. So if you use dd to make an image of a 40gb drive with only 4gb used, your image file will still be 40gb (you could get it smaller with compression, but you'd only get good results if you zeroed your drive first, my guess is). partimage copies only the used blocks on the partition, and will even gzip/bzip2 the image as it goes.
Back to top
View user's profile Send private message
TobiWan
Apprentice
Apprentice


Joined: 07 Jul 2003
Posts: 275
Location: Brussels, Old Europe

PostPosted: Thu Aug 21, 2003 10:40 pm    Post subject: great outcome :-) Reply with quote

Thanks a lot CopterGuy85,

this is exactly what I've been looking for.

I have done some playing around with partimage this afternoon and is really is great. Somebody ought to give that guy a good job :-)

And you're right: dd stores everything, even if it's empty. The only advantage dd has that it'll probably cope better with NTFS than partimage where NTFS is still "experimental".

I really would appreciate your script. My custom modification to it would be to include kernel support for nfs and samba as well some common network modules on the bootable CD. That way, you can rstore images from any samba or nfs share via the network.

This thread is leading exactly into the right direction. I am impressed :D
_________________
Killing for peace is like fucking for virginity.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum