Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Lab of machines
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
longshot
Tux's lil' helper
Tux's lil' helper


Joined: 04 Sep 2003
Posts: 90

PostPosted: Tue Dec 16, 2003 6:16 pm    Post subject: Lab of machines Reply with quote

Just wondering about people's experiences running Gentoo on a laboratory of machines. I've seen people mention they are doing this, but a search didnt seem to turn up relevant threads. I'd really like to hear from people who are actually running Gentoo on labs.

I understand that binaries can be compiled up and emerge --usepkg will use the binary if available. However, it seems the binaries need to be available on the filesystem. Can the binaries be pulled from a server via the network (without using nfs)? Other tools for managing a lab of machines?

We have a teaching laboratory of around 16 machines currently running Redhat 9 and we are evaluating which distribution we are going to upgrade to. Our main candidates are FC 1 and Mandrake 9.2.

Whilst I'm very happy with Gentoo on my desktops (work and home), Gentoo on the lab is probably a stretch for a number of reasons, including sysadmin perspective, and potential ease of installation and maintenance by students (with predominantly windows backgrounds) on home machines.
Back to top
View user's profile Send private message
Deathwing00
Bodhisattva
Bodhisattva


Joined: 13 Jun 2003
Posts: 4087
Location: Dresden, Germany

PostPosted: Tue Dec 16, 2003 8:03 pm    Post subject: Reply with quote

Not directly but indirectly about having Gentoo on labs...

Just told to a telecomunications 'important head' (won't reveal any personal information about this) about this... He is planning to put about 400 machines (mixed Intel and AMD) as servers like gateways, SMTP/POP3 and so on. We were discussing this 'small problem' that comes up when putting Gentoo under heavy production enviroinment, and we saw some viable ways, might they help you...

First: Installation --> There are 400 machines, but there are groups of them with the same specs, so what it could be done is to make a correct 'full install' in each different computer type, then make an image from that disk and inflate it to any other computers of the same type.

Second: Updates --> Updates are hard to perform every day... also cannot risk it to run in 'unstable mode' (i.e. enabling "arch"). The option of nfs with the binary packages is considered if the network is behind a gateway/firewall and so on.

Now, you plan NOT to use NFS (I suppose for security reasons...) well, you should choose one of your computers and make some kind of file server so other computers can reach it... use FTP or SFTP for example. Then all you have to do is as follows:

1) Make the compilation/installation/building in the main computer.
2) Program a script to be run periodically (using the cron daemon, for example) in order to fetch those built binary packages.
3) Fetch the portage tree either from your main computer or directly from gentoo servers. (This doesn't consume much system resources, so it's still viable).
4) Create a script that checks the recently got binary packages, then updates the system. In fact, unless you are very accurate with your scripts, you will be forced to have all binary packages in all of your computers.

Hope this helps :)
Back to top
View user's profile Send private message
Cerement
Guru
Guru


Joined: 14 Jun 2003
Posts: 404

PostPosted: Tue Dec 16, 2003 9:25 pm    Post subject: Reply with quote

Don't know how relevant this may be, but this thread has been around for awhile now ...
HOWTO: Central Gentoo Mirror for your Internal Network
Back to top
View user's profile Send private message
Deathwing00
Bodhisattva
Bodhisattva


Joined: 13 Jun 2003
Posts: 4087
Location: Dresden, Germany

PostPosted: Tue Dec 16, 2003 9:36 pm    Post subject: Reply with quote

Cerement wrote:
Don't know how relevant this may be, but this thread has been around for awhile now ...
HOWTO: Central Gentoo Mirror for your Internal Network


It might be... but our friend does NOT want to use NFS. :)
Back to top
View user's profile Send private message
deathbaz
n00b
n00b


Joined: 02 Oct 2002
Posts: 12
Location: NZ

PostPosted: Wed Dec 17, 2003 12:10 am    Post subject: Reply with quote

We've got a central binary repository and associated install and maintenance scripts ready to roll out to our labs and users soon. Orignally we thought it would be easy but the support for binaries in portage is an afterthought and there are a _lot_ of gotchas.

Using "emerge -G" along with setting the PORTAGE_BINHOST in your /etc/make.conf you can direct client machines to obtain all binaries from a FTP or HTTP server. You also need to sync your clients portage trees to a local tree that is in step with the binaries you provide.

What we found to be the big problem was packages dropping out of portage. When this occurs it breaks all of your binaries that depend on the particular package version that has dropped out. Since packages drop out of portage reasonably regularly this requires constant binary maintenance along with infrequent central portage updates (infrequent so you don't spend all your time upgrading packages).

I've written a python module that helps maintain a binary repository, it removes redundant packages (when you upgrade gnome for instance you'll have lots of old binaries that need cleaning out), advises you about binaries that have dropped out of portage (so you can add replacements before your client machines installs/updates start breaking) and generates a metadata.idx file from the binaries (this is used by clients when using "emerge -G"). For more info see https://bugs.gentoo.org/show_bug.cgi?id=34384

Theres also all the install scripts etc that we'll release at some point (after XMAS 8). The problem we've most recently found (and not yet fixed) is updating clients from the binary repository. Unfortunately "emerge -GU" or "emerge -Gu" won't work since the update flag is inherited by all of the target packages dependencies (a script to work around this is in progress).

So binary based gentoo can be done, its just a bit tricky.
Back to top
View user's profile Send private message
Deathwing00
Bodhisattva
Bodhisattva


Joined: 13 Jun 2003
Posts: 4087
Location: Dresden, Germany

PostPosted: Wed Dec 17, 2003 1:21 pm    Post subject: Reply with quote

deathbaz wrote:
We've got a central binary repository and associated install and maintenance scripts ready to roll out to our labs and users soon. Orignally we thought it would be easy but the support for binaries in portage is an afterthought and there are a _lot_ of gotchas.

Using "emerge -G" along with setting the PORTAGE_BINHOST in your /etc/make.conf you can direct client machines to obtain all binaries from a FTP or HTTP server. You also need to sync your clients portage trees to a local tree that is in step with the binaries you provide.

What we found to be the big problem was packages dropping out of portage. When this occurs it breaks all of your binaries that depend on the particular package version that has dropped out. Since packages drop out of portage reasonably regularly this requires constant binary maintenance along with infrequent central portage updates (infrequent so you don't spend all your time upgrading packages).

I've written a python module that helps maintain a binary repository, it removes redundant packages (when you upgrade gnome for instance you'll have lots of old binaries that need cleaning out), advises you about binaries that have dropped out of portage (so you can add replacements before your client machines installs/updates start breaking) and generates a metadata.idx file from the binaries (this is used by clients when using "emerge -G"). For more info see https://bugs.gentoo.org/show_bug.cgi?id=34384

Theres also all the install scripts etc that we'll release at some point (after XMAS 8). The problem we've most recently found (and not yet fixed) is updating clients from the binary repository. Unfortunately "emerge -GU" or "emerge -Gu" won't work since the update flag is inherited by all of the target packages dependencies (a script to work around this is in progress).

So binary based gentoo can be done, its just a bit tricky.


This is indeed interesting! Are your scripts under the GPL? ;)
Back to top
View user's profile Send private message
deathbaz
n00b
n00b


Joined: 02 Oct 2002
Posts: 12
Location: NZ

PostPosted: Wed Dec 17, 2003 9:54 pm    Post subject: Reply with quote

Deathwing00 wrote:
This is indeed interesting! Are your scripts under the GPL? ;)


They will be - hopefully it'll save someone else having to do it all over again. We'll also publish full instructions about what we did - basically pxe booting with a LTSP kernel, loading the initrd image (containing the auto_install script which partitions the HD, DHCP's then chroots and runs another install script that installs the kernel and the rest of it). Theres also a bit of postinstall configuration based on machine classes and the binary update scripts that keeps the machines in sync with the repository.

Definitely a job for after the holidays 8)
Back to top
View user's profile Send private message
Deathwing00
Bodhisattva
Bodhisattva


Joined: 13 Jun 2003
Posts: 4087
Location: Dresden, Germany

PostPosted: Wed Dec 17, 2003 10:34 pm    Post subject: Reply with quote

And you are right... I'll suggest your scripts to my friend in telecomunications... his 400 machines will be grateful ;)
Back to top
View user's profile Send private message
longshot
Tux's lil' helper
Tux's lil' helper


Joined: 04 Sep 2003
Posts: 90

PostPosted: Fri Dec 19, 2003 1:47 pm    Post subject: Reply with quote

Thanks for the info about the gotchas with binary installs and the forward notice about your scripts. We will most likely go with Fedor or Mandrake, at least until the next upgrade cycle (end of semester, perhaps end of year).
Back to top
View user's profile Send private message
ewan.paton
Veteran
Veteran


Joined: 29 Jul 2003
Posts: 1219
Location: glasgow, scotland

PostPosted: Fri Dec 19, 2003 1:53 pm    Post subject: Reply with quote

with lots of pc's you should also look into distcc and openmosix to shunt cpu load about
_________________
Giay tay nam | Giay nam cao cap | Giay luoi
Back to top
View user's profile Send private message
EvilTwinSkippy
n00b
n00b


Joined: 20 Feb 2003
Posts: 63
Location: Philadelphia, PA

PostPosted: Tue Dec 30, 2003 1:25 pm    Post subject: OPENMOSIX and DISTCC Reply with quote

Using both would be redundent.
<p>
I've had problems with openmosix because most of my tasks are IO intensive. (Mosix farming out samba threads is a disaster.) The rest of my servers never break a sweat, so at best it did nothing for me.
<p>
I've used DISTCC exclusively with good results. I can complete most builds in a few seconds. A rebuild of my entire system takes about 3 days. (With OpenOffice and KDE!)
<p>
I don't doubt that Mosix can help a great many people. But running it for garden variety network applications can lead to strangeness. Better to just deploy an overpowered box.
_________________
I've found that people will take what you say more seriously if you tell them Ben Franklin said it first.
Back to top
View user's profile Send private message
NewBlackDak
Guru
Guru


Joined: 02 Nov 2003
Posts: 512
Location: Utah County, UT

PostPosted: Tue Dec 30, 2003 3:56 pm    Post subject: Reply with quote

We don't have a lab, but do have 18 machines running that are basically just run OpenOffice, and a Reflection clone to interface with an HP-UX billing system. I basically don't upgrae unless there is a security issue, or the speed can be greatly improved. They're all running 2.6 no, because they were running the 2.4 kernel with the security flaw.
_________________
Gentoo systems.
X2 4200+@2.6 - Athy
X2 3600+ - Myth
UltraSparc5 440 - sparcy
Back to top
View user's profile Send private message
deathbaz
n00b
n00b


Joined: 02 Oct 2002
Posts: 12
Location: NZ

PostPosted: Fri Apr 30, 2004 5:48 am    Post subject: Reply with quote

deathbaz wrote:
Deathwing00 wrote:
This is indeed interesting! Are your scripts under the GPL? ;)


They will be - hopefully it'll save someone else having to do it all over again. We'll also publish full instructions about what we did - basically pxe booting with a LTSP kernel, loading the initrd image (containing the auto_install script which partitions the HD, DHCP's then chroots and runs another install script that installs the kernel and the rest of it). Theres also a bit of postinstall configuration based on machine classes and the binary update scripts that keeps the machines in sync with the repository.

Definitely a job for after the holidays 8)


The first phase describing the automated install process and binary repository management are done (along with the scripts). The instructions and code are in a draft form but should be pretty good.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum