Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Kuroo - remote management of Portage trees/packages (IDEA)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  

Remote Portage - bad idea, good idea?
bad!
14%
 14%  [ 2 ]
good!
50%
 50%  [ 7 ]
nice!
35%
 35%  [ 5 ]
*bump* :-)
0%
 0%  [ 0 ]
Total Votes : 14

Author Message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Tue Jul 24, 2007 10:46 am    Post subject: Kuroo - remote management of Portage trees/packages (IDEA) Reply with quote

Hi,

I'm active systems administrator and guess what - I'm using only Gentoo on my machines :-)

As maintaining more then twenty++ different Gentoo machines can be complex and sometimes hard I was wondering if I could use Kuroo for that, just like use it on my workstation.

But Kuroo needs X, a display manager, QT, and many other dependencies. I'm not going to install it on every little virtual server I use.

But what I'm using on all of my servers is ssh. So I was wondering if I could use Kuroo to maintain packages on remote link....

First idea was to do a chroot with X and Kuroo (so i do not b0rk my filesystem ) then mount remote filesystem through sshfs, replace all paths, adjust Portage configs and emerge into the directory... Maybe it could be done in that way, but it would be really much more complex than using Portage in usual way.

The idea: Isn't it better to implement Kuroo as a "console" to a server using Portage commands over ssh? Sth like Vmware Server Console, but limited to
Portage management.

Or in the future such feature could be included as a module in sth like "Landscape for Gentoo".

(landscape for Ubuntu: http://www.linux-watch.com/news/NS5288220442.html - I assume they're using ssh, this is what I would do).

How do you think guys?

Januszzz.


Last edited by Januszzz on Tue Jul 24, 2007 12:01 pm; edited 1 time in total
Back to top
View user's profile Send private message
Errtu
Apprentice
Apprentice


Joined: 12 Nov 2002
Posts: 155
Location: Brazil

PostPosted: Tue Jul 24, 2007 11:07 am    Post subject: Re: Kuroo - remote management of Portage trees/packages (IDE Reply with quote

Januszzz wrote:
How do you fell guys?


With an axe, but that gives a bit of a mess, so i usually just pretend they're not here ... oh.
Back to top
View user's profile Send private message
Clete2
Guru
Guru


Joined: 09 Aug 2003
Posts: 530
Location: Bloomington, Illinois

PostPosted: Tue Jul 24, 2007 7:36 pm    Post subject: Reply with quote

What's wrong with just using Portage to manage your systems? Why do you need the extra front-end?
_________________
My Blog
Back to top
View user's profile Send private message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Tue Jul 24, 2007 8:35 pm    Post subject: Reply with quote

Quote:
What's wrong with just using Portage to manage your systems? Why do you need the extra front-end?


Short answer is: because I have it 8)

I'm not very insane about not using Portage directly, but some people need and like to do it in visual way. That is why ie. Synaptics emerged, and we can see that Ubuntu community (canonical) plans to deliver other, better solution. Nice!

So I thought that Gentoo could have sth like that too. Perfect candidate is Kuroo which is perfectly usable (for me) and could be adapted to work online (maybe, I regret I cannot code enough to do it by myself). For me, when there is sth like sshfs, or in Konqueror, where one can manage virtually all remote filesystem it is surprising that we cannot manage Portage remotely.

Anyway - using Portage on every server and typing emerge -Dupv once a week or reading this in an cron output becomes boring routine, I would like to click - clack sometimes too! :twisted:

Especially connecting authorization with KDE Wallet can really shorten login time (well, I could always SSH use keys, but why do not implement both solutions?)

Januszzz.
Back to top
View user's profile Send private message
wyv3rn
Apprentice
Apprentice


Joined: 18 Aug 2005
Posts: 154
Location: USA

PostPosted: Tue Jul 24, 2007 8:42 pm    Post subject: Reply with quote

This is what Gentoo _really_ lacks. From a management POV, Gentoo doesn't not scale well in the enterprise. A centralized Gentoo/portage management console is needed.
Back to top
View user's profile Send private message
Clete2
Guru
Guru


Joined: 09 Aug 2003
Posts: 530
Location: Bloomington, Illinois

PostPosted: Tue Jul 24, 2007 8:49 pm    Post subject: Reply with quote

Oh, I see what you want. I must have misread your post. It would be nice to have a centralized management system. This would definitely create more appeal to businesses.
_________________
My Blog
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Tue Jul 24, 2007 9:02 pm    Post subject: Reply with quote

Canonicals landscape could be something to take a look at (maybe, just maybe) it has some flexible backend which can be adopted to Gentoos needs.
The feature list seems to be the perfect wet dream for any admin with a bigger number of Gentoo systems :)
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
AllenJB
Veteran
Veteran


Joined: 02 Sep 2005
Posts: 1285

PostPosted: Wed Jul 25, 2007 8:54 am    Post subject: Reply with quote

I'm sure there was work done before on mass deployment for Gentoo previously. I seem to remember a talk at Gentoo UK either last year or the year before. Can't remember who did it or what became of it. welp, edit_21, Uberlord or NeddySeagoon might be able to remember better.
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Wed Jul 25, 2007 5:18 pm    Post subject: Reply with quote

wyv3rn wrote:
This is what Gentoo _really_ lacks. From a management POV, Gentoo doesn't not scale well in the enterprise. A centralized Gentoo/portage management console is needed.

You can use distcc to use a cluster of machines to build packages, you can put the binaries on one (or more) machine(s) for distribution, you can setup cfengine to push (or pull) configuration files and packages to machines on your network. Basically all the ingredients to setup, configure and maintain lots of machines are just there for the taking.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Wed Jul 25, 2007 6:56 pm    Post subject: Reply with quote

Quote:
You can use distcc to use a cluster of machines to build packages, you can put the binaries on one (or more) machine(s) for distribution, you can setup cfengine to push (or pull) configuration files and packages to machines on your network. Basically all the ingredients to setup, configure and maintain lots of machines are just there for the taking.


I admit, tools you've mentioned are just fine...these tools we all know and we love...I could add to your list many more ie. gentoolkit, ccache, screen, ssh with keys authorization and much more... but:
1. distcc fails to compile some programs and needs exactly the same toolchain (or at least gcc as far as I know) so its useless because I want two or more gcc versions, besides it has much more problems (ie when one of your server NIC goes down while compilation you can have your whole machine down because of makeopts="-j40" & oom killer),
2. when I build binary versions of the packages I cannot be sure if those was build against EXACTLY the same libs because my servers are much different. So ie. when I compile KDE against older dbus/hal it will work on half of the computers on the other it will not. Another example is OpenLDAP which compiles with DB (recent change needed maintenance),
3. cfengine is - for me - not usable in complex network, as its maintenance can take much more time than in usual way,

Basically all of those programs are not what I would call "remote Portage console" which I think could be cool thing to use.

And any of the tools you've mentioned do reach the point of enterprise synergy, and in contrary Ubuntu initiative does. For me wyv3rn is right, Gentoo is hard for enterprise where you have to act quickly, and the reason is lack of management tools like the one I proposed.

I just proposed a tool enhancement; if we are stuck in console we would still have vi only and Kate, Quanta, Eclipse would be on other platforms only. Forgive me, but I use Gentoo not only on servers but on workstations too.

Januszz
Back to top
View user's profile Send private message
wyv3rn
Apprentice
Apprentice


Joined: 18 Aug 2005
Posts: 154
Location: USA

PostPosted: Wed Jul 25, 2007 7:28 pm    Post subject: Reply with quote

nixnut wrote:
wyv3rn wrote:
This is what Gentoo _really_ lacks. From a management POV, Gentoo doesn't not scale well in the enterprise. A centralized Gentoo/portage management console is needed.

You can use distcc to use a cluster of machines to build packages, you can put the binaries on one (or more) machine(s) for distribution, you can setup cfengine to push (or pull) configuration files and packages to machines on your network. Basically all the ingredients to setup, configure and maintain lots of machines are just there for the taking.


I use distcc heavily, run my own portage rsync mirror, etc. as you have said. As the next poster said, these also have limitations.

Pushing out binary packages where possible is also good but only works so long as all the machines have similar/same roles/hardware/etc. which is not always the case.

Of course I can/have written some scripts for remote management but they also have limitations and are not nearly as friendly as a centralized management GUI would be. A comprehensive centralized GUI interface to portage on remote machines would rock my world. Kind of like the way vmware-server-console is awesome for centralized management of remote vmware-servers.
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Wed Jul 25, 2007 8:22 pm    Post subject: Reply with quote

I run gentoo on a half-dozen machines, and keeping config files and packages synchronized is really a pain.

My current, half-assed, solution is to

1) keep all relevant config files (/etc/make.conf, /etc/portage/*, and various custom scripts and daemon settings) in a Mercurial repository, so its easy to make sure all machines have the same options set

2) make extensive use of "emerge -k" after building on a central machine (using distcc, of course)

3) pre-package kernels, so *nothing* has to be re-built on each machine (search for my recent posts to see the method and ebuild I use for this)

4) I either NFS a single portage tree, or I use a squashed image of it on all machines, depending on their connectivity


I wish there were a smoother way, especially since portage is so dog-slow with binary tbz2's. Maybe it would be slightly better if I *also* used one of those "run-this-command-on-all-machines" ssh tools.


Also, to simplify everything in this way, I am forced to take the union of all config options of all machines, which almost means I'm making my own i686 binary distribution... uggh. This almost makes me wish Archlinux wasn't so flakey and Ubuntu wasn't so inflexible so I could let someone else do all the work.
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Thu Jul 26, 2007 4:35 pm    Post subject: Reply with quote

Quote:
1. distcc fails to compile some programs and needs exactly the same toolchain (or at least gcc as far as I know) so its useless because I want two or more gcc versions, besides it has much more problems (ie when one of your server NIC goes down while compilation you can have your whole machine down because of makeopts="-j40" & oom killer),
Why do you need gcc on your servers anyway? Sorry, but this doesn't sound like a valid concern in a scenario of lots of servers. If you are talking about development machines that should have several different gcc version then still you don't need to have several gcc versions on your build machines. Distcc may break for some packages, so you find out along which packages these are and build these without distcc.

Quote:
2. when I build binary versions of the packages I cannot be sure if those was build against EXACTLY the same libs because my servers are much different. So ie. when I compile KDE against older dbus/hal it will work on half of the computers on the other it will not. Another example is OpenLDAP which compiles with DB (recent change needed maintenance),
You need KDE on your servers??? Whatever, you certainly can make sure what is installed on your systems. You are the one telling what should be installed, computers don't think for themselves. Build what you want and distribute to any machine you want it installed on.

Quote:
3. cfengine is - for me - not usable in complex network, as its maintenance can take much more time than in usual way,
cfengine came into existence to ease administration of networks consisting of many (heterogeneous) machines, so I very very much doubt administration becomes less of a burden without cfengine than with when the number of machines increases.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Fri Jul 27, 2007 7:22 am    Post subject: Reply with quote

Quote:
Why do you need gcc on your servers anyway? Sorry, but this doesn't sound like a valid concern in a scenario of lots of servers. If you are talking about development machines that should have several different gcc version then still you don't need to have several gcc versions on your build machines. Distcc may break for some packages, so you find out along which packages these are and build these without distcc.


Well, go on, be honest. Have you seen gentoo without gcc?? I've seen several times only, when I created some embedded devices by myself or based on GNAP :) EDIT: and I must say that some of my production servers are also build servers. But why bother, for me compiling time is not too much, I have usually the newest machines possible.

Quote:
You need KDE on your servers??? Whatever, you certainly can make sure what is installed on your systems. You are the one telling what should be installed, computers don't think for themselves. Build what you want and distribute to any machine you want it installed on. "


Yes of course I need KDE on servers. How I could replace it on a machines which are X terminal-over-vpn servers for 80 people? OK, Gnome could do, but my users get used to KDE... I mentionet OpenLDAP too, which is more common on servers.

Quote:
cfengine came into existence to ease administration of networks consisting of many (heterogeneous) machines, so I very very much doubt administration becomes less of a burden without cfengine than with when the number of machines increases


Well, that is right, but once again, I need portage management and my configs are usually different on machines. The same set of packages is much more productive (...of course I wont install postfix on X terminal server and X on mail server, but the core).

That is why assumption that remote Portage manager could ease the work. And - nothing offensive! - still you haven't said enything about a synergy in those products and sth managers call "value add" (hope my woody English won't twist it ;-))

Janusz
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum