Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Use MySQL as a DB backend to Portage
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
arkane
l33t
l33t


Joined: 30 Apr 2002
Posts: 918
Location: Phoenix, AZ

PostPosted: Sat Jul 24, 2004 9:58 pm    Post subject: HOWTO: Use MySQL as a DB backend to Portage Reply with quote

I wanted to make sure this was in the tips'n'tricks section. I didn't write this, however I did make it work following the instructions provided exactly.

This was done by OneofOne.

http://forums.gentoo.org/viewtopic.php?t=175461&postdays=0&postorder=asc&start=0

Here's a copy just to make sure it's documented here.

Instructions:
1. emerge mysql and configure it (if you haven't already)
2. emerge mysql-python
3. /etc/init.d/mysql start
4. put this code in a file and do mysql -u root -p < file.sql

Here's the SQL code to use (file.sql):
Code:

CREATE DATABASE `portage`;
USE mysql;
REPLACE INTO user (Host,User,Password) VALUES ('localhost','portage','');
FLUSH PRIVILEGES;
GRANT SELECT,UPDATE,DELETE,INSERT,CREATE,LOCK TABLES ON portage.* TO portage@localhost;
FLUSH PRIVILEGES;
USE portage;
CREATE TABLE `path_table` (
  p_id int PRIMARY KEY AUTO_INCREMENT,
  name char(255) NOT NULL UNIQUE
);
CREATE TABLE `category_table` (
  c_id int(7) PRIMARY KEY AUTO_INCREMENT,
  name char(255) NOT NULL UNIQUE
) ;
CREATE TABLE `package_name` (

  name char(255) NOT NULL,
  cat_id  int NOT NULL,
  path_id int NOT NULL,
  data TEXT,
  FOREIGN KEY (cat_id) REFERENCES category_table (c_id) ON DELETE CASCADE,
  FOREIGN KEY (path_id)  REFERENCES path_table (p_id) ON DELETE CASCADE,
  #INDEX p_index (name,cat_id,path_id),
  primary key (name, cat_id, path_id)
) MAX_ROWS=99999999;



5. create /usr/lib/portage/pym/portage_db_mysql.py and put this in it :

Code:

# portage_db_mysql.db by oneofone{-a-t-}limitlessfx.com
import MySQLdb,cPickle

import portage_db_template

ex = MySQLdb.escape_string
class database(portage_db_template.database):
   #ought to do ref counting on the number of open connections.  course that's worthless till portage calls database.close().
   connections = {}
   cursor = None
   path_map = {}
   categories = {}
   defaultOptions = {'host':'localhost', 'port': 3306,  'user': 'portage', 'password': "", 'db': 'portage'}
   lastkey = lastval = None # internal caching to make get_values() work a little fast.
   global ex
   def __init__(self,path,category,dbkeys,uid,gid,config_path='/etc/portage/module_configs/'):
      try:
         portage_db_template.database.__init__(self, path,category,dbkeys,uid,gid,config_path)
      except:
         self.path     = path
         self.category = category
         self.dbkeys   = dbkeys
         self.uid      = uid
         self.gid      = gid
         self.config = {}
         self.module_init()
   def __del__(self):
      try:
         self.close()
      except:
         pass
   def module_init(self):
      options = database.defaultOptions.copy()
      if self.config:
         for x in self.config.keys():
            options[x] = self.config[x]
      self.config = options; del options
      from re import sub
      path = sub('/{2,}', '/', self.path)
      self.constr = self.config['host'] + ':' + str(self.config['port']) + '/' + self.config['db']
      if not database.connections.has_key(self.constr):
         try:
            con =  [0, MySQLdb.connect(host=self.config['host'], port=self.config['port'], user=self.config['user'], passwd=self.config['password'], db=self.config['db']) ]
            database.connections.setdefault(self.constr, con)
         except Exception, e:
            raise Exception, "Error connecting to Database using self.config=(%s): exception=(%s)" % (str(self.config), str(e))

      self.con = database.connections[self.constr][1]
      self.db = database.connections[self.constr][1].cursor()
      database.connections[self.constr][0] += 1
     
      try:
         self.db.execute('set AUTOCOMMIT=1')
         if len(database.categories.keys()) == 0:
            self.db.execute('select name,c_id from `category_table`')
            for y, x in self.db.fetchall():
               database.categories[y] = x
      except Exception, e:
         self.check_exception(e)
         raise Exception, "Database Error: failed pulling tables using self.config=(%s), exception=(%s)" % (str(self.config), str(e))
     
      if  not database.categories.has_key(self.category) :
         #create the category table
         try:
            self.db.execute('INSERT INTO `category_table` (`name`) VALUES("%s")' % ex(self.category))
            database.categories.clear()
            self.db.execute('select name, c_id from `category_table`')
            for y, x in self.db.fetchall():
               database.categories[y] = x
         except Exception, e:
            self.check_exception(e)
            raise Exception, "Database error: failed creation table for path('%s'), category('%s'), exception=(%s)" % (self.path, self.category, str(e))
      self.cat_id = database.categories[self.category]
     
      if not database.path_map.has_key(self.path):
         try:
            if len( database.path_map.keys()) == 0:
               if self.db.execute('SELECT name, p_id FROM `path_table` ') > 0:
                  for x,y in self.db.fetchall():
                     database.path_map[x] = y
         except Exception, e:
            self.check_exception(e)
            raise Exception, "database error: %s" % str(e)
         
         if not database.path_map.has_key(path):
            try:
               self.db.execute('INSERT INTO `path_table`(`name`) VALUES("%s")' % ex(path))
               #flush the table, and load it anew
               database.path_map.clear()
               self.db.execute('SELECT name, p_id  FROM `path_table`')
               for path,id in self.db.fetchall():
                  database.path_map[path] = id
            except Exception, e:
               self.check_exception(e)
               raise Exception, "Database error, failed loading path map: exception=%s" % str(e)
      self.p_id = database.path_map[path]
     
   def check_exception(self, e):
      if "Got error 127 from table handler" in str(e):
         if self.db.execute('check table `package_name`') > 1:
            print '`package_name` got corrupted some how, trying to fix'
            self.db.execute('repair table `package_name`')
            check = self.db.execute('check table `package_name`')
            status = self.db.fetchall()
            if ( check == 1 and status[0][3] == "OK"):
               raise Exception, "Database was corrupt be we were able to repair it, please rerun the emerge command"
            else:
               raise Exception, "Database is corrupt, exception=(%s)" % str(e)
     
   def has_key(self,key):
      self.check_key(key)
      try :   
         #self.db.execute('REPAIR TABLE `package_name`')
         return self.db.execute("SELECT data FROM `package_name` WHERE `name` =  '%s' AND `cat_id` = '%d' AND `path_id` = '%d'" %  (ex(key), self.cat_id, self.p_id )) > 0
      except Exception, e:
         self.check_exception(e)
         print "exception in has_key for Key (%s), Error (%s) " % (key, str(e))
         return False
     
   def keys(self):
      ks = {}
      try:
         self.db.execute("SELECT name, data from `package_name` WHERE `cat_id` ='%d' AND `path_id` = '%d'" % (self.cat_id, self.p_id ))
         for x,y in self.db.fetchall():
            ks[x] = cPickle.loads(y)
         return ks
      except Exception, e:
         self.check_exception(e)
         raise KeyError, "Keys are dead :( (%s)" % str(e)
   
   def get_values(self,key):
      self.check_key(key)
      if self.lastkey == key:
         return self.lastval
      try:
         if self.db.execute("SELECT data FROM `package_name` WHERE `name` =  '%s' AND `cat_id` = '%d' AND `path_id` = '%d'" %  (key, self.cat_id, self.p_id )) > 0:
            one = self.db.fetchone()
            if len(one) > 0 and one[0] != None:
               self.lastkey = key; self.lastval =  cPickle.loads(one[0])
               return self.lastval
            return None
      except Exception, e:
            self.check_exception(e)
            raise ValueError, "Value error (%s)" % str(e)
      return None
   
   def set_values(self,key,val):
      self.check_key(key)
      try:
         if self.lastkey == key: self.lastkey = self.lastval = None
         self.db.execute("""REPLACE INTO `package_name` (`name`,`cat_id`, `data`, `path_id`) VALUES ('%s','%d', '%s','%d') """ %  (ex(key), self.cat_id, ex(cPickle.dumps(val)),self.p_id ) )
      except Exception, e:
         self.check_exception(e)
         raise Exception, "Error inserting/updating the database (%s)." % str(e)
         
   def del_key(self,key):
      try:
         if self.lastkey == key: self.lastkey = self.lastval = None
         return (self.db.execute('DELETE FROM `package_name` WHERE `cat_id` = "%d" AND `name`= "%s" AND `path_id` = "%d"' % (self.cat_id, key, self.p_id)) > 0)
      except Exception, e:
         self.check_exception(e)
         print "Error deleting key=(%s), exception= (%s)" % (key,str(e))
         return False
         
   
   def sync(self):
      pass
   
   def close(self):   
      if database.connections.get(self.constr, [0])[0] == 0: return 0 # make sure that .close() doesn't get called twice
      try:
         #print 'in self.db.close(), instcount=%d' % database.connections[self.constr][0]
         database.connections[self.constr][0] -= 1
         self.db.close()
         if database.connections[self.constr][0] == 0:
            #self.db.execute("analyze TABLE `category_table` , `package_name` , `path_table`") # optimize the db on close
            database.connections[self.constr][1].close()
            del database.connections[self.constr]
            return 1
      except Exception, e:
         self.check_exception(e)
         print "Exception in self.db.close() == (%s), ignored." % str(e)
     


6. Create /etc/portage/modules and add :

Code:

portdbapi.auxdbmodule="portage_db_mysql.database"
eclass_cache.dbmodule="portage_db_mysql.database"


7. emerge regen (you dont have to, it'll update the cache when you do emerge -s or something)
8. enjoy!
btw portage won't work if unless mysql is started (of course you can always delete /etc/portage/modules and it'll use the default flat file db system).
This is provided with no warrant, if the computer blows up DON'T BLAME ME!
Credits to everyone on #gentoo-portage (specially carpaski and ferringb) for helping me optimize it and to joecool and redeeman (#love-sources) for testing.
the module : http://oneofone.limitlessfx.com/gentoo-stuff/portage_db_mysql.py
the sql file : http://oneofone.limitlessfx.com/gentoo-stuff/init-portage-db.sql
Back to top
View user's profile Send private message
chipi
n00b
n00b


Joined: 05 Oct 2003
Posts: 62
Location: Israel

PostPosted: Sun Jul 25, 2004 5:59 am    Post subject: Reply with quote

Neat trick! thanks.
Back to top
View user's profile Send private message
John5788
Advocate
Advocate


Joined: 06 Apr 2004
Posts: 2099
Location: 127.0.0.1

PostPosted: Sun Jul 25, 2004 6:57 am    Post subject: Reply with quote

does this make portage faster using a mysql db?
_________________
John5788
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Sun Jul 25, 2004 4:21 pm    Post subject: Reply with quote

John5788 wrote:
does this make portage faster using a mysql db?


I just made the switch and verified that it was working by stopping the MySQL server and seeing what complained.

On my AMD64 3000+ (in 32-bit Linux) with 512 MB RAM I have not noticed any great speed increase, if any at all. Searching seems to take the same amount of time (i.e. forever). The only difference I can see is that now portage requires a MySQL server to be running to work at all. Not worth the trouble, in my opinion.
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
Torin_
Tux's lil' helper
Tux's lil' helper


Joined: 05 Apr 2004
Posts: 114
Location: [PL]Gdynia

PostPosted: Sun Jul 25, 2004 9:26 pm    Post subject: Reply with quote

Hi. I have done all with instructions.
And i've got this errors ? Anybody knows what's are these?
Code:

root@deception /usr/lib/portage/pym # emerge regen
Traceback (most recent call last):
  File "/usr/bin/emerge", line 14, in ?
    import portage
  File "/usr/lib/portage/pym/portage.py", line 6141, in ?
    portdb=portdbapi(settings["PORTDIR"])
  File "/usr/lib/portage/pym/portage.py", line 4424, in __init__
    self.eclassdb = eclass_cache(self.mysettings)
  File "/usr/lib/portage/pym/portage.py", line 4326, in __init__
    self.dbmodule = self.settings.load_best_module("eclass_cache.dbmodule")
  File "/usr/lib/portage/pym/portage.py", line 1305, in load_best_module
    return load_mod(best_mod)
  File "/usr/lib/portage/pym/portage.py", line 34, in load_mod
    mod = __import__(modname)
  File "/usr/lib/portage/pym/portage_db_mysql.py", line 2, in ?
    import MySQLdb,cPickle
ImportError: No module named MySQLdb

I have password for root in mysql is this it ?
_________________
http://www.torin.biz/
Back to top
View user's profile Send private message
xces
Guru
Guru


Joined: 11 Oct 2002
Posts: 515

PostPosted: Sun Jul 25, 2004 10:19 pm    Post subject: Reply with quote

Torin_ wrote:
Hi. I have done all with instructions.
And i've got this errors ? Anybody knows what's are these?

You probably left out 'emerge mysql-python' since the error says it couldn't find the appropriate module.
Back to top
View user's profile Send private message
Torin_
Tux's lil' helper
Tux's lil' helper


Joined: 05 Apr 2004
Posts: 114
Location: [PL]Gdynia

PostPosted: Sun Jul 25, 2004 10:45 pm    Post subject: Reply with quote

i'm so stupid.
i'm going sleep.
thx for help :)
_________________
http://www.torin.biz/
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Mon Jul 26, 2004 12:56 am    Post subject: Reply with quote

AllTom wrote:
On my AMD64 3000+ (in 32-bit Linux) with 512 MB RAM I have not noticed any great speed increase, if any at all.


Actually, I'd like to know if anyone else can verify this. Should using MySQL be speeding things up? Even a little bit?
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
chipi
n00b
n00b


Joined: 05 Oct 2003
Posts: 62
Location: Israel

PostPosted: Mon Jul 26, 2004 1:55 am    Post subject: Reply with quote

Actually, after giving it some tests - I too can't notice any speed improvments at all :(
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Mon Jul 26, 2004 3:59 am    Post subject: Reply with quote

One would think that the whole job of searching would be passed onto MySQL but I guess that is not the case...
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
bravecobra
Tux's lil' helper
Tux's lil' helper


Joined: 26 Dec 2002
Posts: 130
Location: Planet Earth (sometimes)

PostPosted: Mon Jul 26, 2004 4:46 am    Post subject: Reply with quote

AllTom wrote:
Searching seems to take the same amount of time (i.e. forever).

Try esearch
_________________
Brave Cobra
http://www.bravecobra.com
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Mon Jul 26, 2004 4:57 am    Post subject: Reply with quote

bravecobra wrote:
AllTom wrote:
Searching seems to take the same amount of time (i.e. forever).

Try esearch


I would, but now that I've started using packages.gentoo.org and gentoo-portage.com I haven't needed to search from the command line from a while. :) Thanks, though.
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
msimplay
Apprentice
Apprentice


Joined: 20 Jun 2004
Posts: 161

PostPosted: Mon Jul 26, 2004 8:20 am    Post subject: Reply with quote

what are the benefits to doing this modification anyway ?
Back to top
View user's profile Send private message
Torin_
Tux's lil' helper
Tux's lil' helper


Joined: 05 Apr 2004
Posts: 114
Location: [PL]Gdynia

PostPosted: Mon Jul 26, 2004 8:26 am    Post subject: Reply with quote

Try doing emerge -s something.
In my opinion it speeds the operation many times.
_________________
http://www.torin.biz/
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Mon Jul 26, 2004 11:55 am    Post subject: Reply with quote

Torin_ wrote:
Try doing emerge -s something.
In my opinion it speeds the operation many times.


That's the thing. I haven't noticed any difference. Perhaps this only benefits slow(er) computers?
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
arkane
l33t
l33t


Joined: 30 Apr 2002
Posts: 918
Location: Phoenix, AZ

PostPosted: Tue Jul 27, 2004 6:50 pm    Post subject: Reply with quote

AllTom wrote:
Torin_ wrote:
Try doing emerge -s something.
In my opinion it speeds the operation many times.


That's the thing. I haven't noticed any difference. Perhaps this only benefits slow(er) computers?


I have it running on my 1.8ghz P4 (512mb ram), and my 700mhz P3 laptop. It speeds up the emerge -s searches by a large margin.

It's worth trying out. I'd give it a serious workout.. first doing the "emerge regen" to regenerate all the stuff.

do a "time emerge -s perl" with and without this alteration.
Back to top
View user's profile Send private message
AllTom
Tux's lil' helper
Tux's lil' helper


Joined: 26 Nov 2003
Posts: 147
Location: Wherever danger lies...

PostPosted: Tue Jul 27, 2004 11:54 pm    Post subject: Reply with quote

arkane wrote:
AllTom wrote:
Torin_ wrote:
Try doing emerge -s something.
In my opinion it speeds the operation many times.


That's the thing. I haven't noticed any difference. Perhaps this only benefits slow(er) computers?


I have it running on my 1.8ghz P4 (512mb ram), and my 700mhz P3 laptop. It speeds up the emerge -s searches by a large margin.

It's worth trying out. I'd give it a serious workout.. first doing the "emerge regen" to regenerate all the stuff.

do a "time emerge -s perl" with and without this alteration.


Well I followed the directions exactly and verified that portage is, in fact, using MySQL. I did the following test and, as you can see, I didn't notice any difference.

# emerge -s perl
# time emerge -s perl
real 0m2.398s
user 0m1.645s
sys 0m0.128s
# mv /etc/portage/modules /etc/portage/modules-goaway
# /etc/init.d/mysql stop
# emerge -s perl
# time emerge -s perl
real 0m2.344s
user 0m1.672s
sys 0m0.138s

So then I tried using -S for searching the descriptions (which takes a lot longer). This was right after the previous test with nothing else running. First was without MySQL, then with MySQL.

# emerge -S perl
# time emerge -S perl
real 1m16.150s
user 1m8.357s
sys 0m4.828s
# mv /etc/portage/modules-goaway /etc/portage/modules
# /etc/init.d/mysql start
# emerge -S perl
# time emerge -S perl
real 1m24.071s
user 1m9.615s
sys 0m4.243s

So what can I conclude from this? That I did something wrong? That reading from my filesystem is faster than MySQL? That using MySQL as a backend slows things down?

If I were one of the people to notice a speed-up in searching, would that also speed up calculating dependencies? That's really what I want this for. emerge world -vuDp takes a while.
_________________
rtylershaw: "My computer doesn't even work and I love this distro. Weird."
Back to top
View user's profile Send private message
Lews_Therin
l33t
l33t


Joined: 03 Oct 2003
Posts: 657
Location: Banned

PostPosted: Sat Aug 28, 2004 6:42 pm    Post subject: Reply with quote

Out of curiosity...if the mysql server was running on another (faster) machine, could this be used to greatly increase the speed of portage on a slow machine? For example, on my desktop "emerge -pv $PACKAGE" is done before I can respond, but takes >15 seconds for the same package on my server (P166). Would creating a mysql server on my desktop and hooking this script up to it on my server increase performance?
Back to top
View user's profile Send private message
duderonomy
Guru
Guru


Joined: 20 Mar 2004
Posts: 339
Location: SF Bay Area

PostPosted: Sun Aug 29, 2004 3:54 am    Post subject: Reply with quote

I would think that the time spent crunching in python would be the
bottleneck... Does anyone know how to profile this code in python?
That would certainly yield the information we are interested in. :)

Cheers,
D
Back to top
View user's profile Send private message
NewBlackDak
Guru
Guru


Joined: 02 Nov 2003
Posts: 512
Location: Utah County, UT

PostPosted: Mon Aug 30, 2004 12:46 am    Post subject: Reply with quote

I just tried this on my LatitudeCPiA(PII-400, 192MB Ram), and my desktop rig(see sig).
What I noticed is on the desktop it seems unchanged.
On the laptop(very slow disk, 4200RPM) using MySQL is faster.

duderonomy wrote:
I would think that the time spent crunching in python would be the
bottleneck... Does anyone know how to profile this code in python?
That would certainly yield the information we are interested in. :)

Cheers,
D

This is prolly part of it. If I actually search the DB it is MUCH FASTER than doing an emerge -S. If the portage code was updated then this could make searching portage act like a whole new animal.
_________________
Gentoo systems.
X2 4200+@2.6 - Athy
X2 3600+ - Myth
UltraSparc5 440 - sparcy
Back to top
View user's profile Send private message
chratnox
n00b
n00b


Joined: 04 Aug 2004
Posts: 11
Location: Netherlands

PostPosted: Wed Sep 01, 2004 8:25 pm    Post subject: Reply with quote

First off, very very nice! Speeds up emerge sync by about 3 minutes on my box (P2 400 MHz, 256mb ram.. ata66 hdd, mysql running on an ata133 controller)

However, there's a small problem with equery it seems, haven't tried the other commands

Code:
 # equery size gcc
* sys-devel/gcc-3.3.4-r1
           Total Files : 474
           Total Size  : 29301.69 KiB
Exception exceptions.AttributeError: "'NoneType' object has no attribute 'close'" in <bound method Cursor.__del__ of <MySQLdb.cursors.Cursor instance at 0x406c912c>> ignored
Exception exceptions.AttributeError: "'NoneType' object has no attribute 'close'" in <bound method Cursor.__del__ of <MySQLdb.cursors.Cursor instance at 0x4062ce0c>> ignored


Dunno if it's anything serious as of calling it 'problem' though

Running dev-db/mysql-4.0.20-r1, sys-apps/portage-2.0.50-r10
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Thu Sep 02, 2004 12:31 pm    Post subject: Reply with quote

The only thing i hate about portage is the whole emerge sync bit.

The blasted "Updating cache" is bloody annoying. It takes forever.
Now, if MySQL can sort out the emerge sync speed. Then that would be a great improvement.

Everything else is fine about portage. I use esearch, and i also don't mind the speed of emerge -s.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
NewBlackDak
Guru
Guru


Joined: 02 Nov 2003
Posts: 512
Location: Utah County, UT

PostPosted: Thu Sep 02, 2004 4:03 pm    Post subject: Reply with quote

It's the downloading that takes the time.
I think it would be really awesome if there were no emerge sync. You emerge -p system, it hits your favorite mirror to get a list of the most recent packages, and downloads every thing it needs then.
_________________
Gentoo systems.
X2 4200+@2.6 - Athy
X2 3600+ - Myth
UltraSparc5 440 - sparcy
Back to top
View user's profile Send private message
MooktaKiNG
Guru
Guru


Joined: 11 Nov 2002
Posts: 326
Location: London, UK

PostPosted: Thu Sep 02, 2004 6:37 pm    Post subject: Reply with quote

NewBlackDak wrote:
It's the downloading that takes the time.
I think it would be really awesome if there were no emerge sync. You emerge -p system, it hits your favorite mirror to get a list of the most recent packages, and downloads every thing it needs then.


There's a lot of security issues with that.
And you wont be able to do emerge -s, and other search stuff.
You will be a good connection. The server will have to be VERY GOOD, since the whole world is gonna use it.
_________________
http://www.mooktakim.com
Athlon XP 2001, Giga-Byte GA-7VRXP MB, 640Mb DDR RAM 333MHz, MSI Geforce 4800SE 128Mb DDR, 40x12x48 Liteon CDRW drive, Flower Cooler, ADSL Router
Back to top
View user's profile Send private message
eGore911
Apprentice
Apprentice


Joined: 11 Mar 2003
Posts: 290

PostPosted: Thu Sep 02, 2004 7:21 pm    Post subject: Reply with quote

Two things:
1.) emerge sync is not slow because of downloading, i think. I sync over a 100 mbit lan. The issue is the reading from and writing to the fs. So this MIGHT be an improvement using mysql.
2.) I didn't look at the script, so I might be wrong: Moving portage to a DB is worth nothing. You have to build a database concept around that, optimized for searching. This would just cause that it's not save in the DB of the FS, but in another one, nothing else. But since I'm not a database expert, I'd better shut up.

I'd leave it as it is for now.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum