Benchmark: RSync VS. Git for Syncing Portage

[Al-Caveman]

Update: turns out that emerge already has Git support in **9999, and that will
ship in portage 2.2.16. Details: https://github.com/gentoo/portage so it's all
close it seems!

I did some benchmark testing RSync and Git, and in summary I found that that
Git is far more efficient for Portage than RSync. Which is not a surprise as
Git's design is obviously advantagous, but I thought of doing this benchmark
to verify my understanding of the theory behind it (which turned it to confirm
each other).

Test bed description, results, and discussions are here:
https://github.com/Al-Caveman/rsync-vs-git

So far, it seems to me that:
1- Initial Portage copy is better be done by FTP, Torrents, etc.
2- Subsequent updates to be shared by Git instead of RSync.

Update: here are the results (test bed desc. and discussion are in github; too
lazy to paste that too):

Code: Select all

RSYNC:
    sync first (first sync):
        real 12m9.077s
        user 0m4.057s
        sys 0m28.482s
    sync second (~1 month worth of portage sync):
        real 21m48.102s
        user 0m4.687s
        sys 0m26.411s
    sync third (no change):
        real 22m50.308s
        user 0m5.634s
        sys 0m31.522s
GIT:
    pull first (first sync; i.e. git clone):
        real 69m58.406s
        user 0m8.746s
        sys 0m13.861s
    pull second (~1 month worth of portage sync):
        real 0m33.194s
        user 0m1.381s
        sys 0m0.935s
    pull third (no change):
        real 0m0.307s
        user 0m0.166s
        sys 0m0.032s

[tclover]

I am still waiting forever for a git tree. Usage here is to sync on average--once per 2 weeks. But this brings... too many left over laying around. So, sometimes one sync per month is preferred... and then have to get a full copy of tree tarball! What a waste of bandwith. Still waiting... maybe forever.

[Ant P.]

So far, it seems to me that:
1- Initial Portage copy is better be done by FTP, Torrents, etc.
2- Subsequent updates to be shared by Git instead of RSync.

The initial sync would be best done by webrsync-gpg, and the rest had better verify all git signatures: anything less would leave you vulnerable to the gaping security hole Portage is currently infamous for.

[John R. Graham]

Something funny is going on here. Your third case rsync times are wildly longer than what I'm seeing:

Code: Select all

# rm /usr/portage/metadata/timestamp.chk
IPSN-JOHNGRAH2 portage # time emerge --sync
>>> Synchronization of repository 'gentoo' located in '/usr/portage'...
>>> Starting rsync with rsync://134.153.48.2/gentoo-portage...

receiving incremental file list
timestamp.chk
          32 100%   31.25kB/s    0:00:00 (xfer#1, to-check=0/1)

Number of files: 1
Number of files transferred: 1
Total file size: 32 bytes
Total transferred file size: 32 bytes
Literal data: 32 bytes
Matched data: 0 bytes
File list size: 27
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 98
Total bytes received: 135

sent 98 bytes  received 135 bytes  93.20 bytes/sec
total size is 32  speedup is 0.14

receiving incremental file list
metadata/timestamp.chk
          32 100%   31.25kB/s    0:00:00 (xfer#1, to-check=2094/94199)

Number of files: 180736
Number of files transferred: 1
Total file size: 334.32M bytes
Total transferred file size: 32 bytes
Literal data: 32 bytes
Matched data: 0 bytes
File list size: 4.51M
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 26.40K
Total bytes received: 4.62M

sent 26.40K bytes  received 4.62M bytes  134.76K bytes/sec
total size is 334.32M  speedup is 71.91

real    0m38.936s
user    0m2.702s
sys     0m0.905s

Also, since nobody I know pays for CPU time any more, why is it you think that Git is superior here? Real time is longer on both of the interesting cases.

- John

[Al-Caveman]

So far, it seems to me that:
1- Initial Portage copy is better be done by FTP, Torrents, etc.
2- Subsequent updates to be shared by Git instead of RSync.

The initial sync would be best done by webrsync-gpg, and the rest had better verify all git signatures: anything less would leave you vulnerable to the gaping security hole Portage is currently infamous for.

IMO its best to decouple trust from syncing method. Simply, the hash of all
ebuild hashes can be signed in a single file in Portage. Then, emerge can
locally verify it via something like emerge --verifyportage, which is
automatically run when emerge --sync is run for the 1st time. This way, we can
have all the goodies of whatever initial downloading methods that we choose
(e.g. Torrents), while at the same time having all the trust we need.

But this is security. I was intending to discuss efficiency. I think it's best
if we keep them separated concept and indepdent from each other.

[Al-Caveman]

Something funny is going on here. Your third case rsync times are wildly longer than what I'm seeing:

Code: Select all

# rm /usr/portage/metadata/timestamp.chk
IPSN-JOHNGRAH2 portage # time emerge --sync
... <omitted> ...
real    0m38.936s
user    0m2.702s
sys     0m0.905s

I think it's because your network is faster.

Also note that the 1st test with Git is also very very slow. This hardware
limitation seems to be magnifying the issue both ways (good, we can see it).

It just happens that Git is better at what matters (syncing afterwards) while
RSync is better what what doesn't matter (initial download; can do with FTP).

Also, since nobody I know pays for CPU time any more, why is it you think that Git is superior here? Real time is longer on both of the interesting cases.

Obviously CPU time and bandwidth is just some reasons. Another is disk I/O.
Now imagine a busy server: running emerge --sync can be scary as it will
certainly affect processes that are currently serving some users, which costs
money and customer dissatisfaction. Other reasons are: snappiness, saving our
human time, convenience, happiness.

Plus why stick with an inferior syncing method if the superior tool (Git) is
well-tested and exists freely off-the-shelve.

For those who love RSync, emerge can have an option in /etc/portage/make.conf.
Something like EMERGE_SYNCER="rsync" or EMERGE_SYNCER="git".

For your reference, is my rsync daemon command:

Code: Select all

rsync --daemon --config=rsyncd.conf  -v --no-detach --port=22222

This is the config file rsyncd.conf

Code: Select all

pid file = /home/someuser/test/rsyncd.pid
use chroot = no
read only = yes
log file = /home/someuser/test/rsyncd.log
uid=someuser
gid=someuser

[test]
        path = /home/someuser/test
        comment = test 
        read only = yes
[portage_test]
        path = /usr
        comment = test2
        read only = yes

And this is my rsync client command:

Code: Select all

time rsync -r --port=22222 rsync://10.1.0.1/portage_test/portage/* .

[tclover]

Huh... so I am out of date? Not anymore!

Code: Select all

(...)
Receiving objects: 100% (170862/170862), 135.99 MiB | 1.38 MiB/s, done.
(...)

I've just checked the man pages... and noticed the new change--~new~ repos.conf... Scarabeus is the lead of... the git mirror; and a new wiki page related to... pull request. The crew is too thin... only a single member listed on it: so no pull requests! Anyway, I don't want to fork such a big repo... something should be done to remove, at least, the ChanegLog and why not... migrate to thin manifest since the commits are signed!

Well, lets enjoy the migration and avoid downloading a (~50MB) tarball each month then.

[John R. Graham]

I think it's because your network is faster.

You've got a point there. My work network is scary fast: several gigabits/sec onto campus but throttled to 300 mbits/sec to each workstation. My home network is 18 mbits/sec down + 3mbits/sec up. Does that more closely match yours? If so, I'll benchmark from there.

Plus why stick with an inferior syncing method if the superior tool (Git) is well-tested and exists freely off-the-shelve.

If you'll indulge me, could you define "superior", please? So far, your benchmarks don't show what I would call a compelling case.

- John

[Naib]

What was the actual git command used to re-sync?
how much history did the git version contain
did you try: git clone --depth 1 and git pull --depth 1 (or some arbitrary depth that isn't the entire history of changes)

[Al-Caveman]

I think it's because your network is faster.

You've got a point there. My work network is scary fast: several gigabits/sec onto campus but throttled to 300 mbits/sec to each workstation. My home network is 18 mbits/sec down + 3mbits/sec up. Does that more closely match yours? If so, I'll benchmark from there.

I don't think so. My wifi is probably worse. PC1 and PC2 are connected to two
APs (different SSIDs and frequencies), and then the APs are joined via Cat6
over the distance of around, probably 50 meters or so.

Also PC2 (one hosting Git and Rsync repos) has a slow spinning disk (it's a
dated laptop).

That said, I look forward to see your results. So far my logic says that Git
must be faster to sync the diffs in your case than RSync at any time. E.g. if
your RSync is fast, Git will be faster.

Plus why stick with an inferior syncing method if the superior tool (Git) is well-tested and exists freely off-the-shelve.

If you'll indulge me, could you define "superior", please? So far, your benchmarks don't show what I would call a compelling case.

- John

Definition: SYNCER X is said to be superior to SYNCER Y, if the
expected wall-clock time to sync the diffs using X is smaller than that of Y.

Also, 97.477% reduction in diff sync speed is not a compelling case? You can
repeat the tests with your faster hardware and Git will be faster at sync with
statistical significance (although probably less saving than 97.477%, but I
still expect a big beefy chunk of saving nonetheless).

[desultory]

IMO its best to decouple trust from syncing method. Simply, the hash of all
ebuild hashes can be signed in a single file in Portage.

That would necessitate batching of commits, which would in turn force restructuring of the incoming repository for little, if any, actual gain.

[Al-Caveman]

What was the actual git command used to re-sync?
how much history did the git version contain
did you try: git clone --depth 1 and git pull --depth 1 (or some arbitrary depth that isn't the entire history of changes)

Git daemon command:

Code: Select all

git daemon --strict-paths --base-path=/home/someuser/test/ --export-all --reuseaddr portage_old.git/.git/

Git clone command (first sync)

Code: Select all

git clone git://10.1.0.1/portage_old.git

Git pull command (second and third sync)

Code: Select all

cd portage_old
git pull

'portage_old' is a directory where I copied the 1-month-old /usr/portage/* to,
from which I did the Git clone. Then I re-copied /usr/portage/* to it again
after emerge --sync, followed by 'git commit -a' on the server (PC2). Once
that done, then I git pull-ed it using the client (PC1)

As for histories, there is only a single history. So I didn't bother pulling
with '--depth=1'.

Of course, if this is production with repeated use, then I'd use something
like '--depth=1'.

[Al-Caveman]

IMO its best to decouple trust from syncing method. Simply, the hash of all
ebuild hashes can be signed in a single file in Portage.

That would necessitate batching of commits, which would in turn force restructuring of the incoming repository for little, if any, actual gain.

I agree. I should have given it more thought.

[John R. Graham]

Here's why I think your third case is uninteresting: Portage doesn't do it that way. Instead it fetches a single file: timestamp.chk. If that file is the same as the current copy in /usr/portage, then the big rsync is skipped. So, with Portage, your 3rd case is almost instantaneous. Notice that, in an attempt to compare apples to apples, I deleted the timestamp.chk file prior to running "emerge --sync", thus forcing the long rsync.

In any case, it leaves you with only two meaningful benchmarks, both of which show rsync to be superior (your definition) to git.

- John

[Al-Caveman]

Here's why I think your third case is uninteresting: Portage doesn't do it that way. Instead it fetches a single file: timestamp.chk. If that file is the same as the current copy in /usr/portage, then the big rsync is skipped. So, with Portage, your 3rd case is almost instantaneous. Notice that, in an attempt to compare apples to apples, I deleted the timestamp.chk file prior to running "emerge --sync", thus forcing the long rsync.

Yeah we can ignore the 3rd case. Good point.

[In any case, it leaves you with only two meaningful benchmarks, both of which show rsync to be superior (your definition) to git.

I don't see that to be true. The 2nd case is a clear win for Git, which is
also the main reason why we use RSync and the most common use case. I.e. it
reduces 21 minutes down to mere 33 seconds. 97% reduction!

So on average, we indeed spend more time syncing diffs (case 2). Which Git is
the clear winner. So if we calculate the expected value of wall-clock time,
Git's far lower (following the definition).

As for the first case, RSYnc is better than Git. But the first case is
unimportant, because FTP or Torrents will be faster than all of them anyway.

[John R. Graham]

Sorry; you're right. I misread your second case git statistic as minutes, not seconds.

- John

[John R. Graham]

Still, there's still something funny about your numbers. You're using a local RSYNC mirror, right? So am I. Here's a not quite apples-to-apples benchmark time syncing my main home workstation against my local home mirror (the workstation is about 1 week out of date vs. the mirror):

Code: Select all

# time emerge --sync
...
Number of files: 180769
Number of files transferred: 2493
Total file size: 334.37M bytes
Total transferred file size: 12.69M bytes
Literal data: 12.69M bytes
Matched data: 0 bytes
File list size: 4.30M
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 74.18K
Total bytes received: 9.08M

sent 74.18K bytes  received 9.08M bytes  1.67M bytes/sec
total size is 334.37M  speedup is 36.51

real    0m9.166s
user    0m4.977s
sys     0m2.360s

I'll see if I can't restore a backup of the Portage tree that's about a month out of date and repeat but clearly there's something different about your test environment vs. mine. (One thing I know is different is you're not using Portage's internal rsync commands.) A better benchmark might be if you actually set up to run "emerge --sync").

- John

[Naib]

Someone has looked into this (not for portage, but backups)

http://serverfault.com/questions/344546 ... r-git-push

Git is faster, esp for no changes. Rsync wins with large-file changes. Portage is lots of txt files.

[Al-Caveman]

Still, there's still something funny about your numbers. You're using a local RSYNC mirror, right? So am I. Here's a not quite apples-to-apples benchmark time syncing my main home workstation against my local home mirror (the workstation is about 1 week out of date vs. the mirror):

Code: Select all

# time emerge --sync
...<omitted>
real    0m9.166s
user    0m4.977s
sys     0m2.360s

I'll see if I can't restore a backup of the Portage tree that's about a month out of date and repeat but clearly there's something different about your test environment vs. mine. (One thing I know is different is you're not using Portage's internal rsync commands.) A better benchmark might be if you actually set up to run "emerge --sync").

- John

Can you repeat that with Git? Just to see how Git compares to RSync relative
to your setup.

This is emerge's rsync command:

Code: Select all

rsync --recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles -

And the new RSync results are:

Code: Select all

RSYNC GENTOO:
    sync first (first copy):
        real        7m55.153s
        user        0m5.835s
        sys         0m24.568s

    sync second (changes after a month):
        real        22m38.504s
        user        0m6.956s
        sys         0m29.604s

    sync third: canceled cause useless

Still same findings: second case (what matters) is taking too long: 22
minutes. Only exception is first test being 50% faster possibly due to 50%
compression of ebuilds (text files), but who cares about the first test.

Doing "emerge --sync" on PC2 took 1min and 11sec to sync 1 day worth of
changes only. Ironically, this is still 100% slower than when Git synced the
30 days worth of changes. I guess Git could have taken 1-3 seconds instead
to sync the 1 day worth of changes if emerge was using Git.

[Al-Caveman]

Just to clarify a few points:

1. My Git test took 33 seconds to sync 1 month word of data. But this was only
by playing back a single history. So if we do it with a real repo, it would
probably be slower due to playing back a lot more histories than just 1.

2. Will Git still be much faster if there are more histories to be played
back? I guess yes, cause Funtoo people seem happy. But this is just a guess.

3. I am happy that Gentoo didn't rush into Git. But I think Gentoo should at
also give it more thought by having an experimental Portage tree repo as well
as some experimental emerge that uses a Git backend. This way volunteers can
test it and see by themselves.

[EmaRsk]

Since Funtoo and several overlays already use git, it wouldn't be so much "experimental". There may be reasons not to switch to git, but being untested is not one of them.

[steveL]

I am happy that Gentoo didn't rush into Git. But I think Gentoo should at
also give it more thought by having an experimental Portage tree repo as well
as some experimental emerge that uses a Git backend. This way volunteers can
test it and see by themselves.

There's nothing stopping volunteers doing exactly that now. emerge has supported working with git repos for at least a couple of years, which is how funtoo operates. (IIRC that work was done in conjunction with drobbins as use-tester.)

The issue is making up the initial repo, with correct metadata like emails from LDAP for commits. The work was kick-started last year by Patrick, and you can see the latest progress here. By all means clone and see what you think, and then look at maintaining a mirror which keeps up with cvs (the volunteer help.)

I imagine there's still some massaging that needs to be done, but I haven't looked for ages. Good luck :-)

edit: https://github.com/rich0/gentoo-gitmig-2014-09-15 the commit authors look sane, from cursory view.

[Ant P.]

Just to clarify a few points:

1. My Git test took 33 seconds to sync 1 month word of data. But this was only
by playing back a single history. So if we do it with a real repo, it would
probably be slower due to playing back a lot more histories than just 1.

2. Will Git still be much faster if there are more histories to be played
back? I guess yes, cause Funtoo people seem happy. But this is just a guess.

I sync local copies of the full Linux Kernel repo all the time, it's always as fast.

[Al-Caveman]

Just to clarify a few points:

1. My Git test took 33 seconds to sync 1 month word of data. But this was only
by playing back a single history. So if we do it with a real repo, it would
probably be slower due to playing back a lot more histories than just 1.

2. Will Git still be much faster if there are more histories to be played
back? I guess yes, cause Funtoo people seem happy. But this is just a guess.

I sync local copies of the full Linux Kernel repo all the time, it's always as fast.

That's different though. Obviously syncing "all the time" means that you don't
accumulate much commits to play back. So every time you sync, you end up
playing back only a few commits.

But if you wait for a long period, then git sync will end up playing back a
lot of commits. Ultimately, it could become less efficient than RSync in
extreme cases.

But those extreme cases are unrealistic, and -imo- Git should on average be
faster than RSync.

Anyway, turns out that emerge already has Git support in **9999, and that will
ship in portage 2.2.16. Details: https://github.com/gentoo/portage so it's all
close it seems!

[Al-Caveman]

This one is by using 'emerge --sync' itself instead of using my own rsync
commands manually.

The result is that a few hours out of sync portage took 3 s econds to 'git
pull', while it took 56 seconds to 'emerge --sync'. The Git method is much
faster.

Can't wait to get emerge 2.2.16. First thing to do is setting 'sync-type =
git' in repos.conf.

Here are my results. First, using Git to sync via
https://github.com/gentoo/gentoo-portage-rsync-mirror:

Code: Select all

time git pull 
remote: Counting objects: 67, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 67 (delta 37), reused 58 (delta 28)
Unpacking objects: 100% (67/67), done.
From https://github.com/gentoo/gentoo-portage-rsync-mirror
   aa4f71a..92cfe39  master     -> origin/master
Updating aa4f71a..92cfe39
Fast-forward
 app-text/qpdfview/ChangeLog                                              |  7 +++--
 app-text/qpdfview/Manifest                                               | 24 +++++++++++----
 app-text/qpdfview/qpdfview-0.4.13.ebuild                                 |  9 ++++--
 dev-lang/nqp/ChangeLog                                                   |  7 ++++-
 dev-lang/nqp/Manifest                                                    | 24 ++++++++-------
 dev-lang/nqp/nqp-2014.07.ebuild                                          | 61 ------------------------------------
 dev-lang/nqp/nqp-2014.08.ebuild                                          | 61 ------------------------------------
 dev-lang/nqp/nqp-2014.09.ebuild                                          | 61 ------------------------------------
 dev-lang/nqp/nqp-2014.10.ebuild                                          | 61 ------------------------------------
 dev-lang/nqp/nqp-2014.11-r1.ebuild                                       | 65 ---------------------------------------
 dev-lang/rakudo/ChangeLog                                                |  7 ++++-
 dev-lang/rakudo/Manifest                                                 | 25 +++++++--------
 dev-lang/rakudo/rakudo-2014.07.ebuild                                    | 53 -------------------------------
 dev-lang/rakudo/rakudo-2014.08.ebuild                                    | 51 ------------------------------
 dev-lang/rakudo/rakudo-2014.09-r1.ebuild                                 | 51 ------------------------------
 dev-lang/rakudo/rakudo-2014.09.ebuild                                    | 51 ------------------------------
 dev-lang/rakudo/rakudo-2014.10.ebuild                                    | 51 ------------------------------
 dev-lang/rakudo/rakudo-2014.11-r1.ebuild                                 | 56 ---------------------------------
 media-plugins/gst-plugins-ximagesrc/ChangeLog                            | 13 ++++++--
 media-plugins/gst-plugins-ximagesrc/Manifest                             | 30 +++---------------
 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-0.10.31.ebuild | 28 -----------------
 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.3.ebuild   | 28 -----------------
 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.4.ebuild   | 28 -----------------
 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.4.5.ebuild   | 31 +++++++++++++++++++
 media-plugins/gst-plugins-ximagesrc/metadata.xml                         |  6 ++++
 metadata/dtd/timestamp.chk                                               |  2 +-
 metadata/glsa/timestamp.chk                                              |  2 +-
 metadata/md5-cache/app-text/qpdfview-0.4.13                              |  2 +-
 metadata/md5-cache/dev-lang/nqp-2014.07                                  | 15 ---------
 metadata/md5-cache/dev-lang/nqp-2014.08                                  | 15 ---------
 metadata/md5-cache/dev-lang/nqp-2014.09                                  | 15 ---------
 metadata/md5-cache/dev-lang/nqp-2014.10                                  | 15 ---------
 metadata/md5-cache/dev-lang/nqp-2014.11-r1                               | 15 ---------
 metadata/md5-cache/dev-lang/rakudo-2014.07                               | 13 --------
 metadata/md5-cache/dev-lang/rakudo-2014.08                               | 13 --------
 metadata/md5-cache/dev-lang/rakudo-2014.09                               | 13 --------
 metadata/md5-cache/dev-lang/rakudo-2014.09-r1                            | 13 --------
 metadata/md5-cache/dev-lang/rakudo-2014.10                               | 13 --------
 metadata/md5-cache/dev-lang/rakudo-2014.11-r1                            | 13 --------
 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-0.10.31           | 13 --------
 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.3             | 13 --------
 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.4             | 13 --------
 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.4.5             | 14 +++++++++
 metadata/md5-cache/sys-cluster/vzctl-4.8                                 |  4 +--
 metadata/news/timestamp.chk                                              |  2 +-
 metadata/timestamp                                                       |  2 +-
 metadata/timestamp.chk                                                   |  2 +-
 metadata/timestamp.x                                                     |  2 +-
 sys-cluster/vzctl/ChangeLog                                              |  5 ++-
 sys-cluster/vzctl/Manifest                                               | 22 +++++--------
 sys-cluster/vzctl/vzctl-4.8.ebuild                                       |  4 +--
 51 files changed, 155 insertions(+), 989 deletions(-)
 delete mode 100644 dev-lang/nqp/nqp-2014.07.ebuild
 delete mode 100644 dev-lang/nqp/nqp-2014.08.ebuild
 delete mode 100644 dev-lang/nqp/nqp-2014.09.ebuild
 delete mode 100644 dev-lang/nqp/nqp-2014.10.ebuild
 delete mode 100644 dev-lang/nqp/nqp-2014.11-r1.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.07.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.08.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.09-r1.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.09.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.10.ebuild
 delete mode 100644 dev-lang/rakudo/rakudo-2014.11-r1.ebuild
 delete mode 100644 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-0.10.31.ebuild
 delete mode 100644 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.3.ebuild
 delete mode 100644 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.4.ebuild
 create mode 100644 media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.4.5.ebuild
 delete mode 100644 metadata/md5-cache/dev-lang/nqp-2014.07
 delete mode 100644 metadata/md5-cache/dev-lang/nqp-2014.08
 delete mode 100644 metadata/md5-cache/dev-lang/nqp-2014.09
 delete mode 100644 metadata/md5-cache/dev-lang/nqp-2014.10
 delete mode 100644 metadata/md5-cache/dev-lang/nqp-2014.11-r1
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.07
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.08
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.09
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.09-r1
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.10
 delete mode 100644 metadata/md5-cache/dev-lang/rakudo-2014.11-r1
 delete mode 100644 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-0.10.31
 delete mode 100644 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.3
 delete mode 100644 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.4
 create mode 100644 metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.4.5

real	0m3.139s
user	0m0.891s
sys	0m0.370s

Second, using RSync via 'emerge --sync':

Code: Select all

time sudo emerge --sync
>>> Synchronization of repository 'gentoo' located in '/usr/portage'...
>>> Starting rsync with rsync://176.28.50.119/gentoo-portage...
>>> Checking server timestamp ...
Welcome to quetzal.gentoo.org / rsync.gentoo.org

Server Address : 176.28.50.119, 2a01:488:67:1000:b01c:3277:0:1
Contact Name   : mirror-admin@gentoo.org
Hardware       : 4 x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz, 16075MB RAM
Sponsor        : Host Europe, Cologne, Germany, EU

Please note: common gentoo-netiquette says you should not sync more
than once a day.  Users who abuse the rsync.gentoo.org rotation
may be added to a temporary ban list.

MOTD autogenerated by update-rsync-motd on Thu Jul 24 06:32:56 UTC 2014

receiving incremental file list
timestamp.chk

Number of files: 1
Number of files transferred: 1
Total file size: 32 bytes
Total transferred file size: 32 bytes
Literal data: 32 bytes
Matched data: 0 bytes
File list size: 27
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 98
Total bytes received: 660

sent 98 bytes  received 660 bytes  303.20 bytes/sec
total size is 32  speedup is 0.04
Welcome to quetzal.gentoo.org / rsync.gentoo.org

Server Address : 176.28.50.119, 2a01:488:67:1000:b01c:3277:0:1
Contact Name   : mirror-admin@gentoo.org
Hardware       : 4 x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz, 16075MB RAM
Sponsor        : Host Europe, Cologne, Germany, EU

Please note: common gentoo-netiquette says you should not sync more
than once a day.  Users who abuse the rsync.gentoo.org rotation
may be added to a temporary ban list.

MOTD autogenerated by update-rsync-motd on Thu Jul 24 06:32:56 UTC 2014

receiving incremental file list
app-text/qpdfview/ChangeLog
app-text/qpdfview/Manifest
app-text/qpdfview/qpdfview-0.4.13.ebuild
deleting dev-lang/nqp/nqp-2014.11-r1.ebuild
deleting dev-lang/nqp/nqp-2014.10.ebuild
deleting dev-lang/nqp/nqp-2014.09.ebuild
deleting dev-lang/nqp/nqp-2014.08.ebuild
deleting dev-lang/nqp/nqp-2014.07.ebuild
deleting dev-lang/rakudo/rakudo-2014.11-r1.ebuild
deleting dev-lang/rakudo/rakudo-2014.10.ebuild
deleting dev-lang/rakudo/rakudo-2014.09.ebuild
deleting dev-lang/rakudo/rakudo-2014.09-r1.ebuild
deleting dev-lang/rakudo/rakudo-2014.08.ebuild
deleting dev-lang/rakudo/rakudo-2014.07.ebuild
dev-lang/nqp/ChangeLog
dev-lang/nqp/Manifest
dev-lang/rakudo/ChangeLog
dev-lang/rakudo/Manifest
deleting media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.4.ebuild
deleting media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.2.3.ebuild
deleting media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-0.10.31.ebuild
media-plugins/gst-plugins-ximagesrc/ChangeLog
media-plugins/gst-plugins-ximagesrc/Manifest
media-plugins/gst-plugins-ximagesrc/gst-plugins-ximagesrc-1.4.5.ebuild
media-plugins/gst-plugins-ximagesrc/metadata.xml
metadata/timestamp
metadata/timestamp.chk
metadata/timestamp.x
metadata/dtd/timestamp.chk
metadata/glsa/timestamp.chk
metadata/md5-cache/app-text/qpdfview-0.4.13
deleting metadata/md5-cache/dev-lang/rakudo-2014.11-r1
deleting metadata/md5-cache/dev-lang/rakudo-2014.10
deleting metadata/md5-cache/dev-lang/rakudo-2014.09-r1
deleting metadata/md5-cache/dev-lang/rakudo-2014.09
deleting metadata/md5-cache/dev-lang/rakudo-2014.08
deleting metadata/md5-cache/dev-lang/rakudo-2014.07
deleting metadata/md5-cache/dev-lang/nqp-2014.11-r1
deleting metadata/md5-cache/dev-lang/nqp-2014.10
deleting metadata/md5-cache/dev-lang/nqp-2014.09
deleting metadata/md5-cache/dev-lang/nqp-2014.08
deleting metadata/md5-cache/dev-lang/nqp-2014.07
deleting metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.4
deleting metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.2.3
deleting metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-0.10.31
metadata/md5-cache/media-plugins/gst-plugins-ximagesrc-1.4.5
metadata/md5-cache/sys-cluster/vzctl-4.8
metadata/news/timestamp.chk
profiles/use.local.desc
sys-cluster/vzctl/ChangeLog
sys-cluster/vzctl/Manifest
sys-cluster/vzctl/vzctl-4.8.ebuild

Number of files: 181222
Number of files transferred: 24
Total file size: 335.04M bytes
Total transferred file size: 742.57K bytes
Literal data: 742.57K bytes
Matched data: 0 bytes
File list size: 4.30M
File list generation time: 0.002 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 26.86K
Total bytes received: 5.15M

sent 26.86K bytes  received 5.15M bytes  96.83K bytes/sec
total size is 335.04M  speedup is 64.67

 * IMPORTANT: 1 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.


real	0m55.640s
user	0m1.300s
sys	0m0.748s