Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
how to download all the files from a gentoo mirror
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Jackie Lin
Tux's lil' helper
Tux's lil' helper


Joined: 31 May 2017
Posts: 115

PostPosted: Fri Jan 19, 2018 6:34 am    Post subject: how to download all the files from a gentoo mirror Reply with quote

Hi, there.
I am trying to make a local gentoo mirror. I have learned how to make a rsync server.
but the nearest mirror for me does not support rsync. it support only http.
I wonder how I can copy all the files on the http server to local path.
Could anyone help me? Thanks in advance!
_________________
peace, focus.
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30915
Location: here

PostPosted: Fri Jan 19, 2018 7:46 am    Post subject: Reply with quote

An idea, but keep in mind you need much disk space and I don't know if the mirror where you download all packages will be happy
Code:
for package in $(EIX_LIMIT=0 NAMEVERSION="=<category>/<name>-<version>\n" eix --format '<availableversions:NAMEVERSION>' --in-overlay gentoo  '*' | head -n -4); do
     emerge -f --nodeps ${package}
done

_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W

PostPosted: Fri Jan 19, 2018 12:30 pm    Post subject: Reply with quote

Jackie Lin,

A full mirror is about 250G, then you need to get the fetch restricted packages yourself.

Don't worry too much about distance to the mirrors, speed is more important.
Audition mirrors using mirrorselect.

Why do you need a full mirror?
Gentoo provides http-replicator so that you need only download source files once.
You point all your systems to your instance of http-replicator and it fetches things that it doesn't already have.
This saves a lot of internet bandwidth and your HDD space.

Be friendly to the mirror system too. Spread the load around a bit.

-- edit --

If you intend to copy just a single mirror, email the admin and ask first.
If the admin knows the load is expected and is happy with it, you won't get blocked.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Jackie Lin
Tux's lil' helper
Tux's lil' helper


Joined: 31 May 2017
Posts: 115

PostPosted: Sun Jan 21, 2018 8:08 am    Post subject: Reply with quote

Thanks a lot,fedeliallalinea :)
_________________
peace, focus.
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30915
Location: here

PostPosted: Sun Jan 21, 2018 8:36 am    Post subject: Reply with quote

Jackie Lin wrote:
Thanks a lot,fedeliallalinea :)

Remember NeddySeagoon words
NeddySeagoon wrote:
If you intend to copy just a single mirror, email the admin and ask first.
If the admin knows the load is expected and is happy with it, you won't get blocked.

_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Jackie Lin
Tux's lil' helper
Tux's lil' helper


Joined: 31 May 2017
Posts: 115

PostPosted: Mon Jan 22, 2018 2:22 am    Post subject: Reply with quote

Thanks, NeddySeagoon :)
_________________
peace, focus.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum