Possibility that emerge would be blocked by a firewall?

[alphatronics]

I'm on a campus whose firewall is extremely picky about what it lets through. This may be whats preventing me from completing an emerge sync (as I was told to do in the install docs).
I am able to visit websites (I even regged my forum account here over lynx) and other various internet activities, but emerge doesnt seem to be able to connect to the server.

Is there a way to do an emerge over http or ftp? (The firewall has a tendency of blocking most everything else)

[aja]

I can't say for sure without knowing your campus configs, but I work behind a pretty restrictive fire wall, and I wouldn't be surprise if the following don't apply to you as well:

You will probably not be able to emerge sync. However the emerge-webrsync script from the gentoolkit will probably work (it uses HTTP rather than rsync, I believe). Most ebuild downloads use ftp, so once you have a current portage tree by getting daily snapshots using emerge-webrsync, you should be able to use emerge as usual.

Warning: emerge-webrsync doesn't merely sync your tree - it replaces it with the lates snapshot. So it can be a bit bandwidth-intensive. Might want to reserve it for once-a-week during low load time.

[arkhan_jg]

You could also try asking your campus firewall maintainers nicely to allow traffic out to port 873. tcp and udp.

Entirely depends what their policy is on controlling outgoing ports as to how successful you'll be. Depending upon their linux knowledge, you could point out that it's somewhat similar in principle to windows update, so you need to do it for security updates (and new software installs, but they don't need to know that). It's also probably worth mentioning that you can do the updates using the web port, but it'll eat up more bandwidth than doing it incrementally through port 873.

[alphatronics]

They don't know shit about linux here (hell, they use MS Exchange as an email server, I have to visit a website to check email if I don't buy a copy of Outlook), but I'll try using the "security updates" line.

If that doesn't work, is emerge-webrsync included on the install cd or do I need to download it from somewhere?

[_SkeLeToN_]

They don't know shit about linux here (hell, they use MS Exchange as an email server, I have to visit a website to check email if I don't buy a copy of Outlook), but I'll try using the "security updates" line.

If that doesn't work, is emerge-webrsync included on the install cd or do I need to download it from somewhere?

emerge-webrsync come with the gentootoolkit package and from what I remember it is on the cd.

[Foolhardy]

Alpha,

I'm in the exact same boat as you man, and websync didn't work for me Still don't have a complete gentoo install...

Tried both rc3 and rc4 and had fewer probs with 3, but couldn't complete regardless of stage chosen.

[alphatronics]

I was able to do the websync thing, I was afraid that it had crashed (there was nothing telling me that it was in the process of downloading), but it eventually finished after about 15-20 minutes (broadband connection, but slow at times).

[ebrostig]

It may be very easy to fix.

I'm behind a very nazi firewall too.

If you specify an http_proxy in your browser to reach internet, we are all set

Do this:

Code: Select all

export HTTP_PROXY=http://proxy.domain.edu:portnumber
export RSYNC_PROXY=proxy.domain.edu:portnumber (No http:// et the beginning)
emerge rsync

Erik

[Foolhardy]

thank you ebrostig,

I don't know of any http proxies I can use...or is the command as generic as you listed?

[andy_fe]

that's work's for me, I'm using ISA server as proxy, with authentification, and for RSYNC_PROXY cutting the http solves eberything.
Many thx!

[ebrostig]

thank you ebrostig,

I don't know of any http proxies I can use...or is the command as generic as you listed?

Check your browser to see what the proxy settings are. Then use the same names in the setup I gave you.
You can also ask your netadmins for the correct proxy to use.

Erik

[Foolhardy]

i'LL give it a shot ebrostig! Thanks for letting me know about that.

update: I'm installing from a non-firewalled place, and I got past the part I was stuck at. Now it's other stuff to be stuck on

[digicosm]

It may be very easy to fix.

I'm behind a very nazi firewall too.

If you specify an http_proxy in your browser to reach internet, we are all set

Do this:

Code: Select all

export HTTP_PROXY=http://proxy.domain.edu:portnumber
export RSYNC_PROXY=proxy.domain.edu:portnumber (No http:// et the beginning)
emerge rsync

Erik

No dice here. This is what I get:

Code: Select all

mybox linux # emerge rsync
>>> starting rsync with rsync://rsync.gentoo.org/gentoo-portage...
bad response from proxy - HTTP/1.0 403 Forbidden

sigh...

[petervos]

Hi,

I'm having the same problem, and don't think it has anything to do with
a firewall (without a firewall, the errors are still there).
The error messages change, depending on the value of the RSYNC_PROXY
environment var:

if I do 'export RSYNC_PROXY="proxy.pandora.be:8080"', I get the following:
(The whole list of messages for each retry)

bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
bad response from proxy - HTTP/1.0 403 Forbidden
rsync: failed to connect to proxy.pandora.be: Success
rsync error: error in socket IO (code 10) at clientserver.c(83)


But when I supply a numeric address:
'export RSYNC_PROXY="213.224.83.46:8080"',
the list of errors is shorter:

bad response from proxy - HTTP/1.0 403 Forbidden
rsync: failed to connect to 213.224.83.46: Success
rsync error: error in socket IO (code 10) at clientserver.c(83)


Anyone who can explain this? Does rsync use HTTP/1.0?
Can you make it use 1.1?
Some people suggested resolving rsync.gentoo.org might be
the problem, but I can ping it (with response between 110 and 140ms).
I have asked my ISP if their proxy supports rsync, but have not
yet received any answer, so I will use emerge-webrsync for now,
that seems to work.

Peter.

[blasterboy]

Heya !

I am using the same ISP as you (pandora oftewel telenet!) and the solution for you is very simple :

Do NOT use the rsync_proxy line -> pandora does not have a proxy for that.

I only have proxy lines set up for HTTP and FTP - rsync doesn't need them.

Regards,

A

[mmmr7ckl]

Help!

I'm behind a nasty firewall. I want to sync across http.

I tried various methods as above, I can't emerge rsync!

How do I do the web emerge thing? Step by step, exactly, like in the installation guide?

Thanks,

[blasterboy]

If you are stuck behind a firewall that allows you to surf or do ftp from linux but NOT rsync, you might need to do what I did :

Define your proxies (see above mails) and do

Code: Select all

emerge-webrsync 

or before I knew that that existed, do the following...

WARNING FIENDISHLY OVER-COMPLICATED PROCEDURE AHEAD

Download the emerge.tar.gz file (via another machine or using the windows partition on that machine and put it in a FAT32 partition so you can mount it in your linux), untar it in a temp directory and then delete your /usr/portage directory (I alway keep the distfiles though) and then copy it across. OF COURSE YOU CHECK THAT YOUR DOWNLOADED EMERGE FILE IS COMPLETE (and not do what I did - deleting my /usr/portage and then discovering my emerge.tar.gz file was incomplete !).

Once copied, do an 'emerge regen' and it should work. If you want to download updates, and are behind a REALLY 3v1l firewall which blocks everything linux, you need to surf via another (windows) machine to a site where they keep the distfiles on a webserver, download them , transfer em over (via ftp? in your /usr/portage/distfiles, and then launch the emerge).

Quite a complicated procedure to get stuff in ! Another way would be to download them at home and burn them on a cdrom using emerge -f to just 'fetch' them...

The (finnish ?) website where I get the emerge file from was mentioned somewhere in the forums, (I am sitting behind an xp machine for now) but maybe someone here knows it ?

Regards,

A

[Zephaniah]

Couldn't you just set the proxies

Code: Select all

export http_proxy="http://user:pass@proxy:port/"

then use wget to download the latest portage snapshot, as in;

Code: Select all

wget http://public.planetmirror.com/pub/gentoo/snapshots/portage-20030512.tar.bz2

or whatever mirror you use, untar to /usr/portage and then you will have the latest version of portage. Or is there a flaw in this method?

[guero61]

Guys, guys, guys! If you can't rsync or even webrsync, just download that day's tarball! Browse to your favorite mirror and browse to the "snapshots" directory. There you will find a veritable cornucopia of portage tree snapshots!

1. Download snapshot
2. cd /usr
3. tar -xvjpf <path_to_snapshot>
4. Enjoy an up-to-date portage tree!

You could even write yourself a script to do it based off of today's (or yesterday's) date...

[blasterboy]

Wellll, I did say my way of doing it was overcomplicated.

Zephaniah, I didn't even know you could set up your proxy that way... and I am not sure that works if you have to authenticate to a Windows Active Directory server. Anybody knows ?

Untill I met a friendly firewall admin who configured ip-access for my machine on his firewall I was in the situation that I could not get Linux to go out on the web. You either used Windows to authenticate yourself to the proxy or you didn't get out period. Hence the surfing in windows to linux sites, saving to fat32 drives and rebooting in linux to get at the stuff. Painfull.

And of course downloading the snapshots is a good solution (which I didn't know about either - duh!).

Regards,

A

[SuperV]

Our company insists on using Active Directory for everything, including their firewall, which they just decided to implement company-wide. Rather Stalin-istic policy as well.
I just tried what Zephaniah suggested and blamo, emerge-webrsync works like a champ. So, there's the proof! It's in the pudding, or something.

[nejiron]

Heya !

I am using the same ISP as you (pandora oftewel telenet!) and the solution for you is very simple :

Do NOT use the rsync_proxy line -> pandora does not have a proxy for that.

I only have proxy lines set up for HTTP and FTP - rsync doesn't need them.

Regards,

A

worked PERFECT for me. I couldnt ping the server before, I could do the emerge-webrsync and what not tho. I am behind a firewall also. set up the http and ftp proxy's for my computer. DID NOT SET UP RSYNC_PROXY and it WORKS now. weird that I had the same problem, that the site was forbidden.