Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Updated: Getting Squid, SquidGuard, and DansGuardian working
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
dmoulton
n00b
n00b


Joined: 07 Jul 2003
Posts: 72
Location: Somewhere in the Rockies

PostPosted: Sat Sep 20, 2003 4:12 pm    Post subject: Updated: Getting Squid, SquidGuard, and DansGuardian working Reply with quote

UPDATE:

As you can read below in this thread, it is probably better to be using DansGuardian instead of SquidGuard. I am using it now, and it seems to work great, much better than SquidGuard. I am using the blacklist at urlblacklist.com. This is a very large and updated blacklist that takes, at least on my machine, several minutes (maybe 20 sometimes) to load into SquidGuard. It takes DansGuardian less than 30 seconds to load it on the same machine. Your results may vary of course. DansGuardian has many more features and enjoys current support from its developer.

____________________________________

Spent a day or two getting Squid and SquidGuard to work, so I thought this might be useful to others.

My squid cache is running on Sparc Gentoo, but I assume this will work on other systems as well.

Squid

emerge squid

You will find squid.conf in /etc/squid. Edit this file.

I like to set http_port to 3128. You can of course set it to whatever you want, that isn't in use.

You must also set visible_hostname to your hostname value. Failure to do this will result in squid erroring out in a cryptic way.

/etc/init.d/squid start

tail -f /var/log/squid/access.log

Use your favorite browser and set its proxy to use the IP and port of your cache.

Now if you access a website, you should see the logfile role.

If all you intended to do was get squid going for acceleration, you're done.


SquidGuard

ACCEPTKEYWORDS="~<arch>" emerge squidguard (modify with your arch)

Create /etc/squidGuard/squidguard.conf. Here is mine:


# CONFIGURATION DIRECTORIES
dbhome /etc/squidGuard/db
logdir /var/log/squidGuard

# DESTINATION CLASSES:

dest porn {
domainlist porn/domains
urllist porn/urls
expressionlist porn/expressions
}

dest local {
domainlist local/domains
}

#dest bannerads {
# domainlist bannerads/domains
# urllist bannerads/urls
# redirect http://127.0.0.1/squidGuard/banner.gif
#}

#dest audio-video {
# domainlist audio-video/domains
# urllist audio-video/urls
#}


dest hacking {
domainlist hacking/domains
urllist hacking/urls
}

#dest redirector {
# domainlist redirector/domains
# urllist redirector/urls
# expressionlist redirector/expressions
#}

dest warez {
domainlist warez/domains
urllist warez/urls
}

#dest ads {
# domainlist ads/domains
# urllist ads/urls
# redirect http://127.0.0.1/squidGuard/banner.gif
#}

dest aggressive {
domainlist aggressive/domains
urllist aggressive/urls
}

dest drugs {
domainlist drugs/domains
urllist drugs/urls
}

dest gambling {
domainlist gambling/domains
urllist gambling/urls
}

dest violence {
domainlist violence/domains
urllist violence/urls
expressionlist violence/expressions
}


#dest banned_destination {
# domainlist banneddestination/domains
# urllist banneddestination/urls
# expressionlist banneddestination/expressions
#}

#dest advertising {
# domainlist advertising/domains
# urllist advertising/urls
# redirect http://127.0.0.1/squidGuard/banner.gif
# log /var/log/squidGuard/advertising.log
#}

#dest publicite {
# domainlist publicite/domains
# urllist publicite/urls
# redirect http://127.0.0.1/squidGuard/banner.gif
# log /var/log/squidGuard/advertising.log
#}

# ACLs
acl {
default {
pass !porn !drugs !gambling !violence !aggressive !hacking !local all
redirect http://<your webserver of choice>/blocked.html
}
}


Be sure to edit the redirect line near the end, and create a blocked.html page that matches it. There are several sections commented out which you may want to use. I would recommend getting it to work as is first.

Now, for getting those blacklists.


mdkir -p /etc/squidGuard/temp/local
touch /etc/squidGuard/temp/local/domains
chown -R squid.squid /etc/squidGuard/temp

Edit /ec/squid/squid.conf again, search for redirect_program, and edit:

redirect_program /usr/bin/squidGuard -c /etc/squidGuard/squidGuard.conf


I like to get blacklists once a week or so. Here is the script that I have cronned:


#!/bin/bash

# This script downloads and activiates the squidGuard blacklist

cd /etc/squidGuard/temp

wget -q -nd http://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz

if [ -e blacklists.tar.gz ]; then
tar -xzf blacklists.tar.gz
else
exit
fi

rm -f blacklists.tar.gz
mv blacklists db
/etc/init.d/squid stop
rm -rf /etc/squidGuard/db
mv db /etc/squidGuard/

cp -rp local /etc/squidGuard/db/
chown -R squid.squid /etc/squidGuard/db
/etc/init.d/squid start

#This just gets the cache rolling, so the first hit isn't so long.
wget -q -nd --use-proxy=on http://www.cnn.com/index.html
rm -f index.html

You can also use this for the initial download of the blacklists. After you have executed it, tail /var/log/squidGuard/squidGuard.log. If that has errors, sort them out. If it doesn't then attempt to go to a url that is in the blacklist. It should go to your blocked page.

You can add sites to block by editing /etc/squidGuard/temp/local/domains.

You should also rc-update add squid default

Hope this helps somebody!

My thanks to ciaranm and todd on #gentoo-sparc for their help.


Last edited by dmoulton on Fri Apr 16, 2004 3:47 pm; edited 1 time in total
Back to top
View user's profile Send private message
Cottonee
n00b
n00b


Joined: 23 Sep 2003
Posts: 73
Location: Palmerston North, New Zealand

PostPosted: Sun Oct 05, 2003 4:08 am    Post subject: Reply with quote

Thanks dmoulton, I got everything working in my box. :wink:
Back to top
View user's profile Send private message
magrathea
n00b
n00b


Joined: 26 Jul 2003
Posts: 26
Location: amsterdam

PostPosted: Mon Oct 13, 2003 2:06 pm    Post subject: thx for the help Reply with quote

Just wanted to say:
GREAT HOWTO!!
_________________
magrathea home
mmm...... specs;
mama; AMD XP 2200+,384mb DDR,SCSI IBMx2,Geforce MX440, PCTV, MX700
4ngel;laptop 333mhz, dvd, 256mb RAM, 10gb HD 5400 rpm, ati rage lt pro 4mb
--------------------
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Mon Jan 05, 2004 2:49 am    Post subject: Running Slow Reply with quote

Great instructions however after I got it running and set my proxy settings in my browser all web page access seemed to slow down. This is especially true when going to a web page for the first time. The squid log indicates that everything is working OK. Any suggestions for speeding it up. It is only for web access from one machine. I have pretty basic shorewall firewall and the squid cache on the same machine.

I would really like to get this working and implement squidguard next.

Another question; shouldn't the instructions include rc-update -add squid default command so that squid started when booting?

Thanks
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Mon Jan 05, 2004 3:02 am    Post subject: slow proxy Reply with quote

I believe I found part of the problem. I needed to enable socks in konqueror. Once I did that, things did seem to speed up a bit.
Back to top
View user's profile Send private message
Hackeron
Guru
Guru


Joined: 01 Nov 2002
Posts: 307

PostPosted: Mon Jan 05, 2004 11:50 pm    Post subject: Reply with quote

GREAT GUIDE!, but there are a few mistakes in it, I fixed those mistakes and made an easy way to automate the entire process!
just change:
Code:
dontallow='!ads !aggressive !porn !gambling !violence'


to whatever you want to limit. There is a commented line with all values available.

The script just takes a running squid proxy and adds squidGuard with latest database of blocklists with ability to choose what to block. Please report any problems:

Code:
#!/bin/bash
# Script to configure squidguard to filter common junk V0.5

source /sbin/functions.sh

squidconffile="/etc/squid/squid.conf"
squidguardconffile="/etc/squidGuard/squidGuard.conf"
blockedsiteredirect="http://linux.com"

#SQUID GUARD CONFIGS:
# dontallow='!ads !aggressive !porn !local !audio-video !hacking !warez !drugs !gambling !violence !proxy !mail'
dontallow='!ads !aggressive !porn !gambling !violence'

squidconf() {
if [ "$(cat $squidconffile | grep "redirect_program /usr/bin/squidGuard -c $squidguardconffile")" ]; then
echo "redirect_program /usr/bin/squidGuard -c $squidguardconffile" >> $squidconffile
fi
}

squidguardconf() {

echo '
# CONFIGURATION DIRECTORIES
dbhome /etc/squidGuard/db
logdir /var/log/squidGuard
 

dest ads {
 domainlist ads/domains
 urllist ads/urls
 redirect http://127.0.0.1/squidGuard/banner.gif
 log /var/log/squidGuard/advertising.log
}

dest aggressive {
 domainlist aggressive/domains
 urllist aggressive/urls
 log /var/log/squidGuard/aggressive.log
}

dest porn {
 domainlist porn/domains
 urllist porn/urls
 expressionlist porn/expressions
 log /var/log/squidGuard/porn.log
}

dest local {
 domainlist local/domains
}

dest audio-video {
  domainlist audio-video/domains
  urllist audio-video/urls
  log /var/log/squidGuard/media-video.log
}

dest hacking {
 domainlist hacking/domains
 urllist hacking/urls
 log /var/log/squidGuard/hacking.log
}

dest warez {
 domainlist warez/domains
 urllist warez/urls
 log /var/log/squidGuard/warez.log
}

dest drugs {
 domainlist drugs/domains
 urllist drugs/urls
 log /var/log/squidGuard/drugs.log
}

dest gambling {
 domainlist gambling/domains
 urllist gambling/urls
 log /var/log/squidGuard/gambling.log
}

dest violence {
 domainlist violence/domains
 urllist violence/urls
 expressionlist violence/expressions
 log /var/log/squidGuard/violence.log
}

dest proxy {
 domainlist proxy/domains
 urllist proxy/urls
}

dest mail {
 domainlist mail/domains
 log /var/log/squidGuard/mail.log
}

' > $squidguardconffile

echo "
acl {
 default {
 pass $dontallow
 redirect $blockedsiteredirect
 }
}
" >> $squidguardconffile
 
}


if [ ! "$(emerge -p ">=squidguard-1.2.0-r1" | grep R)" ]; then
rm -Rf /etc/squidGuard &> /dev/null
ebegin "Emerging squidGuard"
emerge ">=squidguard-1.2.0-r1" &> /dev/null && eend
ebegin "Configuring squidGuard"
squidconf
squidguardconf
mkdir /etc/squidGuard/temp
cd /etc/squidGuard/temp && eend
ebegin "Fetching backlist sites list"
wget -q -nd http://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz && eend
else
einfo "SquidGuard already installed"
exit 0
fi

 
if [ -e blacklists.tar.gz ]; then
 tar -xzf blacklists.tar.gz
 rm -f blacklists.tar.gz
 /etc/init.d/squid stop
 rm -rf /etc/squidGuard/db
 mv blacklists /etc/squidGuard/db
 mkdir /etc/squidGuard/db/local
 touch /etc/squidGuard/db/local/domains
 chown -R squid.squid /etc/squidGuard/db
 /etc/init.d/squid start
 else
 eerror "Could not download the blacklist sites list"
 exit
 fi
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Wed Jan 07, 2004 4:58 pm    Post subject: Slow squid Reply with quote

Changing the kongueror to use socks did not seem to help that much. I am still looking help in speeding it up. My kernel has the low latency and preemptbale options enabled and the necessary options to get shorewall firewall working. I have tried squid without the firewall running and it still is slow.

Beyond just general slowness, sometimes working web pages do not come back at all or just take forever and I loose patience.
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Thu Jan 15, 2004 4:03 am    Post subject: Moved to dansguardian Reply with quote

After getting squidguard to work and trying it out I found out that squidguard is basically a dead project. The blacklist URLs are not being updated so about 30% of the xxx websites were not being blocked. I moved to dansguardian and 100% of the sites are blocked without using blacklists. I am very pleased with dansguardian so far. It is fast and it does a good job and is highly configurable.
Back to top
View user's profile Send private message
sulu
Guru
Guru


Joined: 21 May 2002
Posts: 399
Location: Dornbirn/Austria

PostPosted: Thu Jan 15, 2004 2:52 pm    Post subject: Interested in dansguardian Reply with quote

g-man

Very interesting info, indeed.

Care to share your experience with dansguardian with us ?
Is it in portage allready ?
Could you post some config-stuff (if not too private 8)

I'm on my first atempt to setup a proxy with web-content-filtering
and methinks that dansguardian a good choice because it really filters and does not rely only on blacklists.

Sulu
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Fri Jan 16, 2004 4:20 am    Post subject: Using dansguardian Reply with quote

There is a stable version of Dansguardian in portage. The default level of filtering is set up for kids in elementary school which was just fine for me. I have two daughters still at home (ages 13 and 10) and they have to do research using the internet. Before installing dansguardian we had our share of 'surprises'. One I remember was my 13 year old daughter's research using google on Cherokee Indians. Google returned link to a porn site having something to do with Cherokee Women. My daughter clicked on it not knowing any better and my wife (who was standing behind her) quicly through herself between my daughter and the computer screen. But I digress.

Go ahead and emerge squid and get it working first as per this thread.
Make sure in the /etc/squid.conf file you use is set to listen to port 3128. I believe that is the default of squid so you should be OK if you take the default.
After you emerge squid do
rc-update add squid default
/etc/init.d/squid start

You will also need a web server. I emerged boa because it small and fast. The reason you need a web server is because when a site is blocked by dansguardian it needs to display a message that the site is blocked via cgi script. If you already have apache installed you don't need boa. I don't think I had to do any config changes to boa.

After you emerge boa do
rc-update add boa default
/etc/init.d/boa start

emerge dansguardian then edit the /etc/dansguardian/dansguardian.conf file to point to the location of your squid proxy. My squid is on the same box so I use:
proxyip = 127.0.0.1

As for the rest of the configuration, I believe I just took the defaults.

I did need to make changes to /etc/init.d/dansguardian file. Here is the depend part of my /etc/init.d/dansguardian file.


depend() {
need net
need boa
need squid
}

I added the 'need boa' and 'need squid' lines so that dansguardian would load after both of those. It would not start properly on boot until I did this because upon starting, dansguardian tries to test the proxy and that fails unless squid is started first.

After you make the above changes do
rc-update add dansguardian default
/etc/init.d/dansguardian start

In your browser, set it to use a proxy. My konqueror setting is http://127.0.0.1 and the port is 8080. You should also disable the browser cache since both squid and dansguardian does some caching.

Hope this helps.
Back to top
View user's profile Send private message
sulu
Guru
Guru


Joined: 21 May 2002
Posts: 399
Location: Dornbirn/Austria

PostPosted: Fri Jan 16, 2004 5:15 am    Post subject: Reply with quote

g-man you are a treasure !!

You provided exactly the information i need.
Funny thing is that i have the same reason for a web-filter like you. My daughter (5 1/2 years) has recently developed an interest for my computer and i dont want any nasty (uhm for the little ones ... ) surprises when we browse the web for something.

Million thanks
Sulu
Back to top
View user's profile Send private message
g-man
n00b
n00b


Joined: 19 Jan 2003
Posts: 42
Location: Fort Collins CO

PostPosted: Sat Jan 17, 2004 12:10 am    Post subject: More info on dansguardian Reply with quote

Sulu

I forgot to say that the cron needs to be set up to rotate the dansguardian log files. Here is what I did.

Create a new file /etc/cron.weekly/dansguardian_logrotate with the following lines:

#!/bin/sh
/etc/dansguardian/logrotation

After that it is done do this:

chmod 755 /etc/cron.weekly/dansguardian_logrotate
chmod 755 /etc/dansguardian/logrotation

That should rotate the dansguardian logs once a week.
Back to top
View user's profile Send private message
sulu
Guru
Guru


Joined: 21 May 2002
Posts: 399
Location: Dornbirn/Austria

PostPosted: Sat Jan 17, 2004 7:58 pm    Post subject: Reply with quote

Thanks again g-man.

Your tip might prove to be useful when eventually i'll get that far.

I had to sort out some issues how to use my fast box as gentoo-system-development-box and how to tranfer the binaries to my slow-box which acts as internet-gateway.
The gateway works fine now so i may set upt the proxy and the web-filter.

Right in the moment im fuzzing around with squid.
Ups, i'm getting off topic, dont i?

Cheers
Sulu
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum