| View previous topic :: View next topic |
| Should google find the forum? |
| Yes! |
|
100% |
[ 9 ] |
| No! |
|
0% |
[ 0 ] |
|
| Total Votes : 9 |
|
| Author |
Message |
cubancigar11
Guru
Joined: 10 Jan 2005
Posts: 340
|
 Posted: Mon Oct 20, 2008 7:08 pm Post subject: Gentoo and google |
 |
|
Hi!
Why does Gentoo forum does not show up in google at all? Is it disabled by forum administrators or is it google?
Just interested... (I cannot use firefox's "i'm felling lucky" thing) |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54219
Location: 56N 3W
|
| Posted: Mon Oct 20, 2008 7:23 pm Post subject: |
|
|
cubancigar11,
Thats news to me, I use google to search the forums.
Add site:forums.gentoo.org after your search terms
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
cubancigar11
Guru
Joined: 10 Jan 2005
Posts: 340
|
| Posted: Mon Oct 20, 2008 7:52 pm Post subject: |
|
|
That I already do, but my problem is shown here: http://i388.photobucket.com/albums/oo329/cubancigar11/snapshot1.png
This is the result I get when I type gentoo forums in location bar. I am not very knowledgeable about this stuff, but I don't see a reason why the main site shouldn't come on top of a google search.
EDIT: corrected link |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
| Posted: Mon Oct 20, 2008 11:10 pm Post subject: |
|
|
something does seem a little fishy... various searches on google were out of whack. doing a search for "site:forums.gentoo.org" only brought back 90 results... compared to 4.89million reported for "site:ubuntuforums.org". we're on it. 
edit: searching for "site:forums.gentoo.org google" returns 118k results... what in the fudge... 
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
amne
Bodhisattva
Joined: 17 Nov 2002
Posts: 6378
Location: Graz / EU
|
| Posted: Tue Oct 21, 2008 1:38 pm Post subject: |
|
|
Strange - for quite some time a simple query on google for "forums" used to return our forums as number 1 hit (bringing us also a lot of spammers). Probably they changed their algorithms for some reason.
Not being found under the term "gentoo forums" is however a bit annoying.
_________________
Dinosaur week! (Ok, this thread is so last week) |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
| Posted: Wed Oct 22, 2008 11:57 pm Post subject: |
|
|
| Code: | Block all bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "msnbot|spider|snapbot|yodaobot|googlebot|Yahoo! Slurp" [NC]
RewriteRule ^/.*php - [F]
|
| Quote: | | someone's blocked all bots in the vhost config |
figured i'd give you guys an update. that happened in the summer. 
| Quote: | (gentoo-infra-representative-who-i-will-not-name) decided google was evil.
and we need to have squid in the middle before he'll let us be exposed to evil again.
|
sounds like a huge load of crap to me, but i don't run the machines gentoo stuff runs on, so what do i know.
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
spindle
Apprentice
Joined: 01 Dec 2003
Posts: 245
|
| Posted: Sat Oct 25, 2008 2:17 pm Post subject: |
|
|
| bunder wrote: |
figured i'd give you guys an update. that happened in the summer.
| Quote: | (gentoo-infra-representative-who-i-will-not-name) decided google was evil.
and we need to have squid in the middle before he'll let us be exposed to evil again.
|
sounds like a huge load of crap to me, but i don't run the machines gentoo stuff runs on, so what do i know. |
I agree with this guy that Google is pretty evil. But yeah, it sounds like a load of crap to me too.
The problem for me is that searching the forums was something that seems to work much better through a tool like Google's search rather than this site's search. Is there way to update the forum search on the site to something better? Maybe it's just me, but the search tool seems pretty poor here.
While I'd like to see the search tool get better I'd want to see the site open up to bots again. I don't see the point of hiding from bots if Google is evil. I'd rather have stuff show up in Google, even if they are evil, since that's where a ton of people go to for information and there is a lot of useful information here. But In the broader scope, I consider the forums to be a publicly available source of information, hiding it from Google or any other entity is a bad thing. |
|
| Back to top |
|
|
tullygray
n00b
Joined: 16 May 2008
Posts: 31
Location: Melbourne, Australia
|
| Posted: Sat Oct 25, 2008 3:11 pm Post subject: |
|
|
Ha! Next you will be saying that Microsoft is evil...  |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
| Posted: Sat Oct 25, 2008 9:00 pm Post subject: |
|
|
| spindle wrote: | | I'd rather have stuff show up in Google, even if they are evil, since that's where a ton of people go to for information and there is a lot of useful information here. But In the broader scope, I consider the forums to be a publicly available source of information, hiding it from Google or any other entity is a bad thing. |
i wholeheartedly agree. why infra decided to do this is beyond my comprehension. this only will further hurt the gentoo image.
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
aidanjt
Veteran
Joined: 20 Feb 2005
Posts: 1118
Location: Rep. of Ireland
|
| Posted: Thu Oct 30, 2008 12:55 am Post subject: |
|
|
| bunder wrote: | | Code: | Block all bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "msnbot|spider|snapbot|yodaobot|googlebot|Yahoo! Slurp" [NC]
RewriteRule ^/.*php - [F]
|
| Quote: | | someone's blocked all bots in the vhost config |
figured i'd give you guys an update. that happened in the summer.
| Quote: | (gentoo-infra-representative-who-i-will-not-name) decided google was evil.
and we need to have squid in the middle before he'll let us be exposed to evil again.
|
sounds like a huge load of crap to me, but i don't run the machines gentoo stuff runs on, so what do i know. |
I think his beef would be with google crawling through the multigigabyte volumes of postings. Thus the desire for a squiddy-in-the-middle solution to keep the PHP and DB trashing to a minimal when google-bot comes back again.
_________________
| juniper wrote: | | you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault. |
|
|
| Back to top |
|
|
rob1101
n00b
Joined: 03 Apr 2007
Posts: 30
|
| Posted: Thu Oct 30, 2008 7:12 am Post subject: |
|
|
| spindle wrote: |
While I'd like to see the search tool get better I'd want to see the site open up to bots again. I don't see the point of hiding from bots if Google is evil. I'd rather have stuff show up in Google, even if they are evil, since that's where a ton of people go to for information and there is a lot of useful information here. But In the broader scope, I consider the forums to be a publicly available source of information, hiding it from Google or any other entity is a bad thing. |
QFT, I completely agree. I would like to hear the devs reasoning behind this. Maybe they do have a good reason. |
|
| Back to top |
|
|
gentoo-dev
Apprentice
Joined: 24 Jan 2006
Posts: 172
|
| Posted: Thu Oct 30, 2008 11:55 am Post subject: |
|
|
| AidanJT wrote: | | bunder wrote: | | Code: | Block all bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "msnbot|spider|snapbot|yodaobot|googlebot|Yahoo! Slurp" [NC]
RewriteRule ^/.*php - [F]
|
| Quote: | | someone's blocked all bots in the vhost config |
figured i'd give you guys an update. that happened in the summer. 8)
| Quote: | (gentoo-infra-representative-who-i-will-not-name) decided google was evil.
and we need to have squid in the middle before he'll let us be exposed to evil again.
|
sounds like a huge load of crap to me, but i don't run the machines gentoo stuff runs on, so what do i know. :lol: |
I think his beef would be with google crawling through the multigigabyte volumes of postings. Thus the desire for a squiddy-in-the-middle solution to keep the PHP and DB trashing to a minimal when google-bot comes back again. |
Which was a stupid thing to do. Users can't search these forums anymore and will double post as a result. More data, more work for the moderators.
IMO, only bots that do not honor robots.txt directives such as Crawl-delay or request-rate should be blacklisted. |
|
| Back to top |
|
|
aidanjt
Veteran
Joined: 20 Feb 2005
Posts: 1118
Location: Rep. of Ireland
|
| Posted: Thu Oct 30, 2008 7:46 pm Post subject: |
|
|
| gentoo-dev wrote: | | Which was a stupid thing to do. Users can't search these forums anymore and will double post as a result. More data, more work for the moderators. |
Note the empty text box in the top right. Knock yourself out. It would be a different matter if it was b.g.o that blocked google bots, because bugzilla's built-in search is God awfully useless, phpBB on the other hand, is decent.
_________________
| juniper wrote: | | you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault. |
|
|
| Back to top |
|
|
swimmer
Veteran
Joined: 15 Jul 2002
Posts: 1330
Location: Netherlands
|
| Posted: Thu Oct 30, 2008 9:59 pm Post subject: |
|
|
Hmmm let's see ...
I was looking for the thread regarding SteveL's update script and I filled in the empty text box in the top right with "update" and got zarroo results. Nevermind, I'll go to the 'advanced search':
Keyword: update
Author: SteveL
zarroo results.
Decent.
EDIT:
On the other hand a google search on "site:forums.gentoo.org update stevel" lists the topic on first place.
Helpful. |
|
| Back to top |
|
|
aidanjt
Veteran
Joined: 20 Feb 2005
Posts: 1118
Location: Rep. of Ireland
|
| Posted: Thu Oct 30, 2008 10:08 pm Post subject: |
|
|
Update what? A pretty non-specific query, don't you think? If I don't know what you mean, how do you expect string searching to be any better?
_________________
| juniper wrote: | | you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault. |
|
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
| Posted: Thu Oct 30, 2008 10:14 pm Post subject: |
|
|
| swimmer wrote: | Hmmm let's see ...
I was looking for the thread regarding SteveL's update script and I filled in the empty text box in the top right with "update" and got zarroo results. Nevermind, I'll go to the 'advanced search':
Keyword: update
Author: SteveL
zarroo results.
Decent.
EDIT:
On the other hand a google search on "site:forums.gentoo.org update stevel" lists the topic on first place.
Helpful. |
providing this list is still accurate, the word "update" is on the stoplist.
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
swimmer
Veteran
Joined: 15 Jul 2002
Posts: 1330
Location: Netherlands
|
| Posted: Thu Oct 30, 2008 11:14 pm Post subject: |
|
|
Ohhh - thanks for that bunder!
I never was aware of such a list (or I supressed it at least ) ... |
|
| Back to top |
|
|
cubancigar11
Guru
Joined: 10 Jan 2005
Posts: 340
|
| Posted: Fri Oct 31, 2008 10:03 am Post subject: |
|
|
| It doesn't matter. Gentoo search engine is as crappy as it could get, this has been discussed to death on these forums (may I add, without any reply from infra people). For years. And the only solution was to use Google, and now even that is banned. |
|
| Back to top |
|
|
swimmer
Veteran
Joined: 15 Jul 2002
Posts: 1330
Location: Netherlands
|
| Posted: Fri Oct 31, 2008 10:06 am Post subject: |
|
|
| ++ |
|
| Back to top |
|
|
anello
Guru
Joined: 17 Jul 2005
Posts: 557
Location: EU -> DE -> Stuttgart
|
| Posted: Fri Oct 31, 2008 10:16 am Post subject: |
|
|
That sucks.
_________________
Antonino Catinello | http://catinello.eu |
|
| Back to top |
|
|
gentoo-dev
Apprentice
Joined: 24 Jan 2006
Posts: 172
|
| Posted: Fri Oct 31, 2008 11:53 am Post subject: |
|
|
| AidanJT wrote: | | gentoo-dev wrote: | | Which was a stupid thing to do. Users can't search these forums anymore and will double post as a result. More data, more work for the moderators. |
Note the empty text box in the top right. Knock yourself out. It would be a different matter if it was b.g.o that blocked google bots, because bugzilla's built-in search is God awfully useless, phpBB on the other hand, is decent. |
Note the amount of people who have complained about f.g.o search feature suckyness and maybe (but probably not) you'll understand how much searching these forums via google is needed.
Besides you still haven't explained why simple directives in robots.txt were not used. They would have helped, but no, it's so much easier to yell 'google is evil' and ban all bots indiscriminately. |
|
| Back to top |
|
|
Devport
Guru
Joined: 15 Dec 2004
Posts: 361
|
| Posted: Sat Nov 01, 2008 7:04 am Post subject: |
|
|
Sorry wrote some bullshit due to almost being asleep - please delete
Last edited by Devport on Sat Nov 01, 2008 7:56 am; edited 11 times in total |
|
| Back to top |
|
|
desultory
Bodhisattva
Joined: 04 Nov 2005
Posts: 9410
|
| Posted: Sat Nov 01, 2008 7:23 am Post subject: |
|
|
| Devport wrote: | You dismiss being one of the top Google search results in a lot of searches by one of the biggest companies in the world - others pay several thousands of dollars to get that ranking ? Because Google has evil branches like everything else has you shoot into your own foot ?
Do you also kill yourself when you meet the devil so he cant do it ?
Another myth is that the internet is highly addictive and evil - you better turn off your computer right NOW.
Even though I try to add some humor I hope you get the point. |
You seem to have missed humor yourself, the reference to Google being evil was in jest, and probably should never have been reproduced in public, and was a reference to their spider, among others, being overly taxing on certain services, for example Bugzilla. Given the circumstances there were two available options; let what amounted to a denial of service attack continue unchecked and effectively lose the service entirely while wasting resources to attempt to serve some out of control users, or to cut off access by those out of control users and allow the service to remain otherwise functional. Consider which you would do.
The only valid point of contention is whether the forums where under similarly unsustainable strain at the time the block was implemented, which is information I do not presently have.
| Devport wrote: | | I agree that Google has evil parts - yet not everything is bad and I dont think gentoo will rescue the world by sacrificing itself ( due to another ignorant decision ). If you want to change something meet other people and build a community of highly influential people and fight for privacy - but dont abuse the tiny power you have to hurt your own people. |
You missed the point, belaboring yours will not change anything any faster. |
|
| Back to top |
|
|
.yankee
Apprentice
Joined: 24 Feb 2008
Posts: 194
Location: Polska
|
| Posted: Sun Nov 02, 2008 1:17 am Post subject: |
|
|
| desultory wrote: |
Given the circumstances there were two available options; let what amounted to a denial of service attack continue unchecked and effectively lose the service entirely while wasting resources to attempt to serve some out of control users, or to cut off access by those out of control users and allow the service to remain otherwise functional. Consider which you would do.
|
For a short-term thinking, I'd agree. But for long-term - there's always a third option. For instance, start to consider a major change to be done - like merging, or even splitting the FGO. Surely, there are at least some subforums that could be exposed to the evil google teeth, while not doing much harm to the servers (especially, if they were kept on separate one(s)).
And apart from all being said about how things suck with google and how decent the local search is,
| tullygray wrote: | | Ha! Next you will be saying that Microsoft is evil... |
...It's good to know some people here actually have a sense of humour |
|
| Back to top |
|
|
desultory
Bodhisattva
Joined: 04 Nov 2005
Posts: 9410
|
| Posted: Sun Nov 02, 2008 4:42 am Post subject: |
|
|
| .yankee wrote: | | But for long-term - there's always a third option. For instance, start to consider a major change to be done - like merging, or even splitting the FGO. Surely, there are at least some subforums that could be exposed to the evil google teeth, while not doing much harm to the servers (especially, if they were kept on separate one(s)). |
No. More work for even less return is even more pointless. |
|
| Back to top |
|
|
|
Gentoo Forums Feedback
