Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo forum search sucks
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3 ... 12, 13, 14  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Forums Feedback
View previous topic :: View next topic  
Author Message
madpenguin8
n00b
n00b


Joined: 19 Jun 2002
Posts: 38
Location: Detroit MI

PostPosted: Sun Feb 02, 2003 9:15 pm    Post subject: Gentoo forum search sucks Reply with quote

Anyone else think that the gentoo forum search sucks. It comes up with just about nothing relevant to my search. It seems like really popular posts that have absolutely nothing to do with what I am searching for always turn up close to the top. Writing up a dupe sucks, but what can a guy do if search turns up garbage.
_________________
Have you ever stopped to think...............and never start again?
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Sun Feb 02, 2003 9:20 pm    Post subject: Reply with quote

Some tips: make sure your search terms are all 3+ letters long. Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want. Use many anded search terms, and widen the search later. Try to choose obscure words if you can: unusual words that appear in an exact error message are usually very good. Things like "bug", "portage", "broken" are bad.

Do you have any recommendations for improving the search engine while still being able to upgrade phpBB in a timely fashion?


EDIT: To consolidate some search threads, I'm including the following quote from another thread. -- pjp
rac wrote:
Here's how the phpBB search function works. Each post is split into words. First, some characters are replaced. There are three classes of characters here, those that get replaced by spaces, those that get elided, and those that get left alone. Next, whitespace is used to delineate words. All words of less than 3 or more than 20 characters are dropped. Then an entry is made in the dictionary table for every word that is not in the dictionary, so that it can be referenced by number. An entry is made in a colossal table for each and every word in each and every post. That's what gets searched against.

To get back to Reformist's two examples, gnome2 is a word. 'gnome 2' is two words, one of which is impossible to match because it is one character long. "1.1.0" is three words, each of which are impossible to match, because they are one character long. One modification that it might be feasible to make would be to change the status of '.'. If it were left alone, version numbers would become searchable. However, words at the end of sentences, followed by periods, would become unsearchable, because a separate entry would be made including the period. If it were elided, the end-of-sentence problem would go away, but then you would have to search for "abiword and 110", and "2.1" would become "21" and fall under the three-character limit.

_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
PARENA
Guru
Guru


Joined: 06 Jan 2003
Posts: 349

PostPosted: Sun Feb 02, 2003 9:55 pm    Post subject: Reply with quote

Here's an example of a (what I think) good search. Let's say you want to know about using the slit in Openbox:

openbox AND slit

(almost) guaranteed to come up with your answer. Unless it's not on the forum of course. :)
Back to top
View user's profile Send private message
idl
Retired Dev
Retired Dev


Joined: 24 Dec 2002
Posts: 1728
Location: Nottingham, UK

PostPosted: Sun Feb 02, 2003 10:42 pm    Post subject: Reply with quote

would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google :)
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Sun Feb 02, 2003 11:50 pm    Post subject: Reply with quote

port001 wrote:
would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google :)
That, while a nice feature, is completely impossible with the current way the search databases work, because the search match tables have fields for word number and post id only. There is no sense of what words occur next to one another.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
Matje
l33t
l33t


Joined: 29 Oct 2002
Posts: 617
Location: Hasselt, Belgium

PostPosted: Mon Feb 03, 2003 12:35 am    Post subject: Reply with quote

rac wrote:
port001 wrote:
would be cool if we could do the old "kde3.1 emerge failure" note the "" that makes sure it matches the whole string. like on google :)
That, while a nice feature, is completely impossible with the current way the search databases work, because the search match tables have fields for word number and post id only. There is no sense of what words occur next to one another.

It _is_ possible, by making php look into the actual post texts. Would be a query like:
SELECT post_id FROM phpbb_posts_text WHERE post_text LIKE '%$searchstring%'
But this'll butcher the mysql I think with these number of posts on the forum :-)
_________________
Life is like a box of chocolates... Before you know it, it's empty...
Back to top
View user's profile Send private message
xlyz
Veteran
Veteran


Joined: 27 Oct 2002
Posts: 1470
Location: Italy

PostPosted: Mon Feb 03, 2003 2:08 am    Post subject: Reply with quote

rac wrote:
Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want.


please make "AND" default. "OR" is seldom used
Back to top
View user's profile Send private message
gsfgf
Veteran
Veteran


Joined: 08 May 2002
Posts: 1266

PostPosted: Mon Feb 03, 2003 3:01 am    Post subject: Reply with quote

xlyz wrote:
rac wrote:
Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want.


please make "AND" default. "OR" is seldom used

at least for the quick search.
Back to top
View user's profile Send private message
pilla
Administrator
Administrator


Joined: 07 Aug 2002
Posts: 7214
Location: Pelotas, BR

PostPosted: Mon Feb 03, 2003 3:07 am    Post subject: Reply with quote

Moving to Gentoo Forum Feedback.
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin
Back to top
View user's profile Send private message
Matje
l33t
l33t


Joined: 29 Oct 2002
Posts: 617
Location: Hasselt, Belgium

PostPosted: Mon Feb 03, 2003 10:45 am    Post subject: Reply with quote

gsfgf wrote:
xlyz wrote:
rac wrote:
Join all your terms with 'and' or check "search for all terms", otherwise the default is "or", which is probably not what you want.


please make "AND" default. "OR" is seldom used

at least for the quick search.

Indeed, I'd like that too, would be a great improvement.
_________________
Life is like a box of chocolates... Before you know it, it's empty...
Back to top
View user's profile Send private message
David_Escott
l33t
l33t


Joined: 12 Jan 2003
Posts: 952
Location: Boston, MA

PostPosted: Mon Feb 03, 2003 1:37 pm    Post subject: Reply with quote

I have the compelling urge to say the following:

THIS IS A DUPLICATE THREAD.
PLEASE SEARCH

http://forums.gentoo.org/viewtopic.php?t=30782
was the closest I could get, searching was a pain because well it sucks (sorry that was a duplicate thought)
But I do remember seeing one thread where rac may have explained some of the difficulties in trying to make search work a little better, if only I could find it again :(

Ahhh found it
http://forums.gentoo.org/viewtopic.php?p=167526#167526 down the page rac explains some of the difficulties
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Mon Feb 03, 2003 10:13 pm    Post subject: Reply with quote

Thanks for those links, David_Escott. As a journey of a thousand miles begins with a single step, I have changed the default setting of the checkbox in the search screen to "Search for all terms". Hopefully this will improve things in some small measure. Note that this change only affects the purple "gentoo" theme - people still using subSilver will not be affected.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
xlyz
Veteran
Veteran


Joined: 27 Oct 2002
Posts: 1470
Location: Italy

PostPosted: Mon Feb 03, 2003 10:51 pm    Post subject: Reply with quote

quick search is still "search for any words" as default

are you going to change it as well?

TIA
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Mon Feb 03, 2003 11:30 pm    Post subject: Reply with quote

Apparently phpBB was caching the template file, so my changes to quicksearch weren't taking hold. It should be fixed now.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
gsfgf
Veteran
Veteran


Joined: 08 May 2002
Posts: 1266

PostPosted: Mon Feb 03, 2003 11:40 pm    Post subject: Reply with quote

Rac i noticed in yopur other post that version #s aren't searchable. If you convert 2.2.2 to 222 and make search strip periods as well so if you search for kde 3.1 search will treat it as kde 31, that would solve that issue. That may be harder than it looks, though.
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Mon Feb 03, 2003 11:48 pm    Post subject: Reply with quote

gsfgf wrote:
If you convert 2.2.2 to 222 and make search strip periods as well so if you search for kde 3.1 search will treat it as kde 31, that would solve that issue. That may be harder than it looks, though.
It is harder than it looks, for a couple of reasons. Either a period causes a word break or it doesn't. Now what would be best is if it caused a word break only if it wasn't a version number, but that could be a challenging regex. Maybe we could steal it from Portage. I think if we're going to go this far, we might as well get it right and have "version numbers" go into the index.

I just had an idea on how this might be implemented. Details later.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
phong
Bodhisattva
Bodhisattva


Joined: 16 Jul 2002
Posts: 778
Location: Michigan - 15 & Ryan

PostPosted: Wed Feb 05, 2003 4:35 am    Post subject: Reply with quote

Not that tough of a regex... For matching (in a generic way) words of 3+ characters and version numbers, I might use the following for matching words to put into the index (offhand, could be more robust):
(\b\w{3,}\b|\b\d(\.\d){1,2}\b)

What's the current regexp?
_________________
"An empty head is not really empty; it is stuffed with rubbish. Hence the difficulty of forcing anything into an empty head."
-- Eric Hoffer
Back to top
View user's profile Send private message
Lion
Apprentice
Apprentice


Joined: 23 Jun 2002
Posts: 207

PostPosted: Sat Feb 08, 2003 10:07 am    Post subject: Search still broken Reply with quote

I think there is still something basically wrong with search.
I always try to include as many relevant search terms in my query, but still I do not get posts that I know to be available.

Simple example: Search for the word 'world'.
'No topics or posts met your search criteria'.
I know this is not true, because the world file is mentioned in many posts.
Search for 'world AND file'.
Thousands of posts, many of which do NO contain the word 'world'.

So, my question is: what am I doing wrong?
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Sat Feb 08, 2003 7:48 pm    Post subject: Re: Search still broken Reply with quote

Lion wrote:
So, my question is: what am I doing wrong?
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function. Words on the stopword list are not indexed because they are too common. Unfortunately for your example, world is on the stopword list, so nothing shows up, and then when you search for "world and file", you are really only searching for 'file'.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16104
Location: Colorado

PostPosted: Sat Feb 08, 2003 8:02 pm    Post subject: Reply with quote

Maybe emerge, portage, gentoo, gnome and alot of others should be added to the list :D
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
xlyz
Veteran
Veteran


Joined: 27 Oct 2002
Posts: 1470
Location: Italy

PostPosted: Sat Feb 08, 2003 8:59 pm    Post subject: Re: Search still broken Reply with quote

rac wrote:
Lion wrote:
So, my question is: what am I doing wrong?
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function.


what are the words included in the list?
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Sun Feb 09, 2003 12:46 am    Post subject: Re: Search still broken Reply with quote

xlyz wrote:
what are the words included in the list?
http://forums.gentoo.org/language/lang_english/search_stopwords.txt
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 737
Location: US-FL-EST

PostPosted: Thu Feb 13, 2003 2:45 am    Post subject: Re: Search still broken Reply with quote

rac wrote:
xlyz wrote:
what are the words included in the list?
http://forums.gentoo.org/language/lang_english/search_stopwords.txt


I assume that gentoo bugzilla has a similar stopword list? It certainly would explain some of my difficulties in using search there.

Do you have an inkling (and could you share the location) of the search stopword list there?
_________________
People whom think M$ is mediocre, don't know the half of it.
Back to top
View user's profile Send private message
rac
Bodhisattva
Bodhisattva


Joined: 30 May 2002
Posts: 6553
Location: Japanifornia

PostPosted: Thu Feb 13, 2003 6:04 pm    Post subject: Re: Search still broken Reply with quote

dufeu wrote:
I assume that gentoo bugzilla has a similar stopword list?
Bugzilla's completely different software. I don't know off the top of my head whether there's a stopword list. If I get some time I may look into it further.
_________________
For every higher wall, there is a taller ladder
Back to top
View user's profile Send private message
edoloughlin
n00b
n00b


Joined: 02 Oct 2002
Posts: 12
Location: Ireland

PostPosted: Wed Feb 19, 2003 10:16 am    Post subject: Re: Search still broken Reply with quote

rac wrote:
Lion wrote:
So, my question is: what am I doing wrong?
Being unlucky. To help keep the size of the search tables down, there is a "stopword list" in phpBB's search function. Words on the stopword list are not indexed because they are too common. Unfortunately for your example, world is on the stopword list, so nothing shows up, and then when you search for "world and file", you are really only searching for 'file'.


Perhaps a lesson could be learned from Google. Stopwords are identified if they are included in a search, viz:
Quote:
"the" is a very common word and was not included in your search
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Forums Feedback All times are GMT
Goto page 1, 2, 3 ... 12, 13, 14  Next
Page 1 of 14

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum