Rant: Nepomuk and strigi - solution in search of a problem

Message

sam_i_am · Post by **sam_i_am** » Mon Dec 07, 2009 8:06 pm

I've been a long time Linux user (since Linux 0.99) and been using KDE as my primary desktop at work since 2003. Contrary to some other posts, I'm rather happy with KDE 4.3. Sure, there is an occasional crash and things changed for no apparent reason. But, all in all I've been happy and growing to like it. Kudos to KDE developers and Gentoo community.

The only exception to this is nepomuk and strigi. I really don't get why they exist. Yes, I've read about the semantic desktop and all the features they provide. But they are really trying to address a problem that I don't have. I give my file names a sensible name and I have no problem locating them. I have some marginal organization (few folders for specific things like documents, pictures, videos, music etc) and I manage fine. There are about 60G of files (including backups from my laptop) and about 1200 files, but I never felt that I could not find what I needed to find.

I mean, really! Are there any valid use cases to justify all that effort spent on the so called "desktop search"? How does that make my life easier? You mean I have to tag them with something sensible? Why? That is adding more work for me.

Today, I found that nepomuk database for my 60G of files is about 2G. I've been experiencing freezes in Dolphin (I can see CPU usage shooting up for about 5 sec on my 2.4 GHz Core2Quad with 4G ram) whenever I change something. Someone mentioned that removing Dolphin configuration files helped. But, it didn't. My guess is that it is groping through that 2G of its file index and trying to update it.

Every time I start KDE, I get about four messages from Strigi saying it is indexing files and takes up about 10-15 sec.

So, for now, I've disabled both nepomuk and strigi. I really hope that won't break anything I regularly use. I'll keep reading about them to see if ever they make my life easier. But, I won't hold my breath.

Sam
PS: There. I feel better now

yngwin · Post by **yngwin** » Mon Dec 07, 2009 8:19 pm

I completely agree. Useless bloat.

platojones · Post by **platojones** » Mon Dec 07, 2009 8:22 pm

I'll reserve judgment on it's usefulness if and when it's actually half-way stable. The biggest problem I see is that it seems to be alpha-quality code, at best. Every version, while slightly more stable than the last, is riddled with massive bugs and crashes or goes crazy often. I know they want to get feedback on it, but in it's current form, it's virtually unusable and brings the entire experience down...it's a pathetically weak link in an otherwise very nice chain.

gerard27 · Post by **gerard27** » Mon Dec 07, 2009 8:41 pm

I have removed nepomuk.
I masked it too in /etc/portage/package.mask.
I don't use pim and therefore equery depends nepomuk shows a few things
like amarok.
By adding -semantic-desktop in /etc/make.conf this went away also.
Gerard.

NathanZachary · Post by **NathanZachary** » Mon Dec 07, 2009 9:12 pm

I completely agree with your sentiments. As far as I can tell, you can safely remove nepomuk without any repercussions. I'm not sure if you can completely remove strigi without being notified, but that would be nice as well. The bloat was one of the reasons I switched away from KDE as my primary environment (and that was back with KDE3, not even 4). I primarily use Openbox with a bunch of components from LXDE. BUT, I still do like many of the aspects of KDE.

DaggyStyle · Post by **DaggyStyle** » Mon Dec 07, 2009 9:53 pm

I'm trying to get rid of them, adding - semantic-desktop to make.conf returns this:

Code: Select all

!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:                

kde-base/kdebase-runtime-meta:4.3

  ('installed', '/', 'kde-base/kdebase-runtime-meta-4.3.4-r1', 'nomerge') pulled in by                                                                          
    >=kde-base/kdebase-runtime-meta-4.3.4[-kdeprefix,semantic-desktop,handbook] required by ('installed', '/', 'kde-base/kdelibs-4.3.4', 'nomerge')             
    (and 2 more)                                                                

  ('ebuild', '/', 'kde-base/kdebase-runtime-meta-4.3.4-r1', 'merge') pulled in by                                                                               
    >=kde-base/kdebase-runtime-meta-4.3.4[-kdeprefix,handbook] required by ('ebuild', '/', 'kde-base/kdelibs-4.3.4', 'merge')                                   
    >=kde-base/kdebase-runtime-meta-4.3.4[-kdeprefix] required by ('installed', '/', 'kde-base/kdebase-startkde-4.3.4', 'nomerge')                              

  Explanation:

    New USE for 'kde-base/kdebase-runtime-meta:4.3' are incorrectly set.
    In order to solve this, adjust USE to satisfy '>=kde-base/kdebase-  
    runtime-meta-4.3.4[-kdeprefix,semantic-desktop,handbook]'.          

kde-base/kdelibs:4.3

  ('ebuild', '/', 'kde-base/kdelibs-4.3.4', 'merge') pulled in by
    >=kde-base/kdelibs-4.3.4[-kdeprefix] required by ('installed', '/', 'kde-base/libksane-4.3.4', 'nomerge')
    kde-base/kdelibs required by world
    >=kde-base/kdelibs-4.3.4[-kdeprefix] required by ('installed', '/', 'kde-base/ksystraycmd-4.3.4', 'nomerge')
    (and 180 more)

  ('installed', '/', 'kde-base/kdelibs-4.3.4', 'nomerge') pulled in by
    >=kde-base/kdelibs-4.3[semantic-desktop] required by ('installed', '/', 'kde-misc/kbluetooth-9999', 'nomerge')
    >=kde-base/kdelibs-4.2[semantic-desktop] required by ('installed', '/', 'kde-misc/krecipes-2.0_alpha4', 'nomerge')
    >=kde-base/kdelibs-4.3[semantic-desktop] required by ('installed', '/', 'media-video/kdenlive-0.7.6', 'nomerge')
    (and 183 more)

  Explanation:

    New USE for 'kde-base/kdelibs:4.3' are incorrectly set. In order to
    solve this, adjust USE to satisfy '>=kde-base/kdelibs-4.2[semantic-
    desktop]' and '>=kde-base/kdelibs-4.3[semantic-desktop]'.

question is, can this programs work without it? it seems that it is not that easy to get rid of it.

gerard27 · Post by **gerard27** » Mon Dec 07, 2009 10:39 pm

I am sorry it's near midnight here and I want to go to bed.
Nevertheless I tried emerge -ep world and it didn't give me any errors.
So it's not this -semantic-desktop USE flag that's causing your problem.
Plse do

Code: Select all

equery depends nepomuk

If there are packages you can't do without you'll have to leave it in.
Gerard.

DaggyStyle · Post by **DaggyStyle** » Tue Dec 08, 2009 6:27 am

unfortunately, like amarok depends on strigi, kbluetooth (svn version) depends on nepomuk.
thus making it impossible to remove it, I did disbled then both in kde but still nepomuk is running in the memory.

wjb · Post by **wjb** » Tue Dec 08, 2009 10:54 am

You can cut down what its indexing using: System Settings -> Advanced User Settings -> Desktop Search -> Advanced Settings

What IS it for? Anyone ...

gerard27 · Post by **gerard27** » Tue Dec 08, 2009 11:49 am

This is a line from common depends in /usr/portage/net-wireless/kbluetooth:

Code: Select all

semantic-desktop? ( >=kde-base/nepomuk-${KDE_MINIMAL} )"

To the best of my knowledge this means nepomuk is needed if semantic desktop is set in /etc/make.conf.
I am sure there's more like these in other ebuilds.
It will be quite a lot of work but this way you'll be able to find out which progs can do w/o nepomuk.
Gerard.

scotus africanus · Post by **scotus africanus** » Tue Dec 08, 2009 12:09 pm

i mostly agree with the sentiment here, disabling these pesky dependencies was more like an exorcism...
however, do spare a thought for the guys developing this interesting technology, they need the feedback to make it better. i once got it to work (cant remember the spell i cast!) and was pleasantly surprised, but like you, for day to day stuff it gets in the way
sa.

DaggyStyle · Post by **DaggyStyle** » Tue Dec 08, 2009 12:26 pm

gerard82 wrote:This is a line from common depends in /usr/portage/net-wireless/kbluetooth:
Code: Select all
semantic-desktop? ( >=kde-base/nepomuk-${KDE_MINIMAL} )"
To the best of my knowledge this means nepomuk is needed if semantic desktop is set in /etc/make.conf.
I am sure there's more like these in other ebuilds.
It will be quite a lot of work but this way you'll be able to find out which progs can do w/o nepomuk.
Gerard.

mmm, just noticed it is the same, one (using a svn version) will modify it.

DaggyStyle · Post by **DaggyStyle** » Tue Dec 08, 2009 10:17 pm

doesn't soprano is part of the same thing?

jbouzan · Post by **jbouzan** » Wed Dec 09, 2009 6:17 am

So I have nepomuk enabled, I've never really thought about it. Reading what it's supposed to do, I thought it'd be cool to see what it actually does. But I haven't tried desktop searches without nepomuk enabled. Is the find tool in dolphin disabled or just less functional? Is there anything like apple's spotlight tool to perform FAYT searches with nepomuk data?

Just to add my bit to this talk, although I personally never use desktop search (like sam_i_am said, basic organization means it's never necessary), I've seen several people who desperately need such software. In particular, one friend is computer literate in the sense of being able to perform any basic task, but never bothers to give files consistent names or put them into their respective folders. She has folders for every class and subject, but they aren't in a logical tree, and files often end up in a higher-level directory than could be. So it seems to me that some people really do need it, but unless or until there is something like I asked above it doesn't help in the way it's needed.

Aquiles · Post by **Aquiles** » Wed Dec 09, 2009 8:13 am

I partly agree with the original poster. However, I have found a good use to the so called Desktop Search. Although it is easy for me to find what I am looking for without any help other than the logical folder structure of my home directory, it is easier and faster to use the KDE application launcher (Alt+F2 by default) to also launch files. If a file named 'testdocument.txt' is indexed I can Alt+F2, then type testdocument and I will get an entry to open the file. This is more convenient than browsing through the filesystem until you reach the folder where the file lives. Which does not mean that there are no other ways to get quickly to frequently used files.

However, I am not sure about whether this feature pays off. I have to say that the Spotlight engine used in Mac OS X is far less consuming than Strigi. Strigi seems to perform some kind of indexing each time I start KDE, while Spotlight is only noticeable when there are huge changes in the filesystem.

monkeygirl · Post by **monkeygirl** » Wed Dec 09, 2009 5:00 pm

I turned this feature off because it was slow on my netbook, but to be honest, I had no idea what it was for and the nepomuk home page objectives made me go 'wtf?!'

Are the desktop searches like the line commands locate or find or are they supposed to be faster? The slow start up for indexing just seems kind of pointless.

keenblade · Post by **keenblade** » Thu Dec 10, 2009 9:00 pm

monkeygirl wrote:...Are the desktop searches like the line commands locate or find or are they supposed to be faster? The slow start up for indexing just seems kind of pointless.

This technology provides an interesting thing that do not exist in locate or find commands. The text search in the contents of pdf files or office files, even in jpeg vs. (actually you can search text with find command, but it will be dog slow to search all the hard disks.). So for ex: you can find a technical term easily in your pdf files for your scientific work.
This sounds very good to legitimate the use of this tech. But I always feel, this kind of tech is here not to help us, but to help someone to steal what we have on our pc. Conspiracy theory here

Plague.CZ · Post by **Plague.CZ** » Thu Dec 10, 2009 9:40 pm

OMG finally some people with sense

But on a more serious note... I understand that some people need such software, and I'm glad that there are people satisfying ther peoples needs, BUT why the hell do we have HARD dependencies on such bloats? I chose Gentoo because I wanted to select which to install and run, why do many KDE apps these days take this from me? It's not only desktop search, this can be disabled easily, but what about hal (i have it installed, but never run it), akonadi-server and others?

sam_i_am · Post by **sam_i_am** » Fri Dec 11, 2009 6:52 pm

The biggest application area I've seen for this kind of search tools is in indexing personal media libraries (photos, music etc). But then again, to make such a system work, you have to add annotations or ratings. I guess if you have thousands of music tracks, you can rate it while you are listening and then later easily can pick the ones you find interesting.

However, that is a rather specialized use case and there are tools that can handle it if needed.

boris_qd · Post by **boris_qd** » Fri Dec 11, 2009 7:37 pm

I actually like them quite a bit. I have a large collection of science papers/notes/books etc. and use the search functionality to find pieces of information I'm looking for. I think it's great.

My problem is that it doesn't seem to search djvu files. If I could get that working I would really be a happy camper.

cjubon · Post by **cjubon** » Sat Dec 12, 2009 10:41 am

I fully agree with the original poster—apart from being buggy, it's useless if you don't need it
But then, I also fully agree with the most recent answer (boris_qd)—basically, the idea makes sense and it may be quite useful for a lot of people.

IMHO, there are two main issues with the current nepomuk implementation:

It's already been said that the code seems to be quite immature—I experienced regular freezes on my funtoo amd64 x2 machine until I turned that feature off.
My main concern is that the whole thing does not seem to be truely modular: for example, in order to install media-video/kdenlive, I need USE flag semantic-desktop turned on in kde-base/kdelibs.
Further, USE flag "semantic-desktop" does different things for different packages: is pulls in kde-base/nepomuk for dolphin, kget, kmail, gwenview, etc., but does not do so for kde-base/kdelibs.

As far as Gentoo developers are concerned (and not upstream devs), I'd suggest to remove the semantic-desktop flag from kde-base/kdelibs, and leave it for those packages that aktually pull in kde-base/nepomuk.

EDIT: @boris_qd: I think the djvu issue is worth being filed on bugs.kde.org.

albright · Post by **albright** » Sat Dec 12, 2009 1:13 pm

I actually like them quite a bit. I have a large collection of science papers/notes/books etc. and use the search functionality to find pieces of information I'm looking for. I think it's great.

yes that would be good but as others have observed nepomuk+strigi
crashes and/or pegs cpu regularly

But even when I've had the index complete I found an absolute
show stopper: not all files were indexed!!

Files I knew existed did not show up in my search results. That really
makes the thing not only useless but dangerous if you ever start to
rely on it.

jackgentoo · Post by **jackgentoo** » Thu Jan 14, 2010 4:40 am

My machine was swapping like mad. I found that the nepomukstorage process was taking up ALMOST 2GB of RAM. Useless ...

durian · Post by **durian** » Thu Jan 14, 2010 6:53 am

It was the first thing I disabled in KDE...

Should be possible to make it unintrusive though - I have a macBookPro which indexes everything in Spotlight without me noticing anything...

-peter

NightMonkey · Post by **NightMonkey** » Thu Jan 14, 2010 7:53 am

As a Windows desktop admin, I have found that killing:

Microsoft Indexing Service
Google Desktop Search

makes the underpowered (for Windows) machines behave much better. If one REALLY needs to find something, you're willing to wait a bit for a true broad data search. But, to index ALL THE TIME means that the disk is going ALL THE TIME. Every write has a double, sometimes triple disk access penalty. I don't care how Mac OS X hides it in Spotlight, because it can't get around the fact that it has to READ THE DATA to index it, which means that the disk is going to be over-accessed, the filesystem cache is going to be wiped out, etc. etc. in Windows, Mac OS X or Linux.

Granted, here in Linux Desktop land we don't also have to worry about contention with AV software (generally), but in Windows, as many of you know, it is quite a ruse to pull on users. "Gosh, why is my machine is so slooow?"

Rant complete.

EDIT:

P.S. I've been killing the Microsoft Indexing Service since 2000. Ten years of thankful hard drives!

Rant: Nepomuk and strigi - solution in search of a problem

Rant: Nepomuk and strigi - solution in search of a problem

nepomuk/strigi