PFL looking for new owner!

tuxmainy · n00b Joined: 15 Nov 2016 Posts: 8

Hi,
as the subject says: I, the founder and current owner of PFL (https://portagefilelist.de/), am looking for a new owner. You may have noticed the recent lack of maintenance and support of PFL. The reasons are mainly private and shouldn't be discussed here.

So here I am after ~15 years asking for anyone who want's to take over PFL. This includes the domain as well. The software can be found in github (https://github.com/portagefilelist). The website code is missing because it contains code which is not licensed by me. So I guess the web site has to be rewritten. And to be honest the website really needs a refresh

So from my point of view your job would be:
1. install all the server side stuff on your / a server
2. create a new website for doing queries (remember the API used by e-file)
3. take over the domain
4. have fun with maintaining the data and PFL client side stuff

Anyone interested? Otherwise I have to shutdown PFL. Sorry

regards
Daniel

PS: May an admin pin this post to the top? I think this is an important request.

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

I'm sorry to hear you have to move on from the project (it sounds like maybe you would have preferred to not).

I've made the post an Announcement, so it will remain at the top until there is a hand off or you've had to shut it down.

Could you be more specific about the work involved? How is the list of files generated? What would that look like for a person hosting this, or paying for compute time somewhere?

You mention a new front end would be needed. Is the old front end not transferable to the new owner until they are able to create a new one?

Not knowing what is involved, I'd personally like to see something like that provided by Gentoo.
_________________
Quis separabit? Quo animo?

tuxmainy · n00b Joined: 15 Nov 2016 Posts: 8

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

Thanks for clarifying. Hopefully that will be useful information for someone. It's a bit beyond my comfort level and budget (0), otherwise I would consider it.

flexibeast · Posted: Mon May 15, 2023 1:10 am Post subject:

@tuxmainy,

i just want to say thanks for maintaining PFL, and for so long - i regularly make use of it. Hope someone is able to take it over soon!

eccerr0r · Posted: Mon May 15, 2023 1:32 am Post subject:

Curious of the bandwidth required to maintain the service?

Though I love running services on my computer, I have neither the network bandwidth or the funds to maintain the DNS name after it expires, sorry :(
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

spica · Apprentice Joined: 04 Jun 2021 Posts: 290

What are the monthly costs for servers?
How much disk storage needed for database?
If someone can start replicating web ui now, how much time he has?

NeddySeagoon · Posted: Mon May 15, 2023 12:50 pm Post subject:

spica,

Server Auction is one place to look for a whole server. I have one of those.
It hosts my Raspberry Pi arm64 binhost among other things.

Maybe you don't need a whole server and a VM or web hosting would work?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

spica · Apprentice Joined: 04 Jun 2021 Posts: 290

NeddySeagoon,
No, no. I want to know the current spendings of the project, how much it costs now. If I know current numbers then I can compare with my infrastructure and then I can understand can I give a shelter or not.
It's a question about requirements for the service
I guess a $5 t3.micro should be enough

ARomaSH · n00b Joined: 10 May 2005 Posts: 4

tuxmainy · n00b Joined: 15 Nov 2016 Posts: 8

Generally on the hardware: I experienced most trouble with CPU and RAM while inserting new records because of index regeneration and users doing wildcard queries which couldn't be handled be the index (e.g. *foo). The database should have a fair amount of RAM available to handle the indexes. Disk I/O might be a problem but using standard SSDs should be fine.

Using another DBMS which is build for the amount of data might resolve some performance problems (like using NO-SQL?). But my database knowledge is limited to SQL. So...

szatox · Advocate Joined: 27 Aug 2013 Posts: 3172

Banana · Veteran Joined: 21 May 2004 Posts: 1416 Location: Germany

Well I'm from germany, I do have a host and I do know PHP, but the DB size would be a pain.

Are there are historical data in it which could be "archived"?
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.

tuxmainy · n00b Joined: 15 Nov 2016 Posts: 8

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

It may help to post an announcement on the PFL website itself.

Maybe even an announcement / update to the cli tool? I'm not sure how feasible that is.
_________________
Quis separabit? Quo animo?

Banana · Veteran Joined: 21 May 2004 Posts: 1416 Location: Germany

I will have a look. Currently I see the website which is able to do a search and display the results, python on the client side which creates a xml and uploads, python on the client which can query, php on the server side to accept the uploaded xml and another php file to import those files.

My idea would be to create a fork and some new server stuff and go from there. I will respond here if my research/experiment is working or not.
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.

geki · Posted: Sat May 20, 2023 11:56 am Post subject:

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

Hu · Administrator Joined: 06 Mar 2007 Posts: 21773

I believe one of the motivations for user submissions was the prevalence of packages where USE=foo controls whether /usr/bin/foo is built at all. For such packages, any one build may not be representative of all the files the package can generate, so the database needs a sampling of different build configurations. Some packages have many USE flags, so building all possible combinations is infeasible (and not all combinations will actually vary the file list). Arguably, an extension on top of the USE system where the ebuild tracks which flags enable/disable building particular files could help there, but presently Gentoo maintainers are not asked to compile that list in any sort of machine-readable form.

geki · Posted: Sun May 21, 2023 8:52 pm Post subject:

It gets all the more fun when we collect files not only from portage tree packages, but also from user overlay packages. Anyway, it's just something I spend a little of my spare time doing for fun. See, if my variant of NoSQL approach gets somewhere. But I think we better wait for what the other(s) come up with.

_________________
hear hear

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

tuxmainy · n00b Joined: 15 Nov 2016 Posts: 8

Hi,
the current dump of the SQL database can be found at https://portagefilelist.de/pfl-files.sql.gz

PFL is used for 3 different questions:

Which file is own by which package? Sometimes it's not easy to see even if you don't have to enable a special useflag
Which file is build by which useflag and architecture? Use case obvoius, I hope
How did the files changed over time? Sometimes you lose a file after an update and it would be nice to see where it is gone.

Point 2 is exactly where you need a huge user base for collecting because covering all useflags and architectrues won't be possible for a singe person, eh?

The current pfl collect script ignores user repositories and I really recommend to not collect user repositories data. This data is not helpfully as it doesn't help someone to know that the file they are looking for is in a package which they cannot emerge. Also this doesn't seem to be a good idea from a data privacy point of view. Imagine you have written a tool called "mirror my favorite p*rnh*b videos" which then gets collected (not the tool itself but the name) and published on a website. Or even more sensitive data like usernames or hashs which are coded into a filenames.
And to be honest: It happened in an early version of PFL (not intended, just never thought about). So I have seen data from user repositoies. I promise to you, this data is mostly garbage at all

Have a look into the current collect script. I think it's easy to understand (at least I hope so

)

regards
Daniel

Banana · Veteran Joined: 21 May 2004 Posts: 1416 Location: Germany

I've currently "rewrote" some of the stuff to get the idea how each part works. I'll have a working example in the next couple of days.
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.

geki · Posted: Sat May 27, 2023 1:55 pm Post subject:

Banana · Veteran Joined: 21 May 2004 Posts: 1416 Location: Germany

I've got a working example (first draft) over here: https://delta-labs.org/portagefilelist/

What it does: you can search for a filename based on my local data collection. Here are some names you can search for: hugo, apache, kernel, mupdf, vim

What it does NOT (yet): show packages, versions. Data endpoint is disabled right now.
I've just implementend the import and frontend. Clientside will be untouched for now, since I do not see any problems with it so far.

@tuxmainy
How do you plan any further steps? Do you want a complete example before switching over?

@geki
If you have also ideas or examples, or whatever, we can join forces.
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.