http://gentoo-{wiki,portage}.com doesnt work for me?

Message

jagland · Post by **jagland** » Wed Oct 22, 2008 6:06 pm

It's going to be a really disaster if the gentoo-wiki / gentoo-portage owner can't get his data back. Can't someone from Gentoo (e.g. from the Foundation or one of the Trustees) get involved? Personally I think the skiplink lot should go burn in hell

for not giving him the info he needs (I wouldn't be shocked if they are so disorganised that they don't have the info!)

AllenJB · Post by **AllenJB** » Wed Oct 22, 2008 6:36 pm

M wrote:I made a script to fetch pages from google cache, just in case, I got panicked when I found out about this drama You can find script here . My ip was banned while I tested the script but now I started script from work and it fetched about 300 pages so far. Anyway I hope everything will be ok.

In many ways I think it would be hugely beneficial to Gentoo Wiki to start over from scratch. There's a ton of legacy that it would benefit from losing.

jagland wrote:It's going to be a really disaster if the gentoo-wiki / gentoo-portage owner can't get his data back. Can't someone from Gentoo (e.g. from the Foundation or one of the Trustees) get involved? Personally I think the skiplink lot should go burn in hell for not giving him the info he needs (I wouldn't be shocked if they are so disorganised that they don't have the info!)

What exactly is "someone from Gentoo" going to be able to do with regards to the current gentoo-wiki site that the owner can't?

I'd also avoid commenting on situations you know nothing about. We know absolutely nothing about Skiplinks situation. What if they were forced to do this because one of their customers didn't pay up and it's no fault of their own? Would you still wish them to burn in hell? They may have far more important things to deal with at the present time.

jagland · Post by **jagland** » Wed Oct 22, 2008 7:53 pm

AllenJB wrote:What exactly is "someone from Gentoo" going to be able to do with regards to the current gentoo-wiki site that the owner can't?

I'd also avoid commenting on situations you know nothing about. We know absolutely nothing about Skiplinks situation. What if they were forced to do this because one of their customers didn't pay up and it's no fault of their own? Would you still wish them to burn in hell? They may have far more important things to deal with at the present time.

Sorry I'm just a bit flaming and frustrated and was wondering if some contact from someone from Gentoo to this Skiplink company might backup the Gentoo-wiki guys case.

Gentoo-wiki is as far as I'm concerned a very useful resource. I admit that it wasn't always 100% accurate, but what Wiki is? - If you find something thats that wrong then correct it!

I just think skiplink are being very awkward and should at least ensure that their customers can get their data. Particularly as the telecoms co concerned are co-operating with the Gentoo-Wiki owner.

Then there's the issue of there not being an offsite back-up, that was where the Gentoo-Wiki guy went wrong, so I hope the community at large are able to help here..and it's good to here that he has recieved many offers of help!

kernelOfTruth · Post by **kernelOfTruth** » Wed Oct 22, 2008 7:53 pm

AllenJB wrote:
M wrote:I made a script to fetch pages from google cache, just in case, I got panicked when I found out about this drama You can find script here . My ip was banned while I tested the script but now I started script from work and it fetched about 300 pages so far. Anyway I hope everything will be ok.
In many ways I think it would be hugely beneficial to Gentoo Wiki to start over from scratch. There's a ton of legacy that it would benefit from losing.

you really know how to drive away those motivated people every time

1) let them first recover all of the old crufted stuff

2) THEN we can begin porting it to a wiki (it's not really from scratch anymore but it should be much easier to do this as a bonus it's much more motivating)

M · Post by M » Wed Oct 22, 2008 8:24 pm

Well, you can't trick google that easy, now ip at work is banned

And from all downloaded pages only 20 are valid, usable, others are from google redirect page with captcha verification...

kernelOfTruth · Post by **kernelOfTruth** » Wed Oct 22, 2008 8:37 pm

M wrote:Well, you can't trick google that easy, now ip at work is banned And from all downloaded pages only 20 are valid, usable, others are from google redirect page with captcha verification...

how about the internet wayback machine ?

onefriedrice · Post by **onefriedrice** » Wed Oct 22, 2008 8:48 pm

M wrote:Well, you can't trick google that easy, now ip at work is banned And from all downloaded pages only 20 are valid, usable, others are from google redirect page with captcha verification...

Yeah... I thought about warning you, but I figured you knew what you were getting yourself into. It was annoying when the IP address(es) of the company I work for got "banned" by Google. We all had to do a CAPTCHA every 15 minutes or so just to use the search. It was then that I realized how often I use Google search. It was resolved after a couple of days.

Anyway, I think a more worthwhile endeavor would be for the maintainer of gentoo-wiki to contact Google and just ask for the goods (as a last resort if he really can't access his backups). I'm sure if he asks nice enough or pays a high enough processing fee, they'll help him out.

Maybe.

jagland · Post by **jagland** » Wed Oct 22, 2008 8:53 pm

What was your script coded in? Would you be willing to share it in someway? I have access to a fair number of IP's I can use and I do mean Public ones probably about a 100 or so (and it doesn't matter if they get blocked by Google for a short while!)

Kollin · Post by **Kollin** » Wed Oct 22, 2008 9:24 pm

M wrote:I made a script to fetch pages from google cache, just in case, I got panicked when I found out about this drama You can find script here . My ip was banned while I tested the script but now I started script from work and it fetched about 300 pages so far. Anyway I hope everything will be ok.

fingers crossed

AllenJB · Post by **AllenJB** » Wed Oct 22, 2008 11:29 pm

People IP ban people who blindly download a ton of pages for a reason. Don't do it! If there are select articles you know to be good, by all means back up those articles individually, but you are not going to be able to successfully back up the entire wiki from google cache or archive.org. Don't forget that any articles you back up using this method will be in HTML, not wiki markup, and if the worst/best case (depending on your pov) happens, you'll have to retranslate them all back to wiki markup.

Even if you have a range of IPs, these are most likely consecutive, so by the time you get to around host #10 (if that), they'll have just IP banned the entire range.

archive.org looks like it has archives up to around April 2008.

With regards to my earlier comment on starting over, the whole point was not to port, but to rewrite from scratch. There are good articles, but seriously a large amount of gentoo-wiki content is severely outdated, uses bad practices or just plain wrong. And on top of that there's all the structural changes the wiki has been through over the years. Starting over is the best option.

kaldek · Post by **kaldek** » Thu Oct 23, 2008 3:00 am

How weird. We've been working on downloading the cache copy - hacker style - and I think we have about 40% of it and haven't been banned as yet.

We're not using PHP though. As I say, hacker style, we downloaded the cache copies of the index pages and used regex to replace all the hyperlinks with links back to the cache, and then pulled down each page using wget.

P.S. When I say "hacker style" I mean quick & dirty, as in a hack.

cast0r · Post by **cast0r** » Thu Oct 23, 2008 8:38 am

AllenJB wrote:In many ways I think it would be hugely beneficial to Gentoo Wiki to start over from scratch. There's a ton of legacy that it would benefit from losing.

you got a point there! if Gentoo only gave us an official platform, otherwise by all means I don't think is worth it.

jagland · Post by **jagland** » Thu Oct 23, 2008 8:49 am

AllenJB wrote:People IP ban people who blindly download a ton of pages for a reason. Don't do it! If there are select articles you know to be good, by all means back up those articles individually, but you are not going to be able to successfully back up the entire wiki from google cache or archive.org. Don't forget that any articles you back up using this method will be in HTML, not wiki markup, and if the worst/best case (depending on your pov) happens, you'll have to retranslate them all back to wiki markup.

Even if you have a range of IPs, these are most likely consecutive, so by the time you get to around host #10 (if that), they'll have just IP banned the entire range.

archive.org looks like it has archives up to around April 2008.

With regards to my earlier comment on starting over, the whole point was not to port, but to rewrite from scratch. There are good articles, but seriously a large amount of gentoo-wiki content is severely outdated, uses bad practices or just plain wrong. And on top of that there's all the structural changes the wiki has been through over the years. Starting over is the best option.

I know how wrong it is - I'm also aware that it is Google we on about - it's not as if we are hammering some little site. I had a much better idea than just sitting on an IP until they banned me and moving onto the next one. If you take your time with sufficient pauses and IP changes then it would probably fall under their radar, and then it hardly constitutes a Denial of Service.

It would be handy to have the information, even if it has to be quality checked which could be part of the manual conversion process if we can get enough people involved.

AllenJB · Post by **AllenJB** » Thu Oct 23, 2008 9:05 am

cast0r wrote:
AllenJB wrote:In many ways I think it would be hugely beneficial to Gentoo Wiki to start over from scratch. There's a ton of legacy that it would benefit from losing.
you got a point there! if Gentoo only gave us an official platform, otherwise by all means I don't think is worth it.

Why only if it's "official"? I really don't get why everyone is so bent on the wiki being "official" in any way. While it _may_ improve the uptime, it won't improve the quality of the articles or the improve the content in any other way.

kaldek · Post by **kaldek** » Thu Oct 23, 2008 10:37 am

I say: Let's work to help the admin get the site content back up. The fact that we're all talking about rebuilding it excellent; let's still do that, but start with the existing content and clean it up.

There is nothing as truly horrifying to me as a totally blank canvas. Existing material shows the weaknesses which need to be fixed, and doing that is a totally different mental process to rebuilding from scratch. There is also a danger that there could be attempts to put more structure into it, which I think is not necessary and would just end up with arguments about the process, the articles to be included, the format, etc.

AllenJB · Post by **AllenJB** » Thu Oct 23, 2008 11:17 am

I disagree with the wiki not needing more structure. Some sort of category system needs to be used. The old system of a few index pages listing all articles doesn't work well for the number of articles, and even when they are listed it makes it hard to find appropriate articles.

There are a number of people who have been working over the past few years to improve the structure - improving usage of categories and page naming conventions. But it's been a losing battle all the way. I would estimate that atleast 20% of the articles on the existing Gentoo Wiki are no listed on index pages (which the wiki has outgrown long ago, in my opinion).

If you know the location of the article you want, you'll find it. But on the current wiki, looking for articles on a specific topic is a nightmare. It's been much improved by recent work on categories, but there's still so much of the wiki that is not indexed (fortunately these also tend to be the bad articles, tho not always).

I'd also estimate that atleast 50% of the articles on the existing wiki are no longer relevant, out of date, use bad practices to a large extent (I'm excluding single uses of USE on the command line, for example), incomplete or candidate for deletion for one reason or another.

I do not suggest that the wiki be wiped lightly. I've seen the horrors that lie within its depths. The items detailed above are the tip of the iceberg, in my opinion. Yes it's a totally different process to rebuild from scratch - that's the whole point.

cyrius · Post by **cyrius** » Thu Oct 23, 2008 11:44 am

What you forgot to mention is that a part of the standard documentation is coming from the WIKI.
I know Gentoo since 2003 and i saw first solutions in the wiki before the standard documentation (Perhaps not always but i saw it).
Am i wrong by writing that ?
The WIKI is the sandbox of the documentation from my point of view and should be necessary.

AllenJB · Post by **AllenJB** » Thu Oct 23, 2008 12:22 pm

cyrius wrote:What you forgot to mention is that a part of the standard documentation is coming from the WIKI.
I know Gentoo since 2003 and i saw first solutions in the wiki before the standard documentation (Perhaps not always but i saw it).
Am i wrong by writing that ?
The WIKI is the sandbox of the documentation from my point of view and should be necessary.

Use of the wiki by developers is rare. The Java project used to use it, but haven't done so for a long time. The vast majority of documentation is created and updated by users only. Developers tend to post documentation straight to gentoo.org/doc - usually any copies are because other people have copied the official documentation and then altered it.

tylerwylie · Post by **tylerwylie** » Thu Oct 23, 2008 4:22 pm

NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

Drone1 · Post by **Drone1** » Thu Oct 23, 2008 5:16 pm

The wiki should be user driven and here's why.

I view the wiki as a useful example and configuration guide, also containing miscellaneous scripts, and tweaks that users have come across and created. I also know that many OTHER linux distro's reference/d the gentoo wiki for those examples and configurations.

If I want to look at the guts of something, I'll go RTFM, look at the source, or jump on an IRC channel and ask the developer. Look at this from a support level perspective.

As for getting a Gentoo 'Seal of Approval' for gentoo-wiki? Please STOP thinking of Gentoo from a Corporate perspective!

FOSS and Gentoo are community driven.

d2_racing · Post by **d2_racing** » Thu Oct 23, 2008 5:18 pm

At least we have the google cache, because I don't know all the Safe Cflags....

ensarman · Post by **ensarman** » Thu Oct 23, 2008 5:48 pm

what about of this idea

ive seen that much articles in the wiki was outdated, and the idea is create a new wiki with new articles, and put in the new wiki the mos recent articles collected in the GoogleCache.

bunder · Post by **bunder** » Thu Oct 23, 2008 11:25 pm

found this on google cache:

Thursday, October 23rd 3:09PM EST

Well Finally had a real conversation with Skiplink today, according to him TelX has seized his property unlawfully due to lack of payment. he has a new datacenter location that I was one of the people that did not get transfered to before this all went down. from what I'm told he may be able to get my server back soon.

I'm currently wat work and don't want to go into too much detail at this point (I will go into detail later depending on the outcome of some events). but if Skiplink is unable to get the server back soon they will not be able to get them back for a long time (if ever). If I am unable to get the servers back I have a sorta-plan ready to go to get something

I will post again before tomorrow. stay tuned

rbr28 · Post by **rbr28** » Fri Oct 24, 2008 2:16 am

at least he says he will have offsite backups next time. I do offsite backups for "junk" I have at home...not having the wiki backed up offsite is kind of unbelievable. For the Gentoo Wiki I expect the db could have probably been rsynced to a $30 flash drive and carried off site!

gesho · Post by **gesho** » Fri Oct 24, 2008 2:19 am

What is this guy thinking? Gentoo has a large community and sysadmins talk to each other. I think we should let his mailbox feel his behavior.

Removed e-mail address. -- desultory