Anubis Blocking My Gentoo Browsers

Message

flysideways · Post by **flysideways** » Wed Aug 20, 2025 12:23 pm

Haha.

I can no longer get to fgo with Firefox or Epiphany on my Gentoo installs. Getting a hard block from Anubis.

They're not promiscuous enough apparently. Even after allowing cookies for forums.gentoo.org, no entry for you!

So far, from web searching, it seems it also requires javascript to be running, and possibly more.

Funny, Gentoo sets up the default browser installs too securely for the new AI gatekeeper to fgo.

Any advice on the settings and services to change and add to default Gentoo browser installs so I can access fgo with them is appreciated.

Stuck by NeddySeagoon 20-Aug-25

Website · Post by **Zucca** » Wed Aug 20, 2025 12:31 pm

Moved from Gentoo Chat to Gentoo Forums Feedback.

There's a way to avoid anubis... Somehow make your browser send your forum credentials via http POST before trying to load the page. I know this because my forum script still works.
EDIT: No it doesn't look to be that way after all.

Code: Select all

curl -L https://forums.gentoo.org

does not get blocked. So I can only guess that anubis caches whitelisted ips.
I've never encountered anubis on my laptop, but on my phone I've seen it many times.

You could try accessing https://forums.gentoo.org/login.php and see if that page is blocked too...

spica · Post by **spica** » Wed Aug 20, 2025 12:45 pm

Anubis gives some small tasks to the browser. This is a test to see if the browser is real and not just a script. If you only load plain HTML, the test does not work. Without Anubis JavaScript, this check cannot run.

Google’s crawler can still pass, because many websites trust Googlebot. If you change your browser’s user agent to Googlebot, you may see fewer ads and sometimes get access to more pages. But note: some websites also check the real Google IP address, not only the user agent.

flysideways · Post by **flysideways** » Wed Aug 20, 2025 1:14 pm

After looking a bit farther, I find some websites set up to use Anubis, like enlightenment.org, run the test and let me through using my Gentoo standard installed browsers.

There are others that will not let me through, the same as fgo.

I also find an update to Anubis that does not use the javascript method.

I'm guessing it is basically a server side setup and settings issue.

flysideways · Post by **flysideways** » Wed Aug 20, 2025 1:38 pm

https://www.fsf.org/blogs/sysadmin/our- ... ns-of-bots

Some web developers have started integrating a program called Anubis to decrease the amount of requests that automated systems send and therefore help the website avoid being DDoSed. The problem is that Anubis makes the website send out a free JavaScript program that acts like malware. A website using Anubis will respond to a request for a webpage with a free JavaScript program and not the page that was requested. If you run the JavaScript program sent through Anubis, it will do some useless computations on random numbers and keep one CPU entirely busy. It could take less than a second or over a minute. When it is done, it sends the computation results back to the website. The website will verify that the useless computation was done by looking at the results and only then give access to the originally requested page.

At the FSF, we do not support this scheme because it conflicts with the principles of software freedom. The Anubis JavaScript program's calculations are the same kind of calculations done by crypto-currency mining programs. A program which does calculations that a user does not want done is a form of malware. Proprietary software is often malware, and people often run it not because they want to, but because they have been pressured into it. If we made our website use Anubis, we would be pressuring users into running malware. Even though it is free software, it is part of a scheme that is far too similar to proprietary software to be acceptable. We want users to control their own computing and to have autonomy, independence, and freedom. With your support, we can continue to put these principles into practice.

Even though we are under active attack, gnu.org, ftp.gnu.org, and savannah.gnu.org are up with normal response times at the moment, and have been for the majority of this week, largely thanks to hard work from the Savannah hackers Bob, Corwin, and Luke who've helped us, your sysadmins. We've shielded these sites for almost a full year of intense attacks now, and we'll keep on fighting these attacks for as long as they continue.

Protecting infrastructure is a job I do not envy.

flysideways · Post by **flysideways** » Wed Aug 20, 2025 1:42 pm

The idea of a new Gentoo user finishing their first working install, then being denied access to fgo by Anubis, from their shiny new Gentoo install, is, well, a bit depressing.

Website · Post by **Zucca** » Wed Aug 20, 2025 2:15 pm

flysideways wrote:The idea of a new Gentoo user finishing their first working install, then being denied access to fgo by Anubis, from their shiny new Gentoo install, is, well, a bit depressing.

This was on my mind the first time I saw Anubis on fgo.

As for FSF's definition of malware... I don't completely agree on that.
Let's take an example of Signal-desktop. It's an electron app. Some would say it does unnecessary calculations and uses unnecessary amounts of RAM. I'd say it's very bad at handling resources... like every other electron program.
I'd say the same about anubis. It's very bad at handling the identification of a real browser because it uses somewhat unnecessary amounts or CPU cycles for the identification.

But I say that it's still far better alternative to those catchpas or whatnot.

Post by **NeddySeagoon** » Wed Aug 20, 2025 6:07 pm

Team,

Please file bugs for -infra

Include in the bug ferpot
the useragents
browser name and version(s)
whether JS is enabled (it's ok if not, but need to know),
cookies are enabled or not.
How you can read this thread,

Website · Post by **Zucca** » Wed Aug 20, 2025 6:26 pm

@flysideways, since you encounter problems, please open a bug.
I would be nice if you could then edit your opening post to include a link to that bug, so that users who come here can go to the same bug report.

flysideways · Post by **flysideways** » Wed Aug 20, 2025 6:45 pm

Collecting the data now.

XeIaso · Post by **XeIaso** » Wed Aug 20, 2025 7:21 pm

Hey,

Main author of Anubis here. Ask one of the mods to confirm that my email address has this md5 checksum:

Code: Select all

echo -n "my@email.address" | md5sum
43c04e6081bcd39c686e76877c0a00b9

I'm aware of this issue, but none of the reports have ever been consistently reproducible. I want to fix this and I think I have a fix in the main branch, but without replication steps including browser, OS, and device vendors/versions I'm lacking ideas.

I want to fix this, but I can't fix it unless I can make it consistently fail. Annoyingly in my testing across a lot of browsers (even privacy browsers like Mullvad browser, Tor browser bundle, Librewolf, Pale Moon, etc) this never shows up.

Hopefully it's something really dumb so it can be fixed for everyone.

Be well,

Website · Post by **Zucca** » Wed Aug 20, 2025 7:31 pm

I'm running Librewolf 104.0.4. I've never had problems with anubis.

XeIaso wrote:Hey,

Main author of Anubis here. Ask one of the mods to confirm that my email address has this md5 checksum:
Code: Select all
echo -n "my@email.address" | md5sum
43c04e6081bcd39c686e76877c0a00b9

Matches, for what it's worth.

flysideways · Post by **flysideways** » Wed Aug 20, 2025 10:38 pm

It seems it is only Gentoo VM's in UTM/QEMU that are not working. At least not with Firefox or Epiphany. My VMWare Fusion VM's can get in, along with Firefox and Safari in the MacOS.

I got in easy peasy with Links. Here now from an offending UTM/QEMU VM.

For this unreachable UTM/QEMU Gentoo/XFCE VM:

useragent Mozilla/5.0 (X11; Linux x86_64; rv:128) Gecko/20010101 Firefox/128
Firefox Browser
javascript.enabled true

I tried the cookie restrictions as Standard, Strict, and Custom with everything off.

The profiles are the same in the VMWare Fusion VM's that will reach fgo.

It is either UTM/QEMU or something in my VM's, I suppose. I have two that are three years old, one XFCE, the other Gnome. They are updated as of the last few days.

I did start up a new UTM/QEMU Debian Trixie VM, and it can contact fgo with the same useragent and settings as my failing Gentoo VM's.

Post by **Banana** » Thu Aug 21, 2025 7:54 am

XeIaso wrote:Hey,

Main author of Anubis here. Ask one of the mods to confirm that my email address has this md5 checksum:
Code: Select all
echo -n "my@email.address" | md5sum
43c04e6081bcd39c686e76877c0a00b9
I'm aware of this issue, but none of the reports have ever been consistently reproducible. I want to fix this and I think I have a fix in the main branch, but without replication steps including browser, OS, and device vendors/versions I'm lacking ideas.

I want to fix this, but I can't fix it unless I can make it consistently fail. Annoyingly in my testing across a lot of browsers (even privacy browsers like Mullvad browser, Tor browser bundle, Librewolf, Pale Moon, etc) this never shows up.

Hopefully it's something really dumb so it can be fixed for everyone.

Be well,

Thank you for helping us.

Is there a posibility within anubis to create something like a troubleshoot report which the user can self create and use it as a report?
Like "click here to create a troubleshoot report, which gathers information about this case, and use this info for a bug report..."

Website · Post by **Zucca** » Fri Aug 22, 2025 8:09 am

I'd suspect that then the crawler/spammer blocker (anubis) itself would become the target of spammers.

Post by **NeddySeagoon** » Fri Aug 22, 2025 12:14 pm

Zucca,

Its going to be a never ending race between crawlers and crawler blockers.
Much like copy protection copy protection breaking of floppy disk days.
Then we had libdecss(sp) for DVDs.

Blu-Rays have gone a step further. If you can break that, the banking system has been broken too.
Its the same crypto.

I suppose it will end when the AI bubble bursts like dot com and others before it.

Website · Post by **Zucca** » Fri Aug 22, 2025 2:24 pm

Yup.
Like

cheaters vs. anti-cheat
spamers vs. spam blockers
law makers vs. loophole exploiters

;)

CaptainBlood · Post by **CaptainBlood** » Sat Aug 23, 2025 6:50 am

Firefox has a unrecommended & experimental plugin.

Thks 4 ur attention, interest & support

CaptainBlood · Post by **CaptainBlood** » Sat Aug 23, 2025 7:04 am

Just tested working here...
Unsure if I'm gonna keep it by now, as a safety measure.

Thks 4 ur attention, interest & support.

Website · Post by **Zucca** » Sat Aug 23, 2025 1:05 pm

CaptainBlood wrote:Just tested working here...
Unsure if I'm gonna keep it by now, as a safety measure.

Thks 4 ur attention, interest & support.

Does it work with javascript turned off?

CaptainBlood · Post by **CaptainBlood** » Sun Aug 24, 2025 5:09 pm

Zucca wrote:Does it work with javascript turned off?

The splash screen (Anubis?) I have been getting for a while when accessing the forum remains not showing with

Code: Select all

about:config
javascript.enabled=false

set and Librewolf browser restarted.

Thks 4 ur attention, interest & support.

Post by **Ionen** » Mon Aug 25, 2025 12:34 am

For the record, for *most* users with Gentoo's chosen settings, Anubis "should" be giving a non-javascript challenge using metarefresh and still work even if you block javascript -- but it may force the javascript one only if it thinks something is suspicious and is more likely to be a AI crawler -- Edit: changing your user agent from default or using outdated browsers can potentially cause this for example.

If anyone that is having issues has blocked metarefresh somehow that may be your problem though (I don't use librewolf/firefox but I think there's some about:config accessibility setting blockautorefresh or something similar that can disable it, I vaguely recall it may have per-site settings too).

Try with a clean profile without special settings or extensions to be sure it's not due to your settings rather than the browser itself anyhow.

I haven't tried librewolf or most others myself, I use qutebrowser which can access forums fine even with javascript disabled (lynx works too). If a up-to-date legitimate specific browser has issues regardless of settings we could try to work around that from Gentoo's side (not that I'm the one who's been working on that and don't know Anubis that well, it's however very configurable and we've been in touch with Anubis' upstream that is responsive to fixing issues).

Website · Post by **Zucca** » Mon Aug 25, 2025 8:33 am

I use Librewolf with totally custom UA for gentoo.org domain and javascript turned on.
I've never seen anubis popping up.
On my phone I tend to use Firefox Focus (basically an always on private-mode browser) and on that browser anubis pops up about 50% of the time (because the browser discards cookies after closing a tab), but it always passes just fine.

pa4wdh · Post by **pa4wdh** » Mon Sep 01, 2025 4:55 pm

I have an anubis related question, i hope is ok to ask it here:
How long should the the anubis challenge take? My PC is low-end and passively cooled, and the challenge usually takes anywhere between 15 en 30 seconds. Since i'm quite aggressive with deleting cookies it's becoming really annoying.

Website · Post by **Zucca** » Thu Sep 04, 2025 6:15 pm

pa4wdh wrote:How long should the the anubis challenge take?

It takes as long as your PC has "mined" the correct sha answer, unless you disable javascript, then it should just use html meta redirect/refresh if I'm not mistaken.

pa4wdh wrote:My PC is low-end and passively cooled, and the challenge usually takes anywhere between 15 en 30 seconds.

I should try this on my (passively cooled) Celeron N3010 PC too...