Anubis doesn't seem to be working anymore here
Anubis doesn't seem to be working anymore here
Posted Jul 11, 2026 0:58 UTC (Sat) by koverstreet (subscriber, #4296)In reply to: Anubis doesn't seem to be working anymore here by joey
Parent article: An update on the scraper situation
Realistically, the solution needs to be some kind of throttling baking into the webserver, and a more efficient cgit implementation with some caching would help a lot. I've hit quite a few perf issues with git, I'm hoping now that they're starting to use Rust in the core codebase that'll make performance work easier.
Definitely not excited by the prospect of yet more sysadmin work, though.
And. If web access to too many git repos gets shut down I have to imagine the AI companies would switch to just chain cloning repos with no caching. Everything they're doing is just really anti social.
The LWN site is currently under high scraper load, so comment display has been suppressed for anonymous users. If you are a human, you may read the comments by clicking the button below:
Note: you can avoid this step in the future by logging into your LWN account.
