|
|
Log in / Subscribe / Register

Ok, bye bye

Ok, bye bye

Posted Jul 26, 2026 13:05 UTC (Sun) by cladisch (✭ supporter ✭, #50193)
In reply to: Ok, bye bye by dskoll
Parent article: Codeberg: Protecting our FLOSS commons from LLMs

> And I have no idea why the bots think it's a good idea to scrape the Web interface to git instead of just doing a git clone

Because HTML has no standardized mechanism to redirect a scraper to a git repository. It might be possible to set <link rel="alternate" …>, but nobody uses it.

And, of course, those bots are vibe coded; neither the bots nor their human handlers have the ability to evaluate the returned data for correctness or usefulness. The only quality measure they have is the amount of pages and returned data, so they have no incentive to add special code to handle git repositories.


The LWN site is currently under high scraper load, so comment display has been suppressed for anonymous users. If you are a human, you may read the comments by clicking the button below:

Note: you can avoid this step in the future by logging into your LWN account.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds