|
|
Log in / Subscribe / Register

What is LWN using ?

What is LWN using ?

Posted Feb 13, 2026 11:17 UTC (Fri) by leromarinvit (subscriber, #56850)
In reply to: What is LWN using ? by KJ7RRV
Parent article: Poisoning scraperbots with iocaine

I was going to write a comment about how it's probably cheaper to keep querying the comments from a database for recent articles with active discussions, and that there should be a break-even point somewhere. But then I realized that in that mode, *every* page load must do the same work of generating a static page to serve to that single reader. So the only difference would be saving that page to disk, which the database presumably also does already (even if it only writes new comments and therefore less data).

Therefore, the break-even point is reached as soon as saving somewhat larger generated pages for new comments is less expensive than the DB round-trip and dynamic page generation for every request. My totally scientific gut feeling says it would require a ridiculously high comment load to make generating static pages for every new comment *not* cheaper than dynamic rendering for every request.

Of course, this only works if all/most users get served the exact same content. But I think this should be the case here, and in any case, per-user differences can probably be outsourced to a per-user CSS file (which the scrapers probably wouldn't even fetch, since it provides no useful value for LLM training - so this could be dynamic).


to post comments

What is LWN using ?

Posted Feb 16, 2026 8:52 UTC (Mon) by taladar (subscriber, #68407) [Link]

The main issue with active, relatively new pages would probably be some form of locking on the page re-generation process. But that could be skipped if you are willing to just serve the old page until the generation of the new version is complete.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds