Another fun DMCA case

Another fun DMCA case

But nobody said, that anyone put any effort in circumventing the "robots.txt copy control measure". There are only a handful of tools out there that even support robots.txt, as it's a file that has a very special purpose and this only in connection with internet search engines.

E.g. wget as one of the tools that support robots.txt even lets you switch off the support via a command line parameter: -e robots=off

The easiest and probably most used way to copy a webpage is to just use the "save page" feature of your webbrowser. An I know of no webbrowser that even recognizes a robots.txt file.

If I wanted to write a service like archive.org which crawls and saves webpages I'd do it in perl and would have to knowingly put effort in supporting something like a robots.txt. So how can you call something a "copy protection measure", if I have to know that you have it and what way you tell me how to behave and actually write code to behave like you want. My other option is to just do nothing and how can I have done something bad if I have done nothing?