LWN: Comments on "Image "Cloaking" for Personal Privacy"

Image "Cloaking" for Personal Privacy

NYKevin — Fri, 24 Jul 2020 19:08:50 +0000

It is important to remember that, when you are entering a country other than your own, you (usually) do not have the legal right of entry, even if you are fully compliant with all of the legal requirements (and depending on the country, possibly even if you already have a visa, although that is admittedly a rare outcome). If they decide they don't trust you, most countries will simply put you back on the plane and turn it around. If you're inadmissible in the departure country too (e.g. you entered on a single-entry visa, the border officials don't like you, etc.), then things get messy (but you will *probably* make it back to your home country, eventually, and the airline will certainly bill you for their trouble). If you have multiple citizenships, this can get really fun, since you might get sent to the wrong "home country" and have to arrange further transportation yourself. Even when you are entering your home country, you may have difficulty asserting your right to entry. If they don't believe you're a citizen, proving it is generally your problem.

So, in my opinion, doing anything that makes a passport look less valid in the eyes of border officials is a really bad idea, regardless of whether doing so is technically legal or not.

street signs vs. faces

rgmoore — Fri, 24 Jul 2020 15:04:37 +0000

If you add machine-friendly identifiers, you'd better make sure they have the same kinds of legal rules surrounding them that human-readable signs do. Otherwise malicious actors will be able to mess with the system with legal impunity. It could be very bad if people could create new traffic signs only autonomous vehicles knew about.

Image "Cloaking" for Personal Privacy

t-v — Fri, 24 Jul 2020 13:57:39 +0000

And they don't comply with "reproduce copyright notice" for the stuff they use from third parties... Sigh. It's really hard to copy MIT licensed code and not screw up apparently, even though what the license requires might have been common courtesy even if it didn't.

Regarding patent, I must admit I have a hard time telling what the real innovation compared to the now classic adversarial example work (of the Goodfellow... and Carlini... variants implemented in CleverHans) is except uploading the image somewhere. That may or may not mean that one would find prior art.

Image "Cloaking" for Personal Privacy

nilsmeyer — Fri, 24 Jul 2020 09:56:16 +0000

I remember people using microwaves to fry the RFID in their passports or ID cards. I don't necessarily mean by "trouble" that you'll be convicted of a criminal offence, but you may be severely inconvenienced, perhaps even arrested or detained.

We already know this happens to people based on other criteria. It also happens a lot where facial recognition is used, especially if you have a darker skin tone where facial recognition causes a lot more false positives.

Image "Cloaking" for Personal Privacy

nilsmeyer — Fri, 24 Jul 2020 09:44:28 +0000

The situation I had in my head was trying to enter another country through an airport for example. If the facial recognition doesn't work someone may suspect that the passport is fake. "Trouble" doesn't necessarily mean that you end up in prison, but at the very least you could be detained and harassed at the airport.

Image "Cloaking" for Personal Privacy

gdt — Fri, 24 Jul 2020 03:44:02 +0000

Typically when you supply a digital photo for use on your passport you state that the image is a "true likeness". Whereas the image has in fact been altered. Obviously the detail varies by jurisdiction but a "false declaration on the passport application" would be the approach.

Image "Cloaking" for Personal Privacy

sjj — Thu, 23 Jul 2020 19:56:32 +0000

Why and how would you get in trouble?

street signs vs. faces

nilsmeyer — Thu, 23 Jul 2020 19:31:53 +0000

This can also throw a wrench in systems that scrape data from social media to build/train their facial recognition software.

Image "Cloaking" for Personal Privacy

nilsmeyer — Thu, 23 Jul 2020 19:28:47 +0000

Yeah you might get into a lot of trouble with that too. In that case you would just go to the booth with a person behind it (and spend a lot of time in a queue) - if available.

Image "Cloaking" for Personal Privacy

NYKevin — Thu, 23 Jul 2020 19:27:56 +0000

For better or for worse, most of the major tech companies are incorporated in the US, so the industry as a whole has no choice but to care about this problem.

Image "Cloaking" for Personal Privacy

jem — Thu, 23 Jul 2020 19:18:10 +0000

This bites yourself in the arse if you have a cloaked image of yourself in your passport, and the unaltered image of you is one taken by a camera at an automated border check booth.

street signs vs. faces

magfr — Thu, 23 Jul 2020 15:52:52 +0000

We all know that whenever the same information is encoded twice there will inevitably be a discrepancy.

street signs vs. faces

clump — Thu, 23 Jul 2020 14:13:33 +0000

Street signs were designed for humans to understand. This will continue to be the case for a long time. As autonomous vehicles become popular it would be a good idea to augment human-readable signs with machine-friendly identifiers. Why not add a small RF box, or some kind of id that can be scanned by machines?

By all means, continue to learn to read symbols designed for humans. However it should be relatively inexpensive (and safer) to tell a machine "this is a stop sign".

street signs vs. faces

ibukanov — Thu, 23 Jul 2020 12:15:24 +0000

In Norway police has been taking passport photos themselves for at least 5 years in big cities. One cannot bring own photos at all. But Norwegian embassys still ask to bring own photos both when they issues passports and visas.

Image "Cloaking" for Personal Privacy

Wol — Thu, 23 Jul 2020 08:46:20 +0000

Only in the US ...

Cheers,
Wol

street signs vs. faces

Sesse — Thu, 23 Jul 2020 08:44:26 +0000

All the street sign examples I've seen involve pretty heavy tampering with the signs (e.g. several measures of thick tape). Obvious enough that you could really just as well write “80” where it says “30” instead.

street signs vs. faces

smurf — Thu, 23 Jul 2020 07:08:01 +0000

There are lots of mucking-with-street-signs examples out there that don't touch the borders.

Their point isn't to make the image in the manipulated photos unrecognizeable, but to make it not-your-own. The problem I see with their idea is that as soon as there are two sets of images of you out there, any adversary worth their salt will not simply replace the old parameter cloud with the new, as the Fawkes authors assume, but split them off into two sets of clouds which are both recognized as "you".

So this probably works WRT shop surveillance systems that try to find who that repeat customer is, might conceivably defend against run-of-the-mill police surveillance cameras if you can get your passport photos replaced with a Fawkes pic (more difficult as authorities start to insist on taking the pics themselves instead of you walking in with one from the photo booth), but not at all when the opponent is the NSA and their ilk.

Image "Cloaking" for Personal Privacy

Cyberax — Thu, 23 Jul 2020 05:56:13 +0000

You can file for a patent for up to a year after a publication or any other form of unveiling of the invention.

Image "Cloaking" for Personal Privacy

epa — Thu, 23 Jul 2020 05:37:38 +0000

If they have already published the software but not yet filed the patent, surely that means it’s too late? The patent office will probably grant it anyway but it would be invalidated in court. Or is that not how it works?

street signs vs. faces

felixfix — Thu, 23 Jul 2020 02:17:20 +0000

It may be more realistic to say the the enemy has the better hand against bulk data. Individuals who invest some effort in differentiating themselves from the bulk data may hide from the bulk processor. But the very fact of having differentiated themselves may also make them stand out.

street signs vs. faces

gus3 — Thu, 23 Jul 2020 01:53:51 +0000

Street signs tend to have hard image borders. Muck with any of those borders, and any algorithm using them will get knocked about.

Images of human faces are a mix of hard borders and soft shading. A proper facial-recognition system doesn't depend on these; it uses the points of the face (eyes, nostrils, lips, visible teeth, ears, visible hair-line, jaw, cheekbones, musculature) to build a face it can recognize.

The hackers and crackers already have tools against the Fawkes system. The images aren't cloaked, no matter how much you want them to be.

Remember: the enemy always has the better hand. It's your job to close the gap between the enemy's hand and yours.

Image "Cloaking" for Personal Privacy

FLHerne — Wed, 22 Jul 2020 23:46:31 +0000

> The cloak effect is not easily detectable, and will not cause errors in model training. However, when someone tries to identify you using an unaltered image of you (e.g. a photo taken in public), and tries to identify you, they will fail.

This assumes not only that all existing facial-recognition systems are vulnerable to their specific tweaking approach, but that future ones will be too. The modified photos will be out there indefinitely.

A lot of research is being done on solving this class of weakness -- it's a serious problem for self-driving vehicles too, there've been demonstrations of minor changes to street signs that lead to completely different recognition outcomes.

While their list of current systems fooled is impressive, I think the absolute assurance of privacy given here is unwarranted.

Image "Cloaking" for Personal Privacy

NYKevin — Wed, 22 Jul 2020 22:53:44 +0000

It's BSD-licensed, yes, but the "copyright" section of their README says this:

> This code is intended only for personal privacy protection or academic research.
>
> We are currently exploring the filing of a provisional patent on the Fawkes algorithm.

I'm going to assume the first line is just a standard CYA "there's no warranty" disclaimer, and not an actual condition on use (because it would flatly contradict the LICENSE file). However, the patent is a great deal more alarming, and in fact, I'm not sure I can recommend using this thing as long as that sentence remains there. It basically amounts to "You can do what you like with our software, but we could turn around and sue you at any time, once the USPTO rubber stamps our patent."