Security quotes of the week
But what if we don’t have the luxury of a testing lab? What if the app behavior changes based on your location, or interaction with the outside world? For instance, if you use an app to rent a car or unlock a door to a shared workplace, the real-time behavior of the app will be different from what you can replicate in a lab. For these kinds of complex interactions, a roaming Machine-in-the-Middle (MitM) schema is needed. In fact, all three components of the previous schema (test device, interceptor, and control device) will need to be consolidated into a single device running the software required for all three components. If the app being audited is a form of disciplinary technology – that is, a surveillance app that one person installs on the device of another person – then the auditor will also need to surreptitiously capture traffic being sent by the app, which may pose additional testing complications.— Bill Budington describes how to audit Android apps on a rooted phone
The title says it all — really! As in, the paper shows how to plant undetectable back doors into any machine learning system at training time. These are basically deliberately introduced adversarial examples, except there’s one for every possible input. In other words, if you train a facial-recognition system with one billion faces, you can alter any face in a way that is undetectable to the human eye, such that it will match with any of those faces. Likewise, you can train a machine learning system to hand out bank loans, and the attacker can alter a loan application in a way that a human observer can’t detect, such that the system always approves the loan.— Cory Doctorow on a paper titled “Planting Undetectable Backdoors in Machine Learning Models”The attack is based on a scenario in which a company outsources its model-training to a third party. This is pretty common, because training models is really expensive. Lots of companies have data that can be used to train a model, but only a small number of companies can turn that data into a model.