Design for security

Serena Chen began her talk in the Security, Identity & Privacy miniconf at linux.conf.au 2019 with a plan to dispel a pervasive myth that "usability and security are mutually exclusive". She hoped that by the end of her talk, she could convince the audience that the opposite is true: good user experience design and good security cannot exist without each other. It makes sense, she said, because a secure system must be reliable and controllable, which means it must be usable, while a usable system must be less confusing, thus it is more secure.

Chen is a product designer who is interested in the "intersection between security and humans". She thought that most in the room would agree that "everyone deserves to be secure without having to be experts". But our current ways of designing systems are not focused on that. We need to stop expecting our users to be or become security experts.

No one cares

It may be hard for a roomful of people interested in security to hear, she said, but "no one cares about security". In truth, they may care about it in theory, but that is not what they are thinking about or trying to accomplish at any given time. As an example, she referred to the classic "dancing pigs" quote ("Given a choice between dancing pigs and security, users will pick dancing pigs every time."), though she noted that it was written in 1999 and might need its memes updated, perhaps by substituting "cats" for "pigs".

But we expect users to actively think about their security and, when they don't, "we shame them". In the security world, there is a "pervasive culture of shame". She fully admits that she has participated, by making fun of users who post their credit card numbers on the internet or get their password scammed on IRC. Beyond that, there's recommending a completely unusable tool (with a slide of the OpenPGP home page) then "belittling them when they can't figure out how to do it", she said to laughter.

Shaming people is lazy, she said. People wanted to complete a task and we have failed to provide a secure and easy way to do so. It is okay that people don't care about security, she said, because they shouldn't have to. It is our job to build secure systems for everyone. She pointed to the classic Sandwich xkcd comic ("sudo make me a sandwich") and noted that lots of developers probably just add sudo to the front of a failing command without even thinking about it; we are all just trying to get things done.

In her job, she focuses on the end-user experience. Instead of "overwhelming people with complex technical instructions", we can make things more intuitive and friendly. That way, it will actually get used. Something that can help is "design thinking"; it is just another problem-solving tool that should be in your toolkit, she said. Using design thinking, she came up with four things she thinks should be considered the next time there is the "inevitable tug-of-war between usability and security".

Least resistance

The first is "paths of least resistance". We are used to putting up a lot of walls, Chen said, such as popping up warnings ("have you considered not doing the thing?" or "oh, you needed to open up a program to do your job, have you considered getting a new job?"). That comes from security being tacked on at the end of the development process, she said.

Instead of walls, she suggested making rivers: make the path of least resistance be the path that is the most secure. It is the "secure by default" principle; if you do nothing, you get the most secure options. It can be as simple as defaulting the options in a dialog to the one that is more secure. For a real-world example, she pointed to blenders that won't turn on unless their lids are on; "you can't screw that up".

Security actions should be a normal part of the process. It shouldn't take "extra credit" work to be secure. If, for example, a phone number is going to be needed to verify an account, ask for it right up front rather than expecting the user to go add it in some settings screen. If a user visits a site or an app with a specific task in mind, they are not likely willing to go through a whole setup process. But if they are going through a setup process, make sure that all of the pieces that need to go with that are grouped with it.

Humans are good at being economical with their physical and mental resources, she said. That means security will not be on their minds as they will be trying to accomplish something. Therefore it is paramount that the goals of the end user are aligned with the security goals of the developers. If they are not aligned, users will get around the security goals—sometimes on purpose.

The browser warnings about privacy with regard to HTTPS certificates that do not validate, perhaps because they are self-signed, are one example. She noted that the warnings have been getting better, but we all know someone who just breezes right past the warnings "without a care in the world", she said. Their current goal is not to try to figure out which sites are dodgy and which aren't, so they just say "I know how to internet" to themselves and click through.

She asked: "why doesn't friction work here?" The problem is that, when she talks about paths of least resistance, she really means paths of perceived least resistance. If clicking through ten security warnings is how she has learned to get a certain thing done, she will do it every time—and she won't even see the warnings any more. In fact, a study has shown that after two exposures to a warning, people do less visual processing when they "see" it again.

There is a massive vulnerability in what she called "shadow IT": the path that employees actually take that is directly contrary to what the IT department requires because the requirements are too onerous. An example would be password rotation policies, which are known not to work and to lead to various less-secure options (e.g. short passwords, passwords on post-it notes, cycling through slight variations of the same password).

If you keep putting up obstacle courses, she said, users will get good at running them. Another area where IT departments put up walls is around what content can be accessed, but security tools should not be used for non-security purposes. She has had to work around company firewalls so she could listen to Spotify, for example. If employees are spending all of their time on YouTube, "that is a management problem, not a security problem", she said.

When you want to build good paths, don't make the users think, she said. "Again, build rivers, not walls." Make the secure path be the easiest one. A good example of this is the BeyondCorp security model used by Google, which removed the security perimeter from the corporate network, effectively putting the whole thing onto the internet. That requires ensuring that the authentication systems are reliable and that there are good models for all of the roles within the organization including what access they require. More importantly, from her perspective, BeyondCorp had a clear focus on user experience and on making it largely invisible to its users.

Intent

If you want to align your security goals with the goals of users, you need to know what the user's intent is. Developers forget about intent all of the time; that is usually where the tension between usability and security occurs. There is a tendency to fall back on common patterns; designers say "make it easy", while security people say "lock it down". But it is not her job to make everything easy, nor is it the security developers' job to make it all locked down. The job is to make an action that the user wants to take at a specific time and place easy; everything else can be locked down.

Figuring out the user's intent is "easier said than done, of course", but it starts by understanding who the user is, where they are, what time it is, and what kinds of things might they want to do under those conditions. Is there enough information for the application to know those things? Are people sharing login information so that it is difficult to know who the actual user is? Are those things that need to be handled?

Determining what a user would be expected to do—and not do—based on their role, while using the minimum amount of personal data to do so, is the goal. Simple things like country of origin and time of day can inform the software on what that user's intent might be, she said.

(Mis)communication

She challenged attendees to think about communication a bit differently than they usually do; it is not just a "mushy kind of human-based I/O that is a bit of a drag to deal with sometimes". In particular, she wanted them to think about miscommunication as a human security vulnerability; it is what is exploited by social engineering attacks. It is the ambiguity in communication that gives social engineers cracks to exploit.

In their current project, she asked attendees, is there something that the program is unintentionally miscommunicating? For example, the (in)famous green lock in the web browser interface, which is a bug that has thankfully largely been fixed, she said, means that the communication is encrypted and that the domain name is attested to by the certificate authority (CA). But to the average person, it simply means that the site is "secure", because it says so right next to the lock icon. We know that is not necessarily true, though—there is a miscommunication, thus a human security vulnerability.

By way of an example, she mocked up a "pretty convincing" web site that would show the green lock. If she was bored one night and "felt like doing some crime", she could grab an unused domain name (say, "chase-help-and-support.com") for around $20 and then go to Let's Encrypt for a free certificate. With some HTML and CSS hacking, she could set up a pretty convincing phishing site that the browser would explicitly label as "secure". She didn't actually do any of that, but phishers do it all the time and it works really well. The question to ask yourself is: do your end users know what it is you are trying to communicate?

Matching mental models

In order to understand whether users are receiving your communication, you need to understand their mental model of what is going on. Matching the mental models of users and developers is the most important consideration in this process. The user's expectations are what ultimately govern whether a system is secure: if they are met, it is secure, if not, it isn't.

Not all man-in-the-middle instances are an attack, the telephone game is a series of man-in-the-middle "attacks", but it is "just a pointless children's game". In a network context, though, users expect that their communications are able to be read only by the expected recipients and man-in-the-middle attacks violate those expectations.

So in order to make a system that is secure from the user's perspective, we need to figure out what their mental model is. One way to do that is to observe them interacting with the system. Designers are often doing user interviews; sitting in on those or reading their transcripts will be quite helpful in determining what users are thinking. If you don't have access to those activities, observe friends and family. By figuring out why they do the things they do, you can infer their intent.

Another approach is to influence their mental model so that it better matches what is actually happening. Whenever we make something, we teach and whenever someone uses something we make, they learn. The path of least resistance will often simply become the way to accomplish a task. Our software is already influencing the user's mental model. As an example of that, she described something that Apple products used to do; they would semi-randomly pop up a dialog asking the user to log into iTunes. People got so used to that—and just quickly typing their password to dismiss it—that it could easily be used as part of phishing scams.

We should pay attention to what our applications are teaching our users. Are they teaching users to ignore warnings and figure out the easiest way around them? Or that security is a barrier to be surmounted rather than something that helps them? The question "is it secure?" is completely meaningless without some kind of context about who and what it is securing and what it is protecting against.

In summary

As she concluded her talk, she noted that cross-pollination between design and security is rare, which is a missed opportunity. All of our jobs are about "outcomes based on specific goals", which should not rely on the stereotypical patterns of designers' considerations versus those of security developers. The key is to align the user's goals with the security goals.

She closed with a final anecdote: in her company's old building, one of the floors had a light switch that no one knew what it controlled. It had a post-it note over the switch that simply said: "No!". "Can you guess what the first thing I did was?", she said with a laugh as she showed a picture of a finger about to press it.

The first question in the lengthy Q&A was, inevitably, about the switch. She never found out what it controlled though she switched it many times. Another question had to do with security flaws that are difficult to communicate to users and, perhaps, have no fix yet, such as a firmware or processor bug. There are no simple answers there, especially if there is no recommended action that can be offered, she said. In those cases, hopefully automatic updates are taking care of things once there is a fix. Until then, it is not clear what can usefully be communicated to non-technical users.

How to get users to care about the "trust question" was another query. She acknowledged the problem but said that users often do not have time to even think about who they trust and they cannot be bothered to do so. She likened it to voting, where we would like to have people care about the issues and make informed choices, but many simply don't have the time—or take the time—to become informed.

A YouTube video of the talk is available.

[I would like to thank LWN's travel sponsor, the Linux Foundation, for travel assistance to Christchurch for linux.conf.au.]