Development quote of the week

Second, there needs to be a way to revert to older versions temporarily for testing - users need to be able to bisect when a change came in to usefully report things to you. A report of "this was fine yesterday, and is not today" is a small window of commits to look through; a report of "I haven't used this in six months, but it was fine 6 months ago, and is not now" is a lot to look through. You need to be able to convert the latter class of report into "OK, it was fine 42 days ago, but not 41 days ago" in order to track down exactly which change is the issue.

Third, with the above in place, users strongly prefer frequent small changes to infrequent large changes.

User interviews revealed that the reason for this is that changes fall into three categories (from most common to least common):

1. Completely unnoticed changes. If your phone gains support for a new VoLTE codec only used in one province in China, and you never communicate with anyone in China, you're not going to notice this. Similarly, if it's gained support for a new type of hearing aid that you don't own, you're not going to notice.
2. Welcomed improvements. These may not always be visible; for example, a recent change enables my phone to use its microphone to reduce the amount of background noise it transmits on calls when I'm using my headset, which helps my mother understand me when she's using her hearing aids.
3. Unwanted changes. These are the sticking point for users; there's been a noticeable change, and they want it reverted. The reason for wanting a revert doesn't matter - the point here is that the user does not want this particular change, whether it's because something has regressed, because their phone now gives them an electric shock whenever they answer a call, or because a button has gone from green-gray to a teal-gray and they find it ugly.

With infrequent changes, by the time you report an unwanted change, there's a good chance that welcomed improvements are built on top of it, and it can't be fixed; you've just got to live with it, now, because it's ingrained in the system. With frequent changes, you have decent odds of reporting the problem before something else depends on it, allowing the change to be undone, then redone into a wanted improvement or unnoticed change. Additionally, if the report comes in late, if the user will bisect down to a small window of change, you can at least explain why the change was made, because you can identify the commit(s) that led to it, rather than just shrugging because the change is too old to track down now.