User: Password:
|
|
Subscribe / Log in / New account

Ubuntu's new app developer upload process proposal

From:  David Planella <david.planella-AT-ubuntu.com>
To:  ubuntu-devel <ubuntu-devel-AT-lists.ubuntu.com>
Subject:  Proposing a New App Developer Upload Process
Date:  Mon, 03 Sep 2012 19:59:15 +0200
Message-ID:  <5044EFF3.705@ubuntu.com>
Archive-link:  Article

Hi everyone,

As many of you will know, in the last few cycles we've been laying out
the foundations to make Ubuntu a target of choice for app developers. I
am not referring to building the platform here (an area in which we also
keep seeking growth), but rather to enabling app authors to create and
distribute original software, to grow a rich ecosystem of independent
apps for our users.

The resounding success of our latest initiative, the Ubuntu App
Showdown, has not only shown an explosion of high-quality applications
(created in just 3 weeks!), but more importantly, the growing interest
in developing applications for Ubuntu.

As we continue building upon these foundations, we also keep on refining
our processes to identify and improve areas in which we can provide a
better experience for app developers. While doing this, we've been
gathering metrics from different sources –the current queue of apps
pending review and the Showdown results being the main ones. They all
provide clear evidence: the current approach to submit third-party apps
in the Extras repository has already reached the limit in terms of human
review capability and application throughput.

Despite our best intentions and the Ubuntu App Review Board's epic
efforts, we're currently putting a strain on reviewers (who cannot keep
up with the incoming stream of apps) and providing an unsatisfactory
experience for app authors (who have to endure long delays to be able to
upload their apps).

During the course of the last few weeks, after having identified the key
issues and assessed the different options available, Jono Bacon, Michael
Hall and myself have been working to put forward a proposal for a new,
improved app developer upload process. We've been not alone in this
project: we've had the input and help from many others, including the
App Review Board and members of many other Ubuntu and Canonical teams
(Security, Foundations, Desktop, Consumer Apps, just to name a few).

We are now at a point where we consider the spec to be ready for public
review and comment, and we would like to ask for your own contribution
to this process. Ultimately, the goal is to discuss and scope the new
app developer upload process at UDS, but we would like to have a solid
specification to be used in advance as the basis for any UDS sessions
and blueprints. If you'd like to contribute, you can:

* Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
* Provide any feedback either on the Feedback section (preferred) or
  continue the discussion on this thread.

As the people who create Ubuntu, your input is especially valuable, and
we'd really appreciate it if you could spend some of your time to help
app development in Ubuntu succeed.

Thanks!

Regards,
David.



(Log in to post comments)

Sandboxed file access

Posted Sep 5, 2012 0:44 UTC (Wed) by geofft (subscriber, #59789) [Link]

One important aspect of this proposal, which is all-but-glossed-over in the writeup, is that the AppArmor policies that allow uploading "untrusted" apps will specifically deny access to users' home directories. To allow reading and writing documents, there is a helper application (yet to be written, and yet to be designed, as far as I can tell) to allow opening files via a file-chooser dialog box. Superficially, this is similar to the sandboxing requirement of the Mac App Store. This has a couple of implications.

First, applications that depend on accessing multiple files, manipulating directories, etc. basically cannot work with this restriction, since you can only pass them a single file. I recall hearing that a few programmer's editors on the Mac App Store have no version control integration, because that would involve watching a whole directory and its subdirectories. The file chooser API has no way to say "and permit access to this subdirectory".

Second, unlike the Mac App Store, there is no per-application sandboxed homedir that can be accessed without prompting for permission. (On the Mac, a sandboxed application has access to ~/Library/Containers/com.example.whatever.) This means that, for instance, games can't save high scores or paused games, applications can't save preferences, etc. etc. without prompting you to select a preferences file!

Lastly, and most importantly, accessing files this way is a completely different API from what Linux developers are used to. On the Mac, it's easy enough for Apple to modify the standard Cocoa open-a-file dialog box API call to instead go through the helper app, and to internally deal with the returned file handle within the Cocoa libraries. On Linux, there isn't really a standard API -- there's a Gtk+ dialog and a Qt dialog for apps using those frameworks, but the spec implies that many apps may be using neither -- and receiving a file descriptor from a privileged app is a complicated piece of code using UNIX sockets and ancillary messages that very few developers are used to. I'm skeptical that app developers will be able to adapt to this new API and restrictions, and find it easier than just getting their app packaged the "normal" route (through Debian), which is what this proposal is attempting to sidestep.

I genuinely wish them luck with this, since I'm convinced that this is roughly the form that application security will need to take in the future, but I'm not convinced there's enough awareness of how important it is to make this work and work well, given how brief the mentions are.

I have been meaning to read the proposal more carefully and see if there's attempts at solving any of this before commenting on the mailing list, but I figured I'd mention it on LWN if it's getting prominence here.

Sandboxed file access

Posted Sep 5, 2012 1:21 UTC (Wed) by reddit (guest, #86331) [Link]

Wow, looks like that finally someone has decided to create a secure desktop OS.

It only took 30 years, pretty fast I'd say... sigh...

Regarding all the limitations, it seems they are just due to the abysmal design of Mac OS X.

For example, it obviously needs to be possible to request the user to open a directory and get access to all children.

At any rate, I think most file accesses can be handled by:
1. Automatically granting read-only access to system-wide files installed by distribution packages
2. Automatically granting access to ~/.<app>
3. Automatically granting access to files mentioned on the command line
4. Making the GTK/Qt file chooser APIs automagically use the trusted file UI API
5. Perhaps granting access to a file permanently after the first time an application is granted access (so that "Recent files" works)

Note that it is also crucial to redesign the windowing system, so that it is impossible to imitate windows of other applications (as well as making it impossible to simulate input or read input or the cursor position without having focus)

This requires having a trusted process draw the window decorations, and having some way to prevent applications from drawing in their windows in such a way that the user thinks there is a top-level window inside them.

The latter can be done by either:
- Randomizing the color of the title bars every boot, and making it impossible to determine which is the current color (i.e. disallow reading the screen and taking screenshots by untrusted apps)
- Having a bar connect windows to the edge of the screen, as well as an internal border breaking that bar for child windows.
- Always creating new windows maximized and preventing apps from altering the window state (but this requires to teach the user that any window that starts not maximized is fake, which is not the case on current OSes).
- Not allowing windows to be positioned such that a window is fully contained in another (also requires to teach the user about this)

Sandboxed file access

Posted Sep 5, 2012 1:36 UTC (Wed) by geofft (subscriber, #59789) [Link]

Most of this is covered -- and attacked -- in the literature, and in particular the literature of years and years ago. People have been caring about secure windowing for ages; the first reference off the top of my head is this 20-plus year-old paper about extending X, but I'm sure you can find older things if you try.

Most of the attacks are along the lines of tricking the users, or relying on them not to pay attention. Randomized titlebar colors seem like they will work about as well as secure pictures for anti-phishing for banks, i.e., not actually that well.

Also, you should look at Qubes, from the previous post, which is actually attempting to be a secure desktop OS.

Sandboxed file access

Posted Sep 5, 2012 9:33 UTC (Wed) by khim (subscriber, #9252) [Link]

Also, you should look at Qubes, from the previous post, which is actually attempting to be a secure desktop OS.

Well, it tries to build a secure desktop OS, but, as usual, in the end it'll create secure server OS. On desktop the biggest problem is Dancing pigs problem. MacOS and Ubuntu are trying to solve it by making it inconviniet for the end-user to give full access to the homedir (you need to select files one-after-another to give them to the application). This works: user will send comple of dozen of his (or her) files to-god-knows-where but s/he'll quickly become bored and will just close the program without obtaining these valuable dancing pigs. Not an ideal outcome but much better then what we have today. If you'll add the ability to request the user to open a directory and get access to all children developers will start asking for the access to /home/<username> right and left and dancing pig trojans will follow.

Sandboxed file access

Posted Sep 5, 2012 1:26 UTC (Wed) by khim (subscriber, #9252) [Link]

I'm skeptical that app developers will be able to adapt to this new API and restrictions, and find it easier than just getting their app packaged the "normal" route (through Debian), which is what this proposal is attempting to sidestep.

"Usual route" is not acceptable for the wast majority of the applications. I mean commercial closed-source ones. Don't be misled by the stats which show that, e.g., most Android applications are "free": yes, they are free-as-in-beer, but that's because they include ads and these don't really fly in open-source world.

But yes, if there will be money to be had then applications developers will jump through necessary hoops. I'm skeptical too, but for a different reason: Ubuntu attracts a lot of "every app must be free!" guys thus it's not clear if there are enough money on the table to make it an attractive target for the app developers.

Other alternative (webapps) are more interesting: they make it possible to create "kind-of-native" applications by adding just a few tweaks to the existing web apps. This may fly: sure, the advantages are small, but the efforts required are small, too.

For the native apps... well, the effort to recompile everything under Linux is already huge! We've found about this the hard way: initially most NaCl developers were Linux guys, but as time goes on more and more come from MacOS and Windows world. They demand Visual Studio integration, etc. Native Client is cross-paltform so it works for us (not 100% well: core is cross-plaform, but WebGL is not so we frequently see problem with applications which were never even tested on Linux), but for Ubuntu this will be big problem: I doubt Canonical will rush to develop Visual Studio plugin any time soon.

Sandboxed file access

Posted Sep 5, 2012 1:33 UTC (Wed) by geofft (subscriber, #59789) [Link]

My understanding of this initiative is that it's specifically targetting free-software applications. The spec claims that the usual app review process for commercial software (whether gratis or not) is working fine, but the app review process for free software isn't scaling.

"... if the developer is selling their software, they are reasonably well catered: the consumer apps team will take care of them delivering their apps into Ubuntu. For Free Software developers ... they must do more work themselves. The Application Review Board (ARB) are there to review these apps but the ARB, as a team of volunteers, is not able to scale with increases in the queue ...."

In particular, this seems to mean that commercial software will be manually reviewed, not subject to this sort of sandboxing, and also not subject to the Debian / Debian-packaging route, which seems to work well for everyone. I'm not objecting to that.

Sandboxed file access

Posted Sep 6, 2012 9:52 UTC (Thu) by krake (subscriber, #55996) [Link]

"yes, they are free-as-in-beer, but that's because they include ads and these don't really fly in open-source world."

Any specific reason? Do ads services require some secret key on the application side that could not be committed to a source repository?

Sandboxed file access

Posted Sep 6, 2012 9:59 UTC (Thu) by njwhite (guest, #51848) [Link]

There's also the issue that many people in our community find ads somewhat offensive, so would release a build of the program without them, which may well be more popular.

Sandboxed file access

Posted Sep 6, 2012 10:36 UTC (Thu) by krake (subscriber, #55996) [Link]

Well, that would mean somebody else is volunteering to do your packaging :)

Or one could just allow to switch it off. Users who'd like to support you would keep it on, user who don't or prefer direct donations could just treat it like adblock for browsers.

Anyway, the question was if there are technical reasons prohibiting ads supported Free Software

Sandboxed file access

Posted Sep 6, 2012 10:42 UTC (Thu) by nix (subscriber, #2304) [Link]

Certainly many of us find ads offensive *when badly done*. But I'm about as strongly against ads as you can find -- I block them almost everywhere because I have visual problems which mean that any ads displayed next to text will visually overlap the text, so if they're not of very similar colour and 'ink' density (which they never are) they'll make the text unreadable. And ads that leap out at you are unacceptable because they're a distraction and I have enough problems with *that* as it is.

But even for me *some* ads are tolerable. Google's text ads are of the same density as the text and are clearly delineated so I can skip them. Ads on the special-offer Kindle are acceptable because they're only displayed when the device is off and thus not displaying text I want to read anyway. And because the latter ads, at least, are actually targetted to some degree, sometimes those ads *are* useful, in that they tell me about things I would have wanted if I'd known they existed. If you'd told me three years ago that I'd avoid paying to turn off ads because the ads were useful I wouldn't have believed you. But no, it *is* possible to do advertising right enough that even someone with biological reasons to loathe advertising can like them. It's just that most online-ad people are so far from that that it is hard to believe.

Sandboxed file access

Posted Sep 6, 2012 11:37 UTC (Thu) by njwhite (guest, #51848) [Link]

> But I'm about as strongly against ads as you can find
> <snip>
> But even for me *some* ads are tolerable.

You're not as strongly against ads as me, then ;)

I'd argue adverts are never ever truly useful as recommendation agents. They are inherantly driven by who has the money to run them. So they work against small, new, innovative players. Let's not forget, our contemporary problem is *not* that we have too little information about new, interesting things. Advertising serves to shift the balance of the things we hear most about to those of entrenched interests.

Then there's the whole not wanting to be manipulated by said interests any more than you can avoid (and people tend to underestimate the degree to which marketing works, even on them.) Or surveilled by them.

Any business model that relies on ads is one I disapprove of. It pushes the initial economic cost to the reader down to zero, but at too great a cost.

Sandboxed file access

Posted Sep 5, 2012 5:42 UTC (Wed) by dlang (subscriber, #313) [Link]

remember that this is not a set of requirements for all applications in Ubuntu, it's only for those who don't want to be in the main repository and instead want to be in the 'software store' with little review

nothing prevents them from doing a more complete review of other apps and allowing them in the software store without being limited like this.

This seems to me to be a great balance between the "install anything with no review and plan on finding malware after the fact" world of android and the "we control everything, don't make us angry, you wouldn't like us angry" world of Apple (where they still end up catching malware after the fact)

making these sorts of apps distinctly second-class citizens, in exchange for reduced review requirements, seems like a good compromise.

Sandboxed file access

Posted Sep 6, 2012 1:35 UTC (Thu) by geofft (subscriber, #59789) [Link]

Well, sure, but my thesis here is basically that if the open-a-file API is unusable, nobody will use this approach. It's very easy to design secure and unusable systems. So the effort for it would be wasted, and one of the existing submission interfaces, e.g. manually-reviewed Extras or Debian packaging, would be the only things people actually use.

Sandboxed file access

Posted Sep 5, 2012 13:28 UTC (Wed) by walters (subscriber, #7396) [Link]

Any application with access to X11 though can elevate privileges with not that much more effort than access to the home directory gives. This appears to be handwaved away in the Ubuntu page =/

It's a really hard problem of course - I talked about it in 2005; see "SELinux and the Linux Desktop".

Sandboxed file access

Posted Sep 6, 2012 1:32 UTC (Thu) by geofft (subscriber, #59789) [Link]

I believe that the plan is to have AppArmor use XACE hooks, which IIRC are the same hooks that SELinux is using for security-enhanced X. This was mentioned on the spec, but I'm not up-to-date enough on AppArmor to know how good the XACE support is.

Certainly a nicer solution would involve the X security extension, or better yet, an untrusted X proxy that also happens to NAT all global X identifiers. I was working on code for this when I was a grad student, then got distracted and went to industry. Maybe I'll dust it off again and see if I can get something working...

Sandboxed file access

Posted Sep 6, 2012 8:33 UTC (Thu) by dplanella (guest, #57898) [Link]

Thanks for the feedback. I guess I'm missing the context of not having been at the talk, but having looked at the slides you're pointing to doesn't give me much detail about the problem.

Would you mind elaborating the issue and options to fix or mitigate it?

We'd be more than happy to add it to the spec if it's currently not covered in enough detail.

Thanks!

Sandboxed file access

Posted Sep 6, 2012 1:25 UTC (Thu) by mhall119 (guest, #57124) [Link]

The spec has been updated for clarity to address some of your concerns.

While it wasn't previous specified, the default AppArmor policies did in fact give read and write access to user-level configuration directories for the application, so saving games or user settings is possible by default.

The Helpers is more vague, mostly because the discussions around them haven't been going on as long as those around AppArmor itself. I have updated that section to include additional ways we can promote their use among developers.

Sandboxed file access

Posted Sep 6, 2012 1:40 UTC (Thu) by geofft (subscriber, #59789) [Link]

Thanks! Yeah, I did realize after I made this comment that it was unclear to me whether the AppArmor policies defaulted to granting home directory access or not. (I _think_ from the abstractions listed when I checked yesterday that they don't, but I wasn't sure.) Certainly the private-data abstraction seemed redundant, if home directories were outright denied.

I'm still unsure from the most recent edit whether you're doing an Apple-style permission to access one particular subdirectory of the user's home directory, or allowing access to the entire home directory (other than private-data). If the latter, it's unclear to me what the helper dialog would do -- do users commonly have files they want to access that are _not_ in their home directory? (I guess, assuming the default umask hasn't changed, "reading other users' files" is arguably relevant.)

Anyway, I do mean to reply on the mailing list once I have some more coherent thoughts together.

Sandboxed file access

Posted Sep 6, 2012 2:21 UTC (Thu) by dlang (subscriber, #313) [Link]

what it sounds like to me is that the app will have a directory that it can use to save it's own data that can use the 'standard' I/O routines

but if it wants to get access to files outside of that sandbox, it's not going to have any choice other than to use the helper.

Sandboxed file access

Posted Sep 6, 2012 18:59 UTC (Thu) by mhall119 (guest, #57124) [Link]

Apps won't have access to the user's home directory by default, but they will have access to ~/.config/{appname}/ by default, which they can use to store configuration and user data.

My understanding of the private-data abstraction is that using it will prevent access to those directories (~/.gnupg, ~/.ssh, etc) even if some other abstraction (or helper) gives access to a parent directory (~/).

Sandboxed file access

Posted Sep 7, 2012 8:59 UTC (Fri) by krake (subscriber, #55996) [Link]

Shouldn't they also get access too ~/.local/share/{appname} for storing data rather then config?

Ubuntu's new app developer upload process proposal

Posted Sep 5, 2012 6:22 UTC (Wed) by cmm (guest, #81305) [Link]

So, is this about Ubuntu proposing a process for uploading app developers?


Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds