Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

[Posted January 24, 2012 by corbet]

Here's a lengthy posting from Mark Shuttleworth describing the "heads-up display" concept that Ubuntu is pushing toward. "It’s smart, because it can do things like fuzzy matching, and it can learn what you usually do so it can prioritise the things you use often. It covers the focused app (because that’s where you probably want to act) as well as system functionality; you can change IM state, or go offline in Skype, all through the HUD, without changing focus, because those apps all talk to the indicator system. When you’ve been using it for a little while it seems like it’s reading your mind, in a good way."

to post comments

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 15:13 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

That's the semantic desktop KDE folks were talking about...

And from the demo it looks quite cool. It'd be nice if it can be separated from Unity.

HUD from the command-line

Posted Jan 24, 2012 15:39 UTC (Tue) by sladen (guest, #27402) [Link]

See the command-line example in the video (the data is sourced via the existing dbusmenu abstraction):

Introducing the HUD to Ubuntu (YouTube, offset 2:18, "the HUD on the commandline")

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 15:59 UTC (Tue) by patrick_g (subscriber, #44470) [Link] (2 responses)

>> And from the demo it looks quite cool.

What about L10n? The demo is in english but does it work in other languages? And I'm not sure a text interface like this is really usable for non-geek users.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:02 UTC (Tue) by cortana (subscriber, #24596) [Link]

According to something that just popped up on Planet GNOME, the localized menu entries (and other strings) are used.

http://gould.cx/ted/blog/Searching_menus

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:02 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, adding icons to actions and L10N should be fairly straightforward.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:32 UTC (Tue) by renox (guest, #23785) [Link] (1 responses)

> That's the semantic desktop KDE folks were talking about...

I don't think so: does KDE's semantic desktop care about application menues?
AFAIK no.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:34 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

They are _talking_ about it. Not actually doing anything useful.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:02 UTC (Tue) by bluss (guest, #47454) [Link] (3 responses)

Seems to be much like Quicksilver etc, but many years later.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 16:40 UTC (Tue) by zzxtty (guest, #45175) [Link] (2 responses)

I haven't read the article (I'm at work) but to me it sounds very much like the command line!

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 17:31 UTC (Tue) by nye (subscriber, #51576) [Link] (1 responses)

>it sounds very much like the command line

In a sense, but not really because it's search-oriented.

It takes advantage of existing strings in menus and tooltips (and maybe dialogue boxes?) so as to avoid an initial bootstrapping problem, so it sounds like it would be a useful supplement to a traditional menu system (though not a replacement).

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 14:08 UTC (Wed) by gvy (guest, #11981) [Link]

The bootstrap is still needed (moreso than for the "usual" menus) but it's maybe the first thing Shuttleworth makes noise about that might actually pay back for those who prefer conscious approach to "intuitive" disorder. Let's see if that becomes a WinFS or not.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 17:35 UTC (Tue) by vivo (subscriber, #48315) [Link]

http://www.afiestas.org/improving-kde-applications-help-m...

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 18:46 UTC (Tue) by karim (subscriber, #114) [Link] (7 responses)

This HUD demo looks cool. So can I have this WITHOUT Unity?

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 22:58 UTC (Tue) by alecs1 (guest, #46699) [Link]

Indeed, it really looks cool and I can't wait to try it, but I have to wonder when I'll be able to try it in Debian.

Which brings me a bit to the bitter conclusions on the Systemd thread: packages from Ubuntu can't always be easily installed in Debian (and distributions aren't usually compatible in this regard), which meant I had KDE 4.7 in Windows and Ubuntu, but had to wait some 4 months (more?) to get it in Debian. To think that I chose Debian because of the multitude of packages...

I'll consider an Ubuntu subscription, if they promise to stay 99.9% compatible with Debian regarding package installation :)

Yeah, I guess I should move along with my rant, or at least target the right thread.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 23:15 UTC (Tue) by jspaleta (subscriber, #50639) [Link] (5 responses)

It's not in Canonical's best business interest to stand this up as an independent dbus service codebase. It better serves their interest to deeply integrate it into the existing differentiated technology stack that underpins the Unity interface experience.

It's a business decision really. They are, and having been, building a differentiated platform offering. This novel linkage of application functionality to a search oriented dbus service is just part of that differentiation strategy. You'll note that Shuttleworth makes it a point to explicitly say "Unity applications" in his talk about the roadmapping. They are working towards building a SDK and API story for application developers to target Unity explicitly to compete with other retail platforms.

If this implementation ends up being useful at all, it'll get forked or re-implemented in a more modular fashion. There's no reason that a different implementation can't run as a replacement daemon to respond the the dbus messages, as long as Canonical commits to a strong and consistent API for the messages. And they'll have to do that, to make any sore of application developer API credible. if they screw around with the API repeatedly they'll burn bridges with app developers who buy into using this over the summer. Like they did with desktopcouch over the past couple of years.

-jef

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 0:20 UTC (Wed) by kragilkragil2 (guest, #76172) [Link] (3 responses)

Well, once applications offer their menu items via dbus (do they atm?) implementing this is not rocket science.
So if they really get FOSS they should prefer a solution that also works for other enviroments .. let's see. Maybe Canonical really is as unethical as all the other evil megacorps (Intel, MS, Apple) .. we will find out.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 0:40 UTC (Wed) by jspaleta (subscriber, #50639) [Link] (2 responses)

There's no need to suggest there is anything unethical going on, even if you meant it in a tongue in cheek fashion.

I don't see anything unethical with producing a deeply integrated implementation of a UI service if that best suits their needs and the needs of their customers. They are producing an open implementation, so if it ends up being useful as a concept then its easy enough to start from their sourcecode and re-implement if needed. That's a country mile away from questionably ethical.

I'm not personally thrilled with the CLA requirement, but again, since they aren't asking for assignment in the new CLA, its not egregious enough for me to beat the war drums. Though I think they'd benefit if they could drop the CLA entirely and just move to a BSD-alike license with a contributor sign off process.

As for Dbus menus, Canonical's indicator concept is essentially a menu over dbus already. And if you look at where GNOME-Shell is going with its integrated application menu concept in the shell chrome...that's dbus as well. menus over dbus are state-of-the-art design thinking already. The question is just which implementations, which APIs, will application developers have to support. It looks like they'll have to support several as each of the "DEs" of merit continue to further morph and evolve into "platforms" and make technical decisions as to what API to expose for applications to use to integrate with the platform.

Well at least up until the platforms split entirely with the expectations that separate applications will be developed for each platform. A peg that at about a year from now for Unity and Gnome. At some point the differentiation will be big enough where it will just be easier to maintain separate application codebases that integrate well with each platform than to try to do it in the same codebase.

-jef

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 23:16 UTC (Wed) by rickmoen (subscriber, #6943) [Link] (1 responses)

Jef --

I think Canonical, Ltd. deserve acclaim for having reversed course on their prior (pre-Aug. 2011) requirement of copyright assignment. The licence rights that they are now asking in the Harmony CLA seem within reason, and in any event the firm are now completely above-board in using the term 'licence' only for an actual licence rather than a handover of ownership. Credit where due.

Rick Moen
rick@linuxmafia.com

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 23:37 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

Indeed credit where credit is due. I'm on record with saying that I'm willing to sign the their current CLA.

But let me remind you that Canonical made the change quietly with no press release.
http://identi.ca/conversation/77085878#notice-79730276

Isn't that funny. They make so many press releases about vapor or half baked things, and here they punted the opportunity to talk about the new CLA decision.

And...on the same week that Canonical released the updated CLA, without an official announcement of any kind, Mark Shuttleworth wrote a blog entry defending copyright assignment as a necessary business practice, appearing to argue against the corporate policy change.

http://www.markshuttleworth.com/archives/687#comment-361100

Credit where credit is due indeed.

Canonical didn't want to make a big deal about it by announcing it in any official way whatsoever. And in fact the founder of the company and financial backer goes out of his way to stand on his soapbox and argue passionately for assignment even though he knows the change is coming. Well then, I see no reason to make a big deal praising the decision-making. Because to be quite honest with you, it baffles me how Canonical's corporate culture came to decision it did. For all I know they flipped a coin when choosing the final CLA that they did. Not once has _anyone_ inside that corporate fenceline stood up and publicly challenged Shuttleworth's public statements making the case for assignment as a business necessity. And when given the opportunity the corporate entity as a whole punted on making a public announcement concerning the advantages of the new CLA over the old assignment policy. Good for them.

-jef

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 7:33 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Right now the source code is about 1000 LOC, according to Shuttleworth. It's trivially easy to rewrite it for other desktop framework.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 21:04 UTC (Tue) by flewellyn (subscriber, #5047) [Link]

This is a fascinating idea. Though, I think it needs more than just fuzzy string matching. Perhaps a syntax for specifying categories to search?

This is just menu accelerators done correctly

Posted Jan 24, 2012 21:43 UTC (Tue) by dskoll (subscriber, #1630) [Link]

I like this. Instead of having to remember "Alt-F e", I can just start typing "Send link" and have it auto-complete.

It's not a radical change (web browsers have done this sort of thing with URLs for a while), but it's a nice touch.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 24, 2012 22:56 UTC (Tue) by xnox (subscriber, #63320) [Link] (3 responses)

Default key is 'Alt'

Emacs users shout 'NOT THE META KEY'

=( I want to use this with emacs =))))) but I still want meta key!

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 18:37 UTC (Wed) by sorpigal (subscriber, #36106) [Link]

I believe the "windows" super key is the whipping boy of choice for whenever an extra modifier key is needed. As long as you can re-map the modifier everything is cool.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 31, 2012 9:44 UTC (Tue) by wertigon (guest, #42963) [Link]

A bit late here, but M-X already do what you describe for emacs...

A global M-X? Yes please!!!

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted May 15, 2012 16:40 UTC (Tue) by cben (guest, #84642) [Link]

By default Emacs is blacklisted from global menu / HUD:
http://stackoverflow.com/questions/7497440/how-to-enable-...
The good news is HUD doesn't interfere with meta in emacs.
The bad news is you can't have HUD in emacs, but as wertigon says, M-x is even better (in case you're not familiar with its partial completion smartness, try e.g. M-x -f-o-w TAB TAB).

However in gnome-terminal, combos like M-d are slowed down (even if I hold Alt and press d repeatedly, it seems HUD momentarily tries to open each time; the flicker I could live with, but the slowness is annoying).
So I set out to choose another HUD shortcut.
(configurable in CompizConfig Settings Manager → Ubuntu Unity Plugin)

Thank you wertigon for mentioning M-x!
Alt+x would fit the bill perfectly, but turns out if I set that, HUD steals M-x from emacs; but Super+x is also a good mnemonic :-)

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 1:01 UTC (Wed) by jiu (guest, #57673) [Link]

I think it's the 1st Shuttleworth post in a while which I find interesting. It's also the first where I don't get the feeling that he's cleverly manipulating me with words (see his articles on copyright assignment for example).

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 7:23 UTC (Wed) by dront78 (guest, #47603) [Link]

Made myself for awesome windows manager 4 years ago ;) Long Live Linix!

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 8:04 UTC (Wed) by ovitters (guest, #27950) [Link] (2 responses)

Seems interesting. It is pretty great that loads of things are being tried out. Even if this ends up not being usable, you cannot improve without trying.

I don't get the ahead of competition bit though; the ribbon is really good for letting people discover all the options easily. That is really needed within Microsoft Office (too many options).
I guess it relates to providing this searchable interface first. I think similar things existed in some programs, etc. But it is a whole different thing to provide this functionality by default. IMO the providing by default is the best part. Applications can rely on it, hopefully getting a way better experience.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 11:08 UTC (Wed) by xnox (subscriber, #63320) [Link] (1 responses)

Mac OS X has searchable menu items.

See: http://drupal.org/node/1340634

It's part of help for the app. And there is shortcut to start menu item search. But it's per this app only, and doesn't have global stuff.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 12:31 UTC (Wed) by tnoo (subscriber, #20427) [Link]

Emacs has searchable menu items (and all other functions, too). And who needs anything else than Emacs? ;-)

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 8:34 UTC (Wed) by Seegras (guest, #20463) [Link] (1 responses)

Seems nice enough for an add-on. But not as replacement.

Because you won't learn a program without exploring, and I'd rather explore menu-options than play text adventures.

And because if you're mainly working with the mouse, changing to the keyboard to enter commands isn't very productive (the opposite is true as well, of course. There should be no commands you need to click on with the mouse when you're mainly working with the keyboard).

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 29, 2012 11:54 UTC (Sun) by k8to (guest, #15413) [Link]

It's a frequently missed point.

Organization and search work *together*. In fact they work far better together than separately. Organized data can be searched efficiently without clumsy slow indexing passes that make everyone tired and produce bad results.

Keep them both.

Or just keep organization if that's too tall an order.

Shades of COM

Posted Jan 25, 2012 18:42 UTC (Wed) by sorpigal (subscriber, #36106) [Link] (3 responses)

Step 1: Make menu items of the current application invokable by searching for them.
Step 2: Make menu items of open applications invokable via the command line.
Step 3: Make some menu items of non-open applications invokable via the command line.
Step 4: IPC forever.

It's starting to smell kind of DCOPy in here.

Shades of COM

Posted Jan 25, 2012 21:42 UTC (Wed) by flewellyn (subscriber, #5047) [Link] (1 responses)

Pipes and redirection are also methods of IPC, you know. And it's based on DBus, which was based on DCOP. So, yes, IPC.

I don't see why that's a problem.

Shades of COM

Posted Jan 27, 2012 14:52 UTC (Fri) by sorpigal (subscriber, #36106) [Link]

There is no problem; I love IPC, DCOP and dbus. This is just a really good idea.

Shades of COM

Posted Jan 27, 2012 2:26 UTC (Fri) by foom (subscriber, #14868) [Link]

Mac OS X has had all 4 of those steps for a long while too:

cat <<EOF | osascript -
tell application "Finder" to activate
tell application "System Events" to click menu item "as Icons" of process "Finder"'s menu bar 1's menu bar item "View"'s menu "View"
EOF

That'll launch the app if it's not running yet, too.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 25, 2012 19:46 UTC (Wed) by cmccabe (guest, #60281) [Link] (6 responses)

What we need is a text-based interface that allows users to easily select the command they want, without wading through menus and dialog boxes. Then maybe we can have tab-completion for this command, so that the user doesn't have to type the entire command! If he wants to use the previous command, he can press the UP arrow key to see it. Lastly, we'll have a dynamically typed scripting language to make the interface extensible.

Has anything like this ever been invented before? Hmmm... nope, can't think of anything.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 26, 2012 7:33 UTC (Thu) by tnoo (subscriber, #20427) [Link] (5 responses)

> Has anything like this ever been invented before? Hmmm... nope, can't think of anything.

Emacs?

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 26, 2012 8:07 UTC (Thu) by neilbrown (subscriber, #359) [Link] (4 responses)

Yes, Emacs and the shell have some good stuff for quick access to commands, and sarcasm is lots of fun, but there is something new here.

That newness is adding weighting to the commands - both static and dynamic.

If, in emacs, I type
M-X d TAB
I get shown a list of all commands that start with 'd', much like the example. However the example matches commands where any word starts with 'd'. You can get something a bit like that with
M-X - d TAB
but then you don't get command which start with 'd'.

But more importantly, the example just lists a few commands with a word starting with 'd', and they seem like the ones most likely to be relevant here. *That* is the clever bit. And it even learns what commands you like using and gives them a higher priority.... I'm not sold on that one though I haven't tried it yet - I like predictability.

And even that isn't entirely new - people have been weighting choices based on passed experience for years.

So yes, of course things like this have been invented before. This shouldn't be seen as a new invention. Rather it is bringing together some established ideas and making them work well in a new context. This is the right way to innovate - stand on giant shoulders.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 26, 2012 8:33 UTC (Thu) by tnoo (subscriber, #20427) [Link]

OK, I concur that the weighting, especially the dynamical one, is clever and innovative. And also that this should work across applications.

So the next big thing would be that applications advertise their main menu entries, together with a short help string and a longer help (like Emacs functions do). This, together with HUD, really seems to be a way to boost productvity greatly. I, for one, would love to see this functionality in Office applications and GIMP, where menu entries often are in non-obvious places for the occasional user.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 29, 2012 1:19 UTC (Sun) by cmccabe (guest, #60281) [Link] (1 responses)

Yeah, I guess I shouldn't be so snarky. It's good that they're still trying new things.

However, I tend to dislike UIs where there is not predictability. It sounds like the this system will do some amount of guessing as to what you want, which I usually dislike.

I also dislike change in UIs. I don't feel like I have to feel guilty about this, either. UIs take time to master, and that time has a value. This consideration has been strangely absent from the debates about new UIs.

I guess at some point I'll have to install Unity somewhere and give it a try.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 29, 2012 3:30 UTC (Sun) by sbergman27 (guest, #10767) [Link]

If you dislike the UI guessing, you'll hate Unity. I used it for months. Never could figure out how you were really supposed to use it. It did work. But I never felt in control.

It's what drove me from Ubuntu back to the the RHEL world and Scientific Linux. It's good not to have to worry about my desktop being rethought from the ground up for at least the next 6 years.

The Linux desktop world is suffering from mass insanity. Hopefully temporary. And hopefully it will all have blown over by 2017.

Shuttleworth: Introducing the HUD. Say hello to the future of the menu.

Posted Jan 31, 2012 20:38 UTC (Tue) by robbe (guest, #16131) [Link]

Dynamically changing the weights of commands ("bubbling"?) certainly has its uses. But like the thing google does, it will dig you further and further into your rut, whether it is optimal or not (a local maximum, if you like).

For example, imagine a user copyediting text who searches for "delete". She will find a mass of commands, and for some reason pick "delete word". Because she has to delete a lot of these, she uses it again and again, reinforcing the preference for this command. A potentially more useful "delete sentence" command will be demphasised, as well as the even more apropos "delete clause".

Ok, now we need "users who picked this menu item also liked ..." There's probably a glut of patents to cut through.

Well this is different

Posted Jan 25, 2012 22:20 UTC (Wed) by drdabbles (guest, #48755) [Link]

It doesn't seem like the LWN community is lighting the torches yet, so this is interesting. I'll echo the sentiment that this is one of the first big projects that has piqued my interest in a while from the Ubuntu/Cannonical camp.

I've gotten used to (but still loathe) the Unity UI. But that's mostly because the application menus feel like an afterthought, and often times they don't work the way you'd expect or want. But, Unity combined with this could result in a clean, stripped down UI that provides tons of functionality without having to hunt for the *%&# menu item you want. :)

It's clear that this was probably part of the cannonical vision all along. And as briefly mentioned, completing this project would pave the way for voice control, which is a massive win for those with disabilities. Right now Apple kills that market, but with the right framework and a large enough effort (read: money and community), it should be possible to surpass them in every way.

Bonus points for the project being so small and built on top of DBUS- It looks like it could even be re-implemented easily if a competing distro didn't like the licensing terms.