LWN.net Logo

Apple attempts to patent audio interfaces

For today's chapter on the ongoing software patent debacle, let us have a look at Apple's patent application #981993. This application, filed in November, 2004, has to do with providing an audio interface to a computing device. In particular, claim 1 reads:

A method for providing an audible user interface for a user of a computing device, the method comprising: receiving a selection of a user interface control on the computing device; selecting an audio file associated with the selected user interface control; and playing the selected audio file at the computing device such that an audio prompt is audiblized for the user, the audio prompt describing the selected user interface control or a displayed user interface item corresponding to the selected user interface control.

The additional, dependent claims make this technology more specific to media players in particular. There is another independent claim which reads like this:

A method for creating an audio file at a host computer system, the method comprising: receiving a text string at a text to speech conversion engine; creating an audio file based upon the text string; and associating the audio file to a media file.

Numerous other claims assert ownership over various combinations of the two above techniques. In summary, what Apple is claiming is the ability to create voice files for a media player device, load them onto that device, and have the device play those files in response to user actions.

This patent would appear to cover a relatively obvious technology. Speaking computers are not particularly new; corporate voice mail systems have operated in this way for quite some time. Experience shows, however, that this sort of prior art often carries little weight in the patent office. Unless something happens, the chances of Apple winning this patent would appear to be fairly good.

The Rockbox project has produced a GPL-licensed firmware distribution which runs on a wide variety of media players from a number of vendors - including Apple. Rockbox adds a number of interesting and useful features; see this LWN review from last January for more information. One feature of particular interest at the moment, however, is the voice interface capabilities built into Rockbox. This feature would appear to be well described by the Apple patent application; it uses voice files generated on a host system to allow navigation through the menus in an audible manner. When the voice mode is enabled, Rockbox's prompts are indeed "audiblized" for the users.

Rockbox has had this feature since early 2004. That is prior to the filing of this patent (though not the requisite one year prior), but Apple's application references an earlier one, filed in 2003. So Rockbox cannot serve as prior art in this case.

One of the most encouraging and heartening things your editor has seen over the last year has been the stream of blind users showing up on the Rockbox mailing lists. By making this feature available, Rockbox has made media players accessible to a broad community of users who have been ignored by the manufacturers of these devices. It is a beautiful example of how the free software community can meet the needs of a user community which is not seen as being profitable in the proprietary world. Apple may have been busy filing patents back in 2003, but it was Rockbox which first brought a voice interface to the iPod.

The voice menu feature in Rockbox has been an empowering addition for a number of people. The idea that it could be shut down by this patent is appalling. But Apple will have a clear incentive to do exactly that: Rockbox turns the competition's players into much nicer devices. Should Apple's near-monopoly on media players begin to erode (and there is no real reason why it should last forever), Apple will, beyond doubt, reach for legal weapons which might inhibit competing offerings. Apple has done that before, after all.

This particular weapon should be neutralized before it becomes a real threat. It is a fight which should be winnable - the idea of an audio interface was not first conceived in 2003. But without some determined resistance, Apple may well obtain the patent it is asking for. At that point, the free software community will (in the U.S., at least) be fenced out of an area which it explored before - and better than - anybody else.


(Log in to post comments)

prior art: phatbox

Posted May 5, 2006 17:28 UTC (Fri) by bryanlarsen (guest, #26230) [Link]

This sounds a lot to me like what the PhatBox does. http://www.phatnoise.com/products/digitalmediaplayers/ind.... I'm not sure how long phatboxes have been around or when they gained this feature, but I would be surprised if it didn't predate 2003.

Bryan

Blind users have had audio interfaces for years

Posted May 5, 2006 17:57 UTC (Fri) by dwheeler (guest, #1216) [Link]

Blind users have had audio interfaces for years. The cynical could easily view this as an attempt by Apple to prevent blind users from using non-Apple devices, even the ones they already have.

Blind users have had audio interfaces for years

Posted May 5, 2006 18:46 UTC (Fri) by trutkin (guest, #3919) [Link]

Yes, that is what Apple is trying to do. Prevent blind people from using their computers.

Blind users have had audio interfaces for years

Posted May 6, 2006 4:20 UTC (Sat) by bronson (subscriber, #4806) [Link]

...prevent blind people from using non-apple computers.

Blind users have had audio interfaces for years

Posted May 7, 2006 1:17 UTC (Sun) by jzbiciak (✭ supporter ✭, #5246) [Link]

I thought Apple users bought Macs for their looks?
</troll>

(I'm just kidding. Now put down those stylish brushed metal ball bats.)

Seriously, there has got to be prior art for this somewhere.

Apple attempts to patent audio interfaces

Posted May 5, 2006 18:44 UTC (Fri) by kleptog (subscriber, #1183) [Link]

There was a patent reform act in the US that was going to do away with this one-year-prior business by moving to a first-to-file system. My guess is this hasn't happened yet, right?

If Rockbox can really serve as prior art for this patent prior to the filing date, that would appear to make it safe anywhere outside the US.

Additional reading

Posted May 5, 2006 19:16 UTC (Fri) by corbet (editor, #1) [Link]

For another look at this issue, see this posting on BlindConfidential, a site dedicated to issues relevant to people with vision impairments. Suffice to say, they are not impressed.

Additional reading

Posted May 5, 2006 20:57 UTC (Fri) by tjc (subscriber, #137) [Link]

I find it kind of odd that a site for the vision-impaired uses a 6 or 8-point font for their articles...

Additional reading

Posted May 5, 2006 23:13 UTC (Fri) by JoeF (subscriber, #4486) [Link]

When you have a screen reader, it doesn't quite matter which font size is used.

Difficult reading

Posted May 7, 2006 22:37 UTC (Sun) by man_ls (subscriber, #15091) [Link]

Many semi-blind people don't have screen readers; they can manage with high-contrast, big text on the screen.

Screen Magnifiers --- Irony

Posted May 8, 2006 17:16 UTC (Mon) by AnswerGuy (guest, #1256) [Link]

This highlights an irony of using a screen magnifier to read "normal" text on one's screen.

From a UI perpspective if I have to position a cursor/pointer over text to read it anyway, then I might as well make the unmagnified text on my screen as small as possible. The will mean less fussing with the scroll bars and PgDn/PgUp keys. So long as the tiny size of the print doesn't exceed the granularity of my mouse movements it will actually be more efficient for me to use really small print since I have to run my cursor along every line of it to read it anyway.

Largely the same applies to those who are completely blind ... the voice reading software won't care about text display size, and minimizing paging may also minimize the discontinuties caused by hitting the end of the text display region (when the screen reader starts spouting off the labels from any buttons or widgets that are below that text).

Of course it's also possible that the maintainers of that site are simply unaware of what it looks like to sighted people or that it's their own privately little joke (along the lines of "give 'em all a little taste of what we go through").

JimD

Additional reading

Posted May 8, 2006 16:22 UTC (Mon) by kirkengaard (subscriber, #15022) [Link]

Eh. Firefox (at least) lets you set a minimum font resolution in the font preferences. Set it to 10, for example, and no page will render with less than 10pt fonts. As long as your UI is rendered with large enough fonts, this should be easy to set, and may solve this sort of problem.

Additional reading

Posted May 9, 2006 11:27 UTC (Tue) by Klavs (subscriber, #10563) [Link]

ctrl-+ fixes that ;)

Apple attempts to patent audio interfaces

Posted May 8, 2006 11:05 UTC (Mon) by rrw1000 (guest, #31035) [Link]

I don't have time to read the patent just now, but FWIW, Microsoft Research had a DJ system in the form of a talking parrot in the late 1990s which probably invalidates an awful lot of the claims. They may have prior patents, of course ...

Apple attempts to patent audio interfaces

Posted May 9, 2006 14:02 UTC (Tue) by mitchskin (subscriber, #32405) [Link]

Well, IANAL, but I have read through parts of the Manual of Patent Examining Procedure related to precedence. AFAICT, the fact that this patent application is "related" to another, earlier application doesn't necessarily mean that this one gets the priority benefit from the earlier one.

If you go to http://portal.uspto.gov/external/portal/pair and enter 10/981,993 (this application's number) into the search box, and click on the "continuity data" tab, you'll see that there's no parent continuity data listed, which AFAICT means that Apple isn't trying to use the priority date from any earlier application.

Which, again AFAICT, means that the earlier application doesn't necessarily kill Rockbox's usefulness as potential prior art.

Apple attempts to patent audio interfaces

Posted May 11, 2006 9:20 UTC (Thu) by kleptog (subscriber, #1183) [Link]

I think the point is that under a first-to-invent system you can claim to have invented it up to a year prior to the actual filing of the application. Given that Rockbox doesn't beat the filing date by a year, you'd have to have an actual court case to decide who "invented" it first.

I'm most places in the world, it's the filing date that counts, which is (IMHO) a much saner system.

Apple attempts to patent audio interfaces

Posted May 10, 2006 21:20 UTC (Wed) by filker0 (guest, #31278) [Link]

I don't know if this applies, but back in the mid-1980s, when I was at DEC, I was marginally involved in an effort to make the Kertzwell synth UI (which was buttons and an LCD/LED display) work for Stevie Wonder. This was done with a text-to-speech device (DecTalk) and a hardware mod in the synth to send the display data to a serial port connected to the DecTalk. A program was used to "scrape" the important information from the display data and "speak" it. This handled menus and other things displayed on the UI of the synth, and seems to be prior art that "audiblized" an otherwise graphic user interface back in the 1980s.

It would be really cool to get Stevie Wonder involved!

Posted May 11, 2006 4:50 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

Imagine the press, and the bad publicity, if the good guys could get a statement from Stevie Wonder objecting to Apple trying to make things worse for the blind just to line their own pockets! And as a blind musician, I'm sure that he has plenty of experience with all kinds of prior art, since he's had to deal with electronic equipment for decades and find technical workarounds or have others do it for him.

Apple attempts to patent audio interfaces

Posted May 11, 2006 12:32 UTC (Thu) by nix (subscriber, #2304) [Link]

s/Kertzwell/Kurzweil/?

Apple profits because of BSD license

Posted May 11, 2006 2:22 UTC (Thu) by shemminger (subscriber, #5739) [Link]

The lack of patent protection is one of the problems with the whole BSD license model.

Apple profits because of BSD license

Posted May 18, 2006 10:36 UTC (Thu) by ringerc (subscriber, #3071) [Link]

The vast majority of licenses currently don't offer any patent limits or retaliation clauses. The GPL's share-alike provision, for example, has no bearing on patents, though IIRC the GPLv3 will present an optional retailiation clause.

That's not to say there aren't licenses that do attempt to address the issue in various ways. It's just that the issue is very far from specific to the BSD license, and all the most widely used free/open licenses have the same flaw in so far as it's percieved to be so.

Apple attempts to patent audio interfaces

Posted May 11, 2006 18:14 UTC (Thu) by npitre (subscriber, #5680) [Link]

In 1994 I was working for a company called VisuAide which was
commercializing a product called Magnum. This was a voice recording
device using a 3.5po floppy for about 50 minutes of voice storage
aimed to replace the traditional cassette tape recorder often used by
blind people to take notes. This device had 3 "applications": a
notepad, a calendar, and an address book to organize voice recordings
into structured sets. And obviously all
the user interface was audio based with navigation prompts, etc.
Entering "VisuAide" and "MagNum" in a Google search will turn up many
hits about it.

In 1996, also at VisuAide, I architected the software for the first
version of Victor, a portable digital book reader also for the use of
blind people as well as sighted people (which by the way was based on
Linux). It of course followed the same principle of having audio
prompts to help navigation through the book structure and operation of
extra functions. Later models of the Victor Reader are still being
produced and sold today.

Another company, Plextor, is producing a similar device called
Plextalk also using audio prompts and has been doing
so for the last 10 years as well.

Apple attempts to patent audio interfaces

Posted May 18, 2006 12:47 UTC (Thu) by wljonespe (guest, #37810) [Link]

Texas Instruments had audio interfaces to various devices, including the TI-99/4 Computer, before 1984.

Apple attempts to patent audio interfaces

Posted May 12, 2006 16:20 UTC (Fri) by aegl (subscriber, #37581) [Link]

"an audio prompt is audiblized for the user"

Patent should be denied on the grounds that "audiblized" is not a real word.

Perhaps we can defeat the patent by making up our own meaning for it:

Audiblized (verb): To have ones eardrums injured by excessive volume. E.g. Steve is in the hospital after getting audiblized by his iPod.

Copyright © 2006, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds