LWN.net Weekly Edition for April 5, 2007

Of course, attaining this goal will open up so many opportunities that one might consider the opening of these opportunities as the real goal. Some of these are concrete -- improving accessibility of Flash-based web applications, tighter integration with the desktop, automatic blocking of advertisements. Others are more nebulous -- now that you have a feature-rich, graphics-oriented managed platform, what can you do with it.

Our major strengths are portability, and real support for small embedded systems. Gnash currently supports all the BSD and GNU/Linux distributions, as well as 64 bit systems, and embedded systems like the OpenEmbedded, and the OLPC. Gnash also runs on WinDoze and Darwin as well. We also support creating custom extensions to Flash, so it's now possible to embed a device driver for lets say, hardware buttons on your smartphone, to trigger event handlers within a Flash movie.

I'd like to improve the completeness of our ActionScript support, there are some classes that exist only as stubs. That and adding Flash v9 support. This is all on the roadmap, but we need more developers to make better progress.

And developers should of course join the project because they want to and hacking on Flash is fun. For me it's not just wanting to play Flash files, but it's also that Flash is a platform. You can work on anything: a script language interpreter, rendering, multimedia, accessibility or security. And code that you write is immediately leveraged -- you make one thing work, and 100s of flash files start working.

But I don't think it's solely the distributions' job to point out and work on deficiencies in the Free software world. It is also the job of the community. Distributions often just package what the community makes available to them. But it seems the community has also has been content with the closed solution.

There is also a lot of coordination on things like test cases, specifications, and documentation that are shared between the two projects. So why is this a problem for some people is beyond me...

Adobe has declined to comment on this issue, since the confusion benefits their lockin of the market. Although Adobe has said they support Open Source projects, and donated Tamarin to Mozilla, we'd love to see a public statement that Gnash developers won't be subject to a lawsuit. It's very difficult to find developers that have never installed the Adobe software ever, which is what we've been doing to maintain our clean room approach.

Flash provides features that are unmatched in the current Free software world, even when comparing it to the newest projects in the realm like SVG. And Flash is a pretty simple format. People have done great-looking sites, games or applications with Flash in your browser that are currently hard or impossible to achieve on any desktop. And they did that 5 years ago. So even if the Free software world were to not use Flash it certainly can learn a lot from it.

Another often overlooked thing is that there are a lot of content creators that create new multimedia content daily. Other people like Lawrence Lessig have already said that it is important to introduce them to Free content. In providing Free solutions to those artists and developers, the Free software community has a lot of mindshare to gain.

Trying to build 9 APIs in a community process with 14 game companies proved to be just insane. So after a year and half of doing this we walked away from that process and decided to take the core APIs, which were the Java bindings for OpenGL, Java bindings for OpenAL for audio, and input, and we released those to the open source community, about the end of 2003, beginning of 2004.

If we're not going to actually to stand up the organization to go after that market, which would be a huge investment from Sun's perspective, we said, Why don't we go out and give it to the community, because they're the ones that are actually innovating in gameplay? The independent game development scene is important and it's going to continue to be important as nextgen consoles come and start tapping into that market. It was the open sourcing of those APIs that really helped further establish Java as a viable platform.

One of the things I like to point out is that really the biggest online multiplayer game in the world is Wall Street. If you look at everything that goes into building a realtime transaction-based system like a trading system, it's almost one for one what is required to build an online massively multiplayer game. So we said, OK, how do we take the things that we've learned about building those sorts of systems and apply them in a way that a game developer can get their heads around the technology without having to understand all the underlying components? What we have put together is a game-agnostic, platform-agnostic, server technology for online multiplayer gaming.

Darkstar is a wholly new way of building back-end technologies for online gaming. I'll give you an example of the type of things that exist currently in the games industry, and what we had to do differently to bring this technology forward.

[The virtual world] Second Life uses a geography-based grid system, which means that every square of their geography has to be physically represented by a server on the back-end - they've got 3000 of these servers. What that means is that if I have resources available on 80% of the servers because they're only running at 20% capacity, there's no way for me to grab those idle resources and apply them to other servers that are being just pounded because people are trying to get into that geography within the game world.

Another model is the sharded model, and this is the one that's been most widely adopted by the massively multiplayer game marketplace. A series of servers that represents the entire video game state for a fixed amount of players. So I may have between 7 and 20 servers that run the entire game of Everquest II for 5000 players. And when I want to add another 5000 players I build another seven to 20 servers and I basically just replicate all the data and put the next 5000 on the second stack of servers.

Again, you have problems. If I have 5000 people on one stack and I want to add 10 more people, I have to build a stack to service 5000 people, just to facilitate the introduction of 10 more people into the environment. Which then means that that second stack is costing 10 times the amount to run than the first stack, because you're not utilizing it. I cannot share dynamically those resources.

More importantly, I can only see the 4999 people within my shard: I can't see the next 5000 people that have joined the Everquest II service. This becomes a problem if you and I are next door neighbors and we're sitting down to play a game of Everquest II and we log in to different servers. That's it, we can't play with each other.

These are the things we have been addressing with Darkstar. Rather than assigning compute resources to a particular geography of a particular world, or a particular group of people, we are in essence assigning compute resources to the individual that connects to the system. As they move from area to area within the game we are basically moving those resources with them dynamically.

Say I have the foo forest and the bar desert, and in the foo forest I can handle 100 people and in the bar desert I can handle a 100 people, and they're at 100% capacity. As people move out of the bar desert into the foo forest, they take those compute resources with them. All of a sudden the foo forest can handle 150 players and the resources being assigned to the bar desert have been decreased to handle only 50.

So what we're doing is moving dynamically compute resources with the player as they move from interaction to interaction. And we're able to do this in a seamless way, we're able to do this with an incredibly high degree of data integrity because of the way the database system works within Darkstar.

For example, if you're playing on the server, and you're doing a transaction, and the server goes down in the machine room, what we're able to do is on the fly say: OK, this thread that you were just occupying died; find any other available thread anywhere else in this entire thousand-node system, and just re-perform the operation. And it just does it automatically. That means the players never see the server go down, you're never disconnected from the service.

And even if they are, because of the way that we actually do these transactions and we're pulling things into memory, we maintain a very high degree of persistence. So let's say you logged off your machine by accident - because you kicked the power cord out. When you log back in, we're able to persist all of the data that was there when you abruptly logged out, because the system knew that something happened, there was a disconnect here, it wasn't shut down as the client is supposed to, so persist the data.

The interface that we've built to the system is through a set of APIs. So it's not as if [developers] have to understand how it's doing failover and what it's pulling into in memory, and all these other things. They just interface through the APIs.

So how do we make money here? Because I give you a server, for free, and open source that required years of development, millions of dollars in funding, and some of the brightest people that we have in Sun Microsystems to build, when it comes time to service that, are you, as a game company, going to go out and hire four guys and have them sit down and try to understand all of the inner workings of everything? Or do you come to me, and I can give you a service contract that has an SLA in there, that guarantees certain things to the company, and it's reasonably priced?

Because Darkstar is game agnostic, and I can run multiple games simultaneously in the same stack, what you start looking at then is the ability to build a model that very much mirrors what the cable industry built against terrestrial television, which is a single infrastructure provisioning many channels of content. We can do the same sort of thing with Darkstar. One large infrastructure may handle many different channels of game content. So it's not unreasonable to look at the possibilities of setting up a complete service where a game developer or publisher never buys a single lick of hardware, they put their games onto a service, and then they're charged for usage.

Looking further down the road, what if you're so good at collecting money and doing the billing that basically you wind up collecting the payments for them? Instead of the developer getting a bill from you every month, they get a check, and [you] take the burden of the operation of these things off the shoulders of content creators.

And the third thing to look at is that we also happen to make some pretty awesome hardware, and wouldn't it be great if you had these kind of certified products that came out already preloaded that you could just slap into your data centre, expand your Darkstar infrastructure on the fly?

Most people go: Ah well, how many people are actually playing those dungeons and dragons games anyway? That's an extremely small segment of what I believe is massively multiplayer games. The next-generation demand for these are not going to come from people like you and I at our ages, they're going to come from our kids. In fact, the biggest growth area right now I believe for online gaming is children between the ages of 6 and 12. One only has to look at sites like Neopets to get an inkling of where this thing is going to go.

So do I believe that this could be a significant revenue source for Sun Microsystems? There is no doubt in my mind. I also believe that because of the nature of the technology, we're going to wind up seeing Darkstar in places other than pure online games - anything that's doing high volumes of transactions across many, many people simultaneously could leverage this technology. For example, at the Game Developers Conference we demonstrated a concept space called MPK20 - basically Sun looking at game-type technologies but applying them to business practices.

MPK20 was a virtual office, and one of the things we integrated into the environment that you don't see in multiplayer games is positional audio. The core thing about it was that we were able to integrate this voice bridge technology that Sun has, which is a VoIP and audio technology, directly into this environment. As you're walking into the environment you can hear the conversations of the other participants as you walk by, and as you turn your head you hear them move.

Let's say there's a fourth person we wanted to be in this conversation, but they weren't able to be there. We can actually play back the entire script and they can view it at their leisure later on. Because it's just a script of a 3D engine, they can actually play it back nuance for nuance, word for word, audio for audio, movement for movement, at any other time. And because they're looking at it from a third-party perspective, they can actually move around the rest of the environment and hear other things that are going on, and see other things that were occurring at the same time.

That MPK20 demo happens to be entirely written in Java - it went from paper to demo in six weeks, with four engineers. But Darkstar, the technology, does not require that your game is built in Java. So if you're writing your entire game in C or C++ we have the same APIs that will let you connect to Darkstar as somebody who's building a Java game. And both of those games written in those two different languages can connect to the same Darkstar server at the same time and use its resources.

What we're saying is, from the client perspective, pick your client, we don't care. If you want to go out and build the game in Java, we'd love it. If you want to build it in C++, knock yourself out, and there are APIs to allow both of those technologies to leverage the platform. We're not saying that it has to be just Windows and OSX, it can be anything you want, including mobile.

Right now, we're the only company that has built the technology that's designed to be multiplatform for network-based gaming. I think we're coming to the market at the right time with the right message. Open sourcing it means the community can grow and build the pieces that they really want and we're going to be there to support it, grow on top of it, and build the business around it.