Interview with Sun's Chris Melissinos

It was about '98 when I started writing emails around the company, asking why we were not focusing on the games industry, how the oncoming surge of connected gaming was really going to require enterprise-level cross-platform technologies to really move the industry forward. Selling videogames to an enterprise company who was the dot in dotcom during the Internet boom days was almost a religious experience. About two years later, after trying to force this, I finally went directly to Scott McNealy and said: Why aren't we doing this? And he said: OK, well, you do it.

It was formally established in 2003. What we've done is gone out to the games industry and said we want your help in making Java a terrific platform for game development. So what's wrong with it? Why aren't you using it? We held a summit and walked away with a blueprint and from that assembled an experts group of 14 game companies and submitted the largest submission to the Java Community Process, called JSR 134, which was the Java Game Profile. Basically the attempt was to build the equivalent of DirectX in Java, and provide a complete cross-platform stack for media and game development.

Trying to build 9 APIs in a community process with 14 game companies proved to be just insane. So after a year and half of doing this we walked away from that process and decided to take the core APIs, which were the Java bindings for OpenGL, Java bindings for OpenAL for audio, and input, and we released those to the open source community, about the end of 2003, beginning of 2004.

Consumers aren't going out and downloading a million different business applications or productivity applications, they're not going out and downloading a new email client every other week. But what they are downloading are new games, all the time. So in order to continue improving Java adoption on the consumer side, we really need to focus on entertainment.

If we're not going to actually to stand up the organization to go after that market, which would be a huge investment from Sun's perspective, we said, Why don't we go out and give it to the community, because they're the ones that are actually innovating in gameplay? The independent game development scene is important and it's going to continue to be important as nextgen consoles come and start tapping into that market. It was the open sourcing of those APIs that really helped further establish Java as a viable platform.

It's all about mitigating risk and cutting cost. As the budgets of these games continue to climb, they're looking for hardened, tested, proven technologies rather than building it themselves. Today, nobody wants to build their own tools anymore. It's too much money, it's too time-consuming - how many times do you have to rebuild chat? So they're looking towards things that they can get access to for low cost, test against, prove that it works and build on top of it.

We said, OK, we've got these APIs done and we're moving forward, but there are some much bigger issues coming down the road with regard to network games and online gameplay that the games industry is ill-equipped to address. Being able to build enterprise-grade, scalable, fault-tolerant systems for online gaming is something the games industry just is not equipped to do. Why? Because they're game developers. Game developers don't want to deal with the pipes, the underlying network infrastructure. They want to build a game, they want to build an experience in art and story and music.

One of the things I like to point out is that really the biggest online multiplayer game in the world is Wall Street. If you look at everything that goes into building a realtime transaction-based system like a trading system, it's almost one for one what is required to build an online massively multiplayer game. So we said, OK, how do we take the things that we've learned about building those sorts of systems and apply them in a way that a game developer can get their heads around the technology without having to understand all the underlying components? What we have put together is a game-agnostic, platform-agnostic, server technology for online multiplayer gaming.

Darkstar is a wholly new way of building back-end technologies for online gaming. I'll give you an example of the type of things that exist currently in the games industry, and what we had to do differently to bring this technology forward.

[The virtual world] Second Life uses a geography-based grid system, which means that every square of their geography has to be physically represented by a server on the back-end - they've got 3000 of these servers. What that means is that if I have resources available on 80% of the servers because they're only running at 20% capacity, there's no way for me to grab those idle resources and apply them to other servers that are being just pounded because people are trying to get into that geography within the game world.

Another model is the sharded model, and this is the one that's been most widely adopted by the massively multiplayer game marketplace. A series of servers that represents the entire video game state for a fixed amount of players. So I may have between 7 and 20 servers that run the entire game of Everquest II for 5000 players. And when I want to add another 5000 players I build another seven to 20 servers and I basically just replicate all the data and put the next 5000 on the second stack of servers.

Again, you have problems. If I have 5000 people on one stack and I want to add 10 more people, I have to build a stack to service 5000 people, just to facilitate the introduction of 10 more people into the environment. Which then means that that second stack is costing 10 times the amount to run than the first stack, because you're not utilizing it. I cannot share dynamically those resources.

More importantly, I can only see the 4999 people within my shard: I can't see the next 5000 people that have joined the Everquest II service. This becomes a problem if you and I are next door neighbors and we're sitting down to play a game of Everquest II and we log in to different servers. That's it, we can't play with each other.

These are the things we have been addressing with Darkstar. Rather than assigning compute resources to a particular geography of a particular world, or a particular group of people, we are in essence assigning compute resources to the individual that connects to the system. As they move from area to area within the game we are basically moving those resources with them dynamically.

Say I have the foo forest and the bar desert, and in the foo forest I can handle 100 people and in the bar desert I can handle a 100 people, and they're at 100% capacity. As people move out of the bar desert into the foo forest, they take those compute resources with them. All of a sudden the foo forest can handle 150 players and the resources being assigned to the bar desert have been decreased to handle only 50.

So what we're doing is moving dynamically compute resources with the player as they move from interaction to interaction. And we're able to do this in a seamless way, we're able to do this with an incredibly high degree of data integrity because of the way the database system works within Darkstar.

For example, if you're playing on the server, and you're doing a transaction, and the server goes down in the machine room, what we're able to do is on the fly say: OK, this thread that you were just occupying died; find any other available thread anywhere else in this entire thousand-node system, and just re-perform the operation. And it just does it automatically. That means the players never see the server go down, you're never disconnected from the service.

And even if they are, because of the way that we actually do these transactions and we're pulling things into memory, we maintain a very high degree of persistence. So let's say you logged off your machine by accident - because you kicked the power cord out. When you log back in, we're able to persist all of the data that was there when you abruptly logged out, because the system knew that something happened, there was a disconnect here, it wasn't shut down as the client is supposed to, so persist the data.

The interface that we've built to the system is through a set of APIs. So it's not as if [developers] have to understand how it's doing failover and what it's pulling into in memory, and all these other things. They just interface through the APIs.

Those are things that we are currently addressing. We are working to make sure that we have a dedicated community landing spot and place for people to engage directly with the Darkstar team at Sun Microsystems. To help incubate the games that will be built on top of the technology, we have announced a program called the Darkstar Playground. Basically, developers will have access to a set of resources operated and paid for by Sun Microsystems, free of charge.

I have to tell you that I was not a big believer in open source, originally. It took me a while to understand exactly what the benefits are of open source, and that the benefits clearly outweighed any other model we looked at.

So how do we make money here? Because I give you a server, for free, and open source that required years of development, millions of dollars in funding, and some of the brightest people that we have in Sun Microsystems to build, when it comes time to service that, are you, as a game company, going to go out and hire four guys and have them sit down and try to understand all of the inner workings of everything? Or do you come to me, and I can give you a service contract that has an SLA in there, that guarantees certain things to the company, and it's reasonably priced?

Because Darkstar is game agnostic, and I can run multiple games simultaneously in the same stack, what you start looking at then is the ability to build a model that very much mirrors what the cable industry built against terrestrial television, which is a single infrastructure provisioning many channels of content. We can do the same sort of thing with Darkstar. One large infrastructure may handle many different channels of game content. So it's not unreasonable to look at the possibilities of setting up a complete service where a game developer or publisher never buys a single lick of hardware, they put their games onto a service, and then they're charged for usage.

Looking further down the road, what if you're so good at collecting money and doing the billing that basically you wind up collecting the payments for them? Instead of the developer getting a bill from you every month, they get a check, and [you] take the burden of the operation of these things off the shoulders of content creators.

And the third thing to look at is that we also happen to make some pretty awesome hardware, and wouldn't it be great if you had these kind of certified products that came out already preloaded that you could just slap into your data centre, expand your Darkstar infrastructure on the fly?

Without really giving out hard numbers, because it's not something that I can do, I can tell you that the online multiplayer segment of the games industry is not only the fastest-growing segment of the $40 billion industry, but is proving to be among the most critical pieces of the games industry. You look at the fact that every game console coming out, every one of them now is network enabled. From the Nintendo DS to the PS3, all of them have a network strategy.

Most people go: Ah well, how many people are actually playing those dungeons and dragons games anyway? That's an extremely small segment of what I believe is massively multiplayer games. The next-generation demand for these are not going to come from people like you and I at our ages, they're going to come from our kids. In fact, the biggest growth area right now I believe for online gaming is children between the ages of 6 and 12. One only has to look at sites like Neopets to get an inkling of where this thing is going to go.

So do I believe that this could be a significant revenue source for Sun Microsystems? There is no doubt in my mind. I also believe that because of the nature of the technology, we're going to wind up seeing Darkstar in places other than pure online games - anything that's doing high volumes of transactions across many, many people simultaneously could leverage this technology. For example, at the Game Developers Conference we demonstrated a concept space called MPK20 - basically Sun looking at game-type technologies but applying them to business practices.

MPK20 was a virtual office, and one of the things we integrated into the environment that you don't see in multiplayer games is positional audio. The core thing about it was that we were able to integrate this voice bridge technology that Sun has, which is a VoIP and audio technology, directly into this environment. As you're walking into the environment you can hear the conversations of the other participants as you walk by, and as you turn your head you hear them move.

There is a complement to MPK20, which called Project Wonderland, basically an open source framework for doing the integration of applications into these sorts of environments. Look at this from a business perspective. Three of us are all connected to this virtual environment, and you and I are standing in your office, and on your wallpaper is your entire Linux desktop - you can use it from inside this 3D environment. And because you and I have sufficient permission to actually see that content, I'm looking at this screen as you're manipulating it. But then [someone] comes walking in and she does not have the right level of permissions, so what does she see? Just a blank window - she sees out to the mountains.

Absolutely. I could put something up, we could each have a different-colored pen to draw on it, in this space in realtime. I can work on a piece of code, pop it up onto the wallpaper, and say: Hey, will you run this application when you come in, in the morning, since we're twelve hours behind each other in timezones, and just post your thoughts? And when I come back in the next day, you've run the application, all your notes are scribbled right there on the whiteboard in this virtual space.

Let's say there's a fourth person we wanted to be in this conversation, but they weren't able to be there. We can actually play back the entire script and they can view it at their leisure later on. Because it's just a script of a 3D engine, they can actually play it back nuance for nuance, word for word, audio for audio, movement for movement, at any other time. And because they're looking at it from a third-party perspective, they can actually move around the rest of the environment and hear other things that are going on, and see other things that were occurring at the same time.

That MPK20 demo happens to be entirely written in Java - it went from paper to demo in six weeks, with four engineers. But Darkstar, the technology, does not require that your game is built in Java. So if you're writing your entire game in C or C++ we have the same APIs that will let you connect to Darkstar as somebody who's building a Java game. And both of those games written in those two different languages can connect to the same Darkstar server at the same time and use its resources.

What we're saying is, from the client perspective, pick your client, we don't care. If you want to go out and build the game in Java, we'd love it. If you want to build it in C++, knock yourself out, and there are APIs to allow both of those technologies to leverage the platform. We're not saying that it has to be just Windows and OSX, it can be anything you want, including mobile.

Right now, we're the only company that has built the technology that's designed to be multiplatform for network-based gaming. I think we're coming to the market at the right time with the right message. Open sourcing it means the community can grow and build the pieces that they really want and we're going to be there to support it, grow on top of it, and build the business around it.