A new article in MIT Technology review discusses the benefits and challenges of "sharding," the process by which large userbases in online worlds are distributed across smaller copies of the world to evenly distribute server load and prevent overcrowding. This process means that users of the same virtual world aren't always able to play together -- something not all companies feel is the best situation for their users.

One such company is CCP, who strives to keep its entire userbase integrated in EVE Online. CCP CEO Hilmar Petursson told MIT Technology review that, given EVE's sci-fi futuristic setting, keeping everyone together has additional significance. "When you watch a typical sci-fi movie, you have the sense that you're watching a small part of a larger world," he said, adding that a single-shard environment also makes for more cultural diversity within the game, creating a more precise simulation of a large-scale economy. EVE Online in particular has prized a realistic in-world economy, even hiring a real-world economist to manage EVE's transactions this year. Now, according to the article, up to 400 of EVE's 200,000 users can do battle together in a single location; contrast this with World of Warcraft, whose 9 million users max out at 40-on-40 brawls.

It hasn't been easy for CCP to accomplish this; it recently announced it will shift its clusters to IBM blade servers in search of more storage density. CCP has deployed a cluster of more than 420 CPU cores housed in IBM System x and BladeCenter servers to position it for further rapid growth; EVE Online launched in 2003 and has seen that 200,000 user growth only in the period since then. IBM has said CCP's supercomputing cluster manages more than 150 million daily database transactions, and Petursson tells MIT Technology Review that he hopes the company's new supercomputing clusters will add enough capacity for 70,000 concurrent users.

MIT Technology Review elaborates further on this trend of shard-aversion:

But the problem of avoiding shards isn't specific to EVE Online. Wu-chang Feng, an associate professor of computer science at Portland State University, says, "I would say it's something most MMOs will go to, if they can." The challenge, he believes, is to create an elegant experience for users at the front end while constructing and managing servers on the back end that can keep up with the pressure of large numbers of players online at once. Other companies working on the problem include IBM, which works with EVE Online and other MMOs; Australian game developer BigWorld Technology; and Linden Lab, makers of Second Life.

IBM global technical lead for games and interactive entertainment George Dolbier chimes in:

To scale up to support millions of concurrent users, MMOs will need to make use of the technologies behind Visa's database, or those that support the NASDAQ stock market. NASDAQ, he says, can actually be thought of as a very large MMO, supporting very large numbers of "players" performing billions of transactions daily in a graphically intense environment, all within a single shard.

Dolbier says that it's not a one-way street: other industries could learn from the technology that's being developed for MMOs. For example, he says, a common problem for game companies is how to recognize and manage "hot spots": small areas that suddenly attract large numbers of players, such as a battleground. To keep the game running smoothly, the servers need to detect movement toward the hot spot and react in real time, rezoning the area of activity so that more servers are responsible for supporting it. Technologies that solve this problem effectively, Dolbier says, will have applications in any industry that requires spotting and reacting to trends, or "anything where behavior is dynamic and you need to move resources around rapidly."

[Via MIT Technology Review]