Virtual Places Chat - Technical Details

Technical Details

The VPchat protocol uses a TCP connection to the server on port 1533. To help circumvent problems if this port is not open in a firewall, FTP port 21 can be used instead. This is a per-client option.

There is also a separate buddy list/instant messenger client which can be used as a stand alone client or in conjunction with the chat client. There is a button in the chat client for launching the buddy list so it appears to be a sub window of the client, however it can remain running after the chat client closes and the user is connected to the chat server a second time through the buddy list.

Originally the buddy list was designed as a separate system, not necessarily related to chat rooms. Users signed in to the buddy list using an email address and password. As the clients are used now at vpchat.com, the fact that the user is signed in twice is somewhat hidden. The system creates the buddy list name automatically by appending “@buddy” to the user’s chat name and they share the same password. This dual login works well to allow the buddy list to exist with or without the chat client.

The buddy list client also supports a multi-user chat conference, similar to a chat room but without avatars. People participate in the conferences by invitation from the person who opens the conference. The rooms do not have names that appear in the public chat room list so uninvited users cannot find them and enter.

The chat protocol is proprietary, although Ubique at one time documented a subset and offered it as an Internet standard for buddy list and instant messaging. It was not adopted as a standard. In the late 90s, Ubique was purchased by the Lotus division of IBM, and a second generation protocol was developed which is now in use by the Lotus Sametime instant messenger.

A major feature of the protocol is efficient support for relatively slow speed connections, e.g. dial-up. There is very low overhead associated with chat traffic. The avatars, up to 16K bytes each, are a potential source of performance problems. When a chatter first enters a room, which may contain many other chatters, he is sent all their avatars. This can be a major source of “lag,” which is addressed by sending the avatars asynchronous to the conversation text. A chatter will begin seeing the room conversation immediately, and he can participate in the conversation before any avatars are loaded. While avatars are loading the chatter will see “hour glass” graphics in place of peoples’ avatars. As the avatars are downloaded, interleaved with conversation, one by one the hour glasses convert to individual pictures. On a slow connection this can take a while, but with a fast connection it is barely noticeable.

Each chat connection from client to server is a persistent connection. The TCP socket remains open for the duration of the chat session. This greatly assists with implementing the idea of “presence” in the community, as the server knows who is connected and where they are chatting at all times. A downside of persistent connections is the proliferation of server side connections as the number of chatters grows. Many chat systems deal with scale of connections by using non-persistent UDP based connections, at the expense of accurate, up-to-date presence information for all the chatters. The VPchat server deals with this by using a two layered system.

The developers observed that a large amount of processing overhead is consumed by the server managing all the connections, at the socket level. A layer of one or more multiplexors (muxes) is implemented, each of which does little more than manage a large group (several thousand per mux) of TCP sockets. The muxes make a periodic pass through all the sockets and gather all the incoming messages into a large bundle, or meta message, which is passed to the chat server. The server gathers the incoming bundles, breaks them apart and analyzes them, then builds new outgoing bundles which it sends to the muxes. The muxes then take care of distributing the individual messages out through the client connections. In this architecture the server only has one TCP socket per mux, which is orders of magnitude (e.g. 10,000 to 1) less than the client connections. Thus a single server can easily scale up to a large number of client connections. New muxes can be added as needed. Given the performance of CPU technology of the late 90s, Excite and Ubique estimated that a single VP server could manage a community up to about 100,000 chatters. In buddy list/messenger applications, in which the level of traffic per user is much less than for chat rooms, the server handles hundreds of thousands of simultaneous users.

However to scale up to millions of users, as handled by chat systems such as Yahoo, MSN, or AOL, the single central server would have been a limitation. The Ubique and Excite developers were working on a multi-server enhancement to handle larger traffic, but the demise of Excite and the purchase of Ubique ended that effort. The Ubique engineers continued their efforts with Sametime, which now supports multiple central servers. For the much smaller level of traffic seen at vpchat.com, the single server technology is not an issue.

To ease the load on the central server, many auxiliary services are offloaded to specialized servers which can run on separate machines. For example, the user name and password authentication at login is offloaded to a server which works with a SQL database. The conversations of logged in chatters are not slowed while new chatters are authenticated. Also, management of presence – who is in which room – is maintained in a separate server, and searching for a user by name is offloaded to yet another server. There are also separate servers for managing buddy lists, game and tournament scoring, managing the chat auditoriums, and for miscellaneous statistics gathering.

The data management aspects of the chat service are handled with an SQL database. Individual chatters have a chat name and password. Optionally they can have profile information which is saved on the server. Avatars and buddy lists are saved on the client side, and uploaded to a cache on the server when a chatter sign in. This works well for scaling up the size of the system, but is a drawback when a chatter uses different computers as his avatars and buddy lists are not readily available.

The SQL database is also used for managing customer accounts. Users may purchase accounts that can have 2, 5, or 10 chat names associated with them. Any or all of the names can be used at the same time, for example family or friends can share an account. One person is responsible, however, for paying the monthly subscription fee.

The database assists community management by keeping track of privileges, penalties and warnings. Selected users can be given server privileges which include the ability to eject someone temporarily from a chat room, to “gag” the person for a period of time (i.e. prevent anything they type from being shown in the chat room), to prevent them from using an offensive avatar (i.e. their avatar is forced to an avatar of a baghead), or to eject them from the community entirely. Short of applying one of these penalties, a privileged user may officially “warn” another user about behavior. The use of penalties and warnings (who gave them out and who received them) is recorded so that the community managers can track behavior of troublemakers and also detect abuse of privileges. The system also lets individual users “ignore” the behavior of another user. The avatar and conversation from an ignored user cannot be seen by the ignoring user.

In addition to the chat, instant message, buddy list, avatar, tour, game, and auditorium features of VP chat, users can also share files and engage in voice chat with each other. Files smaller than 64K bytes are shared through the TCP server connections, but larger files and voice connections are implemented as peer-to-peer messages between clients.