Master servers for network games

There are plenty of complexities involved with writing a game that has online multiplayer. Even if you solve all the challenges to keep the game state in sync between all your players, there is one problem left: how do the players connect in the first place?

Usually we begin by using direct IPs. Many older games actually worked this way, as it is something that doesn't need somebody to negotiate the connection. Using IPs doesn't really scale well, though. It only works if you already have a friend to play with, they are technical enough to find their IP, make sure their router forwards to their local machine, and disable the firewall on that specific port. This is hardly release-worthy in the era where in most games, connecting is as simple as right-clicking a name in your friends list to join them.

We can avoid all this complexity by only ever having a single endpoint we need to talk to. If we could have a server that manages everything for us, we'd be good.

There are two models here: dedicated servers, and master servers. On a dedicated server, the game simulation actually runs on a server we control, and just receives the inputs from and sends the results to the players. This is a model often used by games where cheating is important to prevent, or where there is no clear lobby owner.

Master servers only negotiate connections between clients. Once the connection is made, they back out and let the clients figure it out themselves. The actual game simulation usually ends up being done using a host-client model (where the host simulates everything), or a peer-to-peer model (where each client simulates their own things). Today, we are only going to look at how a master server works.

Master servers

Let me try to explain master servers using a metaphor. Connections are like a piece of rope connected to two metal cans through which you can talk to each other. In a direct connection, you try to throw over the can to your buddy to talk directly, but you have to know where they are, and the firewall has to be out of the way. In the master server model, you throw your can to the master server instead, which often has an obvious position (e.g. a web address or an IP coded into the game logic) and a firewall open to these connections.

Once the connection with the master server is initiated, a host can tell the master server that is has a lobby. The master server will then keep track of that locally. If clients ask for open lobbies, the master server can respond. When the client decides to join a certain lobby, the interesting part happens: since the two sides of the connection have initiated a connection already, there's an exact address on each side, and a hole through the firewall to let messages through. The master server sends one can on each end of the rope to each of the peers through the existing connection, and a direct connection is established. From then onwards, the peers communicate directly, and the master server can drop the connections itself. This makes master servers especially relevant for games that can't afford to maintain dedicated servers, since the computational power needed is very limited.

This process is often called the NAT punchthrough or a hole punch.

The protocol

The first step in implementing a master server is to decide on a protocol with which your clients will talk to the master server. You could use text, JSON files, or some binary format you come up with. For Bearded.TD, a tower defence game I am currently working on, I decided to go with Google's protocol buffers, which are a protocol for sending serialized messages, with converters for many languages out there, (including C#). This would give me the freedom to work with other languages in the future if needed.

The implementation

For the implementation, we will be using Lidgren.Network. If you need an introduction to how it works, I recommend this blogpost on GameDev.

The first step is to initialize the NetPeer so we are available for connections.

public void Run() {
    var config = new NetPeerConfiguration(options.ApplicationName) {
        Port = options.Port
    };
    config.EnableMessageType(NetIncomingMessageType.UnconnectedData);
    peer = new NetPeer(config);
    peer.Start();

    // Do stuff here
}

Note that we enable UnconnectedData as a message type. This allows us to receive payloads without having to set up a permanent connection.

We could set the master server up to respond only when it receives a message using an event-based system, but I haven't been able to get it set up so that we receive an event from Lidgren whenever a network message comes in. Instead, we will just use a small loop. The added advantage this gives us, is that it allows us to regularly check if the registered lobbies are still active.

public void Run() {
    // Set up code here

    while (true) {
        while (peer.ReadMessage(out var msg)) {
            handleIncomingMessage(msg);
        }

        updateLobbies();

        Thread.Sleep(100);
    }
}

Now the only thing left to do is to handle incoming messages, and update the lobbies. Let's start with the incoming messages: we are going to need some boilerplate to deal with all the different message types Lidgren sends, and the request types we set up ourselves in the protocol file.

private void handleIncomingMessage(NetIncomingMessage msg) {
    switch (msg.MessageType) {
        case NetIncomingMessageType.UnconnectedData:
            var request = Proto.MasterServerMessage.Parser.ParseFrom(
                msg.ReadBytes(msg.LengthBytes));
            handleIncomingRequest(request, msg.SenderEndPoint);
            break;

        // Other message types
    }
}

Based on the type of the message that comes in, we may need to do different things. For example, we want to add the lobbies to a central database of lobbies (I use a dictionary for that), and we will also need a way to return a list of all current lobbies.

Let's take a look at a registerLobby method:

private void registerLobby(
        Proto.RegisterLobbyRequest request, IPEndPoint endpoint) {
    var lobby = new Lobby(
        request.Lobby,
        new IPEndPoint(
            new IPAddress(
                request.Address.ToByteArray()), request.Port), // internal
        endpoint // external
    );

    lobbiesById.Add(request.Lobby.Id, lobby);
}

You can see we keep track of the lobbies in a dictionary by their ID. This allows us to keep track of which lobby is which. It is important that active lobbies keep pinging the master server, since that allows us to get rid of lobbies that weren't active for some time.

The interesting part here is the IP endpoints we keep track of. This is needed for the hole punching later. The endpoint Lidgren gives us for this message is the external IP. We will also need the IP endpoint as the client sees it (the internal IP), so we actually have the clients send that information as part of the request. This is how the client code will send the necessary information to the master server:

var request = new Proto.RegisterLobbyRequest {
    Lobby = lobbyInfo,
    Address = ByteString.CopyFrom(
        NetUtility.GetMyAddress(out _).GetAddressBytes()),
    Port = 24680 /* can be any free port */
};

It is worth mentioning that we never actually set up a persistent connection with the master server. All data is sent without that, which is why we needed to support the UnconnectedData message type earlier.

peer.SendUnconnectedMessage(msg, masterServerEndPoint);

For sending the lobbies to the clients from the master server, the setup is very similar. The client sends a request to the master server that it wants to receive the lobbies, and the master server starts writing them to the endpoint that requested it.

The interesting part comes when we want to actually initiate a connection, i.e. do the hole punch. To initiate a connection, the client that wants to connect to a lobby sends its own endpoints, but it also sends a connection token. It doesn't matter much what the token is, so you could hardcode it.

Now that we have both endpoints, we can do the actual NAT punchthrough in the master server. Luckily, Lidgren.Network actually has built-in NAT punchthrough capabilities. It can be done using the Introduce method in the master server.

private void introduceToLobby(Proto.IntroduceToLobbyRequest request, IPEndPoint endpoint)
{
    if (lobbiesById.TryGetValue(request.LobbyId, out var lobby))
    {
        var clientInternal = new IPEndPoint(
            new IPAddress(request.Address.ToByteArray()), request.Port);
        peer.Introduce(
            lobby.InternalEndPoint,
            lobby.ExternalEndPoint,
            clientInternal,
            endpoint,
            request.Token);
    }
    else
    {
        // Deal with lobby not found
    }
}

When this is successful, the client will get a network message of type NatIntroductionSuccess (so make sure you enable this message type on the NetPeer!). When receiving that message, the client can initiate a normal Connect, for which it should use the endpoint from the message it just got, since that allows it to use the hole the introduction made.

Conclusion

This was a really quick run-through on how to build your own master server. To keep this post to a reasonable length, I had to cut out a lot of details. If anything is still unclear, I recommend you take a look at the full implementation of the Bearded.TD master server. You can find the logic that lives in the actual game here, and the master server itself here.