Analyzing Disconnects

When you build an online multiplayer game, you have to be aware that sometimes connections between clients and servers fail.

Disconnects might be caused by software or hardware. If any link of a connection fails, messages get delayed, lost or corrupted and the connection need to be shut down.

If this happens frequently you can usually do something about it.

Contents

• Disconnect Reasons
  • Disconnects By Client
  • Disconnects By Server
• Timeout Disconnect
• Traffic Issues And Buffer Full
• First Aid
  • Check The Logs
  • Try Another Project
  • Try Another Server Or Region
  • Try Another Connection
  • Try Alternative Ports
  • Enable CRC Checks
• Fine Tuning
  • Check Traffic Stats
  • Tweak Resends
  • Check Resent Reliable Commands
  • Send Less
  • Try Lower MTU
• Tools
  • Wireshark
• Platform Specific Info
  • Unity
• Recover From Unexpected Disconnects

Disconnect Reasons

Client SDKs provide disconnection callbacks and a disconnect cause. Use those to investigate the unexpected disconnects you are having. Here we list the major disconnect causes and whether they are caused on the client or the server side.

Back To Top
 

Disconnects By Client

• Client-side timeout: no/too late ACKs from the server. See "Timeout Disconnect" for more details.
• Client socket exception (connection loss).
• Client connection fails on receive (buffer full, connection loss). See "Traffic Issues and Buffer Full".
• Client connection fails on send (buffer full, connection loss). See "Traffic Issues and Buffer Full".

Back To Top
 

Disconnects By Server

• Server-side timeout: no/too late ACKs from client. See "Timeout Disconnect" for more details.
• Server send buffer full (too many messages). See "Traffic Issues and Buffer Full".
• License or Subscription CCU limit hit.

Back To Top
 

Timeout Disconnect

The timeout disconnect is the most frequent issue, aside from problems to connect "at all".

There is no single point of failure when you run into frequent timeouts but there are a few common scenarios that cause issues and some ways to fix them.

Here is a quick checklist:

• Is the game running in background.
• A Unity app is not always running the main loop, when loading
• Check if you can reproduce the issue on other hardware and on another network. See "Try Another Connection".
• Check the amount of data you are sending. If there are spikes or if your messages/sec rate is very high, this can affect the connection quality. Read "Send Less"
• You can adjust the number and timing of resends. See "Tweak Resends".
• If you want to debug your game using breakpoints and all, read this.

Back To Top
 

Traffic Issues And Buffer Full

Photon servers and clients usually buffer some commands before they are actually put into a package and sent via the internet. This allows us to aggregate multiple commands into (fewer) packages.

If some side produces a lot of commands (e.g. by sending lots of big events), then the buffers might run out.

Filling buffers will also cause additional Lag: You will notice that events take longer to arrive on the other side. Operation responses are not as quick as usual.

Read "Send Less".

Back To Top
 

First Aid

Check The Logs

This is the first check you need to do.

All clients have some callback to provide log messages about internal state changes and issues. You should log these messages and access them in case of problems.

You can usually increase the logging to some degree, if nothing useful shows up. Check the API reference how to do this.

If you customized the server, check the logs there.

Back To Top
 

Try Another Project

All client SDKs for Photon include some demos. Use one of those on your target platform. If the demo fails too, an issue with the connection is more likely.

Back To Top
 

Try Another Server Or Region

Using the Photon Cloud, you can also use another region easily.

Hosting yourself? Prefer physical over virtual machines. Test minimum lag (round-trip time) with a client near the server (but not on the same machine or network). Think about adding servers close to your customers.

Back To Top
 

Try Another Connection

In some cases, specific hardware can make the connection fail. Try another WiFi, router, etc. Check if another device runs better.

Back To Top
 

Try Alternative Ports

As there are some cases where our default UDP port range (5055 to 5058) is blocked, we support an alternative range in all Photon Cloud deployments: Instead of using 5055 to 5058, the ports start at 27000. Switching to these "Alternative Ports" is easy.

The Name Server has port 27000 (was 5058), the Master Server 27001 (was 5055) and the Game Server becomes 27002 (was 5056).

Back To Top
 

Enable CRC Checks

Sometimes, packages get corrupted on the way between client and server. This is more likely when a router or network is especially busy. Some hardware or software is outright buggy corruption might happen anytime.

Photon has an optional CRC Check per package. As this takes some performance, we didn't activate this by default.

You enable CRC Checks in the client but the server will also send a CRC when you do.

Photon clients track how many packages get dropped due to enabled CRC checks.

Back To Top
 

Fine Tuning

Check Traffic Stats

On some client platforms, you can enable Traffic Statistics directly in Photon. Those track various vital performance indicators and can be logged easily.

In C#, the Traffic Stats are available in the LoadBalancingPeer class as TrafficStatsGameLevel property. This provides an overview of the most interesting values.

As example, use TrafficStatsGameLevel.LongestDeltaBetweenDispatching to check the longest time between to consecutive DispatchIncomginCommands calls. If this time is more than a few milliseconds, you might have some local lag. Check LongestDeltaBetweenSending to make sure your client is frequently sending.

The TrafficStatsIncoming and TrafficStatsOutgoing properties provide more statistics for in- and outgoing bytes, commands and packages.

Back To Top
 

Tweak Resends

More repeats don't guarantee a better connection though and definitely allow longer delays.

Back To Top
 

Check Resent Reliable Commands

You should begin to monitor ResentReliableCommands. This counter goes up for each resend of a reliable command (because the acknowledgement from the server didn't arrive in time).

If this value goes through the roof, the connection is unstable and UDP packets don't get through properly (in either direction).

Back To Top
 

Send Less

Back To Top
 

Try Lower MTU

With a setting on the client-side, you can force server and client to use an even smaller maximum package size than usual. Lowering the MTU means you need more packages to send some messages but if nothing else helped, it makes sense to try this.

The results of this are unverified and we would like to hear from you if this improved things.

Back To Top
 

Tools

Wireshark

This network protocol analyzer and logger is extremely useful to find out what is actually happening on the network layer of your game. With this tool, we can have a look at the facts (networking wise).

Wireshark can be a bit intimidating but there are only a few settings you have to do when we ask you to log our game's traffic.

Install and start. The first toolbar icon will open the list of (network) interfaces.

You can check the box next to the interface which has traffic. In doubt, log more than one interface. Next, click "Options".

We don't want all your network traffic, so you have to setup a filter per checked interface. In the next dialog ("Capture Options"), find the checked interface and double click it. This opens another dialog "Interface Settings". Here you can setup a Filter.

A filter to log anything Photon related looks like so:

(udp || tcp) && (port 5055 || port 5056 || port 5057 || port 5058 || port 843 || port 943 || port 4530 || port 4531 || port 4532 || port 4533 || port 9090 || port 9091 || port 9092 || port 9093 || port 19090 || port 19091 || port 19093 || port 27000 || port 27001 || port 27002)

When you press "Start", the logging will begin when you connect. After you reproduced an issue, stop the logging (third toolbar button) and save it.

In best case, you also include a description of what you did, if the error happens regularly, how often and when it happened in this case (there are timestamps in the log). Attach a client console log, too.

Mail the .pcap and other files to us and we take a look.

Back To Top
 

Platform Specific Info

Unity

PUN automatically keeps the connection for you. To do so, it calls DispatchIncomingCommands and SendOutgoingCommands on Unity's main loop.

However, Unity won't call Update while it's loading scenes and assets or while you drag a standalone-player's window.

To keep the connection while loading scenes, you should set PhotonNetwork.IsMessageQueueRunning = false.

Pausing the message queue has two effects:

• A background thread will be used to call SendOutgoingCommands while Update is not called. This keeps the connection alive, sending acknowledgements only but no events or operations (RPCs or sync updates). Incoming data is not executed by this thread.
• All incoming updates are queued. Neither RPCs are called, nor are observed objects updated. While you change the level, this avoids calling RPCs in the previous one.

If you use our Photon Unity SDK, you probably do the Service calls in some MonoBehaviour Update method.

To make sure Photon client's SendOutgoingCommands is called while you load scenes, implement a background thread. This thread should pause 100 or 200 ms between each call, so it does not take away all performance.

Back To Top
 

Recover From Unexpected Disconnects

To Document Top