Timeframes and Time
Time in Multiplayer Games
Time in networking can often be confusing and very difficult to pin down. There is rarely one "true" time, but rather relative times and resulting time paradoxes which the game maker has to determine how to resolve or hide.
The main aspect of online multiplayer gaming which all complexities stem from is network latency. Peers can only know about the past of other Peers.
There are two primary Timeframes that exist because of this latency:
- Remote Timeframe
- Local Timeframe
Remote Timeframe
The Remote Timeframe describes the Ticks (and their associated Object States) which have arrived on a Client from the Server. This Timeframe is the most recent confirmed past.
This Timeframe only exists on Clients, and is not a specific "Time" in itself, but rather a more general description of the recent verified server States.
NOTE: There is no Remote Timeframe on the Server - the only Server Timeframe is Local.
Local Timeframe
The Local Timeframe describes the State of Objects as they exist locally, after all simulation has completed (including re-simulation and prediction if applicable). On clients this is also referred to as the Predicted State. On the Server/Host this is the only Timeframe that exists.
Clients must exist AHEAD of the Server, in order for them to supply the Server with their Inputs (for Server Modes) and States (for Shared Modes) BEFORE the server simulates the associated Tick. As such, the Local Timeframe on Clients is always ahead of the Remote Timeframe.
The State of Network Objects as they exist after forward Simulation. This Timeframe only exists if the Object is part of Simulation (Object.IsInSimulation == true).
Hiding the Remote/Local Discrepancy
Clients have two main mechanisms for hiding the difference between these Timeframes, and the paradoxes produced:
- Prediction: All Objects are extrapolated into the Local Timeframe (Prediction). This way all Network Objects are always approximately in the state they will be in for that Tick on the Server. Inevitable errors are constantly corrected through reconciliation and re-simulation. More about prediction can be found here.
- Lag Compensation: Fusion has a built in lag compensation system which allows for cross Timeframe queries (raycasts, etc). Lag Compensation lets a Player interact with Objects which are in the Remote Timeframe, and then relies on the Server to account for the Client rendering Objects in the Remote Timeframe when reproducing these queries. This results in the Server performing queries against the past when simulating the Players inputs, to account for how the Player saw the world at the time. More about Fusion's Lag Compensation system can be found here.
Example Time Problem: Tennis
A simple example of the time paradox we are solving for is a two player Tennis game.
You as Player 1 hits the ball at your opponent. You see the ball reach your opponent, but you will not know for 100 milliseconds if your opponent:
- Swings and hits the ball back at you, or
- Swings and misses, or
- Did not even swing
What do you do show Player 1?
You cannot accurately predict your opponent here, since their possible actions all would produce extremely different outcomes, but we won't know about their response to the ball until it is too late.
For this example, we are assuming Client-Server Mode and NOT Shared Mode, as this problem is even more difficult to solve in Shared Mode.
Solution 1: No Prediction
This handling is the easiest, but also provides the worst user experience. In this case ALL objects are presented in the Remote Timeframe - including the Input Authority (the Local Player object).
| Object | Predicted | Render |
|---|---|---|
| Local Player | No | Remote Timeframe |
| Ball | No | Remote Timeframe |
| Opponent | No | Remote Timeframe |
The Player sends inputs to server, but only moves locally once the server state changes arrives. This creates 100% accurate and consistent results across all clients, but creates unacceptable input latency for all players other than the Host.
NOTE: There are variants of this that can be made to work where you can mask the input latency with game mechanics. For example, you could make swinging require a windup where the player must start the swing 150ms before the actual stroke occurs. The player's swing input would immediately be sent to the Server, and by the time the swing executes on the local player the confirmed hit from the server will have arrived. It should be noted though that this is a not trivial solution, as RTT is variable and some custom prediction and correction would likely need to be employed.
Solution 2: Full Prediction
The opposite of the previous solution, here we attempt to predict everything. More about prediction can be found here.
The benefit of prediction is that the player gets immediate results from inputs, and all objects are still in the same timeframe and therefor are time aligned. However, we have a very critical part of this equation which CANNOT be predicted accurately - the opponents inputs.
| Object | Predicted | Render |
|---|---|---|
| Local Player | Yes | Local Timeframe |
| Ball | Yes | Local Timeframe |
| Opponent | Yes | Local Timeframe |
Most aspects Local Player can accurately be predicted (we know the exact inputs), however the opponent actions still cannot be predicted accurately, and as a result the ball cannot be predicted accurately once it reaches the other player. The result using Full Prediction to solve this problem will be pretty extreme corrections to the ball and opponent's positions, which is not ideal.
Solution 3: Lag Compensation
This leads to the first mixed time approach for dealing with this paradox. In this case the opponent and the ball are kept in the Remote Timeframe to avoid prediction error issues, but the Local Player remains in the Local Timeframe. This mixing of times however means that the Local Player is seeing a world for the current tick that does not match the Server or other Clients world. Lag Compensations allows players to interact with Object Hitboxes in the Remote Timeframe using queries (such as raycasts) as if they were in the Local Timeframe. The Server is able to account for this when simulating each Tick, by rewinding Hitbox states to how the Player saw them when the query was made.
More about Fusion's Lag Compensation system can be found here.
| Object | Predicted | Render |
|---|---|---|
| Local Player | Yes | Local Timeframe |
| Ball | No | Remote Timeframe |
| Opponent | No | Remote Timeframe |
The problem this creates is that the Local Player is now interacting with a Ball that is NOT where it will be on the Server when the Local Player attempts to hit it. So on the Local Player the prediction will result in the player hitting the ball. On the Server however when the Tick arrives where that Player swung, the Server will see the ball as having long flown past the player. The Server will already have simulated the miss and already has sent Ticks after that miss to all clients with the Ball well past the player.
Lag Compensation would allow for the Server to rewind the Ball Hitbox to where the player saw it at the time they swung, and drastically correct the ball back to where it would now be had the player actually hit it in the past. But that would create a massive and jarring correction to the ball. So Lag Compensation won't help here.
Solution 4: Timeframe Shifting
This leads to the second mixed time approach for dealing with this paradox. In this case the opponent is kept in the Remote Timeframe to avoid prediction error issues, and the Local Player remains in the Local Timeframe. The Ball will be predicted into the Local Timeframe, HOWEVER it will be Rendered in a blend of the Local and Remote Timeframes. When the Ball is near the Local Player it will Render in the Local Timeframe, showing the predicted position. When the Ball is near the Opponent it will Render in the Remote Timeframe. As it moves between the two it will lerp from one Timeframe to the Next.
| Object | Predicted | Render |
|---|---|---|
| Local Player | Yes | Local |
| Ball | Yes | Variable based on distance |
| Opponent | No | Remote |
Because the Ball is Predicted, it will be in the correct position for the Local Player's Tick when it comes time to swing. The Ball's predictions will be wrong it is near the Opponent (predictions will have the opponent in outdated location and failing to swing), but the error will be corrected during re-simulations when the Opponent's State arrives. However, because the Ball will not be rendering this prediction (it is rendering the Ball in the Remote Timeframe) all of the missed predictions will not be seen by the player.
The resulting artifact in this case is that the ball will appear to move toward the Opponent at a slower rate of speed than the actual speed of the ball, and the ball will come at the player at an increased apparent speed - due to the Render time-shift.
Simulation vs Render
Another important time concept to understand the difference between Simulation and Rendering.
- Simulation: At a regular interval (
Runner.DeltaTimewhich is1f / TickRate) Fusion callsFixedUpdateNetwork()on all Network Behaviours that are participating in Simulation. The State of all Network Behaviours after simulation is then captured and this is referred to as a Snapshot. These snapshots can be thought of as keyframes. - Rendering: The screen refresh of Unity will very rarely line up with the interval of Simulation. Presenting the latest result of Simulation to the user every Unity Update would create a very hitchy result, especially in the case of movement. Interpolation solves this problem by always tweening between two previous simulated frames.
Local and Remote Timeframes
Multiplayer game development is largely about solving for issues caused by latency and loss. Because of latency, there is no true singular time. All time becomes relative - and these relative times are what we call Timeframes. Specifically Local and Remote Timeframes:
- Remote Timeframe: This Timeframe only exists on Clients, and represents received Server States. This is not a specific Time in itself, but rather a more general description of recent verified server States. NOTE: There is no Remote Timeframe on the Server - the only Timeframe is Local.
- Local Timeframe: The State of Network Objects as they exist after forward Simulation. This Timeframe only exists if the Object is part of Simulation (
Object.IsInSimulation == true).
It is up to the game developer to choose which timeframes to present to players in order to produce the best experience.
The primary time paradox in networking results from the requirement to give players a sense of immediacy to their inputs and actions. When a player presses a button or moves a joystick, the expectation is an immediately response.
Without prediction in a basic Client-Server architecture, a player would press a key, and that keypress would be sent to the server. The server would respond to that keypress by changing the state of the game simulation. It would then send out an updated games tate to all players. This would result in a perfect agreement between all players of the state of things.
However, this would induce massive amounts of input lag which is not acceptable for most games. To counter this most networking architectures employ either Prediction or Lag Compensation to hide this latency.
However, in networking there is always only one Peer that is source of truth (the State Authority) for the state of any Network Entity. If that authoritative Peer IS NOT the the local player's Peer, then a prediction must be made.
Server Mode and Timeframes
In Server and Host Modes, Clients can be thought to exist in a predicted possible future (its Local Timeframe), and Clients only know the true state of Objects once the server snapshots arrive for past Ticks (the Remote Timeframe).
Player Inputs are recorded and sent to the server every Tick, and these inputs are used by the Server in Simulation of that Tick.
In Client-Server Modes (Server/Host/Single), the Server is the State Authority for all Network Objects. As such, it is the only peer with a Local Timeframe that is not Predicted and is the only Peer that exists in the true present. All changes that the Server makes to the State (Networked Properties) of Network Objects in FixedUpdateNetwork() is authoritative and final.
Clients however need to supply their Player Input to the Server for each Tick and have it arrive at the Server BEFORE the Server simulates each Tick. The Server cannot revise history, so if Player Input for a Tick does not reach the Server in time for that Tick Simulation, the input is lost and the Server will have to either extrapolate a previous input or use default input values for that player.
For this reason Clients MUST exist in the future - in order for each Player to collect inputs for each Tick and send them to the Server well in advance of the Server simulation of that tick (regardless if that Client is predicting Network Objects or not). How far ahead the Client must be is a function of the average Round Trip Time (RTT). The greater the RTT/Ping, the greater the client prediction window required. This prediction window the primary cause of time paradoxes in Client-Server game networking, as Clients never know the true state of things for their current Local Timeframe.
There are two strategies for hiding/resolving this paradox of mixed Timeframes.
- Extrapolate all Objects into the local present (Prediction). This way all Network Objects are always approximately in the state they will be in for that Tick on the Server.
- Use Lag Compensation to allow cross timeframe interactions. Lag Compensation lets the player interact with Objects that are in the Remote Timeframe, and relies on the Server to account for the Client seeing older Network Object states when it runs its simulation.
Shared Mode and Timeframes
In Shared Mode the player can be thought to exist in the present (Local Timeframe), and all remote objects exist in the past (Remote Timeframe).
Shared Mode departs from how Client-Server Modes operate, in that Clients are assigned State Authority rather than just Input Authority. This distinction however is mostly semantic, as Shared Mode essentially treats State Authority as a variant of Input Authority.
Rather than the Server collecting inputs each Tick from Players and using them to Simulate, the Shared Mode Server collects Network Object States from Players. This is because the Shared Mode Server is incapable of simulation itself - and relies on Clients to simulate and make state changes.
This difference however, does not change the Remote and Local Timeframe paradigm. Players with State Authority over a Network Object apply changes directly to the Local Object immediately. This is just not called "Prediction" because the Server will accept the results from the client.
Remote Objects (Network Objects the Client does not have State Authority over), exist in the relative past and their states come from other Peers.
Remote Timeframe
Remote refers to the time associated with snapshots that are arriving from the server. On Clients these Tick snapshots will be
Local Timeframe
In the case of a Player Object in Server Mode, the Server is actually the State Authority, but the player is the Input Authority. When the Player presses a movement
- Remote From Time
- Remote To Time
- Remote Render Timeframe (a lerp between Remote From and Remote To)
- Local From Time
- Local To Time
- Local Render Timeframe
- Local Update Time
Last updated on
Back to top