This module demonstrates how to use the Photon Video SDK to share the user screen, captured through the uWindowCapture third party module.
- Module logic
- Screen sharing emission
- Stop emitting
- Screen sharing reception
- Install notes
- Supported topologies
The Photon Video SDK is a special version of the Photon Voice SDK, including support for video streaming.
More details are available in its dedicated README, available on the SDK download page Photon video SDK.
The capture of video streams can be made with recorders implementing
IVideoRecorderPusher, and the playback can be handled by players implementing
In this module, the screen content is captured by uWindowCapture.
uWindowCapture offers specific windows capture, or full desktop capture. In this sample, for simplification, only full desktop capture is supported.
uWindowCaptureRecorder class implementing the
IVideoRecorderPusher interface allows to collect frames for the PhotonVideo SDK.
To enable it,
U_WINDOW_CAPTURE_RECORDER_ENABLE has to be added to the define symbols.
Screen sharing emission
To be able to emit, the scene must contain:
ScreenSharingEmittercomponent: it starts and stops the capture and screen sharing emission.
UwcManagercomponent: it allows the screen to be capture by uWindowCapture
uWindowCaptureRecordercomponent: it collects the uWindowCapture texture and pass it to the PhotonVideo SDK
- some component calling
DisconnectScreenSharing()on the ScreenSharingEmitter component
FusionVoiceClientcomponent on the
NetworkRunnergameobject, that starts the voice session (see Voice - Fusion Integration)
To be able to send video, a Photon Voice connection has to been initialized. The module suppose this is already being done classicaly, and looks in the
Update() to know when the session is started.
ScreenSharingEmitter will first wait for the voice session initialization to be finished.
Then, it will also wait for the uWindowCapture initialization to be finished. This is done through the
When everything is ready, a transmission channel (a "voice") is created by calling
VoiceClient.CreateLocalVoiceVideo on the
FusionVoiceClient: from now on, the recorder will stream the desktop capture.
IEmitterListener is provided, it will be warn of the emission start with the
Note: in multiscreen situations, it is possible to chosse the desktop shared by calling
SelectDesktop with a screen id, starting at 0.
DisconnectScreenSharing() disconnect the video streaming voice.
IEmitterListener is provided, it will be warn of the emission end with the
Screen sharing reception
To be able to receive screen sharing, the scene must contain:
ScreenSharingReceivercomponent: it detects new video connections, then creates the video player and the associated texture.
ScreenSharingScreencomponent: it describes the receiving renderer, and ensure to pass the proper data required by the Oculus Quest shader.
Optionally, it is possible to set up a LOD screen handler, to enable mip mapping (usefull for low pixel density screens, like on VR headsets).
The receiver watch for new voice connections, with the
VoiceClient.OnRemoteVoiceInfoAction callback, whose code is a video codec.
Upon such a connection, it creates a video player with
Then, when this video player is ready (
OnVideoPlayerReady), it creates a material containing the video player texture, and pass it to the
EnablePlayback: the screen will then change its renderer material to this new one.
Oculus Quest shader
A specific settings requires a custom shader for the Oculus Quest:
- if the URP pipeline is used
- on Android (that is the case for the Oculus quest)
- if single-passed (or multi-view) rendering is used (which is common in VR)
For this configuration, the default Photon Video shader won't work.
This module provides a specific QuestVideoTextureExt3D shader to handle this, and the
ScreenReceiver use it when needed.
To work, this shader needs to receive during each update some additional info about the renderer: it is handled in the
The texture created on the fly by the video SDK cannot provide mip-mapping. On some platofrms with low density of pixels, like VR headsets, it might lead to shivering effects.
To handle that, the
ScreenSharingScreenLODHandler drives a camera, that capture at a slow rate a picture of the video texture, to project it on another renderer, which does support mip mapping.
When installing uWindowsCapture in a project, its asmdef has to be added to the Photon/PhotonVoice/PhotonVoiceApi/PhotonVoice.API.asmdef, so that
uWindowCaptureRecorder manages to access it.
The current version has been tested with:
- Fusion SDK 1.1.8
- Photon Video SDK 2.53
- uWindowCapture 1.1.0
A demo scene can be found in the
Assets\Photon\FusionAddons\ScreenSharing\Demo\Scenes\ folder :
Two clients must be used to test the scene :
- first client is the screensharing emitter : the
Emittergame object must be enabled while the
Receivergame object must be disabled,
- second client is the screensharing receiver : the
Emittergame object must be disabled while the
Receivergame object must be enabled,
StartSharingOnVoiceConnectionAvailable is set to true by default, the emitter starts to share the desktop when the scene is launched.
This addon latest version is included into the addon project
- shared mode
- Version 1.0.4: Change codec settings in the demo scene (VP8 instead of VP9)
- Version 1.0.3: Update for PhotonVoice Video SDK 2.53
- Version 1.0.2: Move QuestVideoTextureExt3D shader in Resources directory
- Version 1.0.1: ScreenShare renamed to screensharing & cleanup + add namespace
- Version 1.0.0: First release