Photon Voice For PUN

Getting Started

Currently Photon Voice is available only as a Unity package downloadable directly from the Asset Store here.
It is an add-on that brings voice chat and audio streaming to PUN.
Photon Voice lets you focus on audio transmission features and can handle the connection workflow for you.
While Photon Voice depends on PUN and extends it, it requires a separate AppId of its own that you can get here.

Photon Voice Unity package, also called PUNVoice, includes PUN (minus PUN demos files and Photon Chat API).
If you want to start using both packages just download and import Photon Voice alone.
If you already use PUN and want to add Voice, it is preferable that you clean up PUN and import Photon Voice only.
This way we guarantee compatibility since PUN and PUNVoice packages are updated separately.

Configure

Once you import Photon Voice to the project, a "Photon Voice Settings" section should appear to PhotonServerSettings just below PUN settings.
There you can enter the AppId of your Photon Voice application.
Remember that you can always reopen the PhotonServerSettings in the Unity Inspector by following these steps: "Window" -> "Photon Unity Networking" -> "Highlight Server Settings".

To tweak advanced audio and voice related settings, make sure a PhotonVoiceSettings script is attached to one object in the scene.
The available settings are described as follows:

1. PUN related settings:
   • AutoConnect: if true, automatically connects Photon Voice client and joins a "voice room" when PUN client is joined to a "PUN room". Default is true.
   • AutoDisconnect: if true, automatically disconnects Photon Voice client when PUN client is disconnected. Default is false.
   • AutoTransmit: if true, starts transmitting audio as soon as Photon Voice client is joined to a "voice room" and a PhotonVoiceRecorder is instantiated. Default is true.
   • WorkInOfflineMode: if true, AutoConnect and AutoDisconnect will work even during PUN's offline mode. Default is false.
2. Microphone and "local" audio recording settings that will be applied to every "recorder" instance created:
   • SamplingRate: a frequency of how many times audio is measured per second. Generally, this defines the audio quality you want.
     Possible values are: 8, 12, 16, 24 and 48 kHz. Default is 24 kHz.
   • FrameDuration: outgoing audio stream encoder delay in milliseconds (buffer size in terms of time). Possible values are 5, 10, 20, 40, 80 and 120 ms. Default is 20 ms.
   • Bitrate: the amount of data (the number of bits) that are processed over a certain amount of time (second). Generally, this defines the compression quality. Default value is 30000 b/s.
   • Encrypt: if true, all outgoing voice streams will be encrypted. Default is false. Read more about "Encryption" here.
   • MicrophoneType: sets the default or global audio input device type to be used by any PhotonVoiceRecorder if the latter has its PhotonVoiceRecorder.MicrophoneType set to PhotonVoiceRecorder.MicAudioSourceType.Settings.
     Default is PhotonVoiceSettings.MicAudioSourceType.Unity.
     Read more about this in the "Microphone Selection" section.
3. "Remote" audio streaming settings that will be applied to every "speaker" instance created:
   • PlayDelayMs: playback delay in milliseconds. Used to compensate incoming packets latency variations.
4. Voice detection feature (will be applied to every "recorder" instance created):
   • VoiceDetection: toggles voice detection feature. Default is false.
   • VoiceDetectionThreshold: minimal signal level to be exceeded to start transmission if voice detection is enabled. 0.01 is the default and recommended value.
5. Miscelleanous:
   • DebugInfo: toggles Photon Voice debug logs in Unity console at info level. Default is false.
   • DebugLostPercent: lost frames simulation ratio.

The "Audio Source" Prefab

Photon Voice for PUN expects each "audio source" object to be represented by a prefab.
The minimal required prefab should contain two script components attached to the same GameObject:

• PhotonView
• PhotonVoiceRecorder

After adding the PhotonVoiceRecorder script, you will notice that an AudioSource component has been added and also a PhotonVoiceSpeaker script.
Both are essential to reproducing received remote audio from other clients joined to the same room.

The prefab should be instantiated only at runtime and after joining a room using PUN.
Scene object instances are not supported. So you should not have any PhotonVoiceRecorder instances in the scene.

The audio source prefab is used for two different purposes:

1. Record and transmit "local" sound:
   This is handled by PhotonVoiceRecorder.
   Check the IsTransmitting property of the same component to know if the recorded audio is being sent or not.
2. Reproduce and play received "remote" sound:
   This is managed by PhotonVoiceSpeaker.
   Use its IsPlaying property to see if a remote received audio is being played.

This audio source prefab can be used in different ways:

1. Voice Chat:
   If no audio clip is assigned to PhotonVoiceRecorder and if a microphone device is detected, audio will be recorded and transmitted to all other joined players.
   In case multiple recording devices are available, a default one will be used or you can set it explicitly. See "MicrophoneType".
2. Streaming Audio Clips:
   This option was introduced to test Photon Voice transmission.
   However, you may use it for other purposes.
   In order to broadcast an audio clip to other players joined to the same room, you need to assign an audio media file supported by Unity to PhotonVoiceRecorder.AudioClip.
   This can be done by a drag and drop to the Inspector from Unity Editor.
   If you want the audio clip to be replayed every time make sure PhotonVoiceRecorder.Loop is set to true.

Microphone Selection

If the PhotonVoiceRecorder.Source is set to PhotonVoiceRecorder.AudioSource.Microphone there is another option that determines the type of the microphone API to be used.
Each PhotonVoiceRecorder can have its own type of microphone input device that can be set using PhotonVoiceRecorder.MicrophoneType.
The three different values are:

Unity

By default, we make use of Unity's Microphone API on all platforms.
Unity exposes all available audio input devices using Microphone.devices array of strings.
The mircophone device can be identified using its string name or its index in that array.
By default the first element of that array will be used.
If you want to set another device explicitly, do it using PhotonVoiceRecorder.MicrophoneDevice or globally using PhotonVoiceNetwork.MicrophoneDevice.
If PhotonVoiceRecorder.MicrophoneDevice is null the global one (PhotonVoiceNetwork.MicrophoneDevice) will be used.

To use Unity's Microphone API simply set the PhotonVoiceRecorder.MicrophoneType to Unity.
Or you can do this for all instances of PhotonVoiceRecorder from PhotonVoiceSettings.MicrophoneType.

Photon

Unity is great for making it easier to target multiple platforms using almost same code base.
However, in some ways, Unity's Microphone API is limited or present some issues here and there.
That is why we introduced our own way of directly interacting with audio input devices and bypassing Unity's Microphone API.
One of the major reasons for this is to make use of devices' hardware built-in echo cancellation.
We have implemented native plugins for four major platforms: Android, iOS, Windows and macOS.

However, enumerating the list of available microphones is available only for macOS and Windows.
On Android and iOS we use the default audio input device used by the system.
You can check the list of available "Photon microphones" using PhotonVoiceNetwork.PhotonMicrophoneEnumerator as follows:

C#

var enumerator = PhotonVoiceNetwork.PhotonMicrophoneEnumerator;
if (enumerator.IsSupported) 
{
    for (int i = 0; i < enumerator.Count; i++) 
    {
        Debug.LogFormat("PhotonMicrophone Index={0} ID={1} Name={2}", i, enumerator.IDAtIndex(i), 
            enumerator.NameAtIndex(i));
    }
}

You can set a microphone using PhotonVoiceRecorder.PhotonMicrophoneDeviceID or globally using PhotonVoiceNetwork.PhotonMicrophoneDeviceID.
If PhotonVoiceRecorder.PhotonMicrophoneDeviceID is -1, global PhotonVoiceNetwork.PhotonMicrophoneDeviceID is used
To make sure a Photon microphone ID is valid you can use PhotonVoiceNetwork.PhotonMicrophoneEnumerator.IDIsValid(id).

To use Photon's Microphone API simply set the PhotonVoiceRecorder.MicrophoneType to Photon.
Or you can do this for all instances of PhotonVoiceRecorder from PhotonVoiceSettings.MicrophoneType.

Settings

If you set the PhotonVoiceRecorder.MicrophoneType to PhotonVoiceRecorder.MicAudioSourceType.Settings then the recorder will use the global microphone type set in PhotonVoiceSettings.MicrophoneType.

Voice Detection

Voice detection is an optional feature that will filter recorded sound and transmits only when a predefined threshold of signal level is exceeded.
This means that voice transmission is automatically paused when you stop speaking and it is resumed when you start talking.
This will also help you avoid sending useless noise and thus reduce traffic.

The default value of VoiceDetectionThreshold is 0,01.
It is the recommended value for common environments as a result of experiments with voice detection calibration and noise level measurements.

Voice Calibration

If you still experience issues when recording your voice even with voice detection is on, you may need voice calibration.
Photon Voice offers an auto calibration mode which is limited in time.
To start calibration mode you should call PhotonVoiceRecorder.VoiceDetectorCalibrate(period).
The default calibration period is 2000 milliseconds but you can always change it.
Using calibration, Photon Voice automatically adjusts silence and loudness thresholds.

Audio Groups

Photon Voice is not for broadcast only.
You can offer your players the possibility to have multiple conversations going on at the same time without interfering with each other.
This can be done using "Audio Groups" which makes use of Photon's "Interest Groups" feature.
An "Audio Group" is identified by a number (byte).
You can subscribe and unsubscribe from multiple groups.
However, you can only transmit audio to one group at a time.

By default, all voice clients are subscribed to the group "0": each player can hear everyone else and talk to everyone else.
If you want to replace this default behaviour, change the "Global Audio Group" using the property with the same name PhotonVoiceNetwork.Client.GlobalAudioGroup.
Only players registered to that group will hear the audio you transmit.

On the other hand, you can select more than one group you want to listen to.
To update the list of groups you're registered to, you can use the following method:

C#

PhotonVoiceNetwork.Client.ChangeAudioGroups(byte[] groupsToRemove, byte[] groupsToAdd);

You should take into consideration the following notes when changing audio groups:

• It can be done only when joined to a "voice room".
• If you don't want to add or remove any group pass a null in the respective parameter.
• If you want to add or remove all existing groups pass an empty array (byte[0]) in the respective parameter.
• You cannot unsubscribe from default audio group (0).

Audio groups can be combined with Push-to-Talk to obtain nice features for your game.

Example Use Case 1: Private Conversations

You can add private voice chat in few steps:

• If n is the number of players create n * (n - 1) audio group numbers and bind each one to a couple of players.
  Example: Player 1 and Player 2 can subscribe to group "12" or "21".
• A player should subscribe to all audio groups shared with another player at once or subscribe on demand for a "phone call like" feature.
• A player can talk to exactly one player by setting the GlobalAudioGroup to the corresponding shared "binary audio group" number.

Example Use Case 2: Team Chat

If you are implementing teams in your game, you can add "team chat" easily as follows:

• Assign a group number per team.
• A player should subscribe to the audio group of the team he/she belongs to.
• Players of the same team can talk to each other by setting the GlobalAudioGroup to the team's audio group number.

Push-to-Talk

It is easy to have a push-to-talk feature with Photon Voice.
You need to manually toggle voice recording and transmission.
It's like turning microphone mute on and off.
To start push-to-talk just set PhotonVoiceRecorder.Transmit to true.
To stop it just set the value back to false.
You can achieve this by binding the property to a user input (UI button or keyboard shortcut).