This document is about: VOICE 2

SWITCH TO

VOICE 1 VOICE 2

Recorder Component

The Recorder is the component that will transmit the audio stream. Using this component you choose what to stream (Input Audio Source), how to stream it (Audio Quality), when to start and stop transmission (Transmission Toggles) and to whom we transmit (Receiver Target). Also the Recorder requires initialization.

Input Audio Source

The Recorder supports various types of input audio feed to stream. The most obvious one is the microphone which is used for voice chat. However, the Recorder supports two other types of input sources.

Microphone

To use the microphone as an input source type for the voice stream to be transmitted by the Recorder, simply set the Recorder.SourceType to Recorder.InputSourceType.Microphone.

Microphone Types

Photon offers two options to choose from for which microphone API to use. Each Recorder can have its own type of microphone input device that can be set using Recorder.MicrophoneType. The two different values are:

Unity:

By default, we make use of Unity's Microphone API on all platforms. Unity exposes all available audio input devices using Microphone.devices array of strings. The microphone device can be identified using its string name or its index in that array. By default, the first element of that array will be used. If you want to set another device explicitly, do it using Recorder.UnityMicrophoneDevice or use the new unified generic way via Recorder.MicrophoneDevice gathered via Recorder.MicrophonesEnumerator.

To use Unity's Microphone API simply set the Recorder.MicrophoneType to Recorder.MicType.Unity.

Photon:

Unity is great for making it easier to target multiple platforms using almost the same code base. However, in some ways, Unity's Microphone API is limited or present some issues here and there. That is why we introduced our own way of directly interacting with audio input devices and bypassing Unity's Microphone API. One of the major reasons for this is to make use of devices' hardware built-in echo cancellation. We have implemented native plugins for four major platforms: Android, iOS, Windows, UWP and macOS.

However, enumerating the list of available microphones is available only for macOS, Windows and UWP. On Android and iOS we use the default audio input device used by the system.

You can set a microphone using Recorder.PhotonMicrophoneDeviceId (or Recorder.PhotonMicrophoneDeviceIdString for UWP) or use the new unified generic way via Recorder.MicrophoneDevice gathered via Recorder.MicrophonesEnumerator.

To use Photon's Microphone API simply set the Recorder.MicrophoneType to Recorder.MicType.Photon.

Microphone Selection

We now have a unified generic way to enumerate available microphones no matter the device type.

var enumerator = recorder.MicrophonesEnumerator;
if (enumerator.IsSupported) 
{
  Debug.LogFormat("Enumerating available microphone devices of type {0}", recorder.MicrophoneType);
  foreach(var device in enumerator.Devices)
  {
    Debug.LogFormat("Microphone device={0}", device);
  }
}

Then you can set which device to use via Recorder.MicrophoneDevice.

Considerations

By default, the Recorder uses the microphone set by the system (independently of microphone type (Unity|Photon)). If you change the device via UnityMicrophoneDeviceName or PhotonMicrophoneDeviceId then it will try to use that one if it's available and valid. If the microphone device chosen is not valid or not available then the Recorder switches to default one if any.

If you switch device or microphone type while recording is ongoing (Recorder.IsRecording is true) make sure to call Recorder.RestartRecording() for this to take effect. Read more on "Restart".

If a microphone device change is detected the Recorder will check if the microphone device is still available and valid or not. If the microphone device chosen is not valid or not available then the Recorder switches switch to default one if any. Read more on "React On System Changes".

While the choice of the microphone API (Unity or Photon) persists between Unity Editor and the output build, the choice of the actual device does not. The list of the microphone devices available in the Unity Editor may not be the same on the machine where the build will be running. The build can be running on a different platform with a different system and with different audio input devices. That is why, if needed, you can switch from a default microphone to another one, if available, explicitly via code at runtime from the build client.

Microphone Access: Recording And Muting

By default, the Recorder component will start recording automatically once initialized. You can disable this by setting Recorder.AutoStart to false.

If you are looking for options to mute the microphone 'locally' ('self-muting'), you can use either Recorder.TransmitEnabled or Recorder.IsRecording.

Recorder.IsRecording can be set at runtime (play mode) once the recorder is initialized.

So if you want to stop recording and "release" the microphone you can set Recorder.IsRecording to false. Set it back to true to resume recording.

By design, when recording is disabled the audio stream is completely gone.

Microphone Type Fallback

If true, when recording fails to start with Unity microphone type, Photon microphone type is used -if available- as a fallback and vice versa.

Streaming Audio Clips

This option was introduced to test Photon Voice transmission. However, you may use it for other purposes. In order to broadcast an audio clip to other players joined to the same room, you need to assign an audio media file supported by Unity to Recorder.AudioClip. This can be done by a drag and drop to the Inspector from Unity Editor. If you want the audio clip to be replayed every time make sure Recorder.LoopAudioClip is set to true or tick "Loop" from Unity Editor for the Recorder's inspector.

Custom Factory

Photon Recorder allows you to have your own custom audio source for whatever use case you have in mind. To know more, read "How to use custom audio source?".

Audio Quality

Codec parameters are the main factor here:

SamplingRate: a frequency of how many times the audio is measured per second.Generally, this defines the audio quality you want. Possible values are: 8, 12, 16, 24 and 48 kHz. Default is 24 kHz.
FrameDuration: outgoing audio stream encoder delay in milliseconds (buffer size in terms of time).Possible values are 5, 10, 20, 40, 80 and 120 ms. Default is 20 ms.
Bitrate: the amount of data (the number of bits) that are processed over a certain amount of time (second).Generally, this defines the compression quality. Default value is 30000 b/s.

Other Stream Settings

Encrypt: if true, all outgoing audio streams will be encrypted. Default is false.Read more about "Encryption" here.
ReliableMode: if true, all outgoing audio streams will be in reliable mode. Default is false.

Transmission Toggles

You can toggle transmission using Recorder.TransmitEnabled. There is also VoiceDetection that can be used to make sure we don't transmit useless background noise.

Push-to-Talk

It is easy to have a push-to-talk feature with Photon Voice. You need to manually toggle voice recording and transmission. It's like turning microphone mute on and off. To start push-to-talk just set Recorder.TransmitEnabled to true. To stop it just set the value back to false. You can achieve this by binding the property to a user input (UI button or keyboard shortcut).

Voice Detection

Voice detection is an optional feature that will filter recorded sound and transmits only when a predefined threshold of signal level is exceeded. This means that voice transmission is automatically paused when you stop speaking and it is resumed when you start talking. This will also help you avoid sending useless noise and thus reduce traffic.

The default value of VoiceDetectionThreshold is 0,01. It is the recommended value for common environments as a result of experiments with voice detection calibration and noise level measurements.

Voice Calibration

If you still experience issues when recording your voice even with voice detection is on, you may need voice calibration. Photon Voice offers an auto calibration mode which is limited in time. To start the calibration mode you should call Recorder.VoiceDetectorCalibrate(period). The default calibration period is 2000 milliseconds but you can always change it. Using calibration, Photon Voice automatically adjusts silence and loudness thresholds.

Receiver Target

This is done via Audio Groups. Optionally you could receive your own streams if DebugEchoMode is enabled.

Interest Groups

Photon Voice is not for broadcast only. You can offer your players the possibility to have multiple conversations going on at the same time without interfering with each other. This can be done using "Interest Groups". An "Interest Group" is identified by a number (byte). You can subscribe and unsubscribe from multiple groups. However, you can only transmit audio to one group at a time.

By default, the recorder's interest group is set to 0 : each player can talk to everyone else. To set the target interest group, set Recorder.InterestGroup.

You can select more than one group you want to listen to. By default, all voice clients are subscribed to group 0 only. To update the list of groups you're registered to, you can use the following method:

VoiceConnection.Client.OpChangeGroups(byte[] groupsToRemove, byte[] groupsToAdd);

You should take into consideration the following notes when changing audio groups:

It can be done only when joined to a "voice room".
If you don't want to add or remove any group pass a null in the respective parameter.
If you want to add or remove all existing groups pass an empty array (byte[0]) in the respective parameter.
You cannot unsubscribe from the default audio group (0).

Audio groups can be combined with Push-to-Talk to obtain nice features for your game.

Debug Echo Mode

This mode, when enabled, allows audio streams transmitted by local Recorder to be received by the same voice client. This is useful for debugging purposes, some use cases:

testing audio quality, delays or setup with one actor only joined to a room, especially during the early development phase
testing one's microphone or speakers
other scenarios that involve the need for own echo or true broadcast

Debug Echo Mode works only with Interest Group 0.

Primary Recorder

Since in most cases, a single Recorder component is used, we have provided a shortcut for a single default Recorder called "Primary Recorder" which can be accessed from VoiceConnection.PrimaryRecorder (or PhotonVoiceNetwork.PrimaryRecorder).You can set it in the Unity Editor or via code.

Initialization

In order to work the Recorder needs to be initialized. The PrimaryRecorder is automatically initialized. Also in PUN integration, Recorders are automatically initialized using PhotonVoiceView. In other use cases, or if the automatic initialization fails somehow, you can manually initialize a Recorder using one of two methods:

VoiceConnection.InitRecorder(Recorder) (or PhotonVoiceNetwork.InitRecorder(Recorder))
Recorder.Init(VoiceConnection)

By default, the Recorder will automatically start recording when initialized. To disable this behaviour and manually start recording using Recorder.StartRecording() after initialization, set Recorder.AutoStart to false. You can stop recording using Recorder.StopRecording().

Restart

You need to set the Recorder settings you want before recording is started, otherwise you will need to restart using Recorder.RestartRecording().

To know if the Recorder needs to be restarted, you can check the 'IsDirty' like flag Recorder.RequiresRestart.

Here is the list of public fields or properties that may require restart when changed in order for them to take effect:

Recorder.UserData
Recorder.AudioSourceFactory
Recorder.UnityMicrophoneDevice
Recorder.MicrophoneDevice
Recorder.PhotonMicrophoneDeviceId
Recorder.PhotonMicrophoneDeviceIdString
Recorder.Source
Recorder.MicrophoneType
Recorder.AudioClip
Recorder.SamplingRate
Recorder.FrameDuration
Recorder.Bitrate
Recorder.TrySamplingRateMatch
Recorder.UseOnAudioFilterRead
Recorder.InputFactory

Here is the list of public fields or properties that won't require restart if changed while recording already started:

Recorder.TransmitEnabled
Recorder.Encrypt
Recorder.ReliableMode
Recorder.DebugEchoMode
Recorder.VoiceDetection
Recorder.VoiceDetectionThreshold
Recorder.VoiceDetectionDelayMs
Recorder.InterestGroup
Recorder.LoopAudioClip
Recorder.UseMicrophoneTypeFallback
Recorder.StopRecordingWhenPaused
Recorder.SkipDeviceChangeChecks
Recorder.RecordOnlyWhenEnabled
Recorder.RecordOnlyWhenJoined
Recorder.AutoStart
Recorder.ReactOnSystemChanges

React On System Changes

This option is ignored on iOS and Android when the Recorder's input source is microphone and the microphone type is Photon because the native plugins handle this.

Sometimes if you add or remove input or output audio device(s) from the system settings or if you switch the default one, Photon Voice can stop working properly. Since Photon Voice 2.5 we have introduced a new property Recorder.ReactOnSystemChanges that, once enabled, will make the Recorder automatically reinitializes itself to recover from such system changes. The default value is false for backward compatibility. You can set this value in edit mode or in play mode.

This relies on Unity's OnAudioConfigurationChanged callback on Photon Voice 2's supported platforms except iOS. On iOS tests have shown that OnAudioConfigurationChanged does not work so we have a separate native implementation for iOS.

By default, the Recorder will restart recording only when the device being used is no longer available or valid, in some cases you may need to force restarts even if the device in use did not change. To enable this set Recorder.SkipDeviceChangeChecks to true.

There is an option to stop recording when the application is paused (e.g. moves to the background or loses focus) and start again when the application is unpaused. To enable this behaviour set Recorder.StopRecordingWhenPaused to true.

Record Only When Enabled

If you would like to use Unity components' enabled / disabled concept for the Recorder component, you could make sure that the Recorder works only when enabled and active in the hierarchy by setting Recorder.RecordOnlyWhenEnabled to true. So when you enable or disable the Recorder from the Inspector in the Unity Editor or by code, the recording could start or stop.

Record Only When Joined

This was introduced to delay recording auto start mainly until after microphone permission is granted or PUN integration is setup properly from recorder's side (set recorder.UserData before starting recording). It's also useful to avoid having microphone access (especially nowadays where most operation systems will notify you about which application is using the microphone) while transmission is not possible yet. The default is true and it makes sense to keep it as is.