AppVoiceExperience component in your scene, configured with a Wit configuration. See Integrating Voice SDK.AppVoiceExperience requests it when the app does not already hold it.AudioClip or raw sample frames.AppVoiceExperience component:Activate() opens the microphone and waits for the volume to cross the Sound Wake Threshold before it starts a request.ActivateImmediately() opens the microphone and starts the request at once, without waiting for the volume threshold.Deactivate() stops listening and sends the audio collected so far.DeactivateAndAbortRequest() stops listening and cancels the request without waiting for a response.ActivateImmediately() with a button press and Deactivate() with the release to get push-to-talk. Pair Activate() with the press to let the player start speaking a moment later.AppVoiceExperience component:appVoiceExperience.enabled = false;
Deactivate() only ends the current activation, not the service.| Property | True when |
|---|---|
Active | The voice experience is listening or has a request in flight. |
IsRequestActive | A request is open with Wit.ai. The microphone can already be closed. |
MicActive | This voice experience is reading from the capture stream. |
Microphone.Start gives one script the device at a time. Voice SDK’s default Mic component does not take and release the device around each activation: its Activate On Enable setting defaults to on, so it calls Microphone.Start when the component is enabled and Microphone.End when it is disabled, and it holds the device for as long as it stays enabled. When another script, such as a chat stack, lip sync, or recorder built on the Unity Microphone API, already holds that device, the default Mic component cannot also acquire it.AudioBuffer component, the shared buffer your AppVoiceExperience reads from. AudioBuffer takes samples from one input source: a component that implements IAudioInputSource. The default Mic component is one. To share your capture, give AudioBuffer an input source that reads from it:MicBase when your capture produces a Unity AudioClip.IAudioInputSource yourself when it does not, for example when a chat SDK hands you raw PCM frames in a callback.AppVoiceExperience with Activate(), ActivateImmediately(), and Deactivate(). AudioBuffer, not your code, calls StartRecording(int) and StopRecording() on your input source, the same way it drives the default Mic. The int is the sample duration in milliseconds.AppVoiceExperience accesses Wit.ai directly from your app, which is the default. When platform integration is on (UsePlatformIntegrations) and the platform supports it, Voice SDK goes through platform services instead and does not read from AudioBuffer, so the steps below do not apply.MicBase is in the Meta.WitAi.Lib namespace, IAudioInputSource is in Meta.WitAi.Interfaces, and AudioBuffer and AudioEncoding are in Meta.WitAi.Data.MicBase is an abstract IAudioInputSource built for a capture that produces an AudioClip. Your component owns the device and supplies Voice SDK with samples from it, instead of Voice SDK opening its own.MicBase overrides four abstract members:public abstract string GetMicName();
public abstract int GetMicSampleRate();
public abstract AudioClip GetMicClip();
public abstract int MicPosition { get; }
MicBase declares for your component to override, not code to paste into a subclass. Return the name, sample rate, clip, and current read position of the capture your app already runs.MicBase keeps every nth sample, where n is your sample rate divided by its 16 kHz encoding rate using integer division. A rate below 16,000 Hz makes n zero, so the sample loop throws. A rate that is not a whole multiple goes wrong quietly: at 44,100 Hz, n is 2, and each 10 ms chunk keeps every second sample of its first 320 input samples and drops the other 120. Voice SDK then receives samples spaced at 22,050 Hz, treated as 16 kHz, with gaps. MicBase reads a multi-channel clip’s interleaved samples as if they were one mono stream, so a multi-channel clip reaches Voice SDK garbled.MicBase runs the sample loop itself. Calling StartRecording(int) starts a coroutine that reads the AudioClip from GetMicClip() at the position from MicPosition, computes a peak level for each buffer, and raises OnSampleReady. Your component does not raise OnSampleReady; it supplies the clip and the position. Voice SDK derives the level it compares against the Sound Wake Threshold from those samples, so a clip that stays silent never crosses it.IsMicListening is true. The default IsMicListening reports whether a Unity Microphone device named by GetMicName() is recording. Override it when your capture does not come from a Unity Microphone device.MicBase:StartRecording(int) and StopRecording() start and stop the sample loop that feeds Voice SDK. Neither call touches the device.Deactivate(), so chat audio survives the end of a voice command.MicBase’s job nor Voice SDK’s: whatever code opened the capture, your own or a chat SDK you are wrapping, closes it.Mic component works the opposite way: with Activate On Enable left on, enabling it opens the device, and disabling it closes it.AudioClip, implement IAudioInputSource directly and forward the frames yourself. These are the interface’s members, not code to paste:event Action OnStartRecording;
event Action OnStartRecordingFailed;
event Action<int, float[], float> OnSampleReady;
event Action OnStopRecording;
void StartRecording(int sampleLen);
void StopRecording();
bool IsRecording { get; }
AudioEncoding AudioEncoding { get; }
bool IsMuted { get; }
event Action OnMicMuted;
event Action OnMicUnmuted;
AudioBuffer relies on this contract:StartRecording(int), make IsRecording return true, raise OnStartRecording, and then begin forwarding frames. AudioBuffer stays in its activating state until it receives OnStartRecording. Keep IsRecordingfalse until StartRecording is called: if it already returns true, AudioBuffer skips StartRecording and treats the input as on. If you cannot start, raise OnStartRecordingFailed.OnSampleReady only between OnStartRecording and OnStopRecording. Pass float samples from -1 to 1. AudioBuffer ignores the int and float arguments and computes the level itself, so any values work there. It reads the whole array, so size the array to exactly your valid samples.AudioEncoding (samplerate, numChannels), and leave encoding at its default, signed. Send mono when you can. AudioBuffer converts your rate to its own 16 kHz encoding, so this path has no whole-multiple-of-16 kHz rule, but the conversion picks samples by index without filtering or mixing. For multi-channel input it keeps the first channel at a whole multiple of 16 kHz, and at other rates the kept samples can come from different channels.OnSampleReady on Unity’s main thread, because AudioBuffer starts a coroutine when samples arrive. If your chat SDK delivers frames on an audio thread, queue them and raise OnSampleReady from Update. Discard frames that arrive while IsRecording is false.StopRecording(), stop forwarding, make IsRecording return false, and raise OnStopRecording. Do not close the device. AudioBuffer treats OnStopRecording as the end of capture for every component recording from it, so raise it only when forwarding stops.IsMuted returns true, AudioBuffer holds back the sample-ready callbacks that pass audio to Voice SDK. If you support muting, raise OnMicMuted and OnMicUnmuted when IsMuted changes. If you never mute, return false.AudioBuffer uses the component in its Mic Input field. Assign your input source there yourself instead of relying on the fallback search:AudioBuffer component to your scene and set its Mic Input field to your component.AudioBuffer is enabled. Voice SDK can create and enable one as soon as an AppVoiceExperience is enabled, and Unity does not guarantee that your script’s Awake runs first, so keep the AppVoiceExperience GameObject inactive until these lines have run.AudioBuffer.instantiateMic = false; AudioBuffer.Instance.MicInput = yourInputSource;
MonoBehaviour. If your scene has no AudioBuffer, Voice SDK creates one the first time it needs it.AudioBuffer is enabled, it searches for an IAudioInputSource: first on its own GameObject and that object’s children, then under the root objects of its own scene. It does not search other scenes, so a component in an additively loaded scene is missed. When the search finds nothing and the static AudioBuffer.instantiateMic is true, which is the default, AudioBuffer adds its own Mic component, which starts opening the device. Assigning a different source later destroys that Mic, and destroying it calls Microphone.End on its device if that device is recording, which can stop capture that other code started on that device through Unity’s Microphone API. Setting instantiateMic to false first keeps it from being added at all.AudioBuffer keeps its input source recording while any component is registered as a recorder, so your code can keep reading after Deactivate(). This works with any input source, including the default Mic.MonoBehaviour, register as a recorder and listen for samples, then unregister when you are done:// Start reading AudioBuffer.Instance.StartRecording(this); AudioBuffer.Instance.Events.OnSampleReceived.AddListener(OnSamples); // Stop reading AudioBuffer.Instance.Events.OnSampleReceived.RemoveListener(OnSamples); AudioBuffer.Instance.StopRecording(this);
OnSamples is your own method that takes (float[] samples, int chunkIndex, float level).AudioBuffer tracks recorders as a set. Deactivate() unregisters only the voice experience. The input source keeps recording until the last recorder calls AudioBuffer.Instance.StopRecording(this). If the input source stops on its own or fails to start, AudioBuffer drops every recorder, yours included: AudioBuffer.Instance.IsRecording(this) returns false, and AudioBuffer.Instance.Events.OnAudioStateChange reports Off. Call StartRecording(this) again to resume.OnSampleReceived hands you the array the input source raised, before any conversion to 16 kHz, at the rate and channel layout in its AudioEncoding. The argument order differs from your input source’s OnSampleReady. The second argument is a running chunk count, not a sample count, so use samples.Length. The third is the peak level from 0 to 1. Copy the samples if you keep them: an input source can reuse the array for its next chunk, and MicBase does.Activate() waits on is a volume gate. It fires on any sound loud enough to cross it, including a cough or a slammed door. It does not recognize a keyword.ActivateImmediately() from Unity’s main thread when it matches. Your detector can read the same capture that feeds Voice SDK, either inside your own input source or through OnSampleReceived as described in Keep reading microphone audio after a deactivation, so no extra microphone is needed.Deactivate() yourself.Activate(string). A custom transcription provider replaces the speech recognition only; it does not stop audio from reaching Wit.ai. Send Audio To Wit defaults to on, and audio keeps going to Wit.ai until that setting is turned off. See Providing voice transcriptions.Activate() does nothing. An activation is already in progress. Activate() returns without starting a second request while one is active. It also returns without activating when the voice experience has no Wit configuration or cannot activate audio right now, and logs the reason to the console.OnError reports “No input source was available.”AudioBuffer has no input source. Assign one as described in Connect your input source to AudioBuffer.Mic instead of using your input source.AudioBuffer did not find yours, for example because it is in an additively loaded scene. Assign it explicitly.AudioBuffer.Instance.AudioState stays Activating. Your IAudioInputSource did not raise OnStartRecording from StartRecording. Raise it before your first OnSampleReady.MicBase: StartRecording never starts, and OnStartRecordingFailed fires. Your GetMicClip() override returned null. Return a valid AudioClip.MicBase: OnSampleReady never fires. Your capture is not a Unity Microphone device, and IsMicListening still uses the default implementation, which checks for a Microphone device by name. Override IsMicListening to report your own capture’s state.Microphone.Start caller. An input source shares the samples your capture produces; it does not open a second device.MicBase reads from a Unity AudioClip. For a capture path that never produces an AudioClip, implement IAudioInputSource directly, as described in Feed Voice SDK from raw sample frames.