Develop
Develop
Select your platform

Live Understanding

The Wit.ai speech endpoint can return a partial Automatic Speech Recognition (ASR) response as it analyzes audio data. Instead of a single ASR response being sent from the speech endpoint after the entire audio transmission was processed, Live Understanding enables you to get an initial response and potentially act upon that response more quickly.
Voice SDK returns OnWitPartialResponse methods with initial ASR response data in a manner similar to how transcriptions are provided via OnPartialTranscription and OnFullTranscription.
To do this, Voice SDK provides an AppVoiceExperience method called AcceptPartialResponseAndCancelRequest. This can be called in an OnPartialResponse event listener and it allows you to mark whether the initial response data should be used as the final response data. If it is marked this way, the VoiceService then stops listening and immediately uses that result as the final response data.
This process has also been integrated with intent/entity matchers for greater effectiveness. Once the confidence score with the matcher reaches the preset threshold, the partial response will be automatically marked as the final response data. This option helps support real-time VR voice use cases that require much lower latency. It doesn’t affect the message endpoint used for text analysis since that endpoint only performs analysis once.
The simplest way to integrate Live Understanding into your scene is to use matchers, which help match intents and entities to the user utterance, and then accept partial responses once they’ve passed the matcher criteria.
A more advanced method, but one that may give you more control over the results, is to add an OnWitPartialResponse event listener, and then call the AcceptPartialResponseAndCancelRequest method if the user response matches the expectation.
To Integrate Understanding into your app
  1. In the Unreal editor, open the AppVoiceExperience Blueprint to edit it.
  2. Click + Add Component and add a VoiceIntentWithEntityMatcher.
  3. On the Details panel, set the Intent and Entity to match the expectation you have for the match. For example, if your app changes the color of items on command, you might set Intent to set_color, and Entity to color:color.
  4. Under Partial Response, select the Is Also Used for Partial Response option.
  5. Select the Auto Accept Partial Response Once Past Matcher Criteria option.
  6. If needed, change the Intent Confidence Threshold and Entity Confidence Threshold to match your needs. The maximum value is 1, minimum value is 0.0.
Did you find this page helpful?
Thumbs up icon
Thumbs down icon
Nav Logo
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
English (US)
© 2025 Meta