Creative ways to measure virtual agent success
Asking users to rate conversations is a time-tested method of obtaining feedback. At the same time, there is no reason to stop at that when more innovative ways to measure conversation relevancy without bothering users are available. Settings in Analytics offers non-intrusive ways to gather feedback. From Settings, you can start a feedback loop and find a way to constantly improve experience for users and your company. Settings offers four ways to gather feedback.
When multiple successful parameter in Settings are enabled, then the failure of any parameter causes the entire conversation to be unsuccessful. In the image, Zero Feedbacks and Zero Coverage are enabled. For a conversation to be successful, an end-user has to give feedback and find stories for all their utterances. If either condition fails, the conversation isn't successful.
Zero Feedback
PREREQUISITE
User Feedback should be turned on. Check out the last section in Story Board and User Feedback for configuration instructions.
Extremely useful when User Feedback has been turned on, any conversation with one or more feedback is considered successful.
| Zero Feedback | User Leaves a Feedback | Conversation |
| Turned On | Yes | Successful |
| Turned On | No | Unsuccessful |
| Turned Off | Yes | No impact |
| Turned Off | No | No impact |
Zero Coverage
PREREQUISITE
Zero Coverage works only when there is at least one story.
When it's turned on and at least once during a whole conversation, the user writers or enters an utterance for which a story is available, then the conversation is successful.
| Zero Coverage | Action | Conversation |
| Turned On | A story is found for at least one utterance. | Successful |
| Turned On | No story is found for all utterances | Unsuccessful |
| Turned Off | A story is found for at least utterance | No impact |
| Turned Off | No story is found for all utterances | No impact |
Case or Discussion Created through Virtual Agent
PREREQUISITE
Enable responses for at least one story. Else, the option remains disabled.
Along with Handover to Live Agent, this toggle is probably the simplest to grasp. When Case or Discussion Created through Virtual Agent is turned on, and an end-user starts a new thread on your community or raises a ticket, then the conversation is unsuccessful.
Handover to Live Agent
When Handover to Live Agent is turned on, and an end-user requests to be connected with a human agent, the conversation becomes unsuccessful. The key here is the end-user intention. The moment the end-user clicks "Connects Me to Agent", the conversation becomes unsuccessful. The actual connection might not happen because of connectivity or other issues. Nonetheless, that doesn't prevent the conversation from becoming unsuccessful.