Settings: Measure the Success of a Virtual Agent (Chatbot)
Asking users to rate conversations is a time-tested method of obtaining feedback. At the same time, there is no reason to stop at that when more innovative ways are available to measure conversation relevancy without bothering users. Settings in Analytics offers non-intrusive ways to gather feedback. From Settings, you can start a feedback loop and find a way to constantly improve experience for users and your company. Settings offers four ways to gather feedback.
When multiple successful parameter in Settings are enabled, then the failure of any parameter causes the entire conversation to be unsuccessful. In the above image, Zero Feedback and Zero Coverage are enabled. For a conversation to be successful, an end-user has to give feedback and find stories for all their utterances. If either condition fails, the conversation isn't successful.
Zero Feedback
PREREQUISITE
User Feedback should be turned on. Check out the last section in Story Board and User Feedback for configuration instructions.
Extremely useful. When User Feedback is turned on, any conversation with one or more feedback is considered successful.
| Zero Feedback | User Leaves a Feedback | Conversation |
| Turned On | Yes | Successful |
| Turned On | No | Unsuccessful |
| Turned Off | Yes | No impact |
| Turned Off | No | No impact |
Zero Coverage
PREREQUISITE
Zero Coverage works only when there is at least one story.
When it's turned on and at least once during a whole conversation, the user writes or enters an utterance for which a story is available, then the conversation is successful.
| Zero Coverage | Action | Conversation |
| Turned On | A story is found for at least one utterance. | Successful |
| Turned On | No story is found for all utterances | Unsuccessful |
| Turned Off | A story is found for at least utterance | No impact |
| Turned Off | No story is found for all utterances | No impact |
Case or Discussion Created through Virtual Agent
PREREQUISITE
Enable responses for at least one story. Else, the option remains disabled.
Along with Handover to Live Agent, this toggle is probably the simplest to grasp. When Case or Discussion Created through Virtual Agent is turned on, and an end-user starts a new thread on your community or raises a ticket, then the conversation is unsuccessful.
Handover to Live Agent
When Handover to Live Agent is turned on, and an end-user requests to be connected with a support agent, the conversation becomes unsuccessful. The moment the end-user clicks "Connects Me to Agent", the conversation becomes unsuccessful even if the customer is unable to connect with a live agent due to connectivity or any other reason.