Settings: Measure the Success of a Virtual Agent (Chatbot)

Asking users to rate conversations is a time-tested method of obtaining feedback. At the same time, there is no reason to stop at that when more innovative ways are available to measure conversation relevancy without bothering users. Settings in Analytics offers non-intrusive ways to gather feedback. From Settings, you can start a feedback loop and find a way to constantly improve experience for users and your company. Settings offers four ways to gather feedback.

When multiple successful parameter in Settings are enabled, then the failure of any parameter causes the entire conversation to be unsuccessful. In the above image, Zero Feedback and Zero Coverage are enabled. For a conversation to be successful, an end-user has to give feedback and find stories for all their utterances. If either condition fails, the conversation isn't successful.

Zero Feedback


User Feedback should be turned on. Check out the last section in Story Board and User Feedback for configuration instructions.

Extremely useful. When User Feedback is turned on, any conversation with one or more feedback is considered successful.

Zero Feedback User Leaves a Feedback Conversation
Turned On Yes Successful
Turned On No Unsuccessful
Turned Off Yes No impact
Turned Off No No impact

Zero Coverage


Zero Coverage works only when there is at least one story.

When it's turned on and at least once during a whole conversation, the user writes or enters an utterance for which a story is available, then the conversation is successful.

Zero Coverage Action Conversation
Turned On A story is found for at least one utterance. Successful
Turned On No story is found for all utterances Unsuccessful
Turned Off A story is found for at least utterance No impact
Turned Off No story is found for all utterances No impact

Case or Discussion Created through Virtual Agent


Enable responses for at least one story. Else, the option remains disabled.

Along with Handover to Live Agent, this toggle is probably the simplest to grasp. When Case or Discussion Created through Virtual Agent is turned on, and an end-user starts a new thread on your community or raises a ticket, then the conversation is unsuccessful.

Handover to Live Agent

When Handover to Live Agent is turned on, and an end-user requests to be connected with a support agent, the conversation becomes unsuccessful. The moment the end-user clicks "Connects Me to Agent", the conversation becomes unsuccessful even if the customer is unable to connect with a live agent due to connectivity or any other reason.

Last updatedWednesday, April 10, 2024

Or, send us your review at