Overview
- Stats to Review Monthly and After Significant Releases:
- Containment
- Intent Match
- MACS
- Actions to Take:
- Review transcripts with unmatched phrases, escalated conversations, or those with low MACS scores.
- Update training phrases monthly and after significant releases.
Important References for Tuning
Improving Training Phrases
Domain Training Steps:
- Regularly train your domain in Intent Manager.
- Always select "Train & Create Version" and then "Release" after making updates.
- Training phrases KB title and intentName should be the same.
- Do not create overlapping intents
- Use real customer utterances, complete thoughts, and diverse utterances, variety of sentence
Testing Training Phrases
- Test training phrases using the tester in Intent Builder.
- Ensure phrases have a high confidence score match to an Intent.
Downloading Intents
- Access “Domain Settings” and click “Export All Data” to download intents in your domain.
Adding a New Knowledge Base (KB)
- Steps to Add a New KB:
- Create a CSV file or Google sheet and fill out article data.
- Access KnowledgeAI and click "Add Knowledge Base."
- Fill out fields, upload your file, and click "Save."
- Steps to Add a KB Article:
- Access KnowledgeAI and select the correct KB.
- Click "Add article" and fill out the necessary fields.
Downloading KB Customer Questions & Bot Responses
- Access “Test & Tune,” choose the KB, select “Answered Only,” and click the download icon.
Important Links
Bot Analytics
- Reviewing Unmatched Phrases:
- Log into Conversational Cloud, navigate to Bot Analytics, select the bot, and download unmatched phrases.
- Handling False Positives:
- Update intents by adding or removing training phrases to reduce false positives.
- Stats: MACS Scores:
- Review and filter conversations to focus on specific issues or low MACS scores.
- Use the Conversation ID to pull up the entire transcript if the bot escalated.
Reviewing Transcripts to find areas to optimize bot flows
Manually review a set of conversation transcripts. Review at least 10 randomly selected transcripts in each category:
- Low CSAT/MACS
- Agent Escalations
- Containments (conversation "resolved" without escalation)
- Random sample (select conversations at random to see patterns that may not show in other categories and to see how "average" conversations are going)
As you read through transcripts, ask the following questions:
- For example, if the bot escalates when it is already tuned to handle the customer’s intent or if a welcome menu/message is showing later in the thread incorrectly.
- Was the bot able to resolve the customer’s issue or intent?
- Did the user trigger the fallback flow?
- If so, was their message something the bot should have recognized?
Reviewing Conversations that were escalated
- How did the conversation go when the agent took over?
- Was the agent able to continue the conversation smoothly, or did they repeat questions the bot had asked or not understand what the customer needed?
- The bot may ask for certain information (such as the customer’s address, name, etc.) that the agent needs to resolve their inquiry. If the bot asks for the information, the agent shouldn’t ask for it again.
- Was the agent able to resolve the customer’s issue?
- If not, was it because the brand’s processes didn’t allow the agent to resolve it, or because the agent didn’t follow the process?
You've now completed this stage. Ready to review the next stage of your Conversational Growth Path journey? Click here to proceed.