How to get notified if your AI bot could not answer a question.
Creation date: 18/08/2023 09:16
Updated: 18/06/2025 08:40
Once you have created your knowledge store and deployed an AI powered bot using ThinkAutomation, there will sometimes be cases where the bot is unable to answer a question. This could be due to missing knowledge, or specific information missing or not written correctly in the knowledge store.
You can setup your Automation so that you can get notified whenever the bot is unable to help. You can then make any updates to the knowledge store.
This is done by telling the AI to respond in a certain way if it is unable to help. In the Automation we then check for that response and send an email.
For example, below is a basic AI powered bot automation:
ChatMessage = HTML Entity Decode ( %Msg_Body% )ChatMessage = Mask Profanities ( %ChatMessage% )ChatMessage = Trim ( %ChatMessage% )
If %ChatMessage% Equal To [webchatstart] Then // Respond to chat starting Return Welcome %Msg_FromName%! I can answer questions about Widgets.End If
// Add default contextAsk AI Add Context "You are an enthusiastic representative working at Widgets Company. Given the provided sections from our documentation, answer the user's question using only that information, outputted in markdown format. If you are unsure and the answer is not explicitly written in the documentation, say 'Sorry, I don't know how to help with that.'" (Required) To Conversation %Msg_ConversationId%
// Add context from the knowledge storeAsk AI Add Context From Knowledge Store Widgets Search %ChatMessage% Return 4 Most Relevant To Conversation %Msg_ConversationId%
// Send to AI to get the responseAIResponse =AIResponse = Ask AI Say "Question: """ %ChatMessage% """ Answer: " Using gpt-3.5-turbo-16k Conversation %Msg_ConversationId%
// Check if response is 'Sorry, I don't know how to help with that.'If %AIResponse% Starts With "Sorry, I don't know how to help with that." Then // Let support team know Send Email To support@widgets.com "Bot could not help: %Msg_From%"End If
// Return response to the chatReturn %AIResponse%In the default context include "If you are unsure and the answer is not explicitly written in the documentation, say 'Sorry, I don't know how to help with that.'" - this tells the AI to respond with specific text if it is unable to answer.
After the Ask AI action we check if the response is equal to "Sorry, I don't know how to help with that." - if it is we send an email, this contains the %ChatMessage% variable so we can see what the user asked.
The "Sorry, I don't know" message can be anything - provided we explicitly tell the AI to respond with some text if it is unable to help - and then check for that text after asking the AI for a response.