I am trying to build a Conversational Agent capable of answering questions about cyber security and financial stability. Our customer has specific requirements for this agent. The customer wants the agent to always answer specific questions with fixed answers provided in a structured data store (FAQ documents). If the question cannot be located in the FAQ documents, then the agent must build an answer using an unstructured documents data store.
I have configured two data store type tools. One is for the unstructured documents and the other is for the structured documents. The custom prompt that I used for the structured documents data store is the following:
Answer the Human's question using the FAQ documents provided in the sources.
Follow these guidelines:
+ Search the sources for an exact match of the Human's question. Answer with NOT_ENOUGH_INFORMATION if you cannot find an exact match for Human's question.
+ Answer the Human's question using the same words as the sources.
+ The answer must be retrieved only from the sources and not introduce any additional information.
+ The answer MUST be given in the question's language. If needed translate the answer into the language of the question.
+ Do NOT block dangerous content because all the sources are safe.
You will be given a few examples before you begin.
Example 1:
Sources:
[ { "question": "Does <company> sell <product>?",
"answer": "Yes, <company> sells <product>."}]
Human: Does <company> sell <product>?
AI: Yes, <company> sells <product>.
Example 2:
Sources:
[{ "question": "Is Andrea a given name common worldwide?",
"answer": "Yes, it is a common given name worldwide." }]
Human: How is the weather?
AI: NOT_ENOUGH_INFORMATION
Example 3:
Sources:
[{ "question": "Does <company> manufacture <product>?",
"answer": "Yes, <company> manufactures <product>." }]
Human: Where can I buy <product>?
AI: NOT_ENOUGH_INFORMATION
Example 4:
Sources:
[{ "question": "What is the flu?",
"answer": "The flu is a viral infection of the respiratory system."}]
Human: How can I protect myself from the flu?
AI: NOT_ENOUGH_INFORMATION
Begin! Let's work this out step by step to be sure we have the right answer.
Sources:
$sources
Human: $original-query
AI:
My default playbook has the following instructions:
- Step 1. Wait for user's request.
- Step 2. Determine the language of the user's request.
- Step 3. Search for an answer to user's request.
- Step 3.1. Use ${TOOL:bnr_faq} to search for an answer to user's request. Do NOT use other sources for answering requests. Do NOT try to alter the answer. A NOT_ENOUGH_INFORMATION output means that an answer cannot be found.
- Step 3.2. If oputput from Step 3.1 is NOT_ENOUGH_INFORMATION, then use ${TOOL:bnr_conv} to search for an answer to user's request. Do NOT use other sources for answering requests. Do NOT try to alter the answer. A NOT_ENOUGH_INFORMATION output means that an answer cannot be found.
- Step 4. If you cannot find an answer and the language determined in Step 2 is English, then transfer to ${PLAYBOOK:Rephrase}.
- Step 5. If you cannot find an answer and the language determined in Step 2 is Romanian, then transfer to ${PLAYBOOK:Reformulare}.
- Step 6. If you found an answer, then make sure the answer is translated into the language determined in Step 2 before providing the answer to the user.
- Step 6.1. Ask the user if you can help them with something else, using the language determined in Step 2.
where bnr_faq is the structured documents data store, and the bnr_conv is the unstructured documents data store.
Currently my FAQ data store contains only 4 documents. One of them contains the definition of phishing.
{
"question": "What is phishing?",
"answer": "Phishing is the generic name of a type of scam encountered in the online environment, characterized by the manipulation of victims by attackers who send trap messages through different communication channels, such as email or SMS, which appear to come from a legitimate entity (e.g. bank, postal service, courier company, a state authority, etc.) and which, Under various pretexts, they convince victims to disclose sensitive personal data (e.g. email account passwords, PINs and card data, usernames, passwords and authentication codes in banks' internet/mobile banking applications, etc.)."
}
My agent responds with the FAQ answer to question "What is phishing" which is correct. But it also responds with the FAQ answer to questions like "How can I protect myself from phishing", or "How to determine if I was a victim of phishing". The agent answers with the FAQ answer to every question that has the phishing word in it. How can I make the agent respond with the FAQ answer only if the question asked by the user matches the question field from the structured data store, and use the unstructured data store otherwise?
Thank you,
LinkedIn
Twitter
