A company has a Microsoft Copilot Studio agent that provides answers based on a knowledge base for customer support.
Users report that, occasionally, the agent provides inaccurate answers.
You need to use metrics from the Analytics tab in Copilot Studio to identify the cause of the inaccuracies.
Which two options should you use? Each correct answer presents part of the solution.
NOTE: Each correct selection is worth one point.
Comprehensive and Detailed Explanation From Agentic AI Business Solutions Topics:
The correct answers are B. session information and session outcomes and E. quality of generated answers.
This scenario is focused on a knowledge base-driven Copilot Studio agent where users report that the agent sometimes gives inaccurate answers. The question asks which Analytics tab metrics should be used to identify the cause of those inaccuracies.
That means you need metrics that help you examine:
how the answer was generated
what happened in the conversation when the bad answer occurred
Why E. quality of generated answers is correct
This is the most direct metric for this scenario.
Because the agent is answering from a knowledge base, the problem is tied to the quality of the generated response itself. The quality of generated answers metric helps assess whether the generated responses are relevant, useful, and accurate enough for the user's request.
From an AI business solutions perspective, this metric is essential because it helps diagnose problems such as:
weak grounding from the knowledge source
irrelevant retrieval
poor answer formulation
hallucination-like behavior
mismatch between user question and available source content
If the issue is inaccurate answers, the first place to investigate is the quality signal tied to generated answers.
Why B. session information and session outcomes is correct
To find the cause of inaccuracies, you also need to inspect the broader conversational context. Session information and session outcomes help you see:
what the user asked
how the agent responded
whether the conversation was resolved
whether the user abandoned, escalated, or retried
where the conversation broke down
This is important because an inaccurate answer may not come only from poor generation quality. It may also come from:
the way the user phrased the request
lack of sufficient grounding context
repeated failed attempts in a session
escalation after an unhelpful answer
patterns in unsuccessful conversations
In other words, quality of generated answers tells you about answer quality, while session information and outcomes help you understand the operational context in which those inaccuracies appear.
Together, these two give the strongest diagnostic view.
Why the other options are incorrect
A . survey results
Survey results can tell you whether users were happy or unhappy, but they do not directly help identify the cause of inaccurate knowledge-based responses. They are more of a feedback signal than a root-cause metric.
C . topic usage and topics with low resolution
This is more relevant for agents built around explicit topics and topic flows. The scenario specifically describes an agent that provides answers based on a knowledge base, so generated-answer analytics are more appropriate than topic-resolution analysis.
D . engagement, resolution, and escalation rates
These are useful high-level operational KPIs, but they are not the best metrics for diagnosing why answers are inaccurate. They show outcome trends, not the direct cause of answer-quality issues.
Currently there are no comments in this discussion, be the first to comment!