Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions

Microsoft AI-103 Exam - Topic 5 Question 1 Discussion

You have a Microsoft Foundry project that uses Azure Al Search to ground an agent in internal documentation.After a recent content update, users report that the agent's answers have become less accurate.You need to identify whether the retrieved content is negatively influencing the model's generated responses.Which observability signal should you review?
B) groundedness evaluation metrics
A) prediction drift metrics
C) latency breakdown traces
D) indexer status and failure history

Microsoft AI-103 Exam - Topic 5 Question 1 Discussion

Actual exam question for Microsoft's AI-103 exam
Question #: 1
Topic #: 5
[All AI-103 Questions]

You have a Microsoft Foundry project that uses Azure Al Search to ground an agent in internal documentation.

After a recent content update, users report that the agent's answers have become less accurate.

You need to identify whether the retrieved content is negatively influencing the model's generated responses.

Which observability signal should you review?

Show Suggested Answer Hide Answer
Suggested Answer: B

The correct observability signal is B. groundedness evaluation metrics. In a RAG solution, the key diagnostic question is whether the generated answer is supported by the retrieved context. Microsoft Foundry's built-in evaluator reference defines Groundedness as the metric that measures how grounded the response is in the retrieved context, with scoring that indicates whether the model's claims are supported by the provided source material.

This matches the issue after a content update. If retrieved chunks are stale, misleading, incomplete, or poorly aligned with the user query, groundedness results can show that generated responses are not reliably supported by the retrieved documentation. The RAG evaluator guidance explains that groundedness focuses on whether the response avoids content outside the grounding context, while other process metrics such as retrieval evaluate how relevant the retrieved chunks are. Latency traces are useful for performance troubleshooting, not response accuracy. Indexer status can reveal ingestion failures, but it does not show whether retrieved content is influencing generated answers negatively. Prediction drift is a model monitoring concept and is not the primary signal for RAG grounding quality. Reference topics: Microsoft Foundry observability, RAG evaluators, groundedness, retrieved context, and response quality evaluation.


Contribute your Thoughts:

0/2000 characters
Portia
3 days ago
Wait, isn't prediction drift metrics also important?
upvoted 0 times
...
Elke
8 days ago
Totally agree, that makes the most sense here.
upvoted 0 times
...
Monroe
13 days ago
I think we should check the groundedness evaluation metrics.
upvoted 0 times
...
Louisa
18 days ago
Latency breakdown traces seem less relevant here; I think we should focus on how the content is affecting the responses, which might point us back to groundedness metrics.
upvoted 0 times
...
Roy
23 days ago
I practiced a similar question where we had to assess the impact of content changes, and I feel like groundedness evaluation metrics were the right choice there too.
upvoted 0 times
...
Thaddeus
29 days ago
I'm not entirely sure, but I remember something about prediction drift metrics being important for understanding changes in model accuracy after updates.
upvoted 0 times
...
Asha
1 month ago
I think we might need to look at the groundedness evaluation metrics since they directly relate to how well the agent is using the internal documentation.
upvoted 0 times
...

Save Cancel