AI-500 — frequently asked questions

A prompt-adherence evaluation, because it measures whether the subagent follows its prompt instructions, including the required evidence-gathering tool steps before it answers.

Add a schema-match evaluation that validates each handoff payload against the expected JSON structure, flagging any change that breaks deserialization by the next agent.

No. Those score factual support and goal attainment, so scope drift goes unnoticed; use a responsibility-boundary evaluation that tests in-scope responses and out-of-scope refusals.

Groundedness scores whether the final response is supported by retrieved context; it cannot show that an embedding or analyzer change altered which chunks were retrieved or their rank order.

Both the support agent and the policy agent query the same Foundry IQ knowledge base, so the embedding and analyzer changes degrade retrieval for both, requiring the document retrieval evaluator in each slot.

The scenario says the dashboard includes token usage and TTFT, but the only alert rules configured are for latency anomalies and tool invocation failures, so no alerting coverage exists for those signals.

The single conversation trace plus emitted trace and span IDs let you correlate Agent1 and Agent2 spans, MCP tool invocations, and external REST calls back to one user request for end-to-end tracing.

A message node only outputs text to the conversation and never pauses execution, so the workflow would proceed without any physician decision being captured or enforced.

Hardcoding ER rules in the prompt violates the safety requirement to keep logic out of core prompts; the Ask a question node enforces approval dynamically at the workflow level through human-in-the-loop review.

File grounding requires re-uploading content per agent, breaks the single maintained layer, and does not return structured query answers or citation metadata with each response.

Vector/hybrid search plus the semantic ranker and synonym maps match meaning rather than exact tokens, so conversational follow-ups and unfamiliar terminology still retrieve relevant passages.

OBO passes the signed-in employee's Entra token to the ticket tool, so the tool's existing per-employee authorization still applies to every query.

OBO exchanges tokens through Entra and app-only access uses a managed identity, so no key is stored, and each identity receives only the rights its context needs.

Azure Functions is serverless but typically executes code snippets or small functions rather than full container images with complex state management. Azure Container Apps is better suited for running packaged container images with predefined handoffs.

While technically possible to deploy custom code to App Service, the question specifies 'prompt-defined agents' and 'fully managed environment,' which are best served by Microsoft Foundry Agent Service to leverage built-in orchestration and security features.

Throttling the 30-request batch to reduce concurrency would require pausing/dequeueing fewer items, which directly reduces throughput below the 120 req/min target unless you have infinite parallel resources, which is restricted by the '30 max simultaneous' rule.

Group Chat implies a conversational loop where agents interact with each other iteratively, which adds latency. ConcurrentBuilder is designed for parallel execution of independent tasks that produce a combined output, fitting this scenario better.

Role instructions set abstract agent domains and task boundaries, which helps routing in principle, but they do not demonstrate how to decompose a specific compound request or enforce an exact response schema, so the inconsistent final format can remain.

Per-request cues must be injected by application code on every call, which violates the requirement to minimize code changes and does not provide one durable fix that applies to all future conversations automatically.

Include compound input/output pairs such as a request combining a data-analysis calculation with a literature summary, showing which agent is selected at each step and a schema-compliant final response with consistent section labels and prefixes.

Choose A2A when the requirement involves a peer agent hosted elsewhere, discovering its capabilities, or tracking long-running tasks. Choose MCP when the requirement is calling tools, prompts, or reading resources from a service.

In-process composition requires the agent to run within your solution's process and framework, which contradicts a partner agent that exposes its own endpoint and is operated by another organization.

Yes. A2A defines authentication and authorization between agent hosts and structured message exchange over a protocol, giving auditable interactions while keeping each side's implementation private.

An evaluation in Microsoft Foundry targets a concrete saved version of the agent, so unsaved playground instruction edits have no stable artifact to test. Saving as a new version freezes the updated instructions and gives the synthetic-dataset evaluation a versioned target.

Saving creates an unpublished version visible only to the development team that already has access to Agent1, while publishing exposes it through its own endpoint and identity to other consumers. That extra exposure is why publishing violates a dev-only requirement.

Preview is a manual chat check of the updated instructions, not an automated full evaluation and not a substitute for running a synthetic dataset. Use it for a quick sanity check, then run the real evaluation against the saved version.

A sliding window reduces prompt size by discarding old turns, but it truncates rather than compacts. The earliest symptoms a patient reports can be deleted outright, breaking the requirement to preserve a compacted version of critical triage details.

It lets each new turn reference a stored prior response, so the API appends only the new user input instead of resending the full transcript. This keeps per-turn token growth minimal while retaining server-side conversation state.

Precision alone ignores how many relevant documents were missed (recall) and how far down the list the first relevant hit appears (MRR). A configuration that tops one column but trails on the others is inconsistent, so it is less reliable across real queries.

Precision@10 is the share of the top 10 results that are relevant, Recall@10 is the share of all relevant documents found within those 10, and MRR scores how high the first relevant result ranks. Reliable systems need all three to be strong.

Yes — a shared query set and identical human relevance labels make the metrics directly comparable across configurations, so differences in the table reflect the search configuration itself rather than evaluation noise.

Content Safety primarily filters content at input/output boundaries. It does not natively provide the granular control needed to review and approve intermediate steps, tool calls, or internal prompts within a multi-agent orchestration loop.

Traces capture the detailed sequence of actions, including prompts sent to models, tool invocations, and subagent interactions. They allow reviewers to inspect the full context of an interaction, ensuring that every step complies with governance policies.

Annotate-only still depends on live production traffic arriving after rollout, which lacks pre-labeled ground truth and a clean control point. It cannot reliably count false positives, false negatives, or prove coverage of every workflow path.

The control point runs the workflow without the guardrail's blocking action; the intervention point runs the same labeled cases with the guardrail active. Comparing annotations and logs between the two runs measures how accurately the guardrail intervenes.

A custom guardrail assigned across intake, retrieval, and resolution agents can behave differently at each hop — for example, allowing injected instructions found only in retrieved documents. Testing one entry point hides those gaps.

A tag filter depends on manually tagging chargeback data, which conflicts with the requirement to minimize manual tagging. It also cannot be assumed to split usage on a shared Foundry resource across three projects.

No. Meter groups cost by model or usage meter such as tokens or deployment type, not by the business-unit project that generated the usage.

Saved views let each project or business unit reopen the same Foundry-scoped Cost Analysis with the Resource grouping already applied, avoiding repeated manual filtering while keeping data reconcilable in Cost Management.