Reduce Token Usage in Microsoft Foundry Agents

Analyze, monitor, and tune AI-powered business solutions

Answer Correct answer: A — Chunk documents during indexing to improve retrieval precision and reduce input tokens without sacrificing response quality.

A company has a Microsoft Foundry agent that summarizes customer feedback and recommends products to customers. The agent references data from multiple knowledge sources. Users report that the agent response time is slow. Telemetry data shows that the agent frequently reaches its token usage limit. You need to recommend a solution to reduce token usage without degrading the quality of the generated responses. What should you recommend?

Chunk documents during indexing. Correct Answer
Reduce the number of knowledge sources used by the agent.
Reconfigure the prompts to limit the amount of retrieved content from the knowledge sources.
Lower the maximum token usage limit for the responses.

Community Insight

The exam tests the ability to distinguish between performance tuning (chunking) and quality degradation (truncation). The trap is choosing prompt reconfiguration, which risks losing critical context.

This question addresses optimizing Microsoft Foundry agents by reducing token consumption while maintaining response quality. The correct strategy involves improving retrieval precision through document chunking rather than truncating context.

Many learners choose Option C because it directly limits input size, but this often degrades answer accuracy by excluding necessary grounding data, violating the 'without degrading quality' constraint.

Community Discussion (6 comments)

Joselito79 👍 2 Selected: C

✅ Correct answer: C. Reconfigure the prompts to limit the amount of retrieved content from the knowledge sources. Why C? The telemetry shows the agent is frequently hitting the token usage limit. The agent uses multiple knowledge sources, so excessive retrieved content is likely consuming tokens. Adjusting the prompts to limit the amount of retrieved content reduces token consumption while preserving the most relevant information and therefore response quality.

kevinirl 👍 3 Selected: A

Reconfigure the prompts to limit the amount of retrieved content from the knowledge sources This sounds plausible but is not the best answer. Limiting retrieved content through prompt instructions can reduce tokens, but it may exclude important context needed for accurate answers. The question specifically asks for a solution that reduces token usage without degrading quality. Microsoft's guidance emphasizes solving this at the retrieval/indexing layer through chunking and retrieval optimisation rather than arbitrarily restricting context after retrieval.

timothy123 👍 3 Selected: C

C. Reconfigure the prompts to limit the amount of retrieved content from the knowledge sources. Reduces Input Size: Limiting retrieved content directly decreases the number of tokens sent to the model per request. Improves Speed: Smaller prompts process faster, which fixes the slow response times. Preserves Quality: It filters out unnecessary background data while keeping the most relevant facts for accurate answers. Chunking documents helps with indexing, but does not actively constrain runtime token limits like prompt reconfiguration.

Ninashnash 👍 2 Selected: A

When documents are chunked during indexing, the retrieval system can return only the small, relevant portions of documents instead of large sections or entire documents. This means: Fewer input tokens are sent to the model. Retrieval remains highly relevant. Response quality is maintained (and often improved because the model receives less irrelevant context). This is a standard Retrieval-Augmented Generation (RAG) optimization. All other options impact the quality of responses.

24b8eec 👍 5 Selected: A

Answer: A. Chunk documents during indexing. The key insight: the other three options all reduce tokens by removing information — fewer sources (B), less retrieved content (C), or a truncated response (D) — and each risks degrading quality. Chunking is the only option that cuts tokens by improving retrieval precision: smaller, semantically focused chunks mean the agent pulls in just the relevant content, so token usage drops while answer quality holds (or improves). That's exactly "reduce token usage without degrading quality."

omath 👍 5 Selected: C

✅ Correct Answer: C. Reconfigure the prompts to limit the amount of retrieved content from the knowledge sources This reduces token usage by: - Restricting how much grounding data is pulled into the prompt - Summarizing or chunking retrieved content before passing it to the model - Using more selective retrieval instructions - Reducing unnecessary context while keeping quality intact This is the standard optimization technique for Foundry and Azure OpenAI agents that hit token ceilings. It improves latency and reduces cost without degrading answer quality.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

Option A (Chunk documents during indexing) is the correct solution. Chunking breaks large documents into smaller, semantically coherent segments. When the agent retrieves information, it pulls only the most relevant small chunks rather than entire large documents or excessive text blocks. This significantly reduces the number of tokens sent to the model for processing, thereby speeding up response time and staying within token limits, without removing essential information needed for accurate responses.

Why the Other Options Are Wrong

Options B, C, and D all involve reducing the amount of information available to the agent, which risks degrading the quality of the generated responses. Reducing knowledge sources (B) might remove valid data. Reconfiguring prompts to limit retrieved content (C) acts as a filter that may exclude crucial context, leading to hallucinations or incomplete answers. Lowering the maximum token usage limit (D) forces the model to truncate its own output, resulting in cut-off or incomplete responses.

Community Comment Notes

Community consensus was split between A and C. Some users argued for C, stating it reduces input size directly. However, others correctly pointed out that limiting retrieved content via prompts can exclude important context. As one commenter noted, "Chunking is the only option that cuts tokens by improving retrieval precision." Another user emphasized that limiting retrieved content "may exclude important context needed for accurate answers," validating A as the superior choice for maintaining quality.

Exam Strategy

When asked to optimize AI performance without degrading quality, always look for solutions that improve efficiency (like chunking or caching) rather than those that reduce scope (like limiting sources or output length). Truncation is rarely the correct answer unless explicitly allowed.

Frequently Asked Questions

Why not limit retrieved content via prompts?

Limiting retrieved content via prompts can inadvertently exclude critical context required for accurate answers, thus degrading quality.

Does chunking affect retrieval accuracy?

No, proper chunking improves accuracy by ensuring the model receives semantically focused, relevant snippets rather than noisy, large documents.

More AB-100 FAQ →

Related Analysis

← Back to AB-100 Study Guide