Scaling LLM Verbosity Control via Supervised Fine-Tuning

PMLE By islever Editorial Last updated

Developing ML models using BigQuery ML or AutoML on Gemini Enterprise Agent Platform. Training models.

Short answer

B Use the collected feedback to create a dataset of question-answer pairs and perform supervised fine-tuning on the foundational model to learn the desired verbosity patterns.

Question

You deployed a conversational application that uses a large language model (LLM). The application has 1,000 users. You collect user feedback about the verbosity and accuracy of the model 's responses. The user feedback indicates that the responses are factually correct but users want different levels of verbosity depending on the type of question. You want the model to return responses that are more consistent with users' expectations, and you want to use a scalable solution. What should you do?
  1. Implement a keyword-based routing layer. If the user's input contains the words "detailed" or "description," return a verbose response. If the user's input contains the word "fact." re-prompt the language model to summarize the response and return a concise response.
  2. Ask users to provide examples of responses with the appropriate verbosity as a list of question and answer pairs. Use this dataset to perform supervised fine tuning of the foundational model. Re-evaluate the verbosity of responses with the tuned model. Correct Answer
  3. Ask users to indicate all scenarios where they expect concise responses versus verbose responses. Modify the application 's prompt to include these scenarios and their respective verbosity levels. Re-evaluate the verbosity of responses with updated prompts.
  4. Experiment with other proprietary and open-source LLMs. Perform A/B testing by setting each model as your application's default model. Choose a model based on the results.

Key Takeaways

The trap is choosing prompt engineering or keyword routing for style adaptation; these fail at scale. SFT allows the model to internalize style preferences from examples, providing a robust, scalable solution.

This PMLE exam question explores how to scale LLM response verbosity adjustments beyond brittle keyword rules. The correct approach leverages supervised fine-tuning to embed user preference patterns directly into the model.

Common mistake: Candidates often select Option C (prompt modification) assuming it is the easiest fix. However, enumerating all scenarios in a prompt is not scalable and exceeds practical context limits for dynamic user behavior.

Expert Analysis

The core challenge here is adapting model output style (verbosity) to user intent while maintaining scalability. The prompt explicitly states that factual accuracy is already satisfactory, meaning the model's knowledge base is sufficient; only the presentation needs adjustment.

Option A suggests keyword-based routing. As noted in community feedback, this approach is brittle and unscalable because it relies on rigid string matching rather than semantic understanding. It fails to generalize to new phrasings of 'detailed' or 'concise' requests.

Option C proposes modifying the prompt with specific scenarios. While prompt engineering is powerful, attempting to enumerate all possible user scenarios for verbosity expectations is operationally unsustainable and violates the scalability requirement. Prompts have context window limits and become unwieldy when trying to encode complex conditional logic manually.

Option D involves switching models via A/B testing. Since the current model is factually correct, changing the underlying model does not address the stylistic variance issue and introduces unnecessary cost and latency complexity without guaranteeing a better fit for verbosity control.

Option B, supervised fine-tuning (SFT), is the correct scalable solution. By creating a dataset of question-answer pairs where the answer verbosity matches the desired outcome, the model learns the mapping between input intent and output style. Once tuned, the model handles diverse inputs autonomously without maintaining external rule engines or massive prompt contexts. This aligns with Objective 1.1: Developing ML models using BigQuery ML or AutoML on Gemini Enterprise Agent Platform.

Exam Strategy

When a question asks for a 'scalable' solution to adapt model behavior based on user feedback, distinguish between runtime adjustments (prompts/routing) and model-level changes (fine-tuning). If the behavior requires learning nuanced patterns from many examples, choose fine-tuning. If it is a simple, static rule, choose prompt engineering. Always prioritize long-term maintainability over quick fixes.

Frequently Asked Questions

Why not use prompt engineering for verbosity control?

Prompt engineering works for simple instructions but becomes unscalable when you need to handle many specific scenarios or edge cases. Enumerating all user intents in a prompt leads to bloated context windows and high maintenance costs.

Is fine-tuning always necessary for style changes?

No. For simple, universal style changes (e.g., 'always be concise'), system prompts are sufficient. Fine-tuning is reserved for complex, pattern-based adaptations that require the model to learn from diverse examples, such as varying verbosity by question type.

More PMLE FAQ →

Community Votes

B
43%
C
29%
A
28%

43% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 10 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 10

  1. Fer660 👍 4 Answered A
    Not B: Why would we assume that our users are willing to do all this work? Not C: Same thing -- we can't just ask users for all this, they don't work for us. Not D: This seems to require a whole lot of work. And doing A/B testing for multiple models based on 1000 users seems likely to reach conclusions that are not statistically sound. A: Not perfect, for sure, because we are creating a declarative layer and we might have to keep adding rules to detect the customers' expectations -- seems contrary to the spirit of AI, and also violates the spirit of scalability, but I guess this is the lesser evil here?
  2. dija123 👍 3
    Thinking B (supervised fine tuning ) could be better Answer than A
  3. lokiinaction 👍 2 Answered B
    this is not really possible for users to list all scenarios therefore sharing examples is a good way to go
  4. taylz876 👍 1 Answered B
    Why B is right • The responses are accurate, but the style is wrong. Users want different levels of detail depending on the type of question. That's a behavior the model has to learn, and example pairs of questions and well-sized answers teach exactly that. • Scalable: once the model is tuned, it handles all 1,000 users and new questions on its own, with no hand-written rules that need maintaining. Why the others are wrong • A: A keyword router is brittle. It only triggers on specific words and misses everything else. • C: A prompt listing all the scenarios depends on users anticipating every case, and the list grows without end, so it doesn't scale as well. • D: Switching to other LLMs and A/B testing doesn't target the verbosity problem.
  5. maukaba 👍 1 Answered C
    The feedback shows that accuracy is fine, but verbosity needs to be adjusted dynamically. The most scalable and cost-effective solution is prompt engineering, not retraining or switching models. By explicitly encoding verbosity rules in the prompt, you can guide the LLM to produce responses that match user expectations without additional compute or fine-tuning. This approach is fast to implement, works with existing LLMs, and scales easily for thousands of users.
Show 5 more comments
  1. RogerRabid 👍 1 Answered C
    Keyword based routing is brittle and unscalable. We've already received feedback from the users. Assuming we can do that again we can incorporate this feedback into prompt so the output is more in-line with what users are expecting.
  2. Johnb9682 👍 1 Answered B
    A is incorrect: Keyword matching (e.g., searching for "detailed" or "fact") is brittle, inflexible, and doesn't scale. C is incorrect: Attempting to enumerate all scenarios in system prompts is unscalable.
  3. maukaba 👍 1
    Users are providing feedback hence are collaborative
  4. dija123 👍 1
    A is not scalable indeed!
  5. dija123 👍 1 Answered A
    agree with A

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Monitoring ML Model Performance with Precision and Recall Sampling

S-grade

Optimizing ML Model Recall for Compliance Risk

S-grade

How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

S-grade

Building CLV Models with BigQuery ML in Vertex AI Workbench

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide