Fixing Hallucinated Syntax with Supervised Fine-Tuning
Short answer
B Use the Agent Platform supervised fine-tuning (SFT) service with a dataset of valid MechScript examples to teach the model proprietary syntax that RAG cannot convey.
Question
You built the base Gemini model using Agent Platform RAG Engine and grounded the model with the MechScript reference manual. The model correctly answers questions about the language, but it consistently fails to generate syntactically correct code snippets, often hallucinating Python-like syntax instead. What should you do?
Key Takeaways
RAG provides knowledge; fine-tuning teaches skills. If the model knows the facts but can't produce the format, you need training data, not just context.
When an LLM understands a proprietary language via RAG but fails to generate correct code syntax, supervised fine-tuning (SFT) is the solution to teach new token patterns.
Common mistake: Option C is often chosen incorrectly because users assume more context will fix generation issues. However, adding more reference text does not teach the model the structural rules of a new syntax if those rules weren't present in the initial training corpus.
Expert Analysis
The scenario presents a classic distinction between knowledge retrieval and skill acquisition in LLM applications. The model already possesses the necessary semantic understanding of MechScript, as evidenced by its ability to answer questions correctly when grounded via RAG (Retrieval-Augmented Generation). However, it fails at the generative task of producing syntactically correct code because the specific token patterns and syntax rules for MechScript are absent from its pre-training data.
Increasing the temperature (Option A) would increase randomness, likely worsening hallucinations rather than fixing syntax errors. Increasing retrieved chunks (Option C) adds more context but does not teach the model how to structure the output; if the manual doesn't contain explicit code examples with the desired formatting, more text won't help. Copying the manual into the system prompt (Option D) is inefficient and hits context limits without addressing the fundamental lack of exposure to valid code structures during training.
Supervised Fine-Tuning (SFT) (Option B) is the correct approach here. SFT allows the model to learn new distributions and patterns by being trained on a curated dataset of input-output pairs. By providing examples of valid MechScript code, the model learns the specific syntax and generation patterns required, effectively teaching it the "skill" of writing MechScript, which RAG alone cannot provide.
Exam Strategy
Always distinguish between what the model needs to know (facts) versus what it needs to do (actions/syntax). If the question states the model has the information but generates incorrect formats or structures, look for fine-tuning options. If the model lacks information entirely, look for RAG or prompt engineering.
Frequently Asked Questions
Why doesn't RAG fix syntax errors?
When should I use SFT vs. RAG?
Community Votes
100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 2 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 2
-
taylz876 👍 1 Answered BWhy B is right • The model knows the language but can't write it. RAG grounding with the manual gives it knowledge, which is why it answers questions about MechScript correctly. Writing correct code in a syntax that isn't in public data is a skill the model has to learn. Fine-tuning on valid MechScript examples teaches it that syntax, so it stops falling back on Python-like code. Why the others are wrong • A: A higher temperature makes the output more random, which makes the syntax errors worse. • C: More retrieved chunks give the model more reading material, but the problem isn't missing knowledge. • D: Pasting the manual into the prompt is the same as grounding, which is already tried.
-
RajuMVV 👍 1 Answered BThe correct answer is B. Use the Agent Platform supervised fine-tuning (SFT) service with a dataset of valid MechScript examples. Why this is the best solution: Teaching New Syntax & Token Patterns: Foundation models have zero pre-training exposure to proprietary internal languages like MechScript, causing them to fall back on familiar programming patterns (like Python). While RAG provides knowledge/facts, Supervised Fine-Tuning (SFT) adapts the model's underlying grammar, token transition probabilities, and code generation style to consistently produce syntactically valid code.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →