← All questions
LLMOps & ProductionHard· Debugging· Data Staleness· Format Compliance· Root Cause Analysis

Your assistant returns fluent answers in the wrong form...

It is a mistake to treat format errors and data staleness as a single bug. Format issues are typically query-independent and suggest a problem with the system prompt or output parsing. Conversely, quoting outdated prices is a query-dependent issue related to the data retrieval pipeline. To diagnose this, you should sample the failure cases and inspect the retrieved context. If the current price is missing from the context, the issue is with your retrieval or data freshness. If the current price is present but the model ignores it or formats it incorrectly, the issue lies in the prompt or the model’s instruction following.

Code playground (Python)