What retrieval is good at
Retrieval-augmented generation solved a real problem. A model on its own does not know your specific regulation, and when pressed it will invent one that sounds right. Give it a retrieval step, let it search a corpus of real regulatory text and pull the relevant passage into context, and the invented answer gives way to one grounded in a document that actually exists. That is genuine progress. The model is now reading the real rule instead of a hazy memory of one.
So retrieval is good at a specific thing. It finds the regulation that applies. In a library of hundreds of thousands of rules, that is not a small thing.
The gap it does not cross
But finding the rule and applying it are two different acts, and only one of them is retrieval. Once the relevant passage is in context, something still has to read it, weigh it against the facts of the case, and produce a verdict. The thing doing that is the model, which means the verdict is back to being a probabilistic guess. Retrieval changed what the model is looking at. It did not change what the model is, which is a system that predicts a plausible next word, including a plausible wrong one.
And the citation makes the guess look trustworthy. A verdict that arrives with a real article number attached reads as grounded and checked. Underneath, the decision can still be wrong, and now it is wrong with a footnote. A confident wrong answer is bad enough. A confident wrong answer wearing a correct citation is harder to catch, because it has the surface texture of diligence.
Conditioned is not constrained
It is tempting to think that enough context closes the gap. Pull in the article, the guidance, the precedent, and surely the model is now boxed into the right answer. It is not boxed in. It is conditioned. The retrieved text makes some answers more likely and others less, but nothing in the machinery stops the model from returning a verdict the text does not support. There is no moment at which the regulation gets to veto the model. It only gets to lean on it.
A decision in regulated work cannot rest on a lean. It has to be a consequence. The answer must follow from the rule the way a sum follows from its numbers, not the way a sentence follows from a prompt.
Where retrieval belongs
None of this is an argument against retrieval. It is an argument about which job retrieval should hold. Use it for what it does well. Finding which rules are in play, narrowing a vast corpus to the handful that apply to this case. That is a search problem, and search is exactly its strength.
Then hand those rules to something that decides deterministically. The verdict comes from the rule's conditions evaluating against structured facts, not from the model's reading of a paragraph. The citation is not decoration bolted onto a guess after the fact. It is the rule that produced the decision, and it traces straight back to its source because it is the source.
The point
Retrieval can put the right regulation in front of the model. It cannot make the model's answer follow from it. In consumer search that gap is invisible and harmless, because no one audits a summary. In regulated work the gap is the whole risk, because the decision is the product, and a decision you cannot guarantee is not a product you can sell to a bank. Retrieve to find the rules. Do not retrieve to make the call.