Model Routing and Fallback Design

Group tasks by shape: classification, extraction, transformation, retrieval answer, creative generation, long reasoning, code edit, and tool-using agent. Each group has different model requirements.

A router can be a simple rules file at first. For example, route schema extraction to a fast model, legal or financial synthesis to a stronger model, and low-confidence retrieval answers to a human review queue.

Confidence should come from observable checks: valid JSON, citation coverage, retrieval score, policy match, evaluator score, or whether the model asked for missing information.

Do not rely only on the model saying it is confident. Self-reported confidence can be persuasive but wrong.

Fallbacks should preserve the user experience. If the first model fails, the system might ask a clarifying question, retrieve better context, escalate to a stronger model, or show a safe partial answer.

Avoid silent degradation. Users should not receive a lower-quality answer just because a premium model is unavailable.