Automatic failover from GPT-4o through GPT-4o-mini, Claude 3 Sonnet, and Claude 3 Haiku.
Define a cross-provider fallback chain that routes requests through progressively cheaper models when the primary model is unavailable. Starts with GPT-4o and falls back through GPT-4o-mini, Claude 3 Sonnet, and Claude 3 Haiku.
Ordered list of models to try. Add or remove models as needed
action_on_chain_exhausted
DENY
Action when all models fail. Options: DENY, MONITOR
Fallback models may have different capabilities and pricing. Ensure your application handles capability differences gracefully — for example, a model with a smaller context window may truncate long prompts.