Model Router Designer

İleridevelopmentEn az 32K bağlam

Designs routing layers that dispatch requests across multiple LLMs based on task type, difficulty, latency, cost, and reliability. Covers classifier-based and heuristic routing, fallbacks and retries across providers, quality scoring, and A/B evaluation of routing policies. Helps teams get frontier quality where it matters and cheap models everywhere else.

llmroutingorchestrationfallbackcostreliability

Kullanım alanları

  • Routing between a frontier model and a cheaper open model
  • Adding cross-provider fallback for reliability
  • Classifying request difficulty to pick a model
  • Evaluating routing policies with offline metrics
  • Building a router with cost and latency budgets

Örnek prompt

Design a model router for a coding assistant that must balance cost and quality.
Provide:
1. A routing strategy (classifier vs heuristics) with rationale
2. Tiers mapping task types to models
3. Fallback and retry logic across providers
4. Quality and cost metrics to monitor
5. An offline evaluation plan for the routing policy

Önerilen modeller

claude-opus-4-7gpt-5-5-progemini-3-1-prodeepseek-v4

Uyumlu araçlar

claude-codekirocursorany

Modaliteler

Giriş: text, code
→
Çıkış: text, code

İlgili Skills

llm-cost-optimizerai-agent-builderllm-eval-harness-builder

Yazar

OpenModels Community

@openmodelsrun