Routing
The routing pattern classifies input and directs it to a specialized follow-up task. By separating tasks into distinct categories and routing them to appropriate processes, this pattern enables separation of concerns and highly optimized handling for each input type.
Without routing, optimizing a system for one type of input (e.g., simple queries) can degrade performance for other types (e.g., complex or edge-case queries).
Why It Matters
Section titled “Why It Matters”- Specialization improves performance — Each task type is handled by a tailored process, ensuring higher accuracy
- Efficiency — Tasks go to the most appropriate system, optimizing resource use and reducing computational costs
- Scalability — Varied input types are handled effectively by directing them to the right models or systems
- Reduced bottlenecks — No single model becomes overwhelmed by diverse inputs it isn’t optimized for
Key Components
Section titled “Key Components”| Component | Purpose | Example |
|---|---|---|
| Router (LLM Call) | Classifies incoming tasks and directs them to the appropriate follow-up | Determines if a customer query is general, refund-related, or technical |
| Specialized Process 1 | Handles one specific task category | Responds to general inquiries (store hours, product info) |
| Specialized Process 2 | Handles a different task category | Processes refund requests (checking order history, issuing refunds) |
| Specialized Process 3 | Handles complex or specialized tasks | Resolves technical support issues with troubleshooting steps |
| Output | Consolidates results into the final response | Delivers the resolved answer to the customer |
When to Use It
Section titled “When to Use It”- Diverse input types — When the system handles inputs that differ in complexity or nature
- High-performance requirements — When optimizing for one input type risks degrading performance for others
- Resource optimization — When simple tasks can use smaller models while reserving powerful models for complex cases
Example: Customer Service Automation
Section titled “Example: Customer Service Automation”A company automates customer service across varied query types:
- Router classifies each query:
- General questions (“What are your store hours?”)
- Refund requests (“I want a refund for my last order.”)
- Technical issues (“My device isn’t turning on.”)
- Process 1 handles general inquiries with predefined responses
- Process 2 processes refunds by accessing order history, validating claims, and initiating refunds
- Process 3 resolves technical issues with product-specific troubleshooting
- Output delivers the final response to the customer
How to Implement
Section titled “How to Implement”- Define categories — Identify the distinct task or input types your system will handle
- Train or configure the router — Implement a classification mechanism (LLM or traditional classifier) to route inputs correctly
- Develop specialized workflows — Create tailored workflows or prompts for each category
- Test and refine — Evaluate routing accuracy and each specialized workflow’s effectiveness
Based on Building Effective Agents by Anthropic.
Related
Section titled “Related”- Workflow Architecture Patterns Overview
- Prompt Chaining — sequential processing within a single path
- Parallelization — running multiple paths simultaneously
- Design Your AI Workflow