Architecture guide · Deployment · 7 min guide
Local versus cloud is a workload decision, not a belief system.
Use a cloud API when frontier quality, fast iteration and low operational burden matter most. Consider a local or private model when data boundaries, offline use, latency, predictable high volume or model control justify the engineering cost. Many production systems should be hybrid: keep sensitive retrieval and deterministic actions controlled while using the best permitted model for bounded generation.
Begin with the information boundary
Classify what the workflow sends, stores and returns. Separate public, internal, confidential and regulated information. Then check data residency, retention, provider controls, access logging and whether sensitive fields can be removed before inference.
Measure quality on your work
Model size and benchmark scores do not decide fitness. Build a representative evaluation set in the languages, formats and edge cases your users create. Compare accuracy, citation support, abstention, tool use and consistency.
- Arabic and English quality where required
- Failure behavior on missing evidence
- Structured output and tool-call reliability
- Latency at realistic concurrency
Price the operating system, not only the token
Cloud pricing includes usage and vendor dependence. Local deployment includes hardware, serving, monitoring, upgrades, security, capacity planning and specialist time. Compare total cost at realistic volume and quality—not the cheapest model in isolation.
Use hybrid boundaries deliberately
A hybrid design can keep identity, permissions, retrieval, business rules and consequential actions under direct control while routing only approved context to a model. Different tasks can use different models without changing the product experience.
Choose using seven constraints
- 01Privacy and residency
- 02Task quality
- 03Latency and availability
- 04Volume and total cost
- 05Integration and tool use
- 06Operational capability
- 07Control and portability