AI Engineering and Agent Orchestration: How to Choose the Right Architecture
One agent or many? What should be a skill or a tool, and what should stay deterministic? How I make architecture decisions in AI Engineering and Agent Orchestration.
Getting information about AI has never been easier. There are countless resources on LLMs, AI agents, skills, MCP, frameworks and different orchestration approaches. But finding information is not the same as choosing the right information, building the right architecture and turning it into a working system.
In this article I describe the questions I use to make architecture decisions in AI Engineering and Agent Orchestration projects. The goal is not to recommend a framework. The goal is to know why a system was designed the way it was, and to make sure it produces reliable results.
Problem first, model second
Most AI projects start with "which model?". I start with a different question: which process is slow, costly or error-prone? How is it measured today? What is a better result worth? Without those answers, model selection turns into guesswork.
Model choice comes after that. Quality, latency and cost are weighed together for each task. Keeping providers swappable is an architecture decision too. If users can bring their own OpenAI, Claude, Gemini or Perplexity key, or run Llama on their own llama.cpp server, the model choice belongs to the user and not to the system.
One agent or several?
My default is a single agent. One agent with a clear goal and well-defined tools handles most work. Multiple agents make sense only when:
- The tasks are genuinely independent and can run in parallel
- Different tasks need different permissions or different context
- One agent's context grows too large to carry the whole job
A multi-agent design always has a cost: more model calls, data passed between agents and harder debugging. You need a concrete reason to pay it. "It looks more modern" is not a reason.
Skill, tool or plain code?
A simple way to decide how to define a task:
- Tool: an operation the agent calls, with clear input and output. For example, querying a record or creating a draft.
- Skill: instructions and context packaged for a specific kind of work. It tells the agent when to act, in what order and under which rules.
- Plain code: steps whose result must always be the same.
Tools should be exposed through explicit, permissioned interfaces, and the agent's state should live outside the model. That way you can see what happened, reproduce a failure and limit permissions per tool.
Which steps are deterministic, and which are LLM decisions?
This is where most mistakes happen. Some decisions should never be left to an LLM: who gets access to what, whether a payment is charged again, whether an action is recorded. Those run as rules and code.
LLM decisions add value where uncertainty really exists: understanding free text, suggesting the best fit among several options, drafting content. A practical rule: if you can verify the result, you can leave it to the LLM. If you cannot, add human approval.
Human approval for irreversible actions
Publishing, sending and spending cannot be undone. I do not let agents do these on their own. In Seliee's AI Office, publishing, sending and spending always wait for the user's approval. The MCP server, with more than 230 tools, gives external agents the same controlled access.
No agent goes to production without evaluation and monitoring
The difference between a working demo and a reliable system is measurement:
- Test sets built from real questions, with offline scoring before each release
- Regression tests that show a prompt or model change did not break earlier behaviour
- Per-request tracing: token usage, latency, error rate and cost per feature
- Design against prompt injection and data leakage, tenant isolation and an audit trail
Success criteria (accuracy, grounding in sources, refusal of out-of-scope requests, latency and cost per request) are agreed before the work starts. A criterion set afterwards is just a criterion fitted to the result you already have.
How I make these decisions
I follow the same order on every project: business problem, measurable goal, the smallest AI solution that can prove value, integration with existing systems, and governance. Model and framework choice come at the end of that order.
I offer technical assessment, architecture design and implementation consulting in AI Engineering, Agentic AI and Agent Orchestration. To describe your project briefly, you can fill in the form. You can also see all of my AI engineering work on the AI Engineering page.
Frequently Asked Questions
Should I use one agent or multiple agents?+
Start with a single agent by default. Consider multiple agents when tasks are truly independent, need different permissions or context, or one agent's context cannot hold the whole job. Multi-agent designs add cost and make debugging harder.
Which steps should run as deterministic logic?+
Decisions that must always give the same result and cannot be undone, such as authorization and payments, should run as rules and code. Use LLM decisions where uncertainty really exists and the result can be verified.
When do agents need human approval?+
For irreversible actions such as publishing, sending and spending. These steps should wait for approval even when the agent is otherwise running on its own.
What should an agent system be tested with before production?+
Test sets built from real questions, offline scoring before each release, regression tests and success criteria agreed up front. In production you also need per-request tracing and cost monitoring.
