A Multi-Agent Architecture for an Industrial AI Assistant

The context

The same industrial-maintenance assistant as the RAG pipeline: a conversational product built on a single-level LangGraph graph (router and answer profiles). My founding-engineer mission covered both horizons: taking that system from POC to production, and designing the agentic architecture carrying the product’s long-term vision. I turned the product vision into a technical solution, from the architecture specification (monolith decomposition, migration plan, API contract) through to implementing the code. This page describes the architecture and the measurements.

From POC to production

• Chain-of-thought and real-time streaming: I introduced tagged chain-of-thought to improve accuracy (the model structures its thinking, tool actions and final answer in XML tags), and integrated end-to-end streaming for reasoning transparency in the UI and a far better latency UX: a state-machine parser emits those tags as deltas while Bedrock chunks arrive, memory-bounded, with a full-content fallback on malformed streams. Hardened after an incident against silent infinite loops, with a regression test.
• Tested anti-hallucination: the model was inventing image URLs; validation, retry and prompt constraints, shipped with 79% test coverage.
• Fewer tokens in the right place: image tokens removed from the orchestrator call, images attached only to the nodes that need them.
• Prompt-as-code: 13 production chains migrated into typed, composable Python objects (reusable layers, injection-time variable validation), tested like code behind a 95% coverage gate in CI: canonical layer ordering, system/user separation and static parts positioned for prompt caching are all verified by unit tests. Everything versioned to a hub with movable environment tags and immutable version tags.
• End-to-end LangSmith observability: every run is traced and archived with its relevant metadata (node, environment, release, thread), the same trace serving production debugging and the evaluation campaigns that replay and judge it.

The target architecture

• Plan-and-Execute: an orchestrator as the single decision point, planning and dispatching tasks in parallel fan-out with a synchronization barrier on re-entry.
• Isolated ReAct subgraphs: each sub-agent (search, perception, diagnostics) is a compiled subgraph with its worker, its verification critic and its own state, with no access to the parent state.
• A single tool visible to the model: agent_call with progressive skill discovery, each capability being a markdown file, carrying the workflow, constraints and tool-call examples for one precise use case, plus a script. The model’s context stays dynamic and minimal, loading only the relevant skill, to the direct benefit of cost, latency and reliability. Adding a skill touches neither the worker nor the prompt.
• Context treated as an interface: retrieved content is injected with semantic XML tagging so the model cleanly separates sources and ideas, and agents and skills are named with enough semantic distance between them to prevent selection confusion at routing time.
• Auditable human-in-the-loop: native graph interrupt and resume, dual-persisted, in the checkpoint for replay and in the business table for compliance, with a cycle ceiling and hypothesis-tree mutation on resume.
• Budgets everywhere: plan iterations, tool calls per agent, retries, HITL cycles and the global timeout are all bounded in declarative configuration.

Safety and streaming

In this domain, safety has two faces: protecting the chatbot (injection, jailbreak, prohibited content) and protecting the operators, because an approximate maintenance answer can create physical risk. Both are treated in depth:

• Two-tier input guardrail: a zero-cost regex pre-filter short-circuits the obvious cases; a dedicated LLM classifier handles only the rest.
• Graduated output verification: an adjustment verdict injects a safety constraint into the synthesizer rather than refusing outright, and the content_retracted protocol retracts already-streamed content when a guardrail blocks after the fact.
• A three-layer code-execution sandbox: AST validation, emptied builtins, resource limits.
• Industrial-risk prevention: maintenance work exposes operators to physical hazards (chemicals, electrical loads, pressure, rotating parts), so every answer ships with robust, detailed risk prevention built from the documentation’s safety warnings, and a prohibited-query class is exercised in evaluation to verify the system refuses what it must refuse.
• A writer-first SSE contract: 17 typed event kinds emitted directly by the nodes, with the streaming service kept as a transparent pipe; adding an event is one line in a node, not an API change.

Costs, measured

• Model routing by task class: the frontier model only for planning and final synthesis, a fast model for workers and critics, specialized safeguard models for the guardrails. A single user request triggers 5 to 12 LLM calls.
• Adaptive Bedrock prompt caching: a different cache-point strategy for ReAct loops and for the orchestrator’s replan, with per-model gating so the write premium is never paid without reads.
• Measured in real conditions rather than asserted: 51.7% cost reduction and 49% lower latency on the second conversation turn, a derived break-even threshold (13.9% read ratio), and a chain-by-chain cacheability audit excluding those under the 1,024-token minimum.

Continuous evaluation

The production graph was itself under behavioral watch: 240 synthetic conversations played against the real dockerized service over SSE, with the reasoning trajectory reconstructed and judged, not just the final answer. Results: 99.6% correct routing and 96.6% hallucination-free, with LLM judges tiered by judgment difficulty and a self-deduplicating test set including an adversarial class to verify refusals.

Built to be inherited

A founding-engineer mission is also judged by what it leaves behind:

• Executable architecture rules, versioned in the repos: conventions, layer boundaries and named anti-patterns (CLAUDE.md, rules, docs), written so that developers and AI coding agents inherit the same constraints and contributions interlock without drifting.
• Specialized review agents versioned with the code: prompt quality (XML structure, static/dynamic separation for caching, token thresholds) and test writing, my own review standards automated in service of the team.
• Standardized tooling: uv as the single package manager, pre-commit with lint, format and strict typing, conventional commits across all contributions.

What the project demonstrates

An architecture written before it was coded, then held to: verifiable layer boundaries, bounded budgets, inference cost treated as measured data rather than fate, and safety solved at the protocol level instead of sacrificing the streaming experience. This is the core of what I build: agentic systems where every decision, every token and every failure can be explained.