From local dev demos to dependable deployments — lessons from the engineering front lines.
It’s not just the LLM — it’s the whole agent ecosystem
At Predyktable, we use an agentic AI framework to orchestrate industry leading LLMs and our own internal tools to provide dynamic scenario modelling. It’s not about building new LLMs ourselves — it’s about tight prompting, smart tooling, and robust APIs that let the agent do its job effectively.
Prototype demos are one thing. Making that work in production — with real users, unpredictable data, and system constraints — is another story entirely.
The agent is the API
Our core architecture is built around the Agent as an API, responsible for scenario modelling tasks.
- It interfaces with LLM endpoints for reasoning and planning.
- It connects to our internal services for data ingestion, simulation, and feedback.
- Everything is containerised and designed for loose coupling — the agent runs as an independent, scalable service that plugs into the wider application ecosystem.
It’s less about pipelines and more about coordination and orchestration between services and tools in real time.
From prototype to product: Decoupling the all-in-one stack
One of the hardest transitions was moving from a single app agentic prototype (agent + interface + logic all bundled together) to a modular, production-ready architecture.
- The chat interface was migrated into our self-service platform UI.
- The agent logic became a standalone API, running in its own container.
- We built a new local dev environment for testing the agent in isolation — critical for iteration without impacting the full system.
This modularity gave us flexibility — but it also meant rethinking how each part communicates, handles errors, and scales independently.
Observability: Seeing inside the agent’s mind
When an agent is making autonomous decisions, understanding its process is non-negotiable.
We leaned heavily on LangGraph’s OpenTelemetry integration to track each step, each decision node, and each tool invocation.
This approach really paid dividends when the agent made a decision none of us could explain — we traced it back using our telemetry, identified a flaw in the prompt chain, and fixed it. Without that observability layer, we’d be flying blind.
Security & isolation: Building trust by design
Each agent instance is deployed independently per client — ensuring complete data isolation and eliminating the risk of cross-contamination.
We follow standard enterprise security protocols (auth, access control, encrypted transit), but the single-tenant model for agents has been key for client trust, especially when dealing with sensitive operational data.
Human in the loop: The agent as a co-pilot
The agent isn’t autonomous in the traditional sense — it’s designed to support and augment human decision-making.
It responds to planner input, simulates scenarios, presents outcomes, and refines decisions based on user feedback. Everything is designed to keep the human firmly in control, with the agent providing contextual, fast, and relevant guidance.
Fast iteration, production-grade foundations
The tension we live in is this: build fast enough to keep up with change, but solid enough to run in production.
For us, success means we can evolve quickly — testing new tools, tweaking prompts, adapting to new client needs — without compromising on stability, security, or user trust.
That’s the real challenge of productionising agentic AI. And it’s one we’re still learning from every day.