For Engineers: Implementation Details & Stack
Implementation favors boring, inspectable pieces. Conversation state, retrieval, tools, and UI embed remain separate modules with clear contracts. The agent runtime orchestrates turns, stores short-term memory, and calls tools with typed schemes. Failures return structured errors so the UI can offer a retry or human path. We avoid monolithic prompt files that mix policy, style, and tools.
Retrieval uses chunked, tagged documents with freshness metadata. Product tables stay structured when possible so prices are fielded facts, not prose. Embedding refresh runs on change events or nightly jobs, not only when someone remembers. When recommendation logic is required, rules bound the shortlist after candidate recall. That is the same split used for the beauty recommendation path.
Tooling wraps CRM, calendar, and commerce APIs behind idempotent adapters. Retries use backoff and correlation IDs. Dead letters land where ops can replay without re-asking the user. Secrets rotate without bot redeploys when the secret store updates. Local emails for test handoffs never hit production inboxes by accident.
Edge cases get first-class tests. Multi-language users, partial form fills, and doubled button taps all appear in suites. Latency budgets force streaming tokens when channels support them. We rate-limit abuse and fingerprint obvious bots consequently. Observability tags every turn with intent, tool, latency, and token cost for later analysis.
Stack choices stay justified in writing. Managed LLM APIs for speed. Vector store when corpus size merits it; simple keyword for tiny FAQs. Message queues for async side effects. Terraform or equivalent for repeatable environments. Engineers inherit a repo they can run locally with sample chats, not a black-box vendor console alone.
