Clinical questions on a hospital floor do not wait. Resident and intern doctors at Aga Khan University Hospital needed a way to check treatment guidance quickly — and the answers had to come from the guidelines Kenyan clinicians are actually accountable to, not from whatever an internet-trained model happens to believe.
The problem
General-purpose LLMs are fluent but unaccountable: they will answer a clinical question whether or not the answer reflects Kenyan Ministry of Health protocols, and in a hospital that gap is a safety issue, not a UX issue. At the same time, junior doctors are the heaviest users of reference material and have the least time to search for it. The goal was a system that answers only from vetted guideline content, refuses gracefully outside its scope, and is fast and reliable enough to be part of the daily clinical workflow.
Constraints
- Clinical safety and grounding. Every answer had to be traceable to Kenyan Ministry of Health guidelines — retrieval-augmented generation, not open-ended generation.
- Healthcare-grade access control. Retrieval had to respect role-based access controls, with comprehensive audit logging of who asked what and what the system returned.
- Production reliability. A tool doctors consult during care has to hold the same bar as other critical hospital systems — we operate our platforms at 99.9%+ uptime.
- Resource-constrained context. The system serves an East African hospital network: cost per query, latency on real hospital networks, and operational simplicity all mattered.
Architecture
The core design decision is a two-stage pipeline: a fast, cheap classifier in front of a domain-specialised generator. An incoming question first hits Gemini 2.5 Flash, which classifies intent and scope — is this a clinical guideline question the system should answer, and which guideline domain does it belong to? Only then does retrieval run against a ChromaDB vector store of embedded Ministry of Health guideline content, filtered by the user's role, and the retrieved context is handed to MedGemma for grounded generation.
Two-stage retrieval: classify first, retrieve with access controls, generate grounded answers — audit everything.
Why two stages?
Sending every query straight to a medical LLM is both wasteful and unsafe. The Flash classifier is an order of magnitude cheaper and faster than the generator, so out-of-scope questions are rejected before any expensive retrieval or generation happens — and scoping the retrieval by classified domain measurably improves the relevance of what MedGemma is grounded on.
From prototype to production
I started the early architecture and prototyping in my previous role as Lead Data Infrastructure Engineer, and productionising it became a headline deliverable when I took over the data infrastructure, AIOps and cloud engineering function. Getting from demo to daily clinical use meant the unglamorous work: RBAC-based retrieval controls, comprehensive audit logging, query and caching optimisation that brought p95 response times under 300ms on the retrieval path, infrastructure as code with Terraform, and Kubernetes-based serving with the same monitoring, SLAs and incident-response discipline as the rest of the hospital platform.
Impact
The work was recognised externally with the Google GenAI Accelerator Award. More importantly, it established a reusable pattern at AKU for putting LLMs into clinical settings responsibly: ground on vetted local content, control retrieval by role, audit everything, and hold AI systems to platform-grade SLAs.