When I joined Aga Khan University, clinical records going back to 2008 lived in operational systems that research and analytics teams could not safely or systematically use. Three years later, the same institution runs a governed lakehouse with real-time wearable streaming, SLA-managed data products, and an enterprise RAG platform — operated by a distributed engineering team across Kenya and Pakistan.
The problem
AKU is simultaneously a hospital network, a research university, and a partner in international consortia like the NIH-funded Uzima-DS study. Each of those constituencies needs the same underlying data — but with different freshness, governance, and access requirements, in an environment where healthcare compliance and data sovereignty are hard constraints. There was no platform: every research question started with a bespoke extraction, and nothing had an SLA.
Constraints
- Compliance and sovereignty. Millions of healthcare records with strict compliance requirements; research data crossing organisational (not physical) boundaries; AI workloads that must not leak data to public endpoints.
- Mission-critical operations. Hospital analytics and clinical systems needed database operations with disaster recovery and multi-region replication — not best-effort pipelines.
- A distributed, growing team. The platform had to be operable by engineers across Kenya and Pakistan, with practices (postmortems, capacity planning, SLOs) that outlive any individual.
What we built
The foundation: a clinical data repository and multi-country ingestion
The first deliverable was the university's first enterprise clinical data repository, consolidating records from 2008 to the present. Around it grew multi-country ingestion pipelines — including a real-time stream of Fitbit physiological data from 615 healthcare workers in Kenya for the Uzima-DS research programme, built on Kafka and orchestrated with Airflow and Azure Data Factory into Delta Lake.
The platform: a governed lakehouse with SLOs
As usage grew, we standardised on lakehouse architectures with Delta and Iceberg formats across Azure Synapse, Databricks, and PostgreSQL. Partition optimisation and format standardisation cut ETL costs by 20%, and BI workloads now run against p99 data-freshness SLOs — data products with published, measured guarantees rather than informal expectations.
The ground it stands on: landing zone, DR, and operations
We rebuilt the infrastructure underneath as an Azure Landing Zone in Terraform with private endpoints throughout — eliminating public egress for database workloads and cutting environment provisioning time by 60%. Mission-critical databases got automated backups, disaster-recovery strategies, and multi-region replication. Just as importantly, we established the operational culture: on-call rotations, postmortems, capacity planning, and SLO/SLA definitions that reduced MTTR by 40% and eliminated recurring outages.
The AI layer: RAG with sovereignty
On top of the platform we designed a conversational RAG system on Azure OpenAI, PgVector, and Azure AI Search — architected so that all data and retrieval stay inside the university's own cloud boundary. The same foundation later carried Afya Gemma into production.
Sources flow through streaming and batch ingestion into a governed lakehouse; governance and the Terraform landing zone span every layer.
Design decision
We treated the landing zone and the operational practices as part of the platform, not as afterthoughts. Private endpoints and Terraform made sovereignty and repeatability structural properties of the system — every new environment inherits them — rather than things individual engineers have to remember.
Impact
The platform now underpins hospital analytics, multiple international research programmes, and AKU's production AI systems — with a distributed team, defined SLAs, and an operating culture built to last.