About
Sameer Ankalgi — AI infrastructure and distributed systems. GPU inference, vLLM, Kubernetes. Microsoft, Zürich.
I build AI infrastructure and distributed systems — the layer that decides whether a model ships or stalls. I’m an engineer in GitHub AI Engineering at Microsoft in Zürich, working across GPU-based LLM inference and helping enterprise teams design, deploy, and scale AI workloads on Azure.
The work looks like this: profiling AlphaFold 2 on Azure H100 GPU HPC clusters until large-scale structure prediction was actually viable for a pharma customer. Serving DeepSeek across multi-GPU A100 clusters with vLLM and tensor parallelism, tuned for production throughput rather than benchmark throughput. The gap between those two numbers is most of my job.
I work out of Zürich — Microsoft’s office here and the lake within walking distance of it.
The path a token takes
Almost everything I care about lives somewhere on this diagram.
request
│
┌────▼─────┐ auth · quota · routing
│ gateway │
└────┬─────┘
│
┌────▼───────────┐ continuous batching
│ vLLM scheduler │ paged attention
└────┬───────────┘
│
┌─────┴──────┬──────────┬──────────┐
▼ ▼ ▼ ▼
┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
│ TP 0 │ │ TP 1 │ │ TP 2 │ │ TP 3 │ tensor-parallel shards
└──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘ H100 / A100 · NVLink
└───────────┴────┬─────┴──────────┘
▼
┌────────────┐
│ KV cache │ the thing that runs out first
└────────────┘A demo dies at the model. A production system dies at the scheduler, the cache, or the interconnect — and the person on the pager needs to know which.
How I got here
12+ years across the stack, and none of it started with GPUs:
- Microsoft — GPU inference and AI workloads on Azure, in GitHub AI Engineering.
- Swisscom — architected classified-grade cloud infrastructure for the Swiss Army, and built GenAI automation for manufacturing SMEs.
- Peech Care — co-founder and CTO of a speech-therapy platform that detected articulation disorders from voice notes. Built the acoustic-feature ML pipeline, led a team of 5, shipped the MVP, raised €150K.
- Daimler TSS (contract, via Devoteam) — moved the connected-car data platform off an IBM Db2 column-store onto bare-metal MongoDB: ~2 TB, 600M+ records, zero downtime, cut over in a 33–38h window at ~4–5K writes/sec sustained. The migration service was Go; the runtime was Kubernetes.
- BCG Platinion and PwC — enterprise delivery, ML pipelines, and the habit of asking what a system is actually for.
- SAP and BookMyShow — HANA backends, and high-throughput transaction queuing under bad connectivity, which is where I learned that latency is a product decision.
The mix — knowing why a thing has to ship and being able to actually build it — is what I optimize for. Full detail on the CV.
What I think about
- GPU inference at scale — batching, KV-cache pressure, tensor and pipeline parallelism, and the persistent gap between benchmark throughput and served p99.
- Open-weight serving in the enterprise — capacity planning, gateways, TCO. Collected in oss-model-playbook.
- AI safety as engineering practice — containment in the Coming Wave sense: audits, choke points, reversibility. pi-bench is that instinct turned into a benchmark. Paper reviews land under /ai-safety.
- Scalable oversight — how do you keep \(\frac{dO}{dt} \ge \frac{dC}{dt}\) when capability scales exponentially?
I published Reactive Programming Languages — A Survey, which is where the interest in runtime behaviour started.
Recent
- WRK541 — Microsoft AI Tour Zürich — hands-on workshop on real-world code migration with GitHub Copilot Agent Mode
- ABB AI Hackathon with Microsoft
Outside the laptop
Cycling around the Zürichsee in spring/summer, slow miles along the Limmat, occasional travel write-ups.
Find me
If you’re working on GPU inference, AI infrastructure, or the systems behind frontier models and want to compare notes, book a 30-minute chat.