Inside the Megakernel Serving Engine for North Mini Code
Cohere, Tuesday, September 8th, 2026
A technical account of the megakernel approach delivering 1.58x faster LLM serving on H100 devices.
Cohere details the megakernel serving engine behind North Mini Code, reporting 1.58 times faster LLM serving on H100 devices.
The megakernel approach fuses operations that would conventionally launch as separate GPU kernels, reducing launch overhead and memory traffic that dominate at small batch sizes.
The technique is directly relevant to anyone self-hosting models, since serving efficiency rather than model quality usually determines whether an internal deployment is affordable against commercial API pricing.