2 posts tagged with "load-balancing"

If you're running self-hosted AI models on AKS, routing inference traffic is a different problem to routing web traffic. A standard HTTP load balancer doesn't know about GPU queue depth or KV cache utilisation — it just sees HTTP requests. The new inference gateway for Application Gateway for Containers changes that.

Microsoft put this feature into public preview on 24 June 2026. It brings the Kubernetes Gateway API Inference Extension into Application Gateway for Containers, giving you model-aware routing decisions at the ingress layer without building your own sidecar logic.

The short version: your gateway can now pick which model server pod handles a request based on real-time signals from the model itself, not just round-robin or connection count.

Microsoft is retiring Inbound NAT Pools for Azure Virtual Machine Scale Sets on 30 September 2027. This is the VMSS-specific part of Inbound NAT rule V1 on Azure Load Balancer.

That sounds narrow, but it matters if you still use load balancer frontend port ranges to reach scale set instances for SSH, RDP, or similar admin access. Microsoft also says you won't be able to create new Inbound NAT Pools from 15 November 2026.

The key point is simple: single VM Inbound NAT rule V1 isn't being retired. This change is only for the old VMSS pattern, and the replacement is Inbound NAT rule V2 that targets a backend pool instead.