Skip to main content

ExpressRoute construct naming

· 8 min read
Simon Painter
Simon Painter
Cloud Network Architect
Zain Khan
Zain Khan
Cloud Network Engineer

Make it make sense​

I will always be a network engineer and that means that some words have very specific meanings that have taken root in my soul. The terminology within ExpressRoute has bothered me for ages and when speaking to a few people I found that I am not the only one who finds it unintuitive. To me a circuit is a single link but to Microsoft a circuit is the pair of links and the associated peerings! Two thumbs up to that, Microsoft, or rather in your own language 'one ExpressRoute thumb'.

Here's a run down of what's what, starting with the wires.

ExpressRoute Direct (ERD) port pair​

Pair of physical ports in a peering location in the Microsoft Enterprise Edge (MSEE) and enable the physical fibre cross connects at layer 1 to the customer router. Prior to ERD being available as a product these would typically exist only in the form of a cross connect to service providers who offered ExpressRoute connectivity services, but are now offered direct to customers who have the requirement and the necessary supporting infrastructure. Microsoft documents refer to these as ExpressRoute Direct resources to either add or remove confusion with ExpressRoute Direct Circuits.

ExpressRoute Direct resources are provided as a pair of ports in separate MSEE devices to ensure that maintenance outages or failures on a single MSEE do not cause loss of service. It is not possible to buy them with a single link.

ExpressRoute Directs are paired inside a single peering location except when they used in ExpressRoute Metro; in that case the separate ExpressRoute Direct Ports are supplied in two separate peering locations but still function as a pair.

Vlan tagging​

ExpressRoute Direct supports both QinQ and Dot1Q VLAN tagging. Here's the documentation:

QinQ VLAN Tagging allows for isolated routing domains on a per 
ExpressRoute circuit basis. Azure dynamically gives an S-Tag at 
circuit creation that can't be changed. Each peering on the circuit 
(Private and Microsoft) uses a unique C-Tag as the VLAN. The C-Tag 
isn't required to be unique across circuits on the ExpressRoute 
Direct ports.

Dot1Q VLAN Tagging allows for a single tagged VLAN on a per 
ExpressRoute Direct port pair basis. A C-Tag used on a peering must 
be unique across all circuits and peerings on the ExpressRoute Direct 
port pair.

To simplify the choice you either have to maintain unique vlan tags for all peerings in all circuits using an ExpressRoute Direct resource with Dot1Q, or you alternatively choose 802.1QinQ where a separate outer tag is allocated for each ExpressRoute Circuit over the ExpressRoute Direct and that means you only need to have the vlan tags for each peerings unique within that circuit.

ExpressRoute Direct Port​

Nominally a single port on the MSEE or a single fibre pair cross connect to that port. These cannot be bought separately and are supplied as pairs in the same peering location, or in a metro pair in two separate locations for ExpressRoute Metro.

ExpressRoute Direct Circuit​

Exactly the same as an ExpressRoute Circuit however instead of being provided over a telco partner's infrastructure it's provisioned over an ExpressRoute Direct resource. A single ExpressRoute Direct port pair resource can have multiple ExpressRoute Direct Circuits associated with it.

ExpressRoute Circuit​

These are the logical constructs which encompasses two ExpressRoute Links. A circuit can be associated with more than one ExpressRoute Connection to an ExpressRoute Gateway

Each ExpressRoute Circuit is made up of two links, these are the logical connections between customer (or telco) managed equipment and the MSEE.

ExpressRoute Peering​

Each ExpressRoute Circuit consists of a pair of links and each of these can have distinct peerings. A peering is required for each link and a single circuit can support both Azure private peerings and Microsoft peerings.

Peering type: Azure private peering​

Azure compute services, such as virtual machines (IaaS) and cloud services (PaaS), deployed within a virtual network can be connected through the private peering domain. This domain is considered a trusted extension of your core network into Microsoft Azure.

Peering type: Microsoft peering​

Connectivity to Microsoft online services (Microsoft 365, Azure PaaS services, and Microsoft PSTN services) occurs through Microsoft peering. This peering enables bi-directional connectivity between your WAN and Microsoft public cloud services without using the Internet.

Required Subnet Configuration​

  • Two subnets outside of any VNet address space
  • One subnet for primary link, one for secondary link
  • Customer uses first usable IP, Microsoft uses second usable IP
  • Subnet size options:
    • IPv4: Two /30 subnets
    • IPv6: Two /126 subnets
    • Dual-stack: Two /30 and two /126 subnets
  • Public addressing or RFC1918 can be used for Private peering
  • Public addressing must be used for Microsoft peering

VLAN Configuration​

BGP Configuration for Private Azure Peerings​

  • AS number (2-byte or 4-byte supported)
  • Private AS numbers allowed except 65515-65520
  • Optional MD5 hash for session security

BGP Configuration for Microsoft Peerings​

  • AS number (2-byte or 4-byte supported)
  • Private AS numbers allowed except 65515-65520
  • Public AS numbers allowed with proof of ownership
  • Optional MD5 hash for session security

ExpressRoute Connection​

A Connection is the resource that links an ExpressRoute Circuit at the MSEE to an ExpressRoute Gateway. You can have more than one connection on a single ExpressRoute Circuit, A standard Azure ExpressRoute circuit can typically support up to 10 connections to virtual networks, all within the same geopolitical region; however this limit is the default and can be raised.

ExpressRoute Gateway​

An ExpressRoute Gateway is a virtual network gateway that enables private connectivity between your on-premises network and Azure through ExpressRoute Connections.

Key points:

  • Acts as a bridge between your virtual network and on-premises network
  • Handles routing between these networks
  • Supports multiple circuits for redundancy
  • Must be deployed in a dedicated gateway subnet
  • Scales based on selected SKU
  • An ExpressRoute Gateway can connect to more than one ExpressRoute Connection.

Available Gateway SKUs and capabilities​

Gateway SKUFastPathMax Connections
Standard SKU/ERGw1AzNo4
High Perf SKU/ERGw2AzNo8
Ultra Performance SKU/ErGw3AzYes16
ErGwScale (Preview)Yes*4-16**

*FastPath requires minimum 10 scale units ** Up to 4 connections with 1 scale unit (2Gbps), up to 8 with at least 2 scale units (4Gbps) and up to 16 connections with at least 10 scale units.

ExpressRoute Metro​

ExpressRoute Metro is a variation of an ExpressRoute Circuit where the separate ExpressRoute Links are provisioned in different physical peering locations within a metro area. This give additional geographic redundancy for those who do not require two separate resillient ExpressRoute Circuits in two separate locations.

ExpressRoute Metro can be selected when ordering an ExpressRoute Direct and the metro pair appears like any single peering location on the list of available locations.

Summary​