Thanks for returning to vZealand.com to read the latest part in this series for VCAP6-NV.

This is part 7 of 20 blogs I am writing covering the exam prep guide for the VMware Certified Advanced Professional 6 – Network Virtualisation Deployment (3V0-643) VCAP6-NV certification.

At the time of writing there is no VCAP Design exam stream, thus you’re automatically granted the new VMware Certified Implementation Expert 6 – Network Virtualisation (VCIX6-NV) certification by successfully passing the VCAP6-NV Deploy exam.

Previous blogs in this series:

Part 1 – Intro
Part 2 – Objective 1.1
Part 3 – Objective 1.2
Part 4 – Objective 1.3
Part 5 – Objective 2.1
Part 6 – Objective 2.2

This blogs covers:

Section 2 – Create and Manage VMware NSX Virtual Networks
Objective 2.3 – Configure and Manage Routing

Deploy the appropriate NSX Edge (ESG/DLR) device according to a deployment plan
Configure centralised and distributed routing
Configure default gateway parameters
Configure static routes
Select and configure appropriate dynamic routing protocol according to a deployment plan:
- OSPF
- BGP
- IS-IS
Configure route redistribution to support a multi-protocol environment

Let us begin!

Deploy the Appropriate NSX Edge (ESG/DLR) Device

For the exam we will need to know the difference between the ESG and the DLR, where to use them and how to configure. Practice in your lab, use the VMware NSX HOLs and read a stack of information. No doubt in the exam they will give us a scenario where we will need to deploy ESG and DLR and configure them.

The NSX Edge comes in two flavours: the Edge Services Gateway (ESG) and the Distributed Logical Router (DLR). Both offer different features and use cases.

The Edge Services Gateway (ESG) is a router in a VM form-factor. It offers a range of L4 to L7 services such as firewall, NAT, DHCP and load balancing. It also provides L2 and L3 VPNs.

The ESG sits at the perimeter of your software-defined network and connects to the external world. It is responsible for north-south traffic. The ESG supports a maximum of 10 interfaces (uplinks and internal networks) and now also supports trunking. The firewall can enforce rules on traffic flowing between the interfaces.

The ESG can be deployed for high-availability in a 2 node active/standby VM pair. The control and data plane reside inside the VM. Host failure and losing the active ESG will cause an impact to traffic while the standby is activated.

The Distributed Logical Router (DLR) connects logical switches and routes layer 3 east-west traffic in the ESXi host kernel without leaving the host. This means that two VMs each connected to a different logical switch in different network subnets on the same host can be routed removing the need for that traffic to hair-pin up to the top of rack switch and back down again into the host.

The DLR can support a maximum of 1000 logical interfaces (LIFs).

The DLR data plane is distributed across all ESXi hosts and runs in the host kernel. The control plane exists in a VM. The DLR has a firewall but only supports firewall policy on the control and management traffic of the uplinks configured to an ESG.

The DLR can be deployed with or without the control appliance (VM), but is required if you want to utilise dynamic routing protocols, VXLAN to VLAN bridging or the DLR firewall. Note that you cannot add the VM to the DLR once it has been deployed.

The ESG supports static, OSPF, BGP and IS-IS routing protocols. The DLR supports all with the exception of IS-IS protocol.

Have a good read of the Install Guide, the Administration Guide and some of the VMware NSX Blogs.

I will cover below the basics of deploying an ESG and a DLR.

Deploy a NSX Edge Distributed Logical Router (DLR)

Note: Make sure you have met the pre-reqs which can be found in the NSX Install Guide.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Click the green + sign to add a new DLR.

Select the option Logical (Distributed) Router, enter a Name and a Hostname.

Select Deploy Edge Appliance (Control VM) if you want to utilise dynamic routing protocols or the firewall. Otherwise no control VM will be deployed and you will need to use static routes to route L3 traffic between logical switches.

Select High-Availability if you want to deploy the DLR as an active/standby pair. Note: If you are going to use dynamic routing protocols you must select this option.

Below screen shot of the above options:

Enter the name and password for your admin account. Select SSH if you want to enable and set the logging level.

If you are not deploying the control VM then the Configure Deployment section will be greyed out. Otherwise, determine where to deploy the Control VM.

Click the green + sign to add the Control VM. Enter the info for the mandatory fields: Cluster and Datastore.

If you chose to deploy the Control VM then you must select an HA Interface. VMware recommends this is a logical switch. I have selected my App_Switch.

You can add interfaces now or after deployment. Below I will add my Web_Switch and App_Switch. I have already determined the IP addressing and the IPs for interfaces. Both of my logical switches are internal networks.

I am also going to create the uplink interface which will eventually connect to an ESG. I have created a transit switch (DLRtoEdge_Transit) and have already determined the network IP addressing, then I added the interface. When you add the interface make sure you select the Type as Uplink.

The configuration now looks like this.

Next you can configure the Default Gateway. Anything not defined in the routing table will be sent to this interface.

The last screen shows you the configuration. Click Finish to deploy the DLR.

The deployment of the DLR will begin. Based on the options selected above you can see that two HA Edge DLRs have been deployed (DLR01-0 & DLR01-1).

In the vSphere Web Client, under Networking and Security, NSX Edges, you can see the new Edge DLR.

If you double-click the Edge DLR you can get into its settings and configure the firewall, routing, bridging and other options.

As the Web and App networks are directly connected via the Edge DLR, the VMs on either network can PING each other.

Below showing the Web VM (172.16.10.10) on the Web Switch pinging the App VM (172.16.20.10) on the App Switch. Both VMs are on the same host.

Make sure every host in the cluster has automatic startup/shutdown configured for the DLRs.

Deploy a NSX Edge Services Gateway (ESG)

Note: Make sure you have met the pre-reqs which can be found in the NSX Install Guide.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Click the green + sign to add a new ESG.

Select the option Edge Services Gateway, enter a Name and a Hostname.

Make sure Deploy NSX Edge is selected.

Select High-Availability if you want to deploy the ESG as an active/standby pair. Note: If you are going to use dynamic routing protocols you must select this option.

Below screen shot of the above options:

Enter the name and password for your admin account. Select SSH if you want to enable and set the logging level.

Click the green + sign to add the appliance. Enter the info for the mandatory fields: Cluster and Datastore.

You can now configure your interfaces.

I am configuring the ESG with two interfaces.

One internal interface connecting to the DLR via a transit switch.

One external interface connecting to the upstream router via vlan-backed port group.

Your configuration now looks like this, with the transit switch connecting to the DLR.

Next you configure the Default Gateway. Anything not defined in the routing table will be sent to this interface.

Finally, you configure the firewall and any HA options. By default the firewall blocks all traffic. with an explicit deny rule. Configure appropriately.

Select the vNIC to use for HA and options.

The last screen shows you the configuration. Click Finish to deploy the ESG.

The deployment of the ESG will begin. Based on the options selected above you can see that two HA Edge ESGs have been deployed (ESG01-0 & ESG01-1).

Make sure every host in the cluster has automatic startup/shutdown configured for the ESGs.

In the vSphere Web Client, under Networking and Security, NSX Edges, you can see the new Edge ESG.

If you double-click the Edge ESG you can get into its settings and configure the firewall, DHCP, NAT, Routing, Load Balancer, VPN and other options.

Configure Default Gateway Parameters

You can define a Default Gateway on both the Edge Services Gateway (ESG) and the Distributed Logical Router (DLR). The process for configuring is identical.

Anything not defined in the routing table (either a static or dynamic route) will be sent to this interface.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-Click the Edge (ESG or DLR) that you want to configure the Default Gateway on.

Click the Routing tab.

Click Global Configuration. Under the Default Gateway section click Edit.

Enter your Default Gateway IP address and select the interface. Modify the MTU and Admin Distance if required. Click OK.

Make sure you click Publish Changes to apply the configuration.

You can now see the Default Gateway configured.

The process for configuring the Default Gateway on an ESG is exactly the same.

Configure Static Routes

You can define a static route on both the Edge Services Gateway (ESG) and the Distributed Logical Router (DLR). The process for configuring is identical.

Static routes are OK for small sites with not many networks. As the amount of networks increase the use of dynamic routing protocols are beneficial.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-Click the Edge (ESG or DLR) that you want to configure the Default Gateway on.

Click the Routing tab.

Click Static Routes.

Click the green + sign to add a static route.

I am going to add a static route on my DLR to a network that exists upstream of my ESG. The network is 12.12.12.0/24 via 192.168.10.1 (DLR to ESG transit switch) on interface called ‘To Edge’. As shown below.

Make sure you click Publish Changes to apply the configuration.

You can now see the static route configured on the DLR.

I will also need to configure a static route on my ESG to route traffic to my 172.16.0.0/16 networks which are connected to the DLR.

On the ESG I configure the following route:

Make sure you click Publish Changes to apply the configuration.

You can now see the static route configured on the ESG.

I tested pings between VMs connected to the logical switches to confirm this is functional.

Select and Configure Appropriate Dynamic Routing Protocol: OSPF/BGP/IS-IS

Static routes need to be manually defined for every new network. Dynamic routing protocols offer the ability to propagate information between routing neighbors

Configure OSPF

Open Shortest Path First aka OSPF is an interior link-state routing protocol, probably used mostly in data centers. OSPF has to establish a relationship with neighbours before they share information.

OSPF uses the concept of an Area ID. At a very high-level, routers must be configured with the same Area ID for them to become OSPF neighbours and share routes.

In NSX you can configure OSPF dynamic routing on Distributed Logical Routers (DLRs) and between DLRs and Edge Services Gateways (ESG).

As a pre-req, you must make sure that a Router ID has been configured on both routing devices (in my case the DLR and the ESG). When you configure this, it is configured with the routers uplink interface.

To configure the Router ID. This is required for dynamic routing

The process is identical for both the DLR and the ESG.

I am going to configure my DLR first.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-Click the DLR.

Click the Manage tab, then Routing, then Global Configuration.

Under Dynamic Routing Configuration click the Edit button.

The Router ID field should auto-configure with the Uplink.

Make sure you click Publish Changes to apply the configuration.

The DLR Router ID is configured.

I repeat the same change on the ESG.

On your DLR, click Manage, Routing, and then OSPF.

Click the Edit button, Enable OSPF, enter the Forwarding and Protocol Address.

The Forwarding Address is the interface the traffic is sent.

The Protocol Address is an address on the Forwarding Address network. It is used to form partnerships with neighbours.

Under Area Definitions, delete NSSA Area 51.

Click the green + sign and add your specific Area ID. The Area ID can either be an IP address or decimal. I have added Area zero (0).

Under the Area to Interface Mapping, click the green + sign and add the vNic/interface that wil be used for OSPF.

Select the interface for OSPF (will be the uplink on a DLR).

Configure your Area (Area zero for me) and other settings. These settings must be same on both routers for them to be OSPF neighbours.

Click on Publish Changes to apply the configuration.

I then configure the Edge Services Gateway for OSPF with very similar steps.

Below configuration of ESG. The configuration must match the DLR, the only difference being on the ESG you use an internal interface for OSPF (uplink interface on DLR).

Route Redistribution: On both the DLR and the ESG make sure OSPF Route Redistribution is enabled (click EDIT – OSPF and enable) and also the Route Redistribution table is configured to permit OSPF from Connected networks.

Below shows the configuration on the ESG:

Make sure the DLR has a Default Route configured.

I tested pings between VMs connected to the logical switches to confirm OSPF is functional.

Configure BGP

BGP a.k.a Border Gateway Protocol is a layer 4 dynamic routing protocol. BGP is widely used on the Internet.

Routers configured with BGP exchange information with peers that are configured with an Autonomous System (AS) number. Autonomous Systems connect and aggregate different networks

The local router is configured with a Local AS number and the neighbour Remote AS number.

BGP normally operates at your perimeter . When using BGP externally its called eBGP, and internally iBGP. It is designed for scale and can have slower convergence times.

Both the Edge Distributed Logical Router (DLR) and the Edge Services Gateway (ESG) support the BGP routing protocol.

To Configure BGP on an Edge Distributed Logical Router (DLR)

Note: Each router must have a Router ID configured. See the OSPF section above to configure.

The configuration process is identical on both the DLR and the ESG.

I am going to configure my DLR first.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-Click the DLR.

Click the Manage tab, then Routing, then BGP.

Click Edit, select Enable BGP and Enable Default Originate. Enter the Local AS number (in my case I enter 64698).

Click the green + sign under Neighbours to Add a neighbour router.

The IP Address is the neighbour router.

The Forwarding and Protocol Addresses are the same concept as in OSPF (see OSPF section above).

Add the neighbour Remote AS number.

Click Publish Changes to apply configuration. DLR configuration as below.

Repeat exactly the same process on the Edge Services Gateway (ESG), just make sure you configure the Local and Remote AS numbers correctly. The ESG Neighbour IP Address is the Protocol Address configured on the DLR, NOT the IP address of the DLR. Make sure the ESG has a Router ID.

ESG as below.

I tested pings between VMs connected to the logical switches to confirm BGP is functional.

Configure IS-IS

The IS-IS dynamic routing protocol is only supported on the Edge Services Gateway (ESG), not the DLR.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-Click the ESG.

Note: Each router must have a Router ID configured. See the OSPF section above to configure.

Click the Manage tab, then Routing, then IS-IS. Click Edit.

Select Enable IS-IS.

Select the IS Type.

Enter the Domain Password and Area Password.

Click the Areas: Edit button. Enter the areas.

Click the Interface Mapping: Edit button. Select the Interface and Circuit Type.

Click Publish Changes to apply configuration.

Configure Route Redistribution to Support a Multi-Protocol Environment

Routers share routing information that run the same routing protocol. If more than one protocol is in use then Route Redistribution must be configured.

Log into the vSphere Web Client.

Click Networking and Security, then NSX Edges.

Double-click either the ESG or DLR.

Click Manage, then Routing then Route Redistribution.

Click the Edit button

Select the Routing Protocols in use.

Click the green + sign under IP Prefixes: and add the Name and IP/Network.

Select the Learner Protocol: which chooses the protocol to learn routes from other protocols.

Under Allow Learning From: select the protocols from where routes will be learnt.

Click Publish Changes to apply the configuration.

That’s all for this blog. I read a huge amount of content to create this blog post and I learnt an amazing amount of finer detail. I recommend reading the VMware NSX Install and Admin guide around the stuff covered here. Additionally, I have linked two excellent VMware routing videos that are available from YouTube.

Video 1: Distributed Routing in a VMware NSX Environment

Video 2: Routing in the VMware NSX Edge Services Gateway (ESG)

Part 8 will be based on Section 3 – Deploy and Manage VMware NSX Network Services. Objective 3.1 – Configure and Manage Logical Load Balancing.

Thanks for reading. Please share!