AWS Transit Gateway and VMware Cloud on AWS

Gilles Chekroun
Lead VMware Cloud on AWS Specialist
---

With the release of AWS transit Gateway, connecting VPCs to VMware Cloud on AWS became much more easy.
This blog post will go through the detailed setup for the TGW and Firewall rules on VMC.

NSX-T SDDC and the Transit Gateway

Since November 2018, all new SDDCs deployed will be NSX-T based.
I am assuming that the reader is familiar with that deployment and will have an SDDC ready.
The goal of the AWS Transit Gateway is to allow easy, scalable and performant connectivity between multiple VPCs.
Our lab setup will be something like the schema below.

Lab Setup

On VMware Cloud on AWS side we have two Logical segments (192.168.1.0/24 and .2.0/24). On segment 1, a small Linux machine (.9) and ubuntu machine (.17). On segment 2 just a small Linux machine (.2) so we can do ping tests.
On the side AWS side, I deployed two VPCs (VPC2 and VPC3) with CIDR of 172.100.0.0/16 and 172.101.0.0/16.
In each one I have a small EC2 instance (.159 on VPC2 and .174 on VPC3).
The AWS Transit Gateway connects the two VPC but also the VMC side over VPN.

Transit Gateway Deployment

Let's go to AWS console in Frankfurt and start to deploy the TGW:

Create Attachments

Verify Attachments
The fist one is over VPN and we will see that in a minute. The other two are VPC attachments.

Update Route Tables in VPCs with TGW 

Although the TGW is learning all routes, it doesn't update the route tables in the VPC attached and we need to do that manually. That gives some level of control over the connectivity as well.
Verify EC2 connectivity from VPC2 to VPC3 and back


Setup VPN Attachment

On the AWS console, create a VPN Attachment on the Transit Gateway
We can leave all "Tunnel Options" empty and download the VPN configuration for the VMC side.
Config file important parameters:
By default, AWS creates 2 tunnels. The Parameter for the first tunnel are in the picture above and similarly for the second tunnel.

Configure VPN on VMC side

Check BGP routes learned in the TGW routing table
Check connectivity from DSL02 to EC2 in VPC2 and back

Check connectivity from DSL02 to EC2 in VPC3 and back
Ping from EC2 in VPC3 to vCenter

Traffic Engineering

The default Firewall rule for VPN in VMC is "Drop".
I removed that rule on the previous tests so all communication is open.
We can now put it back and selectively open specific flows of communications

Create Groups by IP addresses

Allow "segment1" to VPC2 only

Similarly, add multiple FW rules to allow specific traffic to specific locations
Remember that FW rule is one way - we need the reverse way too.

Conclusion

The AWS Transit Gateway is really a very good way for customers that have large number of VPC and want interconnection. The previous "Transit VPC" design was very difficult to manage and to scale. We have now a beautiful solution that combines VMware Cloud on AWS and AWS Transit Gateway in a flexible and scalable design.

Comments

  1. thanks for sharing informative information.

    ReplyDelete
  2. Dear Gilles,
    I would like to thank you for this tutorial.
    I'm trying to do the same steps as you explained here. I'm having issues while the configuration VPN on VMC side. I downloaded the config file from AWS, I did exactly the steps you mentioned. However, I always have the first tunnel's IPSEC IS UP and the status is down in AWS, and in VMC the status is up and BGP status is in Progress. The second tunnel is always down in AWS as well as in VMC with the following error:

    ([Routing] Subnet should not overlap with other logical router port of same logical router. Subnet [169.254.13.168/30] overlaps with logical router port(s) [LRPort/cd8ad34b-42da-4483-9d91-4556d35b5199].Found errors in the request. Please refer to the related errors for details.
    [Routing] Subnet should not overlap with other logical router port of same logical router. Subnet [169.254.13.168/30] overlaps with logical router port(s) )

    Could you please help?

    ReplyDelete
    Replies
    1. Make sure the VTI firewall rule is open. Default is drop.
      Make sure the ASN on VMC match the one defined in AWS. Email me for more details and Thanks for the comment

      Delete
  3. base on your diagram when you work with transit gateway Is the connection using ENI is unnecessary ?

    ReplyDelete
    Replies
    1. Today VMC connectivity to TGW is only with route based IPSec VPN. Although you can setup multiple tunnels and use ECMP to aggregate, you cannot really reach the 25 GBPs of the ENI.

      Delete
  4. thank you very much for the quick answer

    ReplyDelete

Post a Comment

Populars

AWS Transit Gateway and Multiple Accounts

Understanding VMware Cloud on AWS Account Structure and Data Charges

Deploy VMware Cloud on AWS Route Based VPN with API