AWS Transit Gateway and VMware Cloud on AWS
Gilles Chekroun
Lead VMware Cloud on AWS Specialist
---
With the release of AWS transit Gateway, connecting VPCs to VMware Cloud on AWS became much more easy.
This blog post will go through the detailed setup for the TGW and Firewall rules on VMC.
I am assuming that the reader is familiar with that deployment and will have an SDDC ready.
The goal of the AWS Transit Gateway is to allow easy, scalable and performant connectivity between multiple VPCs.
Our lab setup will be something like the schema below.
On the side AWS side, I deployed two VPCs (VPC2 and VPC3) with CIDR of 172.100.0.0/16 and 172.101.0.0/16.
In each one I have a small EC2 instance (.159 on VPC2 and .174 on VPC3).
The AWS Transit Gateway connects the two VPC but also the VMC side over VPN.
The fist one is over VPN and we will see that in a minute. The other two are VPC attachments.
Verify EC2 connectivity from VPC2 to VPC3 and back
We can leave all "Tunnel Options" empty and download the VPN configuration for the VMC side.
Config file important parameters:
By default, AWS creates 2 tunnels. The Parameter for the first tunnel are in the picture above and similarly for the second tunnel.
Check connectivity from DSL02 to EC2 in VPC2 and back
Check connectivity from DSL02 to EC2 in VPC3 and back
Ping from EC2 in VPC3 to vCenter
I removed that rule on the previous tests so all communication is open.
We can now put it back and selectively open specific flows of communications
Remember that FW rule is one way - we need the reverse way too.
Lead VMware Cloud on AWS Specialist
---
With the release of AWS transit Gateway, connecting VPCs to VMware Cloud on AWS became much more easy.
This blog post will go through the detailed setup for the TGW and Firewall rules on VMC.
NSX-T SDDC and the Transit Gateway
Since November 2018, all new SDDCs deployed will be NSX-T based.I am assuming that the reader is familiar with that deployment and will have an SDDC ready.
The goal of the AWS Transit Gateway is to allow easy, scalable and performant connectivity between multiple VPCs.
Our lab setup will be something like the schema below.
Lab Setup
On VMware Cloud on AWS side we have two Logical segments (192.168.1.0/24 and .2.0/24). On segment 1, a small Linux machine (.9) and ubuntu machine (.17). On segment 2 just a small Linux machine (.2) so we can do ping tests.On the side AWS side, I deployed two VPCs (VPC2 and VPC3) with CIDR of 172.100.0.0/16 and 172.101.0.0/16.
In each one I have a small EC2 instance (.159 on VPC2 and .174 on VPC3).
The AWS Transit Gateway connects the two VPC but also the VMC side over VPN.
Transit Gateway Deployment
Let's go to AWS console in Frankfurt and start to deploy the TGW:Create Attachments
Verify AttachmentsThe fist one is over VPN and we will see that in a minute. The other two are VPC attachments.
Update Route Tables in VPCs with TGW
Although the TGW is learning all routes, it doesn't update the route tables in the VPC attached and we need to do that manually. That gives some level of control over the connectivity as well.Verify EC2 connectivity from VPC2 to VPC3 and back
Setup VPN Attachment
On the AWS console, create a VPN Attachment on the Transit GatewayWe can leave all "Tunnel Options" empty and download the VPN configuration for the VMC side.
Config file important parameters:
By default, AWS creates 2 tunnels. The Parameter for the first tunnel are in the picture above and similarly for the second tunnel.
Configure VPN on VMC side
Check BGP routes learned in the TGW routing tableCheck connectivity from DSL02 to EC2 in VPC2 and back
Check connectivity from DSL02 to EC2 in VPC3 and back
Ping from EC2 in VPC3 to vCenter
Traffic Engineering
The default Firewall rule for VPN in VMC is "Drop".I removed that rule on the previous tests so all communication is open.
We can now put it back and selectively open specific flows of communications
Create Groups by IP addresses
Allow "segment1" to VPC2 only
Similarly, add multiple FW rules to allow specific traffic to specific locationsRemember that FW rule is one way - we need the reverse way too.
thanks for sharing informative information.
ReplyDeleteDear Gilles,
ReplyDeleteI would like to thank you for this tutorial.
I'm trying to do the same steps as you explained here. I'm having issues while the configuration VPN on VMC side. I downloaded the config file from AWS, I did exactly the steps you mentioned. However, I always have the first tunnel's IPSEC IS UP and the status is down in AWS, and in VMC the status is up and BGP status is in Progress. The second tunnel is always down in AWS as well as in VMC with the following error:
([Routing] Subnet should not overlap with other logical router port of same logical router. Subnet [169.254.13.168/30] overlaps with logical router port(s) [LRPort/cd8ad34b-42da-4483-9d91-4556d35b5199].Found errors in the request. Please refer to the related errors for details.
[Routing] Subnet should not overlap with other logical router port of same logical router. Subnet [169.254.13.168/30] overlaps with logical router port(s) )
Could you please help?
Make sure the VTI firewall rule is open. Default is drop.
DeleteMake sure the ASN on VMC match the one defined in AWS. Email me for more details and Thanks for the comment
base on your diagram when you work with transit gateway Is the connection using ENI is unnecessary ?
ReplyDeleteToday VMC connectivity to TGW is only with route based IPSec VPN. Although you can setup multiple tunnels and use ECMP to aggregate, you cannot really reach the 25 GBPs of the ENI.
Deletethank you very much for the quick answer
ReplyDeleteThanks! AMAZING...
ReplyDeleteI thought you access your Native AWS services across the ENI's? Wouldn't you incur egress charges across the VPN tunnel to the TGW?
ReplyDeleteYes it’s correct. The idea here was to connect to TGW with VPN. Yes you will pay charges for TGW process and VPN as well.
DeleteThanks for the response. I'm trying to design our VMC environment and I want to connect my SDDC to an existing VPC with native AWS services in it. There is already a TGW connected to this VPC with a VPN connection to our on-prem environment.
DeleteYou can use this VPC as the connected VPC for your SDDC. You will be able to have VMs in your SDDC accessing AWS services in that VPC via the ENI. There is no transitive routing so if you need VMs to access your on-prem we will have to do a different design. Email me : gchekroun at VMware dot com Depending on your timeframe, we have new designs that will be helpful
DeleteHello Gilles! I just read this article and asked me if the VPN connection to the TGW is mandatory? We want to create our VMC and I want to use an existing VPC as connected VPC for my SDDC. On the other hand I have a TGW and it would be great if there is some possibility to use the VPC attachments versus the TGW? So the connection speed is not limited to 1.25Gbps (I know we can create an ECMP multi link). What do you think? :-) Thanks & Regards
ReplyDelete@zwergi - Today the only way to connect to a TGW is VPN. Very soon (weeks) we will bring a new Networking concept called SDDC groups with our own managed TGW. You will be then able to connect your SDDC as VPC attachment to the VMware TGW.
DeleteVPN supports 4 tunnels and with ECMP that will give you about 5Gbps.
Cheers
Hi Gilles, just gone through this article. Very well explained. Thanks!
ReplyDelete