Considerations on vTGW to TGW Peering Link

-

  Gilles Chekroun


Lead VMware Cloud on AWS Solutions Architect
---

Recently I was talking with a Customer on designing their VMware Cloud on AWS environment and linking it to their existing AWS infrastructure.
The idea was to have 2 SDDCs, Test and Prod, and link them with a VMware Managed TGW and peer it to their existing TGW.
The customer requirements were:
  • VMs in each SDDC should have internet access from the SDDC out
  • VMs from Test should be able to talk to VMs in Prod
  • VMs in any SDDC should be able to access their own vCenter but also the other one
  • Some VMs any SDDC should have a Public IP and NAT rule to be accessible from outside
  • VMware traffic should stay within VMware SDDCs
  • Other traffic, like on-prem, should go via the Customer TGW

Test Setup

Step1 - Create SDDC Group

Create an SDDC Group and attach the 2 SDDCs Prod and Test
Make sure the proper FireWall rules are open in each SDDC Compute GW.

Step 2 - Attach the Customer TGW

In the SDDC Groups, under External TGW TAB, add the details from the customer side:
  • AWS Account
  • TGW ID
  • TGW Region
  • SDDC Region
  • Peer Link prefixes (up to 250)
Note the RFC 1918 prefixes on the Peering Link

Step 3 - Accept the peering attachment in AWS console

Accept the peering attachment and wait until "AVAILABLE" status is shown.
Verify the connectivity under SDDC Groups in VMC console

Note that it can take 10-15 mins until "CONNECTED" status

Step 4 - Check the routing tables

In the SDDC Groups, look for the Routing Tab.
Check also each SDDC Learned routes in the "Networking & Security" "Transit Connect" area.

Step 5 - Time for some Ping tests

VMs in SDDC_Prod and SDDC_Test can :
  • ping each other
  • ping the respective vCenters
  • ping external IP like amazon.com
  • use Public IP to SSH or HTTP/HTTPS
Prod Virtual Machine IP : 10.123.160.2
Test Virtual Machine IP  : 10.123.192.5

From Prod Virtual Machine:
Ping SDDC_Test VM and amazon.com
Ping SDDC_Test vCenter
Request a public IP and create a NAT rule
Verify access from outside

What about a 0.0.0.0/0 on the Peering Link?

Having RFC 1918 address on the peering link is great and covers about 2.3 Billion addresses.
Since we have all static routes entries in the routing tables, the most specific entries will take precedence.

If we code a 0.0.0.0/0 on the peering link, many things happen:
  • VMs in any SDDC can still communicate
  • VMs in any SDDC can still access the vCenters
  • VMs in any SDDC can NOT ping external IPs like amazon.com
  • Accessing VMs from outside with Public IPs will not be allowed
Basically anything that is NOT SDDC prefixes will be sent to the customer TGW.
Usually that is what customers wants and they will have the ability to control internet access on an Egress VPC and/or inspect traffic using a NGFW product.
vTWG route table

Conclusion

According to what the Customer wants to achieve it is very important to plan the Peering Link prefixes properly.
Today VMware Managed TGW has a soft limit of 250 prefixes.
All routes are static and even if the Customer TGW has a route table populated with all their prefixes, there is no protocol on the Peering Link that will allow these routes to be transferred to the vTGW.
It's been over 3 years that AWS is eluding about supporting BGP on the peering link and even strongly suggest to use different Autonomous System Numbers for each TGW, but until now, it's still static routes.

AWS. . . are you here???

Comments

  1. AnonymousOctober 14, 2022 at 11:54 PM

    Hi, just wondering if vTGW supports VPN attachment now?

    ReplyDelete

Post a Comment

Populars

Egress VPC and AWS Transit Gateway (Part1)

-
Image
Gilles Chekroun
 Lead VMware Cloud on AWS Solutions Architect --- Usually my blog posts are customer driven and recently I have been working on a design that would include an Egress VPC and AWS Transit Gateway. This customer is going to use both VMware Managed Transit Gateway and also AWS Transit Gateway. I will split this post in 3 parts: The Egress VPC - this article Adding a VMC SDDC to the Egress VPC here Adding VMware Managed Transit Gateway here Why do we need an Egress VPC? Numerous posts on AWS site  will describe how to build an Egress VPC and the subtleties of the various route tables of the TGW and the Egress VPC itself. The main goal is to have ONE Internet gateway only  that will allow workloads to go out to internet on the Egress VPC. One of the most important point is redundancy and multi-availability zones. Applications usually reside in private subnets, while NAT Gateways reside in a public subnet. NAT Gateways To focus the Internet access to a single point, we can cre
Read more

AWS Transitive routing with Transit Gateways in the same region

-
Image
Gilles Chekroun Lead VMware Cloud on AWS Specialist --- At the AWS Re:invent 2019 conference, the long waited TGW peering functionality was announced and available in a few AWS regions. This is an INTER-REGION peering only meaning that the Transit Gateways need to be in different regions. Hoping that AWS will soon release an INTRA-REGION capability I discussed with a few AWS Solutions Architects in Las Vegas, among them Tom Adamski , about using a VPC as a bridge between two TGWs in the same region. Tom assured me that it is possible opening for the first time the transitive routing capability in AWS networking. Well . . . I needed to test that and see by myself. Test bed For a simple test, I will use 2 TGWs in the same regions with 2 VPCs attached each and another VPC as "bridge" connected to both TGWs. Yes you can connect a VPC up to 5 TGWs . The whole idea is to use this "bridging VPC" and point the default route of the TGWs to it. To do that I c
Read more

Costs analysis for Data Transfer via VMware Managed TGW

-
Image
Gilles Chekroun
 Lead VMware Cloud on AWS Solutions Architect --- Every customer design is different but all of them should include a costs analysis specifically when using VMware managed Transit Gateway or AWS Transit Gateway. The AWS page here has very useful information for us to be able to understand costs. Transit Gateway Costs TGW pricing is split in 2 components: A fixed price for infrastructure connectivity per hour. Depending on the attachments, various account owners are charged.  A variable price for processing data via the TGW. Price is per GB. The sending account is charged. Prices depend on regions and range from $0.05 to $0.09 per attachment per hour. Who is charged? See AWS page VPC attachments The VPC account owner is charged and is billed hourly. VPN attachments The TGW account owner is billed hourly. Site-to-Site VPN connection pricing still applies in addition to the VPN TGW attachments. Direct Connect Gateway attachments The DXGW account owner is billed hourly.
Read more