Infrastructure as Code : Build your VMware NSX Cloud lab in AWS with Terraform

Gilles Chekroun
Lead NSX Systems Engineer - VMware Europe
---



Why using Code for Infrastructure?

Infrastructure as code is a way to define Compute, Network and Security infrastructure through source code.
Such code can be kept in source controlled environment to allow deployment of tested infrastructure avoiding lots of human mistakes.
It is becoming a very strong trend in our industry for pushing Network, Compute and Security infrastructure to cloud providers like Amazon Web Services.

This article describes the use of Terraform to build the infrastructure for my VMware NSX Cloud lab in AWS.

What is Terraform?

Terraform enables you to create, change, and improve infrastructure. It is an open source tool from Hashicorp that codifies APIs into declarative configuration files.
The idea for my lab is to build the AWS infrastructure with Terraform and configure it with Ansible. I am using VMware NSX Cloud as an example of how we can create the proper AWS VPCs, rules, gateways, load balancer, 3Tier application using native AWS RDS and then configure the servers/database with Ansible.

What is the compelling event for doing that?

When you log in AWS console, you will see so many options and you realize that a lot of "learning" has to be done.

Soon you start your first EC2 instance. Nice!! 
But then you realize that you need much more . . .

• Autoscaling groups 
• Different Availability Zones
• Load Balancer
• Database with Master/Replica/StandBy
• Internet Access through an Internet Gateway
• Access to S3
• Monitoring
• DNS route etc...

And of course, you want three different environments: Dev-Test ; Staging ; Production

. . . It's becoming complicated !!

How to Deploy and Manage Infrastructure Simply

Terraform will help us to build code to deploy and manage our infrastructure.
It describes end-state in a high-level syntax. The code is HCL (HashiCorp Language - fully JSON compatible)
Terraform creates execution plans. There is a planning step before generating an execution plan according to what is already running. There is a possibility to create a graph of all resources and this is built as a dependency map.

Terraform keeps a STATE of what is already deployed and applies only changes to the infrastructure.

Special files

Secret variables in “terraform.tfvars”

This file is one single place for sensitives variables like AWS Credentials, Key_pair, Data Base Credentials etc…

Subnets ranges defined in “ variables.tf ”

This file is also one single place for all variables definitions like Compute1 and Compute2 subnets, RDS subnets etc.

HCL code is very easy to read and understand.

Terraform Directory Structure

Run terraform from main directory.

• All variables in variables.tf
• Secret variables like AWS credentials in terraform.tfvars

Commands:

• terraform init (initialize aws provider)
• terraform get (read all modules)
• terraform plan (checks against AWS resources)
• terraform apply (creates or destroys resources)
• terraform output (shows output parameters for Ansible use)

To destroy the lab

• terraform destroy 

Code example for tagging EC2 Network interfaces

Output parameters

Each module can have output variables that other modules can use. I also created a main “output.tf” that gathers all outputs needed by Ansible 

• IP addresses
• RDS endpoint name
• compute VPC ID
• subnets IDs
• vRNI credentials etc. . . 

A nice way to pass these parameters is to convert the output file to JSON with the simple command:

terraform output -json > output.json

and include this file as variable for Ansible playbooks.

Ansible Directory Structure

Edit credentials.yaml

• read the file comments and run aws configure command line.

Check ~/aws/credentials file then run the Ansible playbook called deploy-lab.yaml

    ansible-playbook deploy-lab.yaml 

You can add –v or –vv or –vvv for different debugging levels.

Ansible Playbooks

The main play-book is deploy-lab.yaml. It contains the files for other modules.

Terraform Apply

After a few minutes,  Terraform code is applied in AWS and we get the output variables.

If an output is sensitive, it can be hidden like DBpass for example.

This output will be converted to JSON format for using in Ansible.

Note the 49 resources added.

Created Infrastructure

Run Ansible main Playbook.

Once the infrastructure is deployed, we can run our Ansible playbook to configure it.

The main playbook will pause so you can deploy Public Cloud Gateway for NSX before installing hosts agents.

NSX Configuration

On the NSX Cloud dashboard, select NSX Manager.

Select "Switching" / "Ports" and check that all our instances are NSX managed.

Check our Elastic Load Balancer . . .

. . . and verify the Web servers side.

Update the NSX DFW rules if necessary.

Destroying the lab.

Untag the Eth(0) interfaces or terminate NSX managed EC2 instances.

Un-deploy NSX Public Cloud Gateway.

Run terraform destroy -force command to wipe AWS resources.

Note the 49 resources destroyed. 

Code in GitHub here.