Use Python and VMware Cloud on AWS APIs to create logs in a Slack Channel

-
Gilles Chekroun
Lead NSX Systems Engineer - VMware Europe

--- In this article, I will describe how to use a simple AWS Lambda function to create a logging mechanism using a Slack channel and report the status of my VMware Cloud on AWS SDDC Status every 1 min.
For that, I will use a simple Python function that will call the necessary APIs in VMC and build the output for the Slack channel.
I will also use a CRON job (every 1 mins) to trigger the Lambda function.

Install Python

You can install python 3.6 for Windows here or for Mac here.
In Mac OS X, Python is there by default. My version is:
$ python --version
Python 2.7.10
I really want to use version 3.6.4. For that i will install "virtualenv" to create a virtual environment and install the proper release there without interferences to my Mac setup.
$ virtualenv ~/code/vmc-api --no-site-packages
$ brew install python3
$ virtualenv --python=/usr/local/bin/python3 vmc-api
Once done, you can activate the new environment by doing:
$ cd code/vmc-api/bin
$ source activate
and now check the new version:
(vmc-api) $ which python
/Users/gilleschekroun/code/vmc-api/bin/python
(vmc-api) $ python --version
Python 3.6.4
Note the (vmc-api) prompt, telling you that you are working in a virtual environment.

The "SKYSCRAPER" Python script

Built by Matt Dreyer and demo'd at AWS Re:invent 2017, this script is very easy to install and to demo. No knowledge of Python is required.

 The script provides data about the VMC SDDC through API Calls.
You can download it from here.
To use that script, we will need a couple of variables from our VMC environment.

1- Refresh Token

Click on OAuth Refresh Token.
Make a note of your Refresh Token, we will need it later.

2 - Org ID and SDDC ID

In the "Support" TAB of your SDDC, make a note of the Org ID and the SDDC ID.
In the same folder where you downloaded the script, create two new files:
access-token.txt
Paste the OAuth Refresh Token  and save. The file should only contain the token.
default-org.txt
Paste the Org ID and save. The file should only contain the Org ID.
Assumptions: 
  • pip and python have already been installed.
  • Use homebrew if pip is not available.
  • You have access to a VMC Org, VMC SDDC and have a Refresh Token.

3 - Install packages to import

Open a Terminal Window and navigate to folder where Python script was downloaded and install the following packages:
pip install requests -t . --upgrade
pip install simplejson -t . --upgrade
pip install certifi -t . --upgrade
pip install prettytable -t . --upgrade
pip install colorama -t . --upgrade
Run some scripts:

    python skyscraper.py show-sddcs

    python skyscraper.py show-users

    python skyscraper.py show-orgs (yes, I am part of 3 Orgs)

AWS Lambda, Slack and VMC integration

The Lambda script leverages a combination of AWS CloudWatch Events and AWS Lambda to capture the state of the SDDC and posts the result on a Slack channel as logging mechanism. 

Create a Slack WebHook

During this step, we will configure a Slack webhook.
Incoming Webhooks are a simple way to post messages from external sources into Slack.

1 - Log on to Slack and Create a Channel
Set up the Slack Incoming Webhook here.
Once done, select the channel you want to push the notifications to and you will get the WebHook URL:

Download the Lambda Function and edit some fields

The Python Lambda function is  here:
Make sure it’s renamed to lambda_function.py
Edit the following fields in: 
  • sddcID = "XXXX-XXXXX-XXXXX-XXXX-XXXX-XXXX"
  • tenantID = "YYYY-YYYYY-YYYYYY-YYYYY-YYYYY”
  • strAccessKey = "XXXX-XXXXX-XXXXX-XXXX-XXXX-XXXX"
  • slackURL = https://hooks.slack.com/services/TXXXXXXXX/BXXXXXX/XXXXXXXXXXXXX
tenantID is your OrgID.
strAccessKey is your OAuth Refresh Token. 
SlackURL should be your Slack URL WebHook.

Don't forget to rename to lambda_function.py! This is how AWS Lambda will execute our code.

Open a Terminal Window and navigate to the folder where the Lambda function was downloaded and install the following packages. The packages will be created in the local directory.
pip install requests -t . --upgrade
pip install simplejson -t . --upgrade
pip install certifi -t . --upgrade
pip install pyvim -t . --upgrade
pip install datetime -t . –upgrade
Zip the Python file and newly created directories into an archive (Archive.zip)

Create a new Lambda Function

Log in to AWS console. Go to ”Compute” / “Lambda” / ”Create function”.

Create CloudWatch CRON Rule 

Select "CloudWatch" / "Rules" and create a new rule.

Create an IAM Lambda Role

Go to ”Security” / “IAM” / ”Roles” and create a new role.
Click Next: Permissions. Attach the following “AWSLambdaBasicExecutionRole” policy. Choose a name for the role and create the role. You will need it during the next step.

Go back to the Lambda function definition and configure the trigger (our CRON job)
Save 
Click on the "SlackChannelTest" in the center and choose "Upload a .ZIP file". Get the previously saved Archive.zip and save.

Watch the results in your Slack Channel

In the CRON job we used 1 min interval (just for testing) so you should get a Slack Post every minute
ENJOY !!

Comments

Post a Comment

Populars

Egress VPC and AWS Transit Gateway (Part1)

-
Image
Gilles Chekroun
 Lead VMware Cloud on AWS Solutions Architect --- Usually my blog posts are customer driven and recently I have been working on a design that would include an Egress VPC and AWS Transit Gateway. This customer is going to use both VMware Managed Transit Gateway and also AWS Transit Gateway. I will split this post in 3 parts: The Egress VPC - this article Adding a VMC SDDC to the Egress VPC here Adding VMware Managed Transit Gateway here Why do we need an Egress VPC? Numerous posts on AWS site  will describe how to build an Egress VPC and the subtleties of the various route tables of the TGW and the Egress VPC itself. The main goal is to have ONE Internet gateway only  that will allow workloads to go out to internet on the Egress VPC. One of the most important point is redundancy and multi-availability zones. Applications usually reside in private subnets, while NAT Gateways reside in a public subnet. NAT Gateways To focus the Internet access to a single point, we can cre
Read more

AWS Transitive routing with Transit Gateways in the same region

-
Image
Gilles Chekroun Lead VMware Cloud on AWS Specialist --- At the AWS Re:invent 2019 conference, the long waited TGW peering functionality was announced and available in a few AWS regions. This is an INTER-REGION peering only meaning that the Transit Gateways need to be in different regions. Hoping that AWS will soon release an INTRA-REGION capability I discussed with a few AWS Solutions Architects in Las Vegas, among them Tom Adamski , about using a VPC as a bridge between two TGWs in the same region. Tom assured me that it is possible opening for the first time the transitive routing capability in AWS networking. Well . . . I needed to test that and see by myself. Test bed For a simple test, I will use 2 TGWs in the same regions with 2 VPCs attached each and another VPC as "bridge" connected to both TGWs. Yes you can connect a VPC up to 5 TGWs . The whole idea is to use this "bridging VPC" and point the default route of the TGWs to it. To do that I c
Read more

Considerations on vTGW to TGW Peering Link

-
Image
   Gilles Chekroun
 Lead VMware Cloud on AWS Solutions Architect --- Recently I was talking with a Customer on designing their VMware Cloud on AWS environment and linking it to their existing AWS infrastructure. The idea was to have 2 SDDCs, Test and Prod, and link them with a VMware Managed TGW and peer it to their existing TGW. The customer requirements were: VMs in each SDDC should have internet access from the SDDC out VMs from Test should be able to talk to VMs in Prod VMs in any SDDC should be able to access their own vCenter but also the other one Some VMs any SDDC should have a Public IP and NAT rule to be accessible from outside VMware traffic should stay within VMware SDDCs Other traffic, like on-prem, should go via the Customer TGW Test Setup Step1 - Create SDDC Group Create an SDDC Group and attach the 2 SDDCs Prod and Test Make sure the proper FireWall rules are open in each SDDC Compute GW. Step 2 - Attach the Customer TGW In the SDDC Groups, under External TGW TAB, add t
Read more