Use Python and VMware Cloud on AWS APIs to create logs in a Slack Channel

-
Gilles Chekroun
Lead NSX Systems Engineer - VMware Europe

--- In this article, I will describe how to use a simple AWS Lambda function to create a logging mechanism using a Slack channel and report the status of my VMware Cloud on AWS SDDC Status every 1 min.
For that, I will use a simple Python function that will call the necessary APIs in VMC and build the output for the Slack channel.
I will also use a CRON job (every 1 mins) to trigger the Lambda function.

Install Python

You can install python 3.6 for Windows here or for Mac here.
In Mac OS X, Python is there by default. My version is:
$ python --version
Python 2.7.10
I really want to use version 3.6.4. For that i will install "virtualenv" to create a virtual environment and install the proper release there without interferences to my Mac setup.
$ virtualenv ~/code/vmc-api --no-site-packages
$ brew install python3
$ virtualenv --python=/usr/local/bin/python3 vmc-api
Once done, you can activate the new environment by doing:
$ cd code/vmc-api/bin
$ source activate
and now check the new version:
(vmc-api) $ which python
/Users/gilleschekroun/code/vmc-api/bin/python
(vmc-api) $ python --version
Python 3.6.4
Note the (vmc-api) prompt, telling you that you are working in a virtual environment.

The "SKYSCRAPER" Python script

Built by Matt Dreyer and demo'd at AWS Re:invent 2017, this script is very easy to install and to demo. No knowledge of Python is required.

 The script provides data about the VMC SDDC through API Calls.
You can download it from here.
To use that script, we will need a couple of variables from our VMC environment.

1- Refresh Token

Click on OAuth Refresh Token.
Make a note of your Refresh Token, we will need it later.

2 - Org ID and SDDC ID

In the "Support" TAB of your SDDC, make a note of the Org ID and the SDDC ID.
In the same folder where you downloaded the script, create two new files:
access-token.txt
Paste the OAuth Refresh Token  and save. The file should only contain the token.
default-org.txt
Paste the Org ID and save. The file should only contain the Org ID.
Assumptions: 
  • pip and python have already been installed.
  • Use homebrew if pip is not available.
  • You have access to a VMC Org, VMC SDDC and have a Refresh Token.

3 - Install packages to import

Open a Terminal Window and navigate to folder where Python script was downloaded and install the following packages:
pip install requests -t . --upgrade
pip install simplejson -t . --upgrade
pip install certifi -t . --upgrade
pip install prettytable -t . --upgrade
pip install colorama -t . --upgrade
Run some scripts:

    python skyscraper.py show-sddcs

    python skyscraper.py show-users

    python skyscraper.py show-orgs (yes, I am part of 3 Orgs)

AWS Lambda, Slack and VMC integration

The Lambda script leverages a combination of AWS CloudWatch Events and AWS Lambda to capture the state of the SDDC and posts the result on a Slack channel as logging mechanism. 

Create a Slack WebHook

During this step, we will configure a Slack webhook.
Incoming Webhooks are a simple way to post messages from external sources into Slack.

1 - Log on to Slack and Create a Channel
Set up the Slack Incoming Webhook here.
Once done, select the channel you want to push the notifications to and you will get the WebHook URL:

Download the Lambda Function and edit some fields

The Python Lambda function is  here:
Make sure it’s renamed to lambda_function.py
Edit the following fields in: 
  • sddcID = "XXXX-XXXXX-XXXXX-XXXX-XXXX-XXXX"
  • tenantID = "YYYY-YYYYY-YYYYYY-YYYYY-YYYYY”
  • strAccessKey = "XXXX-XXXXX-XXXXX-XXXX-XXXX-XXXX"
  • slackURL = https://hooks.slack.com/services/TXXXXXXXX/BXXXXXX/XXXXXXXXXXXXX
tenantID is your OrgID.
strAccessKey is your OAuth Refresh Token. 
SlackURL should be your Slack URL WebHook.

Don't forget to rename to lambda_function.py! This is how AWS Lambda will execute our code.

Open a Terminal Window and navigate to the folder where the Lambda function was downloaded and install the following packages. The packages will be created in the local directory.
pip install requests -t . --upgrade
pip install simplejson -t . --upgrade
pip install certifi -t . --upgrade
pip install pyvim -t . --upgrade
pip install datetime -t . –upgrade
Zip the Python file and newly created directories into an archive (Archive.zip)

Create a new Lambda Function

Log in to AWS console. Go to ”Compute” / “Lambda” / ”Create function”.

Create CloudWatch CRON Rule 

Select "CloudWatch" / "Rules" and create a new rule.

Create an IAM Lambda Role

Go to ”Security” / “IAM” / ”Roles” and create a new role.
Click Next: Permissions. Attach the following “AWSLambdaBasicExecutionRole” policy. Choose a name for the role and create the role. You will need it during the next step.

Go back to the Lambda function definition and configure the trigger (our CRON job)
Save 
Click on the "SlackChannelTest" in the center and choose "Upload a .ZIP file". Get the previously saved Archive.zip and save.

Watch the results in your Slack Channel

In the CRON job we used 1 min interval (just for testing) so you should get a Slack Post every minute
ENJOY !!

Comments

Post a Comment

Populars

Egress VPC and AWS Transit Gateway (Part1)

-
Image
Gilles Chekroun
 Lead VMware Cloud on AWS Solutions Architect --- Usually my blog posts are customer driven and recently I have been working on a design that would include an Egress VPC and AWS Transit Gateway. This customer is going to use both VMware Managed Transit Gateway and also AWS Transit Gateway. I will split this post in 3 parts: The Egress VPC - this article Adding a VMC SDDC to the Egress VPC here Adding VMware Managed Transit Gateway here Why do we need an Egress VPC? Numerous posts on AWS site  will describe how to build an Egress VPC and the subtleties of the various route tables of the TGW and the Egress VPC itself. The main goal is to have ONE Internet gateway only  that will allow workloads to go out to internet on the Egress VPC. One of the most important point is redundancy and multi-availability zones. Applications usually reside in private subnets, while NAT Gateways reside in a public subnet. NAT Gateways To focus the Internet access to a single point, we can cre
Read more

AWS Transitive routing with Transit Gateways in the same region

-
Image
Gilles Chekroun Lead VMware Cloud on AWS Specialist --- At the AWS Re:invent 2019 conference, the long waited TGW peering functionality was announced and available in a few AWS regions. This is an INTER-REGION peering only meaning that the Transit Gateways need to be in different regions. Hoping that AWS will soon release an INTRA-REGION capability I discussed with a few AWS Solutions Architects in Las Vegas, among them Tom Adamski , about using a VPC as a bridge between two TGWs in the same region. Tom assured me that it is possible opening for the first time the transitive routing capability in AWS networking. Well . . . I needed to test that and see by myself. Test bed For a simple test, I will use 2 TGWs in the same regions with 2 VPCs attached each and another VPC as "bridge" connected to both TGWs. Yes you can connect a VPC up to 5 TGWs . The whole idea is to use this "bridging VPC" and point the default route of the TGWs to it. To do that I c
Read more

Build a VMware Cloud on AWS Content Library using AWS S3

-
Image
Gilles Chekroun Lead NSX Systems Engineer - VMware Europe --- What is a Content Library? DISCLAIMER: External content libraries are not supported by VMware GSS ! vSphere Content Library is a repository where you can store VM templates, vApp templates and other kind of files like ISO.  Administrators can use the templates to deploy VMs and vApps. Each item library can contain multiple files. The basis is  .ovf   together with the description files like .mf and .vmdk If you have .ova files, you can simply un-compress them using the linux command:  tar -xvf SomeFile.ova There are two kind of Libraries: Local and Subscribed. Local Library Local Library stores items in a single vCenter. VM and vApp templates are stored as OVF file format. Subscribed Library You can create a Subscribed Library in the same vCenter and to ensure the content is up-to-date, the library can automatically sync with the source on a regular basis. You can also use the option to download co
Read more