Creating Paramiko and Cryptography Virtual Environment to Run On AWS Lambda

A prior blog post explained how to obtain the dependencies to successfully build Paramiko and Cryptography on an AWS EC2 instance in a virtual environment. This post will show how to package up those dependencies for a Lambda function using EC2.

I've created some networking in order to automate deployment of a WatchGuard Firebox Cloud which I am using for my EC2 instance below. If you are unsure how to set up the networking that securely allows Internet access you could run the CloudFormation templates in my FireboxCloudAutomation Github repo and use that networking to complete the steps below. Stay tuned for more networking information here and on Secplicity. It is highly recommended that you strictly limit any SSH access to instances in your VPC and ideally remove that access over the network when not in use. You can also create a bastion host.

For now I will manually create an EC2 instance. Might automate this later. I am not deploying production systems here simply testing. I would automate all of this if actually using it in a production environment.

First instantiate the EC2 instance.

Choose the AWS Linux AMI which has the necessary dependencies matching what is on a Lambda function.

Choose your instance type. Smallest is probably fine.

Configure networking - this is where I am using the networking I created for the WatchGuard Firebox Cloud as noted above so I will have SSH access to my new instance without having wide open networking. Choose the Firebox VPC, the Public Subnet which allows Internet access, and auto-assign an IP. Can't connect without that.

Tag your instance with a name which is helpful for finding it in the console.

Create or use a restrictive SSH security group. The ONLY port we need open for this is SSH port 22 and I only need to be able to access it from My IP address as selected below. Then in theory the only way someone could get to this instance would be to get onto my network (which could be done of course, but we are limiting the attack vector as much as possible). Also I haven't thoroughly reviewed these software packages. If for some reason they had some malware that would reach out to a C2 server, it wouldn't be able to reach that server due to my network rules so I feel a bit safer with this configuration.

Select a key that will be used to SSH into this instance. KEYS ARE PASSWORDS. Protect them.

Wait for the EC2 instance status checks to pass and the indicator below to turn green.

Right click on the instance to get the command to connect to the instance. 

Follow the instructions to connect to the instance. If having problems read this blog post on connecting to EC2 instances using SSH.

Once connected to EC2 instance in terminal:

Run the commands from the my post that explains how to build Paramiko and Cryptography.

Note that you will likely want to use Python 2.7 due to inconsistencies between EC2 (Python 3.4 or 3.5) and Lambda function (Python 3.6). You can probably make it work but this will get you up and running faster:

sudo yum update -y

sudo pip install virtualenv --upgrade

cd /tmp

virtualenv -p /usr/bin/python2.7 python27

source python27/bin/activate

sudo yum install gcc -y

#probably don't need these but just in case libs are missing

#sudo yum install python27-devel.x86_64 -y

#sudo yum install openssl-devel.x86_64 -y

#sudo yum install libffi-devel.x86_64 -y

pip install --upgrade pip

pip install paramiko

Commands to create virtual environment also found here with comments:

https://github.com/tradichel/FireboxCloudAutomation/blob/master/code/execute/paramiko.sh

Following these directions I zip up the files in the lib/python2.7/site-packages dir

http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

Zip the files as explained on this page:

http://docs.aws.amazon.com/lambda/latest/dg/with-s3-example-deployment-pkg.html#with-s3-example-deployment-pkg-python

#change to the site-packages directory

cd python27/lib/python2.7/site-packages

zip -r9 /tmp/lambda.zip . -x \*__pycache__\*

We also need the files in the lib64 directory:

cd ../../..

cd lib64/python2.7/site-packages

zip -g -r9 /tmp/lambda.zip . -x \*__pycache__\*

Now we should have a lambda.zip file in the tmp directory on the EC2 instance:

Now I can deactivate the virtual environment because I'm done with it.

deactivate

Run this from local machine to copy down the zip we just created to the folder where your python file lives that you want to include in the zip file for the Lambda function.

#scp username@ipaddress:pathtofile localsystempath

#change the key and IP address below

scp -i "[yourkey].pem" ec2-user@[ip-address]:/tmp/lambda.zip lambda.zip

Now I can add my own python files to the zip for use in a lambda function, again as explained on this page to run SSH commands from a lambda function:

http://docs.aws.amazon.com/lambda/latest/dg/with-s3-example-deployment-pkg.html#with-s3-example-deployment-pkg-python

Heres the code I use to add my fireboxconfig.py to the lambda.zip file I downloaded. I actually copy the lambda.zip to a new file, add my python file and upload it to an S3 bucket so the zip file can be used in my Lambda CloudFormation Template.

For more on that check out my next blog post where I'll explain how I use it in the rest of the code on GitHub to automate configuration of a WatchGuard Firebox Cloud.  I'm using the Lambda function with Paramiko to connect to a WatchGuard Firebox Cloud and configure via the Firebox CLI (command line interface). For more on the WatchGuard CLI check out the latest version of the WatchGuard Firebox Cloud CLI Commands.

Questions or Suggestions - DM me on twitter @teriradichel