This blog post describes how to run SSH jobs from an AWS Lambda function:
https://aws.amazon.com/blogs/compute/scheduling-ssh-jobs-using-aws-lambda/
It seemed like that would be the solution for running SSH in an AWS Lambda function for the purposes of automating configuration of an WatchGuard Firebox Cloud.
The only issue was when attempting to run the code I realized that additional libraries are required. I started with Python 3.6 because why not? It's the most up to date version of Python on Lambda and sounds like Paramiko will work with that version. Turns out Paramiko must be packaged up in the Lambda code. That in turn requires the cryptography package which in turn uses some C libraries. Packaging this up on a mac or Windows machine would include OS specific libraries that wouldn't work in a Lambda function, which presumably runs on something like an AWS EC2 Linux instance.
Always looking for a quick fix I reached out to my friend, Google. There I found some recommendations suggesting creation of the virtual environment on an EC2 instance. However that wasn't as straightforward as one might hope. The required libraries were not all installed by default and the names of the libraries are different than the documentation and various blog posts on the topic. Basically in order to create a python virtual environment you'll need to install gcc, as well as specific versions of python-devel and openssl-devel. I describe how to find and install those libraries in a bit more detail in my previous posts.
Here's what I came up with. Looks so simple now...and by the way by the time I wrote this something changed so make sure you check what packages are available to install as noted in my recent blog posts. I also show how to use the list command below to find all the packages with python3 in the name.
#update the ec2 instance
sudo yum update -y
#see what versions of python 3 are available
sudo yum list | grep python3
#install the one we want
sudo yum install python35.x86_64 -y
#switch to temp directory
cd /tmp
#create virtual environment
virtualenv -p /usr/bin/python3.5 python35
#activate virtual environment
source python35/bin/activate
#install dependencies
sudo yum install gcc -y
sudo yum install python35-devel.x86_64 -y
sudo yum install openssl-devel.x86_64 -y
sudo yum install libffi-devel.x86_64 -y
#install cryptography and parmaiko paramiko
pip install cryptography
pip install paramiko
And finally - it works.
Successfully installed asn1crypto-0.22.0 cffi-1.10.0 cryptography-1.8.1 idna-2.5 packaging-16.8 paramiko-2.1.2 pyasn1-0.2.3 six-1.10.0
Great but guess what. Tried running this on Lambda and get missing library errors.
Digging further I figured out how to find out what versions of Python are available on Lambda by using this blog post:
https://www.linkedin.com/pulse/running-python-3-aws-lambda-lyndon-swan
Ran this code in my lambda function:
args = ("whereis","python3")
popen = subprocess.Popen(args, stdout=subprocess.PIPE)
popen.wait()
output = popen.stdout.read()
print(output)
Looks like only Python 3.4 and Python 3.6 are available and all of the above is based on 3.5.
b'python3: /usr/bin/python3 /usr/bin/python3.4m /usr/bin/python3.4 /usr/lib/python3.4 /usr/lib64/python3.4 /usr/local/lib/python3.4 /usr/include/python3.4m /var/lang/bin/python3.6m /var/lang/bin/python3.6-config /var/lang/bin/python3 /var/lang/bin/python3.6m-config /var/lang/bin/python3.6 /usr/share/man/man1/python3.1.gz'
Options would be go back to 2.7 or try to use 3.4 since 3.6 doesn't appear to be available on EC2 instances. *sigh*. Let's see if we can build a 3.4 virtual environment.
#see what versions of python 3 are available on EC2 instance
sudo yum list | grep python3
#output gives us python34.x86_64
#install the one we want
sudo yum install python34.x86_64 -y
#create virtual environment
virtualenv -p /usr/bin/python3.4 python34
#activate virtual environment
source python34/bin/activate
#install dependencies
sudo yum install gcc -y
sudo yum install python34-devel.x86_64 -y
sudo yum install openssl-devel.x86_64 -y
sudo yum install libffi-devel.x86_64 -y
#install cryptography and parmaiko paramiko
pip install paramiko
Installing collected packages: pyasn1, paramiko
Successfully installed paramiko-2.1.2 pyasn1-0.2.3
Great. But it didn't run on Lambda either.
{ "errorMessage": "No module named '_cffi_backend'", "errorType": "ModuleNotFoundError"}
Presumably I need to set up my lambda function to use 3.4 as noted above but lets roll back to 2.7 and see if that works. Since EC2 instances use 2.7 by default we won't hopefully need all the extra packages.
#update the ec2 isntance
sudo yum update -y
#switch to temp directory
cd /tmp
#create virtual environment
virtualenv -p /usr/bin/python2.7 python27
#activate virtual environment
source python27/bin/activate
#install dependencies
#sudo yum install gcc -y
#sudo yum install openssl-devel.x86_64 -y
#sudo yum install libffi-devel.x86_64 -y
#install cryptography and parmaiko paramiko
pip install paramiko
Successfully installed asn1crypto-0.22.0 cffi-1.10.0 cryptography-1.8.1 enum34-1.1.6 idna-2.5 ipaddress-1.0.18 paramiko-2.1.2 pyasn1-0.2.3 pycparser-2.17
And... testing it on a 2.7 Lambda function, it works. No missing libaries.
Read on if you want to see how the Lambda function is set up to use Paramiko and Cryptography to connect to configure a WatchGuard Firebox Cloud via the Command Line Interface and SSH.
