When your instances run on AWS by default they will reach out to the Internet to an NTP service to update the clock that is used to create all the timestamps in system logs, and other time related functions. A more detailed explanation in found here on the Amazon Web Site:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html
For better security you'll want to limit which subnets have Internet access by limiting the route tables that all an Internet Gateway. On the other hand, it is really important to have all your instances synchronized with accurate times, so that in the case of a security incident, the logs can be correctly correlated.
To overcome this issue we can use a WatchGuard Firebox or similar device to get the time from the Internet and have all the instances check the WatchGuard Firebox on the internal network to get time stamps.
To configure the WatchGuard Firebox as an NTP server the NTP option and the option to use the Firebox as an NTP server must be enabled.
http://www.watchguard.com/help/docs/fireware/11/en-US/Content/en-US/basicadmin/NTP_server_enable_add_c.html
I have some code for automated deployment of a WatchGuard Firebox here:
https://github.com/tradichel/FireboxCloudAutomation
There's a Lambda function that connects to the Firebox to make configuration changes here:
https://github.com/tradichel/FireboxCloudAutomation/blob/master/code/resources/firebox-lambda/fireboxconfig.py
The following commands can be added to that script to enable NTP:
#make Firebox an NTP server
command="ntp enable\n"
channel.send(command)
time.sleep(3)
command="ntp device-as-server enable\n"
channel.send(command)
time.sleep(3)
Note the space after ntp.
Once you have your Firebox set up as an NTP server, go back and update the instances as explained in the article at the top of the page to use the Firebox as an NTP server instead of the Amazon default NTP servers.
You'll need to ensure port 123 is open to and from the NTP server for the UDP protocol.
Of course you'll want to configure your Firebox Cloud in a high availability configuration if you want to ensure that you always have NTP available. This blog post presents one way to create a high availability NAT:
https://aws.amazon.com/articles/2781451301784570
Thinking about other options for a scalable, HA configuration of a Firebox Cloud for future blog posts.
