Authenticating Users with SASL for Postfix

Authenticating Users with SASL for Postfix

Continued from setting up Postfix - SMTP and TLS - on an Amazon Cloud EC2 instance:

http://websitenotebook.blogspot.com/2013/03/configuring-tls-on-postfix-redhat-aws.html

Simple Authentication and Security Layer 

http://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer

Simple Authentication and Security Layer (SASL) is a framework for authentication and data security in Internet protocols. It decouples authentication mechanisms from application protocols, in theory allowing any authentication mechanism supported by SASL to be used in any application protocol that uses SASL.

saslauthd is not a part of Postfix per se. It is a used by and integrated with Postfix to support authentication of users (users login functionality). 

Postfix works with two different implementations of SASL: Cyrus and Dovecot

Not a fan of the curly braces configuration files in Dovecot so going to try Cyrus.

Basically when you type testsaslauthd you aren't testing Postfix. You are just testing that a user can authenticate with saslauthd.  If you don't have the correct hook up between Postfix and saslauthd then you'll authenticate but won't get you to Postfix functionality. 

Now let's try to decipher the Postfix documentation...

Configuring Cyrus saslauthd implementation to work with PAM for Postfix

http://www.postfix.org/SASL_README.html 

Configuring Cyrus SASL

There may be a different configuration file for each application Cyrus provides authentication for within Postfix (SMTP, POP, IMAP, etc).

The first step configuring Cyrus SASL is to determine name andlocation of a configuration file that describes how the PostfixSMTP server will use the SASL framework.

The name of the configuration file (default: smtpd.conf)is configurable.

The configuration file used by Cyrus is a concatentation of the value in the configuration file (smtpd by default) and .conf.

i.e. by default the value in your config file is smtpd + .conf so the default configuration file is smtpd.conf

Configuration variables in main.cf for smtpd:

/etc/postfix/main.cf:    # Postfix 2.3 and later    smtpd_sasl_path = smtpd    # Postfix < 2.3    smtpd_sasl_application_name = smtpd

Cyrus SASL configuration file directory

The location where Cyrus SASL searches for the named file dependson the Cyrus SASL version and the OS/distribution used. [Worked fine on my AWS ec2 instance. Skipping the details]

Note Cyrus SASL searches /usr/lib/sasl2/ first. If itfinds the specified configuration file there, it will not examineother locations. [On my Amazon EC2 instance -- I did find sasl2 in this directory.]

Postfix to Cyrus SASL user databases

The SASL library uses one of the following for user database which are discussed in the Postfix documentation.

authentication backend	password verification service / plugin
/etc/shadow	saslauthd
PAM	saslauthd
IMAP server	saslauthd
sasldb	sasldb
MySQL, PostgreSQL, SQLite	sql
LDAP	ldapdb

saslauthd - Cyrus SASL password verification service

Communication between the Postfix SMTP server (read: Cyrus SASL'slibsasl) and the saslauthd server takesplace over a UNIX-domain socket.
 
saslauthd usually establishes the UNIX domainsocket in /var/run/saslauthd/ and waits for authenticationrequests. The Postfix SMTP server must have read+execute permissionto this directory or authentication attempts will fail.

Important Some distributions require the user postfix to bemember of a special group e.g. sasl, otherwise itwill not be able to access the saslauthd socketdirectory. [Note: I did not see this group on my AWS EC2 instance]

The following example configures the Cyrus SASL library tocontact saslauthd as its password verification service:

/etc/sasl2/smtpd.conf:    pwcheck_method: saslauthd    mech_list: PLAIN LOGIN

Important Do not specify any other mechanisms in mech_listthan PLAIN or LOGIN when usingsaslauthd! It can only handle these two mechanisms,and authentication will fail if clients are allowed to choose othermechanisms.

Important Plaintext mechanisms (PLAIN, LOGIN)send credentials unencrypted. This information should be protectedby an additional security layer such as a TLS-encrypted SMTP session [see my last post:(http://websitenotebook.blogspot.com/2013/03/configuring-tls-on-postfix-redhat-aws.html)]

Additionally the saslauthd server itself must beconfigured. It must be told which authentication backend to turnto for password verification. The backend is selected with asaslauthd command-line option and will be shown in thefollowing examples.

Note Some distributions use a configuration file to provide saslauthdcommand line options to set e.g. the authentication backend. Typicallocations are /etc/sysconfig/saslauthd or/etc/default/saslauthd.

Using saslauthd with /etc/shadow

Access to the /etc/shadow system password filerequires root privileges. [I stopped here. Next option...]

Using saslauthd with PAM

Cyrus SASL can use the PAM framework to authenticate credentials.saslauthd uses the PAM framework when started likethis:

% saslauthd -a pam

In other words, stop saslauthd if already running and start it again as shown above

sudo service saslauthd stop
sudo saslauthd -a pam

Note PAM configuration for the Postfix SMTP server is usually givenin /etc/pam.d/smtp and is beyond the scope of thisdocument.

See section "Testing saslauthdauthentication" for test instructions.
 

Using saslauthd with an IMAP server

[Not doing this...so skipping]

Testing saslauthd authentication

[
Um...it would have explained that users for this option are system users on the Linux box, unlike other options listed in the documentation.

useradd newuser
passwd newuser [enter new password at prompt).

For this option you do *not* use saslpasswd or saslpasswd2
]

Cyrus SASL provides the testsaslauthd utility totest saslauthd authentication.

[For the record the user name should *not* have the domain name on it like other options. You may find some documentation for other variants which indicate domain name is needed - not in this case]

% testsaslauthd -u username -p password0: OK "Success."

[the above worked for me. I didn't need any of the following mumbo jumbo]

Note Sometimes the testsaslauthd program is not distributedwith a the Cyrus SASL main package. In that case, it may bedistributed with -devel, -dev or-debug packages.

Specify an additional "-s smtp" if saslauthdwas configured to contact the PAM authentication framework, andspecify an additional "-f /path/to/socketdir/mux"if saslauthd establishes the UNIX-domain socket in anon-default location.
 
If authentication succeeds, proceed with the section "Enabling SASL authentication and authorizationin the Postfix SMTP server".

Cyrus SASL Plugins - auxiliary propertyplugins

Didn't use any of these...

The sasldb plugin

Or this..tried it with no success but I can tell you that you should add a 2 to the end of most commands if using the latest version.

Enabling SASL authentication andauthorization in the Postfix SMTP server

By default the Postfix SMTP server uses the Cyrus SASLimplementation.

[skipping Dovecot text here]

Enabling SASL authenticationin the Postfix SMTP server

Regardless of the SASL implementation type, enabling SMTPauthentication in the Postfix SMTP server always requires settingthe smtpd_sasl_auth_enable option:

/etc/postfix/main.cf:    smtpd_sasl_auth_enable = yes

After a "postfix reload", SMTP clients will see the additionalcapability AUTH in an SMTP session, followed by a list ofauthentication mechanisms the server supports:

% telnet server.example.com 25...220 server.example.com ESMTP PostfixEHLO client.example.com250-server.example.com250-PIPELINING250-SIZE 10240000250-AUTH DIGEST-MD5 PLAIN CRAM-MD5...

However not all clients recognize the AUTH capability as definedby the SASL authentication RFC. Some historical implementations expect theserver to send an "=" as separator between the AUTHverb and the list of mechanisms that follows it.
The broken_sasl_auth_clients configuration optionlets Postfix repeat the AUTH statement in a form that these brokenclients understand:

/etc/postfix/main.cf:    broken_sasl_auth_clients = yes

Note Enable this option for Outlook up to and including version 2003and Outlook Express up to version 6. This option does not hurt otherclients.

After "postfix reload", the Postfix SMTP server will propagatethe AUTH capability twice - once for compliant and once for brokenclients:

% telnet server.example.com 25...220 server.example.com ESMTP PostfixEHLO client.example.com250-server.example.com250-PIPELINING250-SIZE 10240000250-AUTH DIGEST-MD5 PLAIN CRAM-MD5250-AUTH=DIGEST-MD5 PLAIN CRAM-MD5...

Postfix SMTP Server policy- SASL mechanism properties

The Postfix SMTP server supports policies that limit the SASLmechanisms that it makes available to clients, based on the propertiesof those mechanisms. The next two sections give examples of howthese policies are used.

Property	Description
noanonymous	Don't use mechanisms that permitanonymous authentication.
noplaintext	Don't use mechanisms that transmitunencrypted username and password information.
nodictionary	Don't use mechanisms that arevulnerable to dictionary attacks.
forward_secrecy	Require forward secrecy betweensessions (breaking one session does not break earlier sessions).
mutual_auth	Use only mechanisms that authenticateboth the client and the server to each other.

Unencrypted SMTP session

The default policy is to allow any mechanism in the Postfix SMTP serverexcept for those based on anonymous authentication:

/etc/postfix/main.cf:    # Specify a list of properties separated by comma or whitespace    smtpd_sasl_security_options = noanonymous

Important Always set at least the noanonymous option.Otherwise, the Postfix SMTP server can give strangers the sameauthorization as a properly-authenticated client.

Encrypted SMTP session (TLS)

A separate parameter controls Postfix SASL mechanism policyduring a TLS-encrypted SMTP session. The default is to copy thesettings from the unencrypted session:

/etc/postfix/main.cf:    smtpd_sasl_tls_security_options = $smtpd_sasl_security_options

A more sophisticated policy allows plaintext mechanisms, butonly over a TLS-encrypted connection:

/etc/postfix/main.cf:    smtpd_sasl_security_options = noanonymous, noplaintext    smtpd_sasl_tls_security_options = noanonymous

To offer SASL authentication only after a TLS-encrypted session has beenestablished specify this:

/etc/postfix/main.cf:    smtpd_tls_auth_only = yes

Enabling SASL authorization in the PostfixSMTP server

After the client has authenticated with SASL, the Postfix SMTPserver decides what the remote SMTP client will be authorizedfor. Examples of possible SMTP clients authorizations are:

• Send a message to a remote recipient.
  
• Use a specific envelope sender in the MAIL FROM command.
  

These permissions are not enabled by default.

Mail relay authorization

With permit_sasl_authenticated the Postfix SMTPserver can allowSASL-authenticated SMTP clients to send mail to remote destinations.Examples:

# With Postfix 2.10 and later, the mail relay policy is# preferably specified under smtpd_relay_restrictions./etc/postfix/main.cf:    smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination

# Older configurations combine relay control and spam control under# smtpd_recipient_restrictions. To use this example with Postfix ≥# 2.10 specify "smtpd_relay_restrictions="./etc/postfix/main.cf:    smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination ...other rules...

Skipping other options

Testing SASL authentication in the Postfix SMTP Server

To test the server side, connect (for example, withtelnet) to the Postfix SMTP server port and you shouldbe able to have a conversation as shown below. Information sent bythe client (that is, you) is shown in bold font.

% telnet server.example.com 25...220 server.example.com ESMTP PostfixEHLO client.example.com250-server.example.com250-PIPELINING250-SIZE 10240000250-ETRN250-AUTH DIGEST-MD5 PLAIN CRAM-MD5250 8BITMIMEAUTH PLAIN AHRlc3QAdGVzdHBhc3M=235 Authentication successful

And...let the errors begin...checking the logs I see:


I set up port 25 with no tls to test plain login (since testing on a secure TLS port can't see anything). I didn't exactly read the above and with TLS you first must establish the TLS connection, then do the authentication, which you can't do through Telent (see next test).

First I get this error related to mismatch between some options:

The problem is a mis-match between smtpd_sasl_security_options
(e.g., noplaintext) and the available server mechanisms (e.g.,
plaintext only). 

In otherwords remove noplaintext from main.cf because we're using TLS to encrypt traffic. OK now I can connect again...

And I see AUTH PLAIN LOGIN when I telnet to the server on port 25 and do an EHLO (see previous post for more info)

Next error. Tried test above to test a login from Telnet

AUTH PLAIN AHRlc3QAdGVzdHBhc3M=

Ok darnitalll now we have a new error...

I test using the telnet example shown above and I get this error:

535 5.7.8 Error: authentication failed: no mechanism available.

I found that in my smtpd.conf file I had a space instead of an underscore in pw

Restart - no dice.

Out of curiosity I search for smtpd.conf and find three versions:

/usr/lib/sasl2/smtpd.conf
/var/run/saslauthd/smtpd.conf
/etc/sasl2/smtpd.conf

Which one is being used? The documentation above says the first in the list.

Not true!! Argh.

/var/run/saslauthd/smtpd.conf was empty. I renamed it and restarted - same error.

Then I checked /etc/sasl2/smtpd.conf and I had altered that file at some point. I changed pwd_check back to saslauthd.

Restart.

Authentication successful! Oh the agony of unhelpful error messages and inaccurate documentation and open source crypticness....yes I just made that word up.

Now. Let's see about authentication over TLS...crossing fingers.

So first I test from an app that was working fine before I moved everything to the Amazon cloud. Sending an email from a web site. No dice.

I can see that my web app is able to connect to and negotiate a connection with the mail server because I see the connection attempt in the maillog

sudo tail -f /var/log/maillog

lost connection after UNKNOWN from unknown 
disconnect from unknown

I can only assume at this point it's some kind of problem with TLS or relay.

 So let's try this example below and see what happens...

To test this over a connection that is encrypted with TLS, useopenssl s_client instead of telnet:

% openssl s_client -connect server.example.com:25 -starttls smtp...220 server.example.com ESMTP PostfixEHLO client.example.com...see above example for more...

Lovely. Again no worky. Does anyone find this as painful as I do yet?

getaddrinfo: Name or service not known
connect:errno=2

Oops. Typed the example and forgot to change domain name.

Here's a page with some error codes:
 

http://tud.at/programm/apache-ssl-win32-howto.php3

ok typing with correct domain it seems to attempt to connect to TLS but times out:

socket: Connection timed out
connect:errno=110

And in the case of a local connection from the same box using domain name it times out so maybe it's VPC/firewall thing. Try the local IP: 127.0.0.1 instead of the domain name. 

Ok something useful at least. Maybe:

139834971027272:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:766:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 228 bytes and written 206 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
 
I found some sites suggesting to take the passphrase off the key -- same error
http://www.mnxsolutions.com/apache/removing-a-passphrase-from-an-ssl-key.html


Could be a problem with OpenSSL library - not sure yet.
http://comments.gmane.org/gmane.mail.postfix.user/228998

Instead of AHRlc3QAdGVzdHBhc3M=, specify thebase64-encoded form of \0username\0password (the \0is a null byte). The example above is for a user named `test'with password `testpass'.

Caution When posting logs of the SASL negotiations to public lists,please keep in mind that username/password information is trivialto recover from the base64-encoded form.

You can use one of the following commands to generate base64encoded authentication information:

• Using a recent version of the bash shell:
  

  % echo -ne '\000username\000password' | openssl base64

  Some other shells support similar syntax.
  
• Using the printf command:
  

  % printf '\0%s\0%s' 'username' 'password' | openssl base64% printf '\0%s\0%s' 'username' 'password' | mmencode

  The mmencode command is part of the metamailsoftware.
  
• Using Perl MIME::Base64:
  

  % perl -MMIME::Base64 -e \    'print encode_base64("\0username\0password");'

Configuring TLS on Postfix - RedHat AWS EC2 instance - Amazon Cloud

Please note. This didn't really end up working. Leaving notes for all the things I did get working however. This is old and not recommended - just use the AWS mail services or some other cloud based mail service and save yourself some pain.

AWS SES
AWS Workmail
Gmail

---
Attempting to set up Postfix on Amazon EC2 instance with TLS and authentication to work with Postini mail security service. Steps taken (Caveat - I have never done this before today and currently re-learning Linux):

If you haven't already created a private key create one becausee you'll need it to connect to Linux server below:
 
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-credentials.html#using-credentials-login-password

If you haven't already created a VPC (virtual private network) create one, or don't use VPC in the example below.

AWS documentation has four scenarios for setting up VPCs for different purposes:

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenarios.html

Configure the VPC security group you use for the mail server below to the IP address of the machine you are using to administer the EC2 instance. You can find your IP address as the outside world sees it (not the local IP address internal to your network) at the top of http://dnsstuff.com You can give this IP access to all ports and protocols or specifically the ports and protocols needed to set up the mail server (22 for SSH, 25 for SMTP, any non-standard ports you want to send mail from, etc).

You'll also need to open up access to Postini and from any servers from which you want to relay mail to Postini on applicable mail ports (25, etc). See link regarding Postini outbound configuration in steps below.

Hint: Before even attempting anything with Postfix, make sure your network connectivity is set up correctly (your VPC and firewalls on machines). If at any point you are trying to send mail from another machine through the Postfix server and no messages are showing up in Postfix logs, it could be a network problem that has nothing to do with Postfix. A connection may have never been made to/from the local or remote machine which has nothing to do with Postfix. You'll need to fix that first. You can test network connectivity by opening up all ports between the various machines and IPs or at least the port and protocol required to ping and test that you can ping and get a reply from the postfix machine on the machine you are trying to send mail from. Once that's working then move on to Postfix configuration.

I checked and by default the Amazon AWS ec2 redhat instances do not have a firewall running. I did have to adjust the firewall on a Windows instance to open the necessary ports to access the Linux machine.

Ok now for the nitty gritty.

The goal here is to send mail from an application server through the mail server to Postini, which will then be verified encrypted and routed to the appropriate party. Using Postini ensures only valid IP ranges can send mail outbound and you can enforce TLS encryption.Postini also checks outbound mail for viruses and spam.


--- Get EC2 Linux RedHat instance from Amazon ---

1. login and go to EC2 Dashboard
2. Click "Launch instance"
3. Choose classic wizard
4. Click continue
5. Click Community AMIs
6. Enter Amazon Linux ID you want to use (ami-8e27adbe for oregon or see bottom of this page http://aws.amazon.com/amazon-linux-ami/)
Note: Make sure you select an ami in the same region where you created your VPC.
 7. Click select
8. Choose size (T1 Micro for my test)
9. Click on EC2-VPC tab (assuming you created a VPC)
10. Choose an Internet accessible subnet for your server where the VPC security group you want to use for your mail server is located.
11. Click continue on next page (I didn't change anything)
12. Update the mail tag to some name that helps you identify this server
13. Click continue
14. Choose a key pair you created previously or create a new one
15. Choose a security group - in my case I created a mail specifc security group that can access my Postini specified IP range for my account and allows access from the servers in my network that are allowed to send mail.

I opened up the ports I have configured my apps to send mail on and 25/465 to Postini (seems to only be sending on 25 at this point but my apps send mail to the mail server on other ports)

16.Click continue
17. Click launch
18. Click close (unless you want to create alarms)
19. Wait for your instance to fire up -- will say "initializing" in your instance list
20. Assign an elastic IP address from your VPC to your instance (you may have problems connecting without it)
20. Right click on the instance and choose connect
21. Enter the path to your private key and connect - you should now be able type commands that execute on your server in the window that pops up.

--- DNS - if you want ---

Configure a domain name to point to your IP address. For exmaple you may want to configure mail.yourdomain.com to point to your elastic IP address.

I'm not going into details on the above - contact your domain name provider, use a distributed, redundant DNS service like http://EasyDNS.com or check out the options from Amazon DNS service which they call "Route 53" http://aws.amazon.com/route53/.

Configure an SPF record in your DNS to indicate that only Postini IPs are valid IPs for sending mail from your domain - to help prevent people from spoofing your emails or having your mails end up in spam mailboxes (hopefully).

Something like this, though I want to go back and review this later to make sure it is the best option:

v=spf1 ip4:64.18.0.0/20 ~all

--- Install and Configure Postfix ---

Install Postfix and get rid of Sendmail if you want.

sudo yum install postfix
sudo yum erase sendmail

Note: If you have problems connecting to AWS repo see this post:
http://websitenotebook.blogspot.com/2013/03/connection-timeout-running-yum-on-ec2.html

You might want to back up the original postfix configuration files:

sudo cp /etc/postfix/main.cf /etc/postfix/bak/main.cf
sudo cp /etc/postfix/master.cf /etc/postfix/bak/master.cf

Figure out how to use vi again if you forgot like me and edit main.cf in the postfix directory

Navigate to /etc/postfix

Use VI to edit main.cf:

sudo vi main.cf

hit "i" to insert text

When you are done editing save by typing ctrl-C then:

:wq!

Edit the main.cf file with the following settings:

myhostname = [mail.yourdomain.com or whatever your mail server hostname is]

mydomain  = [mail.yourdomain.com or whatever your mail domain is]

inet_interfaces = all  [and comment out anything else]

mydestination = $mydomain, $myhostname

mynetworks = [ip addresses/ranges for which you want to allow relay - limit to trusted networks]

relayhost=outbounds[x].obsmtp.com (where [x] is the number in the domain when you login at Postini -- indicating which particular Postini server(s) your account uses)

Type ctrl-C and :wq! to save and exist the file

restart the service:

sudo service postifix restart

--- Add your new instance as a reinjection host at Postini ---

At this point, make sure your VPC is allowing access from the mail server via the VPC security group the mail server is in to the Postini IP addresses for your Postini account.

You can then add your new mail server VPC elastic IP address at Postini so Postini can send bounce messages back to your mail server.

Postini will test connection to your mail server and that it can send mail at this point so you'll know if something is wrong. Check the VPC and your Postfix Configuration if Postini can't connect to your mail server at this point.

Postini outbound configuration and IP addresses:

http://www.google.com/support/enterprise/static/postini/docs/admin/en/admin_ee_cu/ob_setup.html 

--- Send a message ---

...ok at this point you should have mail going over to Postini if your firewalls are set up correctly. You can send a test message with this command:


sendmail [put email address here]
FROM: [put email address here like ec2-user@your.domain.com]
SUBJECT: hello world
this is a test email
.

Check postfix logs to see if your message was sent. 

sudo tail -f /var/log/maillog

If you limit sending to TLS you'll see an error message related not being able to send unless TLS is enabled

451 STARTTLS is required for this sender - psmtp (in reply to MAIL FROM command))

Install mutt to check and see if you get a bounce message.

sudo yum install mutt

To run mutt must type: 

mutt
 
When you're done reading  messages type

q

There won't be a bounce because the message is still in the postfix queue. Check:

postqueue -p
  
You'll see something like this:
 
96D8C200FB      319 Sun Apr  7 05:44:10  ec2-user@mx.domain.com
(host outbounds5.obsmtp.com[64.18.6.12] said: 451 STARTTLS is required for this sender - psmtp (in reply to MAIL FROM command))
                                         user@domain.com

-- 0 Kbytes in 1 Request.

If you log into Postini and change your settings to allow SMTP the message will go through: 

Login, click on outbound servers, choose your domain, click on TLS.

1. Choose how the email protection service accepts outbound messages from your mail server. 

Choose: Accept SMTP and TLS.

Flush the Postmail queue:

sudo postfix flush
     
Now use tail command above and you should see the mail was sent.

Apr  7 06:52:05 ip-10-0-0-112 postfix/smtp[18321]: 96D8C200FB: to=, relay=outbounds5.obsmtp.com[64.18.6.12]:25, delay=4076, delays=4072/0.03/0.15/3, dsn=2.0.0, status=sent (250 Thanks)
Apr  7 06:52:05 ip-10-0-0-112 postfix/qmgr[18226]: 96D8C200FB: removed
 
That's nice. But that's not really what we want. We just sent unencrypted email over the internet which can be read by anyone.

Go back to Postini and choose the option to accept only TLS and save.

This might be a good point to make a back up of your mail config as well.

--- Get an SSL certificate for TLS ---

I  noticed that when I installed Postfix, OpenSSL was installed:

Processing Dependency: libcrypto.so.10

(OPENSSL_1.0.1)(64bit) for package: 2:postfix-2.6.6-2.14.amzn1.x86_64
 
....

Package openssl.x86_64 0:1.0.0j-1.43.amzn1 will be updated
---> Package openssl.x86_64 1:1.0.1e-4.53.amzn1 will be an update
--> Processing

Dependency: make for package: 1:openssl-1.0.1e-4.53.amzn1.x86_64
--> Running transaction check
---> Package

make.x86_64 1:3.81-20.7.amzn1 will be installed
--> Finished Dependency Resolution

Dependencies Resolve

If needed you could install openssl:

sudo yum install openssl

Make a directory for your ssl files and migrate to it:

sudo mkdir /etc/postfix/certs

Digicert has some tools to generate a CSR using openssl:

Info about OpenSSL CSR Creation:

http://www.digicert.com/csr-creation-apache.htm

A form you can fill out to create the code that generates the CSR on your server:

https://www.digicert.com/easy-csr/openssl.htm

The command it creates will look *something* like this. This has my cert specific info so make sure you go to the page above and create a CSR with your specific domain name, location, etc. I added -outform PEM because these things need to be in PEM format and I saw that on another web site. Not sure if it matters.

openssl req -new -outform PEM -newkey rsa:2048 -nodes -out domain_com.csr -keyout star_domain_com.key -subj "/C=US/ST=WA/L=Seattle/O=Radical Sofftware Inc./CN=*.domain.com"

Run the above command in your directory to get the new key and csr files.

Go to Digicert and request a new Apache cert. Paste the contents of the CSR file that was generated in your /etc/postfix/certs directory when you ran the above command into the text box for the request on the web site. See Digicert link with instructions above for more info.

Submit and wait for the new cert to be generated (varies depending on if getting new or existing).

When your cert is ready go to the Digicert web site (I prefer not to use the one that comes in the email), log into your account, and download the new cert.

Choose option for PEM file with all certs in it.

Save it in your /etc/postfix/certs directory as domain.PEM where domain is your domain name.

Download and install putty so you can telnet to your Linux machine.Run or use your telnet tool of choice.

http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Enter your elastic IP and choose Telnet. Click open.
  Type 

EHLO [your mail serer domain here]


For some reason the first time I type it doesn't work. I type it again and get the desired response showing TLS is running:

250-your.mailserver.tld
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250 8BITMIME

Try another test message as described above. You'lll see the same error in the logs when you tail (451...). This is because what we just enabled is for inbound mail.

What we need to do now is encrypt outbound mail.

--- Add outbound mail configuration for TLS ---

mtp_tls_cert_file = /etc/postfix/certs/cert.pem
smtp_tls_key_file = /etc/postfix/certs/key.pem
smtp_use_tls = yes
smtp_enforce_tls = no
 

Check it out to see if our message now gets sent.

sudo postfix reload
sudo postfix flush
sudo tail -f /var/log/maillog

Using tail it looks like the message got sent but there's an error message about an untrusted issuer:

Apr  7 08:09:08 ip-10-0-0-112 postfix/smtp[18657]: certificate verification failed for outbounds5.obsmtp.com[64.18.6.12]:25: untrusted issuer /C=US/O=Equifax/OU=Equifax Secure Certificate Authority

What is odd is that the message does actually get sent. You can verify this by checking the queue and it will be empty:

sudo postqueue -p 
Mail queue is empty
 
What is also odd is that the mail doesn't show up right away in the mailbox I sent it to but does after many hours of delay. (Like I went to bed and in the morning it showed up).

I contacted DigiCert at this point and was told:

Looking at the error, it states your server is pulling an Equifax certificate.  If you have installed and bound our certificate successfully, then all you would need to do is reboot the server for it to start using our certificate.  If the reboot does not fix your problem, you will need to check the bindings on your server and correct them.

Go to ec2 instances in AWS EC2 dashboard. Right click on instance. Choose reboot.

I tried to send a new email and got the same error in the logs.

Went to DigiCert web site and downloaded individual files, zipped.

Copied DigiCertCA.crt to /etc/postfix/certs and renamed with .PEM extension.

Restarted postfix and got an error saying for some reason could not load the ca file so was disabling TLS.

Apr  7 17:03:08 ip-10-0-0-112 postfix/smtp[1687]: cannot load Certificate Authority data: disabling TLS support

I checked and the file was missing the first few letters. I am not sure if this is a problem when pasting text from a windows machine into the MindTerm AWS Linux client or I just copied only part of the file. I fixed that, saved the file, reloaded postfix.

No error. Message sent!

Apr  7 17:07:31 ip-10-0-0-112 postfix/postfix-script[1714]: refreshing the Postfix mail system
Apr  7 17:07:31 ip-10-0-0-112 postfix/master[1549]: reload -- version 2.6.6, configuration /etc/postfix
Apr  7 17:07:36 ip-10-0-0-112 postfix/qmgr[1719]: 6469620158: from=, size=326, nrcpt=1 (queue active)
Apr  7 17:07:39 ip-10-0-0-112 postfix/smtp[1728]: 6469620158: to=, relay=outbounds5.obsmtp.com[64.18.6.12]:25, delay=683, delays=681/0.08/0.31/2.3, dsn=2.0.0, status=sent (250 Thanks)
Apr  7 17:07:39 ip-10-0-0-112 postfix/qmgr[1719]: 6469620158: removed

Sweet. That only took a million tries to get this all working.

Only problem now is that the mail hasn't shown up in the recipient inbox.

What is also interesting is that even though my server indicates the message was sent, I see no indication in the Postini outbound logs for this domain that the message was received.

Next I looked at the quarantine for the user I am sending to and I see that the messages are, in fact there. They have been flagged as spam. Probably tripped up after sending too many requests. Should not be flagged as spam otherwise, because my SPF records are set to include the new AWS elastic IP address.

This would be another excellent point to back up your main.cf file. :)

--- Open other ports if needed ---

Ok now I want to open up some other ports to send mail on from other applications that require specific ports.

sudo vi /etc/postfix/master.cf

Add a line for the port you want to open up (make sure your VPC allows connections on this port). You can see the line I added bellow in bold. Note that the type is inet to allow connections from the Internet and the command is smtpd.

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       n       -       -       smtpd 
468       inet  n       -       n       -       -       smtpd 


Check the logs to make sure everything is groovy. No errors. Like an i at the start of the file when typing i for insert too many times which causes you to get a message about an invalid transport type. Not that I ever did such a thing.

Now open putty as instructed above and test EHLO but this time connect on your new port - 468 in the example above. If you get the expected results with EHLO your mail system is open and listening on that port.

Another fine time to back up master.cf

Now you possibly want to authenticate users (i.e. have them login with a user name and password). That's going to be handled in my next post.

Connection Timeout Running Yum on EC2 instance with VPC

I was getting a connection error like this trying to run Yum on an Amazon AWS EC2 Redhat Linux instance in a public subnet of a VPC with a security group I had set up specifically for this machine.

http://packages.us-west-2.amazonaws.com/2012.09/main/201209eb6a01/x86_64/repodata/repomd.xml: [Errno 12] Timeout on http://packages.us-west-2.amazonaws.com/2012.09/main/201209eb6a01/x86_64/repodata/repomd.xml: (28, 'connect() timed out!') Trying other mirror.

I found that opening up outbound traffic completely for the security group for that server resolved the problem and was able to successfully download packages.

Last night I talked to some Amazon folks at an event recently that told me because the VPC is a stateful firewall it would be OK to open all outbound traffic for that server.

However... If prefer to know that you are getting your updates from a valid Amazon repo or at least an Amazon IP, you can open up your outbound traffic in your security group to the specific IPs or IP ranges for the repo(s) you are trying to connect to.

For instance, if the error message says you are trying to connect to: http://packages.us-west-2.amazonaws.com...

Open a command prompt and ping packages.us-west-2.amazonaws.com
I got IP address: 205.251.235.166

The IP for this repo could change obviously but you could set up your security group to allow outbound traffic to this IP address. If the IP for that repo changes at some point you'll get an error and have to change the IP to whatever Amazon changes the domain to point to in the future.

You can also go to Arin.org and get the complete Amazon IP range for this IP and allow traffic to all Amazon IP adddresses outbound. In this case 205.251.192.0/18

http://whois.arin.net/rest/net/NET-205-251-192-0-1/pft

NetRange 205.251.192.0 - 205.251.255.255
CIDR 205.251.192.0/18
Name AMAZON-05
Handle NET-205-251-192-0-1
Parent NET205 (NET-205-0-0-0-0)
Net Type Direct
Assignment Origin AS AS7224 AS16509 AS39111
Organization Amazon.com, Inc. (AMAZON-4)
Registration Date 2010-08-27
Last Updated 2012-03-02
Comments RESTful Link http://whois.arin.net/rest/net/NET-205-251-192-0-1

When I ping  packages.sa-east-1.amazonaws.com I get a Lacnic IP address:

177.72.244.0

You'd have to go to lacnic.org to look up that IP range:

inetnum:     177.72.240/21
aut-num:     AS53032
abuse-c:     MAAZI67
owner:       A100 ROW SERVICOS DE DADOS BRASIL LTDA
ownerid:     012.147.176/0001-50
responsible: Marla Azinger
country:     BR
owner-c:     MAAZI67
tech-c:      MAAZI67
inetrev:     177.72.240/21
nserver:     pdns1.ultradns.net 
nsstat:      20130329 AA
nslastaa:    20130329
nserver:     pdns2.ultradns.net 
nsstat:      20130329 AA
nslastaa:    20130329
nserver:     pdns3.ultradns.org 
nsstat:      20130329 AA
nslastaa:    20130329
nserver:     pdns5.ultradns.info 
nsstat:      20130329 AA
nslastaa:    20130329
nserver:     pdns6.ultradns.co.uk 
nsstat:      20130329 AA
nslastaa:    20130329
created:     20110816
changed:     20111121

nic-hdl-br:  MAAZI67
person:      Marla Azinger
e-mail:      mazinger@amazon.com
created:     20111114
changed:     20111118

SSL Cert Install Issues - Digicert SSL certs

Ok I just had to do this in February but I already forgot all these little things you have to do to get SSL certificates from Digicert working in various systems. Documenting installation process here to aid my somewhat non-functional short term memory.

login at digicert:
https://www.digicert.com/custsupport/

Go to My orders Tab

Click on Cert

Rekey cert
http://www.digicert.com/ssl-support/ssl-certificate-reissue.htm

Choose 2048 bits

Generate IIS cert on web server:
http://www.digicert.com/csr-creation-microsoft-iis-7.htm

After cert generation, close and go back into IIS or hit F5 to refresh to get the new cert to show up

Add the CSR to your cert at digicert. Wait a few minutes for it to get reissued.

Get the Certificate from digicert (download it) after has been reissued.

When you try to import - if you get ASN1 bad key -
Run digicert util to import the key instead
https://www.digicert.com/util/

Go to site bindings and edit - choose your new SSL cert - shouldn't need to restart the web server.

Test your site - SSL should be working.

Now for IMail mail server...you have to go through some hijinks to get that working with your IIS cert. Instructions here:

http://support.ipswitch.com/kb/IM-20030415-DM01.htm

With these steps IMAIL will work for general email clients like Outlook.

OK on to Java...

If you are connecting to IMail from a Java app and TLS you'll get an error stating that PKIX path building failed. In this case you need to go to the digicert site and get the IIS digicert and CA root certs and add them to the cert file you created in teh step above for IMail. You can download all three certs as a zip file when you click the download button (under the cert box). Put them in the IIS file in this order, type "reset" button. Re-enter password. Hit Apply. Restart all Imail services.

-----BEGIN CERTIFICATE-----
blah blah blah ...your cert
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blah blah blah ...digicert ca cert
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blah blah blah ...root cert
-----END CERTIFICATE-----

At least that worked for me....

Now for the Java web server...
http://www.digicert.com/ssl-certificate-installation-java.htm

Hopefully next time will go a little smoother...unless somebody changes something.

Error occurred during initialization of VM

Good post on resolving the error:

Error occurred during initialization of VM Unable to load dependent libraries: Can't find dependent libraries

The culprit is too many java.exe's and the wrong one being put to use.

http://geekycoder.wordpress.com/2009/07/08/java-tips-adventure-in-resolving-the-java-error-error-occurred-during-initialization-of-vm/

What I am curious about is how did the java.exe get in the Windows directory that caused this error? Not going further with this (I'm busy) but the things I did recently include upgrading to the latest version of Java which I did twice due to putting it in a directory I didn't want it in the first time around, install some software from Fluke Networks, and go to a bunch of Java programming pages on the net. While looking at one of them an error was thrown. Not sure that was related to the web site or just a coincidence.

Oh well, renamed all the java.exe's except the one I wanted so they wouldn't get run and put the path to the one I intended to use on my system path so all good it seems.

Using JavasScript and Ant

If you hate the fact that you are doing programming like tasks with XML in Ant, consider using JavaScript if you don't want to write new custom Java classes to use as targets. I would probably opt for new targets but some people seem to prefer scripting languages so here you go:

JavaScript and Ant

In general I try to keep the configuration in the XML file separate from the executable code, which is why I would opt for custom Java class targets configured by XML, rather than try to incorporate executable code into an XML file. That's just me.

I also read you can integrate Groovy with Ant.

Conditionally Call and Ant Target

If you want to set a target to only run if you set a particular value, easy. Just use the if attribute as follows:

<target name="whatever" if="my.value">
  <echo message="target executed"/>
</target>

If you want the above target to run - set my.value to something in your property file before that target and it will run.

<property name="my.value" value="yada"/>

If you leave the property out of your file altogether the target won't run.

You can also have other tasks set values and use those values to determine if targets should run or not based on whether a file was available, etc. Details about conditional values can be found in the Ant FAQ:

Conditionally Execute Ant Target

Ant - Copy Files That Have Changed

If you want to copy files that have changed from one directory to another use a file set with the "modified" option like this:

 <copy todir="${dir.copy.to}">
  <fileset dir="${dir.copy.from}">
         <modified/>
        </fileset>
        
 </copy>

Ant will set up a cache.properties file which tracks file information so it knows the state of files after each build. Then when it runs a task it can refer to the cache file to determine which files have changed since the last successful build and need to be included in the fileset.

If you want the build to include a file that is getting excluded by the cache file just delete that file from the cache.properties file and it will be included in the file set the next time the ant build is run.