Configuring Linux Mail Servers/I Email is an important part of any Web site you create. In a home environment, a free web based email service may be sufficient, but if you are running a business, then a dedicated mail server will probably be required. This chapter will show you how to use sendmail to create a mail server that will relay your mail to a remote user's mailbox or incoming mail to a local mail box. You'll also learn how to retrieve and send mail via your mail server using a with mail client such as Outlook Express or Evolution.
Configuring Sendmail One of the tasks in setting up DNS for your domain (my-site.com) is to use the MX record in the configuration zone file to state the hostname of the server that will handle the mail for the domain. The most popular Unix mail transport agent is sendmail, but others, such as postfix and qmail, are also gaining popularity with Linux. The steps used to convert a Linux box into a sendmail mail server will be explained here.
How Sendmail Works As stated before, sendmail can handle both incoming and outgoing mail for your domain. Take a closer look.
Incoming Mail Usually each user in your home has a regular Linux account on your mail server. Mail sent to each of these users (
[email protected]) eventually arrives at your mail server and sendmail then processes it and deposits it in the mailbox file of the user's Linux account. Mail isn't actually sent directly to the user's PC. Users retrieve their mail from the mail server using client software, such as Microsoft's Outlook or Outlook Express, that supports either the POP or IMAP mail retrieval protocols. Linux users logged into the mail server can read their mail directly using a textbased client, such as mail, or a GUI client, such as Evolution. Linux workstation users can use the same programs to access their mail remotely.
Outgoing Mail The process is different when sending mail via the mail server. PC and Linux workstation users configure their e-mail software to make the mail server their outbound SMTP mail server. If the mail is destined for a local user in the mysite.com domain, then sendmail places the message in that person's mailbox so that they can retrieve it using one of the methods above. If the mail is being sent to another domain, sendmail first uses DNS to get the MX record for the other domain. It then attempts to relay the mail to the appropriate destination mail server using the Simple Mail Transport Protocol (SMTP). One of the main advantages of mail relaying is that when a PC user A sends mail to user B on the Internet, the PC of user A can delegate the SMTP processing to the mail server. Note: If mail relaying is not configured properly, then your mail server could be commandeered to relay spam. Simple sendmail security will be covered later.
Sendmail Macros When mail passes through a sendmail server the mail routing information in its header is analyzed, and sometimes modified, according to the desires of the systems administrator. Using a series of highly complicated regular expressions listed in the /etc/mail/sendmail.cf file, sendmail inspects this header and then acts accordingly. In recognition of the complexity of the /etc/mail/sendmail.cf file, a much simpler file named /etc/sendmail.mc was created, and it contains more understandable instructions for systems administrators to use. These are then interpreted by a number of macro routines to create the sendmail.cf file. After editing sendmail.mc, you must always run the macros and restart sendmail for the changes to take effect. Each sendmail.mc directive starts with a keyword, such as DOMAIN, FEATURE, or OSTYPE, followed by a subdirective and in some cases arguments. A typical example is. As stated before, sendmail can handle both incoming and outgoing mail for your domain. Take a closer look. FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl
The keywords usually define a subdirectory of /usr/share/sendmail-cf in which the macro may be found and the subdirective is usually the name of the macro file itself. So in the example, the macro name is /usr/share/sendmailcf/feature/virtusertable.m4, and the instruction `\ hash -o /etc/mail/virtusertable.db' is being passed to it. Notice that sendmail is sensitive to the quotation marks used in the m4 macro directives. They open with a grave mark and end with a single quote. FEATURE(`masquerade_envelope')dnl
Some keywords, such as define for the definition of certain sendmail variables and MASQUERADE_DOMAIN, have no corresponding directories with matching macro files. The macros in the /usr/share/sendmail-cf/m4 directory deal with these. Once you finish editing the sendmail.mc file, you can then execute the make command while in the /etc/mail directory to regenerate the new sendmail.cf file. [root@bigboy tmp]# cd /etc/mail [root@bigboy mail]# make
If there have been no changes to the files in /etc/mail since the last time make was run, then you'll get an error like this: [root@bigboy mail]# make make: Nothing to be done for `all'. [root@bigboy mail]#
The make command actually generates the sendmail.cf file using the m4 command. The m4 usage is simple, you just specify the name of the macro file as the argument, in this case sendmail.mc, and redirect the output, which would normally go to the screen, to the sendmail.cf file with the ">" redirector symbol. [root@bigboy tmp]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
I'll discuss many of the features of the sendmail.mc file later in the chapter.
Installing Sendmail Most RedHat and Fedora Linux software products are available in the RPM format. You will need to make sure that the sendmail, sendmail-cf, and m4 software RPMs are installed. (Chapter 6, "Installing RPM Software", will tell you how.) When searching for the RPMs, remember that the filename usually starts with the software package name by a version number, as in sendmail-8.12.10-1.1.1.i386.rpm.
Starting Sendmail You can use the chkconfig command to get sendmail configured to start at boot: [root@bigboy tmp]# chkconfig sendmail on
To start, stop, and restart sendmail after booting, use [root@bigboy tmp]# service sendmail start [root@bigboy tmp]# service sendmail stop [root@bigboy tmp]# service sendmail restart
Remember to restart the sendmail process every time you make a change to the configuration files for the changes to take effect on the running process. You can also test whether the sendmail process is running with the pgrep command: [root@bigboy tmp]# pgrep sendmail
You should get a response of plain old process ID numbers.
How To Restart Sendmail After Editing Your Configuration Files In this chapter, you'll see that sendmail uses a variety of configuration files that require different treatments for their commands to take effect. This little script encapsulates all the required post configuration steps. #!/bin/bash cd /etc/mail make newaliases /etc/init.d/sendmail restart
It first runs the make command, which creates a new sendmail.cf file from the sendmail.mc file and compiles supporting configuration files in the /etc/mail directory according to the instructions in the file /etc/mail/Makefile. It then generates new e-mail aliases with the newaliases command, (this will be covered later), and then restarts sendmail. Use this command to make the script executable. chmod 700 filename
You'll need to run the script each time you change any of the sendmail configuration files described in the sections to follow.
The line in the script that restarts sendmail is only needed if you have made changes to the /etc/mail/sendmail.mc file, but I included it so that you don't forget. This may not be a good idea in a production system. Note: When sendmail starts, it reads the file sendmail.cf for its configuration. sendmail.mc is a more user friendly configuration file and really is much easier to fool around with without getting burned. The sendmail.cf file is located in different directories depending on the version of RedHat you use. The /etc/sendmail.cf file is used for versions up to 7.3, and /etc/mail/sendmail.cf is used for versions 8.0 and higher and Fedora Core.
The /etc/mail/sendmail.mc File You can define most of sendmail's configuration parameters in the /etc/mail/sendmail.mc file, which is then used by the m4 macros to create the /etc/mail/sendmail.cf file. Configuration of the sendmail.mc file is much simpler than configuration of sendmail.cf, but it is still often viewed as an intimidating task with its series of structured directive statements that get the job done. Fortunately, in most cases you won't have to edit this file very often.
How to Put Comments in sendmal.mc In most Linux configuration files a # symbol is used at the beginning of a line convert it into a comment line or to deactivate any commands that may reside on that line. The sendmail.mc file doesn't use this character for commenting, but instead uses the string "dnl". Here are some valid examples of comments used with the sendmail.mc configuration file: •
These statements are disabled by dnl commenting.
dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA') dnl # DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA') •
This statement is incorrectly disabled:
# DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA') •
This statement is active:
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')
Configuring DNS for sendmail Remember that you will never receive mail unless you have configured DNS for your domain to make your new Linux box mail server the target of the DNS domain's MX record. See either Chapter 18, "Configuring DNS", or Chapter 19, "Dynamic DNS", for details on how to do this.
Configure Your Mail Server's Name In DNS You first need to make sure that your mail server's name resolves in DNS correctly. For example, if your mail server's name is bigboy and it you intend for it to mostly handle mail for the domain my-site.com, then bigboy.my-site.com must correctly resolve to the IP address of one of the mail server's interfaces. You can test this using the host command: [root@smallfry tmp]# host bigboy.my-site.com bigboy.my-site.com has address 192.168.1.100 [root@smallfry tmp]#
You will need to fix your DNS server's entries if the resolution isn't correct.
Configure The /etc/resolv.conf File The sendmail program expects DNS to be configured correctly on the DNS server. The MX record for your domain must point to the IP address of the mail server. The program also expects the files used by the mail server's DNS client to be configured correctly. The first one is the /etc/resolv.conf file in which there must be a domain directive that matches one of the domains the mail server is expected to handle mail for. Finally, sendmail expects a nameserver directive that points to the IP address of the DNS server the mail server should use to get its DNS information. For example, if the mail server is handling mail for my-site.com and the IP address of the DNS server is 192.168.1.100, there must be directives that look like this: domain my-site.com nameserver 192.168.1.100
An incorrectly configured resolv.conf file can lead to errors when running the m4 command to process the information in your sendmail.mc file. WARNING: local host name (smallfry) is not qualified; fix $j in config file
The /etc/hosts File The /etc/hosts file also is used by DNS clients and also needs to be correctly configured. Here is a brief example of the first line you should expect to see in it: 127.0.0.1 bigboy.my-site.com localhost.localdomain localhost bigboy
The entry for 127.0.0.1 must always be followed by the fully qualified domain name (FQDN) of the server. In the case above it would be bigboy.my-site.com. Then you must have an entry for localhost and localhost.localdomain. Linux does not function properly if the 127.0.0.1 entry in /etc/hosts doesn't also include localhost and localhost.localdomain. Finally you can add any other aliases your host may have to the end of the line.
How To Configure Linux Sendmail Clients All Linux mail clients in your home or company need to know which server is the mail server. This is configured in the sendmail.mc file by setting the SMART_HOST statement to include the mail server. In the example below, the mail server has been set to mail.my-site.com, the mail server for the my-site.com domain. define(`SMART_HOST',`mail.my-site.com')
If you don't have a mail server on your network, you can either create one, or use the one offered by your ISP. Once this is done, you need to process the sendmail.mc file and restart sendmail. To do this, run the restarting script we from earlier in the chapter. If the sendmail server is a Linux server, then the /etc/hosts file will also have to be correctly configured too.
Converting From a Mail Client to a Mail Server All Linux systems have a virtual loopback interface that lives only in memory with an IP address of 127.0.0.1. As mail must be sent to a target IP address even when there is no NIC in the box, sendmail therefore uses the loopback address to send mail between users on the same Linux server. To become a mail server, and not a mail client, sendmail needs to be configured to listen for messages on NIC interfaces as well. 1) Determine which NICs sendmail is running on. You can see the interfaces on which sendmail is listening with the netstat command. Because sendmail listens on
TCP port 25, you use netstat and grep for 25 to see a default configuration listening only on IP address 127.0.0.1 (loopback): [root@bigboy tmp]# netstat -an | grep :25 | grep tcp tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN [root@bigboy tmp]#
2) Edit sendmail.mc to make sendmail listen on all interfaces. If sendmail is listening on the loopback interface only, you should comment out the daemon_options line in the /etc/mail/sendmail.mc file with dnl statements. It is also good practice to take precautions against spam by not accepting mail from domains that don't exist by commenting out the accept_unresolvable_domains feature too. See the fourth and next to last lines in the example. dnl dnl dnl dnl dnl dnl dnl ... ... ... dnl dnl dnl dnl dnl dnl dnl dnl
This changes sendmail to only listen on the loopback device 127.0.0.1 and not on any other network devices. Comment this out if you want to accept email over the network. DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')
We strongly recommend to comment this one out if you want to protect yourself from spam. However, the laptop and users on computers that do not have 24x7 DNS do need this. FEATURE(`accept_unresolvable_domains')dnl FEATURE(`relay_based_on_MX')dnl
Note: You need to be careful with the accept_unresolvable_names feature. In the sample network, bigboy the mail server does not accept e-mail relayed from any of the other PCs on your network if they are not in DNS. Chapter 18, "Configuring DNS", shows how to create your own internal domain just for this purpose. Note: If your server has multiple NICs and you want it to listen to one of them, then you can uncomment the localhost DAEMON_OPTIONS entry and add another one for the IP address of the NIC on which to wish to accept SMTP traffic. 3) Comment out the SMART_HOST Entry in sendmal.mc. The mail server doesn't need a SMART_HOST entry in its sendmail.mc file. Comment this out with a dnl at the beginning. dnl define(`SMART_HOST',`mail.my-site.com')
4) Regenerate the sendmail.cf file, and restart sendmail. Again, you can do this with the restart script from the beginning of the chapter. 5) Make sure sendmail is listening on all interfaces (0.0.0.0). [root@bigboy tmp]# netstat -an | grep :25 | grep tcp tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN [root@bigboy tmp]#
You have now completed the first phase of converting your Linux server into a sendmail server by enabling it to listen to SMTP traffic on its interfaces. The following sections will show you how to define what type of mail it should handle and the various ways this mail can be processed.
A General Guide To Using The sendmail.mc File The sendmail.mc file can seem jumbled. To make it less cluttered I usually create two easily identifiable sections in it with all the custom commands I've ever added. The first section is near the top where the FEATURE statements usually are, and the second section is at the very bottom. Sometimes sendmail will archive this file when you do a version upgrade. Having easily identifiable modifications in the file will make post upgrade reconfiguration much easier. Here is a sample: dnl ***** Customised section 1 start ***** dnl dnl FEATURE(delay_checks)dnl FEATURE(masquerade_envelope)dnl FEATURE(allmasquerade)dnl FEATURE(masquerade_entire_domain)dnl dnl dnl dnl ***** Customised section 1 end *****
The /etc/mail/relay-domains File The /etc/mail/relay-domains file is used to determine domains from which it will relay mail. The contents of the relay-domains file should be limited to those domains that can be trusted not to originate spam. By default, this file does not exist in a standard RedHat / Fedora install. In this case, all mail sent from my-super-dupersite.com and not destined for this mail server will be forwarded:
my-super-duper-site.com
One disadvantage of this file is that controls mail based on the source domain only, and source domains can be spoofed by spam e-mail servers. The /etc/mail/access file has more capabilities, such as restricting relaying by IP address or network range and is more commonly used. If you delete /etc/mail/relay-domains, then relay access is fully determined by the /etc/mail/access file. Be sure to run the restart sendmail script from the beginning of the chapter for these changes to take effect.
The /etc/mail/access File You can make sure that only trusted PCs on your network have the ability to relay mail via your mail server by using the /etc/mail/access file. That is to say, the mail server will relay mail only for those PCs on your network that have their e-mail clients configured to use the mail server as their outgoing SMTP mail server. (In Outlook Express, you set this using: Tools>Accounts>Properties>Servers) If you don't take the precaution of using this feature, you may find your server being used to relay mail for spam e-mail sites. Configuring the /etc/mail/access file will not stop spam coming to you, only spam flowing through you. The /etc/mail/access file has two columns. The first lists IP addresses and domains from which the mail is coming or going. The second lists the type of action to be taken when mail from these sources or destinations is received. Keywords include RELAY, REJECT, OK (not ACCEPT), and DISCARD. There is no third column to state whether the IP address or domain is the source or destination of the mail, sendmail assumes it could be either and tries to match both. All other attempted relayed mail that doesn't match any of the entries in the /etc/mail/access file, sendmail will reject. Despite this, my experience has been that control on a per email address basis is much more intuitive via the /etc/mail/virtusertable file. The sample file that follows allows relaying for only the server itself (127.0.0.1, localhost), two client PCs on your home 192.168.1.X network, everyone on your 192.168.2.X network, and everyone passing e-mail through the mail server from servers belonging to my-site.com. Remember that a server will be considered a part of my-site.com only if its IP address can be found in a DNS reverse zone file: localhost.localdomain localhost 127.0.0.1 192.168.1.16 192.168.1.17 192.168.2 my-site.com
RELAY RELAY RELAY RELAY RELAY RELAY RELAY
You'll then have to convert this text file into a sendmail readable database file named /etc/mail/access.db. Here are the commands you need: [root@bigboy tmp]# cd /etc/mail [root@bigboy mail]# make
The sendmail restart script we configured at the beginning of the chapter does this for you too. Remember that the relay security features of this file may not work if you don't have a correctly configured /etc/hosts file.
The /etc/mail/local-host-names File When sendmail receives mail, it needs a way of determining whether it is responsible for the mail it receives. It uses the /etc/mail/local-host-names file to do this. This file has a list of hostnames and domains for which sendmail accepts responsibility. For example, if this mail server was to accept mail for the domains my-site.com and another-site then the file would look like this: my-site.com another-site.com
In this case, remember to modify the MX record of the another-site.com DNS zonefile point to my-site.com. Here is an example (Remember each "." is important): ; Primary Mail Exchanger for another-site.com another-site.com. MX 10 mail.my-site.com.
Which User Should Really Receive The Mail? After checking the contents of the virtusertable, sendmail checks the aliases files to determine the ultimate recipient of mail.
The /etc/mail/virtusertable file The /etc/mail/virtusertable file contains a set of simple instructions on what to do with received mail. The first column lists the target email address and the second column lists the local user's mail box, a remote email address, or a mailing list entry in the /etc/aliases file to which the email should be forwarded. If there is no match in the virtusertable file, sendmail checks for the full email address in the /etc/aliases file.
[email protected] @another-site.com
[email protected] [email protected] [email protected] @my-site.com
webmasters marc
[email protected] paul paul error:nouser User unknown
In this example, mail sent to: • • •
[email protected] will go to local user (or mailing list) webmasters, all other mail to another-site.com will go to local user marc. sales at my-site.com will go to the sales department at my-othersite.com. paul and finance at my-site.com goes to local user (or mailing list) paul
All other users at my-site.com receive a bounce back message stating "User unknown". After editing the /etc/mail/virtusertable file, you have to convert it into a sendmailreadable database file named /etc/mail/virtusertable.db with two commands: [root@bigboy tmp]# cd /etc/mail [root@bigboy mail]# make
If these lines look like you've seen them before, you have: They're in your allpurpose sendmail restart script.
The /etc/aliases File You can think of the /etc/aliases file as a mailing list file. The first column has the mailing list name (sometimes called a virtual mailbox), and the second column has the members of the mailing list separated by commas. To start, sendmail searches the first column of the file for a match. If there is no match, then sendmail assumes the recipient is a regular user on the local server and deposits the mail in their mailbox. If it finds a match in the first column, sendmail notes the nickname entry in the second column. It then searches for the nickname again in the first column to see if the recipient isn't on yet another mailing list. If sendmail doesn't find a duplicate, it assumes the recipient is a regular user on the local server and deposits the mail in their mailbox.
If the recipient is a mailing list, then sendmail goes through the process all over again to determine if any of the members is on yet another list, and when it is all finished, they all get a copy of the e-mail message. In the example that follows, you can see that mail sent to users bin, daemon, lp, shutdown, apache, named, and so on by system processes will all be sent to user (or mailing list) root. In this case, root is actually an alias for a mailing list consisting of user marc and
[email protected]. # Basic system aliases -- these MUST be present. mailer-daemon: postmaster postmaster: root # General redirections for pseudo accounts. bin: root daemon: root ... ... abuse: root # trap decode to catch security attacks decode: root # Person who should get root's mail root: marc,
[email protected]
Notice that there are no spaces between the mailing list entries for root: You will get errors if you add spaces. Note: The default /etc/aliases file installed with RedHat / Fedora has the last line of this sample commented out with a #, you may want to delete the comment and change user marc to another user. Also after editing this file, you'll have to convert it into a sendmail readable database file named /etc/aliases.db. Here is the command to do that: [root@bigboy tmp]# newaliases
In this simple mailing list example, mail sent to root actually goes to user account marc and
[email protected]. Because aliases can be very useful, here are a few more list examples for your /etc/aliases file. •
Mail to "
[email protected]" goes to users "peter", "paul" and "mary".
# Directors of my SOHO company directors: peter,paul,mary •
Mail sent to "
[email protected]" goes to users "grandma", "brother" and "sister"
# My family family:
grandma,brother,sister
•
Mail sent to admin-list gets sent to all the users listed in the file /home/mailings/admin-list.
# My mailing list file admin-list: ":include:/home/mailings/admin-list"
The advantage of using mailing list files is that the admin-list file can be a file that trusted users can edit, user root is only needed to update the aliases file. Despite this, there are some problems with mail reflectors. One is that bounce messages from failed attempts to broadcast go to all users. Another is that all subscriptions and unsubscriptions have to be done manually by the mailing list administrator. If either of these are a problem for you, then consider using a mailing list manager, such as majordomo. One important note about the /etc/aliases file: By default your system uses sendmail to mail system messages to local user root. When sendmail sends e-mail to a local user, the mail has no To: in the e-mail header. If you then use a mail client with a spam mail filtering rule to reject mail with no To: in the header, such as Outlook Express or Evolution, you may find yourself dumping legitimate mail. To get around this, try making root have an alias for a user with a fully qualified domain name, this forces sendmail to insert the correct fields in the header; for example: # Person who should get root's mail root:
[email protected]
Sendmail Masquerading Explained If you want your mail to appear to come from
[email protected] and not
[email protected], then you have two choices: •
•
Configure your email client, such as Outlook Express, to set your email address to
[email protected]. (I'll explain this in the "Configuring Your POP Mail Server" section.). Set up masquerading to modify the domain name of all traffic originating from and passing trough your mail server.
Configuring masquerading In the DNS configuration, you made bigboy the mail server for the domain mysite.com. You now have to tell bigboy in the sendmail configuration file sendmail.mc that all outgoing mail originating on bigboy should appear to be coming from my-
site.com; if not, based on our settings in the /etc/hosts file, mail will appear to come from mail.my-site.com. This isn't terrible, but you may not want your Web site to be remembered with the word "mail" in front of it. In other words you may want your mail server to handle all email by assigning a consistent return address to all outgoing mail, no matter which server originated the email. You can solve this by editing your sendmail.mc configuration file and adding some masquerading commands and directives: FEATURE(always_add_domain)dnl FEATURE(`masquerade_entire_domain')dnl FEATURE(`masquerade_envelope')dnl FEATURE(`allmasquerade')dnl MASQUERADE_AS(`my-site.com')dnl MASQUERADE_DOMAIN(`my-site.com.')dnl MASQUERADE_DOMAIN(localhost)dnl MASQUERADE_DOMAIN(localhost.localdomain)dnl
The result is that: •
•
•
•
The MASQUERADE_AS directive makes all mail originating on bigboy appear to come from a server within the domain my-site.com by rewriting the email header. The MASQUERADE_DOMAIN directive makes mail relayed via bigboy from all machines in the another-site.com and localdomain domains appear to come from the MASQUERADE_AS domain of my-site.com. Using DNS, sendmail checks the domain name associated with the IP address of the mail relay client sending the mail to help it determine whether it should do masquerading or not. FEATURE masquerade_entire_domain makes sendmail masquerade servers named *my-site.com, and *another-site.com as my-site.com. In other words, mail from sales.my-site.com would be masqueraded as my-site.com. If this wasn't selected, then only servers named my-site.com and my-othersite.com would be masqueraded. Use this with caution when you are sure you have the necessary authority to do this. FEATURE allmasquerade makes sendmail rewrite both recipient addresses and sender addresses relative to the local machine. If you cc: yourself on an outgoing mail, the other recipient sees a cc: to an address he knows instead of one on localhost.localdomain. Note: Use FEATURE allmasquerade with caution if your mail server handles email for many different domains and the mailboxes for the users in these domains reside on the mail server. The allmasquerade statement causes all mail destined for these mailboxes to appear to be destined for users in the domain defined in the MASQUERADE_AS statement. In other words, if MASQUERADE_AS is my-site.com and you use allmasquerade, then mail for
[email protected] enters the correct mailbox but sendmail rewrites the To:, making the e-mail appear to be sent to
[email protected] originally.
•
•
FEATURE always_add_domain always masquerades email addresses, even if the mail is sent from a user on the mail server to another user on the same mail server. FEATURE masquerade_envelope rewrites the email envelope just as MASQUERADE_AS rewrote the header.
Masquerading is an important part of any mail server configuration as it enables systems administrators to use multiple outbound mail servers, each providing only the global domain name for a company and not the fully qualified domain name of the server itself. All email correspondence then has a uniform email address format that complies with the company's brand marketing policies. Note: E-mail clients, such as Outlook Express, consider the To: and From: statements as the e-mail header. When you choose Reply or Reply All in Outlook Express, the program automatically uses the To: and From: in the header. It is easy to fake the header, as spammers often do; it is detrimental to e-mail delivery, however, to fake the envelope. The e-mail envelope contains the To: and From: used by mailservers for protocol negotiation. It is the envelope's From: that is used when e-mail rejection messages are sent between mail servers.
Testing Masquerading The best way of testing masquerading from the Linux command line is to use the "mail -v username" command. I have noticed that "sendmail -v username" ignores masquerading altogether. You should also tail the /var/log/maillog file to verify that the masquerading is operating correctly and check the envelope and header of test email received by test email accounts.
Other Masquerading Notes By default, user "root" will not be masqueraded. To remove this restriction use: EXPOSED_USER(`root')dnl
command in /etc/mail/sendmail.mc. You can comment this out if you like with a "dnl" at the beginning of the line and running the sendmail start script.
Using Sendmail to Change the Sender's Email Address Sometimes masquerading isn't enough. At times you may need to change not only the domain of the sender but also the username portion of the sender's e-mail address. For example, perhaps you bought a program for your SOHO office that sends out notifications to your staff, but the program inserts its own address as sender's address, not that of the IT person. Web-based CGI scripts tend to run as user apache and, therefore, send mail as user apache too. Often you won't want this, not only because apache's e-mail address may not be a suitable, but also because some anti-spam programs check to ensure that the From:, or source e-mail address, actually exists as a real user. If your virtusertable file allows e-mail to only predefined users, then queries about the apache user will fail, and your valid e-mail may be classified as being spam. With sendmail, you can change both the domain and username on a case-by-case basis using the genericstable feature: 1) Add these statements to your /etc/mail/sendmail.mc file to activate the feature: FEATURE(`genericstable',`hash -o /etc/mail/genericstable.db')dnl GENERICS_DOMAIN_FILE(`/etc/mail/generics-domains')dnl
2) Create a /etc/mail/generics-domains file that is just a list of all the domains that should be inspected. Make sure the file includes your server's canonical domain name, which you can obtain using the command: sendmail -bt -d0.1
Here is a sample /etc/mail/generics-domains file: my-site.com another-site.com bigboy.my-site.com
3) Create your /etc/mail/genericstable file. First sendmail searches the /etc/mail/generics-domains file for a list of domains to reverse map. It then looks at the /etc/mail/genericstable file for an individual email address from a matching domain. The format of the file is linux-username
[email protected]
Here is an example: alert peter apache
[email protected] [email protected] [email protected]
4) Run the sendmail restart script from the beginning of the chapter and then test. Your e-mails from linux-username should now appear to come from
[email protected].
Troubleshooting Sendmail There are a number of ways to test sendmail when it doesn't appear to work correctly. Here are a few methods you can use to fix some of the most common problems.
Testing TCP connectivity The very first step is to determine whether your mail server is accessible on the sendmail SMTP TCP port 25. Lack of connectivity could be caused by a firewall with incorrect permit, NAT, or port forwarding rules to your mail server. Failure could also be caused by the sendmail process being stopped. It is best to test this from both inside your network and from the Internet. Chapter 4, "Simple Network Troubleshooting", covers troubleshooting with TELNET.
Further Testing of TCP connectivity You can also mimic a full mail session using TELNET to make sure everything is working correctly. If you get a "500 Command not recognized" error message along the way, the cause is probably a typographical error. Follow these steps carefully. 1) Telnet to the mail server on port 25. You should get a response with a 220 status code. [root@bigboy tmp]# telnet mail.my-site.com 25 Trying mail.my-site.com... Connected to mail.my-site.com. Escape character is '^]'. 220 mail.my-site.com ESMTP server ready
2) Use the hello command to tell the mail server the domain you belong to. You should receive a message with a successful status 250 code at the beginning of the response. helo another-web-site.org
250 mail.my-site.com Hello c-24-4-97-110.client.comcast.net [24.4.97.110], pleased to meet you.
3) Inform the mail server from which the test message is coming with the MAIL FROM: statement. MAIL FROM:
[email protected] 250 2.1.0
[email protected]... Sender ok
4) Tell the mail server to whom the test message is going with the " RCPT TO:" statement. RCPT TO:
[email protected] 250 2.1.5
[email protected]... Recipient ok
5) Prepare the mail server to receive data with the DATA statement DATA 354 Enter mail, end with "." on a line by itself
6) Type the string "subject:" then type a subject. Type in your text message, ending it with a single period on the last line. For example. Subject: Test Message Testing sendmail interactively . 250 2.0.0 iA75r9si017840 Message accepted for delivery
7) Use the QUIT command to end the session. QUIT 221 2.0.0 mail.my-site.com closing connection Connection closed by foreign host. [root@bigboy tmp]#
Now verify that the intended recipient received the message, and check the system logs for any mail application errors.
The /var/log/maillog File Because sendmail writes all its status messages in the /var/log/maillog file, always monitor this file whenever you are doing changes. Open two TELNET, SSH, or console windows. Work in one of them and monitor the sendmail status output in the other using the command [root@bigboy tmp]# tail -f /var/log/maillog
Common Errors Due To Incomplete RPM Installation Both the newaliases and m4 commands require the sendmail-cf and m4 RPM packages. These must be installed. If they are not, you'll get errors when running various sendmail related commands. •
Sample Errors when running newaliases
[root@bigboy mail]# newaliases Warning: .cf file is out of date: sendmail 8.12.5 supports version 10, .cf file is version 0 No local mailer defined QueueDirectory (Q) option must be set [root@bigboy mail]# •
Sample errors when processing the sendmail.mc file
[root@bigboy mail]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf /etc/mail/sendmail.mc:8: m4: Cannot open /usr/share/sendmailcf/m4/cf.m4: No such file or directory [root@bigboy mail]# •
Sample errors when restarting sendmail
[root@bigboy mail]# service sendmail restart Shutting down sendmail: [ OK ] Shutting down sm-client: [FAILED] Starting sendmail: 554 5.0.0 No local mailer defined 554 5.0.0 QueueDirectory (Q) option must be set [FAILED] Starting sm-client: [ OK ] [root@bigboy mail]#
If these errors occur, make sure your m4, sendmail and senmail-cf RPM packages are installed correctly.
Incorrectly Configured /etc/hosts Files By default, Fedora inserts the hostname of the server between the 127.0.0.1 and the localhost entries in /etc/hosts like this: 127.0.0.1
bigboy
localhost.localdomain
localhost
Unfortunately in this configuration, sendmail will think that the server's FQDN is bigboy, which it will identify as being invalid because there is no extension at the
end, such as .com or .net. It will then default to sending e-mails in which the domain is localhost.localdomain. The /etc/hosts file is also important for configuring mail relay. You can create problems if you fail to place the server name in the FDQN for 127.0.0.1 entry. Here sendmail thinks that the server's FDQN was my-site and that the domain was all of .com. 127.0.0.1
my-site.com
localhost.localdomain
localhost
(Wrong!!!)
The server would therefore be open to relay all mail from any .com domain and would ignore the security features of the access and relay-domains files I'll describe later. As mentioned, a poorly configured /etc/hosts file can make mail sent from your server to the outside world appear as if it came from users at localhost.localdomain and not bigboy.my-site.com. Use the sendmail program to send a sample e-mail to someone in verbose mode. Enter some text after issuing the command and end your message with a single period all by itself on the last line, for example: [root@bigboy tmp]# sendmail -v
[email protected] test text test text .
[email protected]... Connecting to mail.another-site.com. via esmtp... 220 ltmail.another-site.com LiteMail v3.02(BFLITEMAIL4A); Sat, 05 Oct 2002 06:48:44 -0400 >>> EHLO localhost.localdomain 250-mx.another-site.com Hello [67.120.221.106], pleased to meet you 250 HELP >>> MAIL From:
250 ... Sender Ok >>> RCPT To: 250 ... Recipient Ok >>> DATA 354 Enter mail, end with "." on a line by itself >>> . 250 Message accepted for delivery [email protected]... Sent (Message accepted for delivery) Closing connection to mail.another-site.com. >>> QUIT [root@bigboy tmp]#
localhost.localdomain is the domain that all computers use to refer to themselves, it is therefore an illegal Internet domain. Consider an example: Mail sent from computer PC1 to PC2 appears to come from a user at localhost.localdomain on PC1 and is rejected. The rejected e-mail is returned to localhost.localdomain. PC2 sees that the mail originated from localhost.localdomain and thinks that the rejected e-
mail should be sent to a user on PC2 that may not exist. You end up with an error in /var/log/maillog: Oct 16 10:20:04 bigboy sendmail[2500]: g9GHK3iQ002500: SYSERR(root): savemail: cannot save rejected email anywhere Oct 16 10:20:04 bigboy sendmail[2500]: g9GHK3iQ002500: Losing ./qfg9GHK3iQ002500: savemail panic
You may also get this error if you are using a spam prevention program, such as a script based on the PERL module Mail::Audit. An error in the script could cause this type of message too. Another set of tell tale errors caused by the same problem can be generated when trying to send mail to a user (the example uses root) or creating a new alias database file. (I'll explain the newaliases command later.) [root@bigboy tmp]# sendmail -v root WARNING: local host name (bigboy) is not qualified; fix $j in config file [root@bigboy tmp]# newaliases WARNING: local host name (bigboy) is not qualified; fix $j in config file [root@bigboy tmp]#
An accompanying error in /var/log/maillog log file looks like this: Oct 16 10:23:58 bigboy sendmail[2582]: My unqualified host name (bigboy) unknown; sleeping for retry
Fighting SPAM Unsolicited Commercial Email (UCE or SPAM) can be annoying, time consuming to delete and in some cases dangerous when they contain viruses and worms. Fortunately there are ways you can use your mail server to combat SPAM.
Using Public SPAM Blacklists With Sendmail There are many publicly available lists of known open mail relay servers and spam generating mail servers on the Internet. Some are maintained by volunteers, others are managed by public companies, but in all cases they rely heavily on complaints from spam victims. Some spam blacklists simply try to determine whether the email is coming from a legitimate IP address.
The IP addresses of offenders usually remain on the list for six months to two years. In some cases, to provide additional pressure on the spammers, the blacklists include not only the offending IP address but also the entire subnet or network block to which it belongs. This prevents the spammers from easily switching their servers' IP addresses to the next available ones on their networks. Also, if the spammer uses a public data center, it is possible that their activities could also cause the IP addresses of legitimate e-mailers to be black listed too. It is hoped that these legitimate users will pressure the data center's management to evict the spamming customer. You can configure sendmail to use its dnsbl feature to both query these lists and reject the mail if a match is found. Here are some sample entries you can add to your /etc/sendmail.mc file; they should all be on one line. •
RFC-Ignorant: A valid IP address checker.
FEATURE(`dnsbl', `ipwhois.rfc-ignorant.org',`"550 Mail from " $&{client_addr} " refused. Rejected for bad WHOIS info on IP of your SMTP server - see http://www.rfc-ignorant.org/"') •
Easynet: An open proxy list.
FEATURE(`dnsbl', `proxies.blackholes.easynet.nl', `"550 5.7.1 ACCESS DENIED to OPEN PROXY SERVER "$&{client_name}" by easynet.nl DNSBL (http://proxies.blackholes.easynet.nl/errors.html)"', `')dnl •
Spamcop: A spammer blacklist.
FEATURE(`dnsbl', `bl.spamcop.net', `"450 Mail from " $`'&{client_addr} " refused - see http://spamcop.net/bl.shtml"') •
Spamhaus: A spammer blacklist.
FEATURE(`dnsbl',`sbl.spamhaus.org',`Rejected - see http://spamhaus.org/')dnl
Be sure to visit the URLs listed to learn more about the individual services.
Spamassassin Once sendmail receives an e-mail message, it hands the message over to procmail, which is the application that actually places the e-mail in user mailboxes on the mail server. You can make procmail temporarily hand over control to another program, such as a spam filter. The most commonly used filter is spamassassin. spamassassin doesn't delete spam, it merely adds the word "spam" to the beginning of the subject line of suspected spam e-mails. You can then configure the e-mail
filter rules in Outlook Express or any other mail client to either delete the suspect message or store it in a special Spam folder.
Downloading And Installing Spamassassin Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the RPMs, remember that the filename usually starts with the software package name and is followed by a version number, as in spamassassin-2.60-2.i386.rpm. (For help downloading, see Chapter 6, "Installing RPM Software").
Starting Spamassassin You can use the chkconfig command to get spamassassin configured to start at boot: [root@bigboy tmp]# chkconfig --level 35 spamassassin on
To start, stop, and restart spamassassin after booting: [root@bigboy tmp]# service spamassassin start [root@bigboy tmp]# service spamassassin stop [root@bigboy tmp]# service spamassassin restart
Configuring procmail for spamassassin The /etc/procmailrc file is used by procmail to determine the procmail helper programs that should be used to filter mail. This file isn't created by default. spamassassin has a template you can use called /etc/mail/spamassassin/spamassassin-spamc.rc. Copy the template to the /etc directory. [root@bigboy tmp]# cp /etc/mail/spamassassin/spamassassin-spamc.rc /etc/procmailrc
Configuring Spamassassin The spamassassin configuration file is named /etc/mail/spamassassin/local.cf. A full listing of all the options available in the local.cf file can be found in the Linux man pages using the following command:
[root@bigboy tmp]# man Mail::SpamAssassin::Conf
You can customize this fully commented sample configuration file to meet your needs. ################################################################### # See 'perldoc Mail::SpamAssassin::Conf' for # details of what can be adjusted. ################################################################### # # These values can be overridden by editing # ~/.spamassassin/user_prefs.cf (see spamassassin(1) for details) # # How many hits before a message is considered spam. The lower the # number the more sensitive it is. required_hits
5.0
# Whether to change the subject of suspected spam (1=Yes, 0=No) rewrite_subject 1 # Text to prepend to subject if rewrite_subject is used subject_tag *****SPAM***** # Encapsulate spam in an attachment (1=Yes, 0=No) report_safe 1 # Use terse version of the spam report (1=Yes, 0=No) use_terse_report 0 # Enable the Bayes system (1=Yes, 0=No) use_bayes 1 # Enable Bayes auto-learning (1=Yes, 0=No) auto_learn 1 # Enable or disable network checks (1=Yes, 0=No) skip_rbl_checks 0 use_razor2 1 use_dcc 1 use_pyzor 1 # Mail using languages used in these country codes will not be marked # as being possibly spam in a foreign language. # - english
ok_languages
en
# Mail using locales used in these country codes will not be marked # as being possibly spam in a foreign language. ok_locales
en
Be sure to restart spamassassin for your changes to take effect.
Testing spamassassin You can test the validity of your local.cf file by using the spamassassin command with the --lint option. This will list any syntax problems that may exist. In this example two errors were found and corrected before the command was run again. [root@bigboy tmp]# spamassassin -d --lint Created user preferences file: /root/.spamassassin/user_prefs config: SpamAssassin failed to parse line, skipping: use_terse_report 0 config: SpamAssassin failed to parse line, skipping: auto_learn 1 lint: 2 issues detected. please rerun with debug enabled for more information. [root@bigboy tmp]# vi /etc/mail/spamassassin/local.cf ... ... ... [root@bigboy tmp]# spamassassin -d --lint [root@bigboy tmp]
Startup spamassassin The final steps are to configure spamassassin to start on booting and then to start it. [root@bigboy tmp]# chkconfig spamassassin on [root@bigboy tmp]# service spamassassin start Starting spamd: [ OK ] [root@bigboy tmp]#
Tuning spamassassin You can tune the sensitivity of spamassassin to the type of spam you receive by adjusting the required_hits value in the local.cf file. This can be made easier by viewing the score spamassassin assigns a message in its header. In most GUI based email clients this can be done by looking at the email's properties. In this case, a
Nigerian email scam spam was detected and given a score of 20.1 and marked as spam. X-Spam-Status: Yes, score=20.1 required=2.1 tests=DEAR_FRIEND, DNS_FROM_RFC_POST,FROM_ENDS_IN_NUMS,MSGID_FROM_MTA_HEADER,NA_DOLLARS, NIGERIAN_BODY1,NIGERIAN_BODY2,NIGERIAN_BODY3,NIGERIAN_BODY4, RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_SBL,RISK_FREE,SARE_FRAUD_X3, SARE_FRAUD_X4,SARE_FRAUD_X5,US_DOLLARS_3 autolearn=failed version=3.0.4 X-Spam-Report: * 0.5 FROM_ENDS_IN_NUMS From: ends in numbers * 0.2 RISK_FREE BODY: Risk free. Suuurreeee.... * 0.4 US_DOLLARS_3 BODY: Mentions millions of $ ($NN,NNN,NNN.NN) * 0.8 DEAR_FRIEND BODY: Dear Friend? That's not very dear! * 2.2 NA_DOLLARS BODY: Talks about a million North American dollars * 1.8 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net * [Blocked - see ] * 1.1 RCVD_IN_SBL RBL: Received via a relay in Spamhaus SBL * [213.185.106.3 listed in sbl-xbl.spamhaus.org] * 1.4 DNS_FROM_RFC_POST RBL: Envelope sender in postmaster.rfcignorant.org * 1.9 NIGERIAN_BODY3 Message body looks like a Nigerian spam message 3+ * 2.9 NIGERIAN_BODY1 Message body looks like a Nigerian spam message 1+ * 1.4 NIGERIAN_BODY4 Message body looks like a Nigerian spam message 4+ * 1.7 SARE_FRAUD_X5 Matches 5+ phrases commonly used in fraud spam * 0.5 NIGERIAN_BODY2 Message body looks like a Nigerian spam message 2+ * 1.7 SARE_FRAUD_X3 Matches 3+ phrases commonly used in fraud spam * 1.7 SARE_FRAUD_X4 Matches 4+ phrases commonly used in fraud spam * 0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay
Limiting your spam fighting efforts to the required_hits value isn't usually adequate. You will probably need additional spamassassin tools to be more selective and accurate in your tests. This will be covered next.
The Rules du Jour Spamassassin Tool There are groups of open source developers that create and update customized spamassassin configuration files that make the tool even more effective. They have even made life easier for the harried systems administrator by creating a script named rules_du_jour that, on a daily basis, will automatically download the rules you select.
The rules_du_jour script can be downloaded from its website at http://www.exit0.us/index.php?pagename=RulesDuJour which has easy to understand installation instructions, but there are some caveats which need to be explained.
The /etc/rulesdujour/config Configuration File Rules du Jour's configuration file located at /etc/rulesdujour/config has four variables that need to be defined. Each must be enclosed in quotation marks. The first is SA_DIR, which defines the directory in which you have installed spamassassin. The second is MAIL_ADDRESS which defines the address to which Rules du Jour sends its status messages. The third, SA_RESTART, is the command to be used to restart spamassassin each time the rules_du_jour script is run. The final parameter, TRUSTED_RULESETS, is the most complicated. It is a space delimited list of all the rules you wish to use. A full list can be found on the Rules du Jour website but there isn't much explanation about what they do and how sensitive each one is to marking email as being spam. Fortunately, you can get this information from the "Rules" section of the Rules Emporium site (http://www.rulesemporium.com/). It is important to read the notes for each rule they sometimes have sub-groupings of rules that may more suitable for your needs. For example, the SARE_HTML rule includes all the rules in SARE_HTML0, SARE_HTML1, SARE_HTML2, SARE_HTML3 and SARE_HTML_ENG, but according to Rules Emporium, only SARE_HTML0 has a low degree of false positives. Here is a sample of a /etc/rulesdujour/config configuration file that has taken advantage of some of the more popular and reliable rules. # # File: /etc/rulesdujour/config # # # Script information can be found at: # # http://www.exit0.us/index.php?pagename=RulesDuJour # SA_DIR="/etc/mail/spamassassin" MAIL_ADDRESS="[email protected]" SA_RESTART="service spamd restart" TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU SARE_URI0 SARE_URI1 ANTIDRUG SARE_SPOOF SARE_BAYES_POISON_NXM SARE_OEM SARE_RANDOM SARE_FRAUD SARE_HEADER0 SARE_HEADER2 SARE_HTML0 SARE_SPECIFIC SARE_BML SARE_GENLSUBJ0 SARE_GENLSUBJ2 SARE_WHITELIST"
Note: The Rules du Jour and Rules Emporium sites use the terms "spam" and "ham" frequently. Spam is unwanted email, while ham is the opposite.
Installing Rules du Jour Installation is fairly simple, here is how it is done: 1) Download the rules_du_jour script with the wget command, make it executable and place it in the /usr/local/bin directory. [root@bigboy tmp]# wget http://sandgnat.com/rdj/rules_du_jour --10:58:27-- http://sandgnat.com/rdj/rules_du_jour => `rules_du_jour' Resolving sandgnat.com... 208.42.148.125 Connecting to sandgnat.com|208.42.148.125|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 63,448 (62K) [application/octet-stream] 10:58:28 (79.59 KB/s) - `rules_du_jour' saved [63448/63448] [root@bigboy tmp]# chmod 700 rules_du_jour [root@bigboy tmp]# mv rules_du_jour /usr/local/bin
2) Create and edit your /etc/rulesdujour/config configuration file. [root@bigboy tmp]# mkdir -p /etc/rulesdujour [root@bigboy tmp]# vi /etc/rulesdujour/config
3) Run the rules_du_jour script, and then run spamassassin in lint mode to test for errors. There should be none. [root@bigboy tmp]# /usr/local/bin/rules_du_jour exec: curl -w %{http_code} --compressed -O -R -s -S -z /etc/mail/spamassassin/RulesDuJour/rules_du_jour http://sandgnat.com/rdj/rules_du_jour 2>&1 curl_output: 304 ------ TRIPWIRE -----RULESET_NAME=TRIPWIRE INDEX=0 CF_URL=http://www.rulesemporium.com/rules/99_FVGT_Tripwire.cf ... ... ... /^\s*#.*(vers?|version|rev|revision)[:\.\s]*[0-9]/i;' | sort | tail -n 1 CF_MUNGE_SCRIPT= [root@bigboy tmp]# spamassassin -d --lint [root@bigboy tmp]#
4) The final step is to add /usr/local/bin/rules_du_jour to your cron table. In this case, I have just created the file /etc/cron.d/rulesdujour with the following entry: # # Get latest SpamAssassin rules. Runs at 12:23AM every day. # 0 23 * * * root /usr/local/bin/rules_du_jour
5) You may have to restart crond to get this configuration to become active [root@bigboy tmp]# service crond restart Stopping crond: [ OK ] Starting crond: [ OK ] [root@bigboy tmp]#
My experience with Rules du Jour has been very good. Without it, I could only set the required_hits value in the /etc/mail/spamassassin/local.cf file to a relatively insensitive value of 4.5. Anything lower would create too many false positives and valuable mail would be marked as being suspicious. Rules du Jour selectively raises the score of specific types of spam such that I can make the required_hits setting have a very sensitive value of 2.1 with very few false positives.
Using Greylisting To maximize the effect of their efforts, spammers try to send email as quickly as possible. They take note of the emails that bounce, so that they know which addresses to remove from their lists to make their next mailing more efficient. When mail servers receive mail too rapidly for them to handle, they can ask the sender to try again later. Spammers often view resending emails to valid addresses as a waste of computing time that could be used to send mail to brand new addresses that belong to faster mail servers. Emails that need to be resent are usually abandoned. Some emails need reliable delivery to be effective and the senders of these types of messages are willing to resend. These include bank statement notifications, ecommerce purchase confirmations, and subscription newsletters. In a previous section we saw where spamassassin always rejects emails from blacklisted sources. With greylisting, sources are just asked to resend. One of the most popular greylist mail filter (milter) products is the milter-greylist package which also works seamlessly with spamassassin. It is easy to use and I’ll discuss how can be configured on your mail server.
Downloading and Installing milter-greylist Installing milter-greylist is relatively simple, but there are a lot of steps. Here’s how to do it: 1. You will have to first install the sendmail-devel software package. Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the RPMs, remember that the filename usually starts with the software package name and is followed by a version number, as in sendmail-devel8.13.1-2.rpm. (For help downloading, see Chapter 6, "Installing RPM Software"). 2. The next step is to download the milter-greylist tar file which you can download from the milter’s website at http://projects.puremagic.com/greylisting/. In this case we download the version 2.0.2 file using the wget command. [root@bigboy tmp]# wget ftp://ftp.espci.fr/pub/milter-greylist/miltergreylist-2.0.2.tgz --10:48:06-- ftp://ftp.espci.fr/pub/milter-greylist/milter-greylist2.0.2.tgz => `milter-greylist-2.0.2.tgz' Resolving ftp.espci.fr... 193.54.82.47 Connecting to ftp.espci.fr|193.54.82.47|:21... connected. Logging in as anonymous ... Logged in! => SYST ... done. => PWD ... done. => TYPE I ... done. => CWD /pub/milter-greylist ... done. => PASV ... done. => RETR milter-greylist-2.0.2.tgz ... done. Length: 116,459 (114K) (unauthoritative) 10:48:20 (70.79 KB/s) - `milter-greylist-2.0.2.tgz' saved [116459] [root@bigboy tmp]#
3. Now that you have the file, you’ll have to extract the contents using the tar command. [root@bigboy tmp]# tar -xzvf milter-greylist-2.0.2.tgz milter-greylist-2.0.2 milter-greylist-2.0.2/ChangeLog milter-greylist-2.0.2/Makefile milter-greylist-2.0.2/Makefile.in milter-greylist-2.0.2/README … … … milter-greylist-2.0.2/spf.h milter-greylist-2.0.2/sync.c milter-greylist-2.0.2/sync.h [root@bigboy tmp]#
4. Now enter the newly created milter-greylist directory and compile the package with the configure command. Take a look at the README file before doing this as there may be some additional options you require, but for most cases the defaults are sufficient.
[root@bigboy milter-greylist-2.0.2]# ./configure && make && make install checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes ... ... ... /usr/bin/install -c -d -m 755 /etc/mail test -f /etc/mail/greylist.conf -o -f /etc/mail/greylist.except || \ /usr/bin/install -c -m 644 greylist.conf /etc/mail /usr/bin/install -c -d -m 755 -o root /var/milter-greylist [root@bigboy milter-greylist-2.0.2]#
The next step is to configure the package, which will be covered next.
Configuring milter-greylist Configuring milter-greylist requires these four quick steps: 1. Add the milter-greylist statements listed in the README file to your /etc/mail/sendmail.mc file: INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/miltergreylist.sock') define(`confMILTER_MACROS_CONNECT', `j, {if_addr}') define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}') define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}') define(`confMILTER_MACROS_ENVRCPT', `{greylist}')
2. There will be a number of shell scripts in the milter-greylist tar directory that correspond to various versions of Linux. Copy the correct version to your /etc/init.d directory and use the chkconfig command to make sure the milter starts on the next reboot. [root@bigboy milter-greylist-2.0.2]# /etc/init.d/milter-greylist [root@bigboy milter-greylist-2.0.2]# greylist [root@bigboy milter-greylist-2.0.2]# [root@bigboy milter-greylist-2.0.2]# [root@bigboy milter-greylist-2.0.2]# milter-greylist 0:off 1:off 2:on [root@bigboy milter-greylist-2.0.2]#
cp rc-redhat.sh chmod 755 /etc/init.d/milterchkconfig --add milter-greylist chkconfig milter-greylist on chkconfig --list | grep milter 3:on 4:on 5:on 6:off
3. Edit the /etc/mail/greylist.conf configuration file. Here we set the “try again later” to five minutes and use the whitelist command to deactivate the timer for trusted networks so that mail is delivered immediately. # # File: /etc/mail/greylist.conf #
# How long a client has to wait before we accept # the messages it retries to send. Here, 1 hour. # greylist 5m # # Whitelist addresses within my own home/office network # acl whitelist addr 192.168.0.0/16
4. Start the milter with the service command. [root@bigboy milter-greylist-2.0.2]# service milter-greylist start Starting Milter-Greylist: [ OK ] [root@bigboy milter-greylist-2.0.2]#
Your new spam mitigation tool should now be fully functional. Delete the mistergreylist directory in /tmp and you should be ready to go!
Configuring milter-greylist Now that we have milter-greylist installed, we need to be able to do some basic troubleshooting. The /var/log/maillog file should be used to determine what is happening to your mail. Here are two samples of what to expect: Dec 24 00:32:31 bigboy sendmail[28847]: jBO8WVnG028847: Milter: to=, reject=451 4.7.1 Greylisting in action, please come back in 00:05:00 Dec 23 20:40:21 bigboy milter-greylist: jBO4eF2m027418: addr 211.115.216.225 from rcpt : autowhitelisted for 24:00:00
In the first entry, the email received is given a tag (jBO8WVnG028847) based on key characteristics in the mail header and a request is sent to the sender to resend the email in five minutes. Any email that is received with the same calculated key within the autowhite period configured in the greylist.conf file will then be automatically accepted without delay. In the second entry, the email has been resent and immediately accepted. Any other email from that source within the next 24 hours will be accepted without delay. Note: Greylisting is very effective, but you will have to tne its operation to make sure critical emails are not delayed at all. One soluton is to set the autowhite period in /etc/mail/greylist.conf to slightly more than 24 hours especially if you get mail from certain recipients, such as newsletters, on a daily basis. This makes them arrive without interruption.
A Simple PERL Script To Help Stop SPAM Blacklists won't stop everything, but you can limit the amount of unsolicited spam you receive by writing a small script to intercept your mail before it is written to your mailbox. This is fairly simple to do, because sendmail always checks the .forward file in your home directory for the name of this script. The sendmail program then looks for the filename in the directory /etc/smrsh and executes it. By default, PERL doesn't come with modules that are able to check e-mail headers and envelopes so you have to download them from CPAN (www.cpan.org). The most important modules are: • • • •
MailTools IO-Stringy MIME-tools Mail-Audit
I have written a script called mail-filter.pl that effectively filters out spam e-mail for my home system. A few steps are required to make the script work: 1. Install PERL and the PERL modules you downloaded from CPAN. 2. Place an executable version of the script in your home directory and modify the script's $FILEPATH variable point to your home directory. 3. Update file mail-filter.accept, which specifies the subjects and e-mail addresses to accept, and file mail-filter.reject, which specifies those to reject. 4. Update your .forward file and place an entry in /etc/smrsh. Mail-filter first rejects all e-mail based on the reject file and then accepts all mail found in the accept file. It then denies everything else. For a simple script with instructions on how to install the PERL modules, see Appendix II, "Codes, Scripts, and Configurations".
Configuring Your POP Mail Server Each user on your Linux box will get mail sent to their account's mail folder, but sendmail just handles mail sent to your my-site.com domain. If you want to retrieve the mail from your Linux box's user account using a mail client such as Evolution, Microsoft Outlook or Outlook Express, then you have a few more steps. You'll also have to make your Linux box a POP mail server. Fedora Linux comes with the easy to use dovecot IMAP/POP server RPM package which requires very little configuration after installation.
Installing Your POP Mail Server Most RedHat and Fedora Linux software products are available in the RPM format. You will need to make sure that the dovecot software RPM is installed. (Chapter 6, "Installing RPM Software", will tell you how.) When searching for the RPMs, remember that the filename usually starts with the software package name by a version number, as in dovecot-0.99.11-1.FC3.4.i386.rpm.
Starting Your POP Mail Server You can use the chkconfig command to get dovecot configured to start at boot: [root@bigboy tmp]# chkconfig dovecot on
To start, stop, and restart dovecot after booting, use [root@bigboy tmp]# service dovecot start [root@bigboy tmp]# service dovecot stop [root@bigboy tmp]# service dovecot restart
Remember to restart the dovecot process every time you make a change to the configuration files for the changes to take effect on the running process. You can also test whether the dovecot process is running with the pgrep command: [root@bigboy tmp]# pgrep dovecot
You should get a response of plain old process ID numbers:
The /etc/dovecot.conf File You can define most of dovecot's configuration parameters in the /etc/dovecot.conf file. By default dovecot will act as a server for IMAP, secure IMAP (IMAPS), POP and secure POP (POPS). You can limit this list by editing the protocols line in the /etc/dovecot.conf file and then restarting dovecot for the change to take effect. In the example below dovecot is configured to serve only POP3. # # File /etc/dovecot.conf sample # # Protocols we want to be serving imap imaps pop3 pop3s #protocols = imap imaps pop3 pop3s protocols = pop3
You can then use the netstat command to do a simple preliminary test to make sure dovecot is serving POP3 only. [root@bigboy tmp]# netstat -a | egrep -i 'pop|imap' tcp 0 0 *:pop3 *:* LISTEN [root@bigboy tmp]#
How To Configure Your Windows Mail Programs All your POP e-mail accounts are really only regular Linux user accounts in which sendmail has deposited mail. You can now configure your e-mail client such as Outlook Express to use your use your new POP/SMTP mail server quite easily. To configure POP Mail, set your POP mail server to be the IP address of your Linux mail server. Use your Linux user username and password when prompted. Next, set your SMTP mail server to be the IP address/domain name of your Linux mail server.
Configuring Secure POP Mail If you need to access your e-mail from the mail server via the Internet or some other insecure location, you may want to configure POP to work over an encrypted data channel. For this, modify the protocols section of the /etc/dovecot.conf file to include pop3s and then restart dovecot. Encrypted POP runs on TCP port 995, so firewall rules may need to be altered as well. Most POP clients support secure POP. For example, Windows configures it in the Advanced menu of the Outlook Express Account Configuration window.
How to handle overlapping email addresses. If you have user overlap, such as John Smith ([email protected]) and John Brown ([email protected]), both users will get sent to the Linux user account john by default. You have two options for a solution: •
Make the user part of the email address different, [email protected] and
[email protected] for example, and create Linux accounts john1 and john2. If the users insist on overlapping names, then you may need to modify your virtusertable file. •
Create the user accounts john1 and john2 and point virtusertable entries for [email protected] to account john1 and point [email protected] entries to account john2. The POP configuration in Outlook Express for each user should retrieve their mail via POP using john1 and john2, respectively.
With this trick you'll be able to handle many users belonging to multiple domains without many address overlap problems.
Troubleshooting POP Mail The very first troubleshooting step is to determine whether your POP server is accessible on the POP TCP port 110 or the secure POP port of 995. Lack of connectivity could be caused by a firewall with incorrect permit, NAT, or port forwarding rules to your server. Failure could also be caused by the xinetd process being stopped or the configuration files being disabled. Test this from both inside your network and from the Internet. (Troubleshooting TCP with TELNET is covered in Chapter 4, "Simple Network Troubleshooting") Linux status messages are logged to the file /var/log/messages. Use it to make sure all your files are loaded when you start xinetd. Check your configuration files if it fails to do so. This example starts xinetd and makes a successful secure POP query from a remote POP client: (Linux logging is covered in Chapter 5, "Troubleshooting Linux with syslog"). Aug 11 23:20:33 bigboy xinetd[18690]: START: pop3s pid=18693 from=172.16.1.103 Aug 11 23:20:33 bigboy ipop3d[18693]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:40 bigboy ipop3d[18693]: Login user=labmanager host=17216-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:40 bigboy ipop3d[18693]: Logout user=labmanager host=17216-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0 Aug 11 23:20:40 bigboy xinetd[18690]: EXIT: pop3s pid=18693 duration=7(sec) Aug 11 23:20:52 bigboy xinetd[18690]: START: pop3s pid=18694 from=172.16.1.103 Aug 11 23:20:52 bigboy ipop3d[18694]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:52 bigboy ipop3d[18694]: Login user=labmanager host=17216-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:52 bigboy ipop3d[18694]: Logout user=labmanager host=17216-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0 Aug 11 23:20:52 bigboy xinetd[18690]: EXIT: pop3s pid=18694 duration=0(sec)
Conclusion E-mail is an important part of any Web site, and you need to plan its configuration carefully to make it a seamless part of the Web experience of your visitors. Without it, your Web site won't seem complete. A fully functioning Web site is just the beginning. It needs to be maintained to reduce the risk of failure and monitored to help detect potential problems. Chapter 22, " Monitoring Server Performance", discusses many Linux-based tools that you can be use to track the health of your Linux server.
How to set up a mail server on a GNU / Linux system/II
How to set up a mail server on a GNU / Linux system Step by step guide to install Postfix Ubuntu + Postfix + Courier IMAP + MySQL + Amavisd-new + SpamAssassin + ClamAV + SASL + TLS + SquirrelMail/Roundcube + Postgrey Easy to follow howto on setting up a mail server with unlimited users and domains, with IMAP/Pop access, anti-spam, anti-virus, secure authentication, encrypted traffic, web mail interface and more. Based on an Ubuntu distribution platform, but instructions are distro generic. Examples are run on Amazon AWS ec2, but only for demonstration purposes. | More 10th edition
Author Ivar Abrahamsen License: Respect (CC by-sa) Last Update: 2010-06-11 Contact / Discuss
Contents •
Editions List of different versions of this document.
Introduction Brief description of this document. o o o
Aim Research Donate
Software Which software packages are we using and why.
Installation How to install all packages and which ones. o o o o
Distrobution Base Install Repositories Packages
Configuration Post install, what to configure for each section, with full command examples. o o o o o
Firewall (Shorewall) MTA (Postfix) Database (MySQL) Pop/IMAP (Courier) Content Checks (amivisd-new) Anti-Spam(SpamAssassin) Anti-Virus (ClamAV)
Policy Check (PostGrey) Authentication (SASL) Encryption (TLS) Webmail (SquirrelMail) Administration (phpMyAdmin)
o o o o
Data Creating the basic stub of data, and how to add your own. o o
Add users and domains Common SQL
Test Testing and troubleshooting each element. o o o o o o o o
Common problems Test strategy Switch debug on Tail, tail and tail again Telnet is your friend Can postfix receive? Can postfix send? Can courier read?
Initialize If receiving an already setup machine, a list of actions to do to initialize and configure it.
Extend Post working system, detailed instructions on optional features to add. o o o o o o o o o o o o
Remote MX mail backup Relay recipient lookup Local file backup Sender ID & SPF Spam Reporting White/Black lists PGP & S/MIME Relocation notice Pop-before-SMTP Auto Reply Block Addresses Throttle Output Mail Lists
o o o o
Admin software Google Apps / GMail Maildrop, spam folder and vacation messaging Roundcube webmail client
Elastic Compute Cloud (ec2) Amazons' hosting service. Used as examples for this howto. o o o o o
Impressions of EC2 ec2 introduction, tips and hwotos Using EC2 with this howto Amazon EC2 Images: AMIs EC2 Links
o o o o o o o o o o
About author Contact Why References Software Links Difference between Ubuntu versions Download Todo Change Log FAQ
Appendix
Search
Return to top.
Editions Edition
State
Started Updated
Description
1st
Released (outdated) 2004-01 2004-02
Based on Mandrake 9.1.
2nd
Released (outdated) 2004-02 2004-07
Based on Mandrake 10.x, but valid for all distributions. Very thorough. Includes package description, where to get the sources and binaries, how to build them or which RPMs to use, includes many refrences, etc etc. Starts off with a basic working server, then advances, extends and tightens it in
stages.
3rd
Released (outdated) 2005-05 2005-11
Based on Ubuntu 5.04, Hoary Hedgehog. More concise simplified guide to get an advanced server working quickly. Now includes SASL & TLS integration.
4th
Released (outdated) 2005-10 2005-12
Based on Breezy Badger, Ubuntu 5.10. Includes Postgrey
5th
Released (outdated) 2006-05 2006-11
Based on Dapper Drake, Ubuntu 6.06 LTS.
Scrapped
2006-11 2007-10
Was to be based on Edgy Eft, Ubuntu 6.10 or 7.04. include Domain Key signing. include my mail admin or my catchall aliases admin.
2008-04 2009-06
Updated, based on Ubuntu 8.04 LTS Hardy Heron. Using Amazon EC2 as example. (Tested with 8.10 & 9.04 as well)
2009-05 2009-11
Based on Ubuntu 8.10 (intrepid), then tested with 9.04 (jaunty) & 9.10 (karmic) as well. Using official Ubuntu ec2 as examples.
2009-11 2010-05
Based on Ubuntu 9.10 (karmic) using Canonical's cloud images. Added Roundcube webmail option.
2009-12 2010-06
Based on Ubuntu 10.04 LTS (lucid) using Canonical's cloud images.
6th
7th
8th
9th
Released
Released (superseeded)
Released (superseeded)
10th (this) Release (candidate)
Further details available in the change log and below in the introduction. Return to top.
Introduction Aim This is a step by step howto guide to set up a mail server on a GNU / Linux system. It is easy to follow, but you end up with a powerfull secure mail server. The server accepts unlimited domains and users, and all mail can be read via your favourite clients, or via web mail. It is secure, traffic can encrypted and it will block virtually all spam and viruses. Return to top.
Research Dont take my word for it! Research others opinions and methods. Look at my references, look at Postfix.org's howtos, read the excellent books available (E.g. Kyle's or Hildebrandt's), search the web or read the proper documentation. If you refer to this howto in your own document, or find useful links, then let me know.
Donate If you found this howto very useful, spread the word and help others? If this howto was exceptionally useful why not donate me some beer money? Or buy a postfix book using my amazon affiliate links further down? Or buy a t-shirt from my t-shirt shop? Otherwise send me a Thank You note?
Return to top.
UK
US
EU
Software
What software packages have/will I use and why. OS: Ubuntu Linux www.ubuntu.com
Ah the age old distro argument... Thankfully this set up should work on most distros. I used to base this howto on Mandrake(now Mandriva), and I started this new edition on a Gentoo box. But I don't have the patience for Gentoo, nor the money to stay with Mandriva Power editions. Why Ubuntu? Its free, simple and slick. As Ubuntu is derived from debian the installations used
here will be apt-get based. Please refer to my other editions for details on RPM or source based installations.
MTA: Postfix www.postfix.org
Simple, free and slick. Yup I am a sucker for anything that works easily. Postfix is powerfull, well established, but not too bloated, and is security concious from the start.
Pop/IMAP: Courier IMAP www.courier-mta.org/imap/
My first mail server installtion was with Courier. I have not found a reason to change this as again it is simple, and free.
Database: MySQL www.mysql.com
Although I use Firebird for my application development, (or Hibernate/CJDBC hybrids), MySQL is well supported for the sort of lookups required in a mail server.
Content Check: Amavisd-new www.ijs.si/software/amavisd/
Easy plug in solution for spam, virus checking etc.
Anti-Spam: SpamAssassin spamassassin.apache.org
Powerfull renowned spam fighting tool.
Anti-Virus: ClamAV www.clamav.net
Free virus scanner that can be trusted and includes update daemon.
Authentication: Cyrus SASL
www.imc.org/ietf-sasl/
Secure and trusted crypthography technology for authentication of SMTP traffic.
PostGrey isg.ee.ethz.ch/tools/postgrey/
Postgrey is an excellent little script to stop 99% of all spam. All it does is on first contact for specific from-to combinations, tells the sender server to try again in a little while, which most spammers cant afford to do. When proper servers try again after a few minutes it lets it through.
Encryption: TLS www.ietf.org/html.charters/tls-charter.html
Secure and trusted crypthography technology for encryption of SMTP traffic. Not too be confused with client encryption technology like GnuPG and S/MIME. They are covered in the extend section. Formerly referenced as SSL.
WebMail: SquirrelMail or Roundcube www.squirrelmail.org
Easy to set up php based web mail client. Extensive plugin selection. www.roundcube.net
Ajaxified prettier web mail client. Not quite as solid as SquirrelMail.
Platform: Amazon ec2 aws.amazon.com/ec2
This guide can be installed locally, co-located or in the cloud. My preference is ec2, and I provide ec2 based examples, however it makes no difference where you install your mail server. Please see software links appendix for further information about these software packages. In that section there is more links to documentation or forums, and viable alternatives, downloadable packages, versions details etc. Further software and tweaks are discussed in the extension section.
Also review other peoples opinion on these packages via my references. Return to top.
Installation • • • •
Distrobution Base Install Repositories Packages
Distribution This section is different for every distribution and for every version. This howto is based on Ubuntu and its base of debian which uses apt-get. Therefor this section uses apt packages to its fullest. For other installation method please refer to previous edition's software links and your own distribution for the documention for other ways of installing. My 2nd edition(outdated) has instructions for Mandriva, general RPM and tarball compiling. To follow the rest of this howto with another distrobution, you need to ensure all your packages have been installed with the same modules, E.g MySQL lookup on postfix and sasl, php in apache etc. I have set up mail servers using the 32bit and 64bit x86 platforms, and if all the packages are available then other, E.g. Mac platforms should work too.
Base Install With installing Ubuntu you have a choice of which base system to install. You may choose server or desktop image or very basic setups. I will assume a server install, but it should not differ. Or if you have chosen an ec2 based server, you can: • • •
Base it on a Canonical image Apply my ec2 Ubuntu server suggestions Use my prebuilt ec2 public images for this postfix mail server
Ps. Please note that after a while Ill stop specifying the use of sudo, as it is up to yourselves if you use it or use a priviliged user, e.g. root.
Repositories For assistance with repositories, refer to this article on ubuntu's wiki. A previous edition discussed repository configuration in more detail. But basically for this you need main and universe, and I also throw in the other "safe" ones: restricted and multiverse (and partner, when available). sudo vi /etc/apt/source.list If main and universe already is listed, this is a quick find and replace to add the others all over: :%s/main un/main restricted multiverse un/g
As mentioned in the previous edition you also might want to find a repository closer to your server.
Packages You need to install a whole bunch of packages. We will install them bit by bit. But first check your package sources are correctly pointing to main multiverse restricted universe repositories of your current Ubuntu version. sudo vi /etc/apt/sources.list Secondly update your current system: sudo aptitude update sudo aptitude safe-upgrade
MySQL First we'll install MySQL sudo aptitude install mysql-client mysql-server This will prompt you for a root password. Choose someting wise and remember it! For purpose of this tutorial I will set it to rootPASSWORD
Postfix Then we'll install postfix sudo aptitude install postfix postfix-mysql This will prompt you to choose type of email server. Select internet site It will also suggest a server name. Correct this if needed.
SASL sudo aptitude install libsasl2-modules libsasl2-modules-sql libgsasl7\ libauthen-sasl-cyrus-perl sasl2-bin libpam-mysql
ClamAV sudo aptitude install clamav-base libclamav6 clamav-daemon clamavfreshclam (Earlier vesions of Ubuntu may use libclamav5)
Amavis, SpamAssassin, postgrey
sudo aptitude install amavisd-new sudo aptitude install spamassassin spamc sudo aptitude install postgrey
SquirrelMail sudo aptitude install squirrelmail squirrelmail-locales php-pear php5cli
phpMyAdmin sudo aptitude install phpmyadmin
Enter Yes to set it up, enter root mysql password, enter a phpmyadmin mysql user password twice. Accept apache2 as the web server.
ShoreWall sudo aptitude install shorewall-common shorewall-perl # for earlier ubuntu versions use package shorewall instead sudo aptitude install shorewall shorewall-doc
Amazon provides a firewall/ access control for its servers, so not always needed then, but nice to have. And in all others situations; a must have.
Courier sudo aptitude install courier-base courier-authdaemon courier-authlibmysql \ courier-imap courier-imap-ssl courier-ssl will prompt you about
webdirectories. You can say no to this. It will also warn you about the certificate location. Ignore it.
Extras I also install a few other packages that I personally prefer. But nothing todo with the mail server. sudo aptitude install vim mutt lynx
Package status To find out which packages you may have installed, you can use for example: sudo dpkg --list | grep postfix or sudo aptitude search postfix
EC2 Bundle My AMI flurdy-amis/ubuntu-mail-server-clean is based on Canonical's official Ubuntu with these basic mail server packages installed.
Return to top.
Configuration •
Core/Simple o o o o
Firewall (Shorewall) MTA (Postfix) Database (MySQL) Pop/IMAP (Courier)
o
Content Checks (amivisd-new) Anti-Spam(SpamAssassin) Anti-Virus (ClamAV) Policy Check (PostGrey)
Advanced
Secure Authentication (SASL) Encryption (TLS) Webmail (SquirrelMail) Administration (phpMyAdmin) o o
• •
Simple mail server Now lets configure a simple mail server using some of the packages installed.
Firewall
Shorewall Not essential for an EC2 image. It is essential for a normal server. UFW is bundled with recent Ubuntu distributions, but I still prefer Shorewall for servers. Basically at first you want to only allow SSH. Then SMTP and IMAP from your IP only. When you are confident that the mail server is secure, you can open SMTP to the world. If you prefer you can also open IMAP to the world, unless you have a very small client IP range. Later you may open web access to the webmail and admin gui. This you may also restrict to specific IPs.
SSH only
By default Shorewall in Ubuntu has an empty set up. You can find the default values for Shorewall in /usr/share/doc/shorwall-common/default-config. And examples in /usr/share/doc/shorwall-common/examples. We will create a basic set up. First configure which network adapters we are accessing the net. cp /usr/share/doc/shorewall-common/default-config/interfaces /etc/shorewall/ vi /etc/shorewall/interfaces net eth0 detect dhcp,tcpflags,logmartians,nosmurfs
Then we will configure network zones cp /usr/share/doc/shorewallcommon/default-config/zones /etc/shorewall/ vi /etc/shorewall/zones Add the firewall if not there and the internet as a zone. fw firewall # loc ipv4 net ipv4
Then if needed to specify hosts you can do it in this file. E.g. If you wanto specify what is your home IP etc. cp /usr/share/doc/shorewall-common/defaultconfig/hosts /etc/shorewall/ vi /etc/shorewall/hosts # loc eth0:192.168.0.0/24
Then set what is the default policy for firewall access. cp /usr/share/doc/shorewall-common/default-config/policy /etc/shorewall/ vi /etc/shorewall/policy $FW net ACCEPT net $FW DROP info net all DROP info # The FOLLOWING POLICY MUST BE LAST all all REJECT info
For safety in case it goes down. cp /usr/share/doc/shorewall-common/default-
config/routestopped /etc/shorewall/ vi /etc/shorewall/routestopped eth0 0.0.0.0 routeback You may put in a netmask of your ip range if you are more
concerned. Now for the main firewall rules. You can find predetermined macro rules for Shorewall in /usr/share/shorewall. cp /usr/share/doc/shorewallcommon/default-config/rules /etc/shorewall/ vi /etc/shorewall/rules SSH/ACCEPT net $FW
Open for business Once your server is working come back to this step and open up SMTP and Web access to others. vi /etc/shorewall/rules Ping/ACCEPT net $FW # Permit all ICMP traffic FROM the firewall TO the net zone ACCEPT $FW net icmp # mail lines SMTP/ACCEPT net $FW SMTPS/ACCEPT net $FW Submission/ACCEPT net $FW IMAP/ACCEPT net $FW IMAPS/ACCEPT net $FW #web Web/ACCEPT net $FW
Firewall configuring is always risky business, as it is easy to lock yourself out. To test the setup syntax, run shorewall check Restart it with /etc/init.d/shorewall restart
Then to switch it on during boot: vi /etc/default/shorewall startup=1
For more details on IP Tables and Shorewall, look up its website. Return to top.
MTA
Postfix You should put the name of your server in this file sudo vi /etc/mailname Could be something like smtp.domain.name, where domain name obviously is replaced with your domain name. Now will open the main postfix configuration file: sudo vi /etc/postfix/main.cf Debian and Ubuntu already puts in some sensible default values in this file. You may need to comment some of them out if we put the same in as well. First specify the name of your server. # This is already done in /etc/mailname #myhostname= mail.example.com Next is the origin which is the domain appended to email from this machine, this can be your full servername, or domain name. # myorigin=/etc/mailname myorigin=example.com
Then decide what the greeting text will be. Enough info so it is useful, but not divelge everything to potential hackers. smtpd_banner = $myhostname ESMTP $mail_name
Next you need to decide whether to send all outgoing mail via another SMTP server, or send them yourself. I send via my ISP's server, so it has to worry about the queing etc. If you send it yourself then you are not reliant on 3rd party server. But you may risk more exposure and accidentally be blocked by spam blockers. And it is more work for your server. Also many servers block dynamic dns hosts, so you may find your server gets rejected. However choose whichever you are comfortable with. # leave blank to do it yourself relayhost =# or put it an accessible smtp server relayhost = smtp.yourisp.com
Next is network details. You will accept connection from anywhere, and you only trust this machine inet_interfaces = all mynetworks_style = host Next you can masquerade some outgoing addresses. Say your machine's name is mail.domain.com. You may not want outgoing mail to come from [email protected], as you'd prefer [email protected]. You can also state which domain not to masquerade. E.g. if you use a dynamic dns service, then your server address will be a subdomain. You can also specify which users not to masquerade. # masquerade_domains = mail.example.com www.example.com ! sub.dyndomain.com # masquerade_exceptions = root
As we will be using virtual domains, these need to be empty. local_recipient_maps = mydestination =
Then will set a few numbers. # how long if undelivered before sending warning update to sender delay_warning_time = 4h # will it be a permanent error or temporary unknown_local_recipient_reject_code = 450 # how long to keep message on queue before return as failed. # some have 3 days, I have 16 days as I am backup server for some people # whom go on holiday with their server switched off. maximal_queue_lifetime = 7d # max and min time in seconds between retries if connection failed minimal_backoff_time = 1000s maximal_backoff_time = 8000s # how long to wait when servers connect before receiving rest of data smtp_helo_timeout = 60s # how many address can be used in one message. # effective stopper to mass spammers, accidental copy in whole address list # but may restrict intentional mail shots. smtpd_recipient_limit = 16 # how many error before back off. smtpd_soft_error_limit = 3 # how many max errors before blocking it. smtpd_hard_error_limit = 12
Now we can specify some restrictions. Be carefull that each setting is on one line only. # Requirements for the HELO statement smtpd_helo_restrictions = permit_mynetworks, warn_if_reject reject_non_fqdn_hostname, reject_invalid_hostname, permit # Requirements for the sender details smtpd_sender_restrictions = permit_mynetworks, warn_if_reject reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unauth_pipelining, permit # Requirements for the connecting server smtpd_client_restrictions = reject_rbl_client sbl.spamhaus.org, reject_rbl_client blackholes.easynet.nl, reject_rbl_client dnsbl.njabl.org # Requirement for the recipient address smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, permit smtpd_data_restrictions = reject_unauth_pipelining
Further restrictions: # require proper helo at connections smtpd_helo_required = yes # waste spammers time before rejecting them smtpd_delay_reject = yes disable_vrfy_command = yes
Next we need to set some maps and lookups for the virtual domains. # not sure of the difference of the next two # but they are needed for local aliasing alias_maps = hash:/etc/postfix/aliases alias_database = hash:/etc/postfix/aliases # this specifies where the virtual mailbox folders will be located virtual_mailbox_base = /var/spool/mail/virtual # this is for the mailbox location for each user virtual_mailbox_maps = mysql:/etc/postfix/mysql_mailbox.cf # and this is for aliases virtual_alias_maps = mysql:/etc/postfix/mysql_alias.cf # and this is for domain lookups virtual_mailbox_domains = mysql:/etc/postfix/mysql_domains.cf # this is how to connect to the domains (all virtual, but the option is there) # not used yet # transport_maps = mysql:/etc/postfix/mysql_transport.cf
You can (as in my older editions) use a lookup for the uid and gid of the owner of mail files. But I tend to have one owner(virtual), so instead add this: virtual_uid_maps = static:5000 virtual_gid_maps = static:5000
You need to set up an alias file. This is only used locally, and not by your own mail domains. sudo cp /etc/aliases /etc/postfix/aliases # may want to view the file to check if ok. # especially that the final alias, eg root goes # to a real person sudo postalias /etc/postfix/aliases
Next you need to set up the folder where the virtual mail will be stored. This may have already been done by the apt-get. And also create the user whom will own the folders. # to add if there is not a virtual user sudo mkdir /var/spool/mail/virtual sudo groupadd --system virtual -g 5000 sudo useradd --system virtual -u 5000 -g 5000 sudo chown -R virtual:virtual /var/spool/mail/virtual
Note: If using Amazon ec2 you may want to move the mail spool to /mnt or an EBS location. You will need to symlink correctly afterwards. Return to top.
Postfix's MySQL configuration Next we need to set up the files to access the lookups via the database. We will only set up a few now, and the rest later when/if needed: Edit(create) how to find the users mailbox location sudo vi /etc/postfix/mysql_mailbox.cf
user=mail password=mailPASSWORD dbname=maildb table=users select_field=maildir where_field=id hosts=127.0.0.1 additional_conditions = and enabled = 1
Create how to find the email alias: sudo vi /etc/postfix/mysql_alias.cf user=mail password=mailPASSWORD dbname=maildb table=aliases select_field=destination where_field=mail hosts=127.0.0.1 additional_conditions = and enabled = 1
Create how to find the domains: sudo vi /etc/postfix/mysql_domains.cf user=mail password=mailPASSWORD dbname=maildb table=domains select_field=domain where_field=domain hosts=127.0.0.1 additional_conditions = and enabled = 1
If you specify an ip in hosts, (as opposed to 'localhost') then it will communicate over tcp and not the mysql socket. (chroot restriction). Ps. remember to replace the passwords with your chosen mail user password. Return to top.
Database
MySQL Now we will need to create the tables for thos lookups just specified. First you need to create a user to use in MySQL for mail only. Then you need to create the database, Take note of your chosen mail username and password. You will need the password you specified for root during MySQL package installation. # If not already done (in package installation)... mysqladmin -u root password new_password # log in as root mysql -u root -p # then enter password for the root account when prompted Enter password: # then we create the mail database create database maildb; # then we create a new user: "mail" GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON maildb.* TO 'mail'@'localhost' IDENTIFIED by 'mailPASSWORD'; GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON maildb.* TO 'mail'@'%' IDENTIFIED by 'mailPASSWORD'; exit; Obviously replace mailPASSWORD with
your chosen password! Then you will need to create these tables: • • •
aliases domains users
We will create more later on for further extensions, but only these are relevant now. Log in to mysql as the new mail user mysql -u mail -p maildb # enter the newly created password Enter password: Then run this commands to create the tables: CREATE TABLE `aliases` ( `pkid` smallint(3) NOT NULL auto_increment, `mail` varchar(120) NOT NULL default '', `destination` varchar(120) NOT NULL default '', `enabled` tinyint(1) NOT NULL default '1', PRIMARY KEY (`pkid`), UNIQUE KEY `mail` (`mail`) ) ; CREATE TABLE `domains` ( `pkid` smallint(6) NOT NULL auto_increment, `domain` varchar(120) NOT NULL default '', `transport` varchar(120) NOT NULL default 'virtual:', `enabled` tinyint(1) NOT NULL default '1', PRIMARY KEY (`pkid`) ) ; CREATE TABLE `users` ( `id` varchar(128) NOT NULL default '', `name` varchar(128) NOT NULL default '', `uid` smallint(5) unsigned NOT NULL default '5000', `gid` smallint(5) unsigned NOT NULL default '5000', `home` varchar(255) NOT NULL default '/var/spool/mail/virtual', `maildir` varchar(255) NOT NULL default 'blah/', `enabled` tinyint(3) unsigned NOT NULL default '1', `change_password` tinyint(3) unsigned NOT NULL default '1', `clear` varchar(128) NOT NULL default 'ChangeMe', `crypt` varchar(128) NOT NULL default 'sdtrusfX0Jj66', `quota` varchar(255) NOT NULL default '', `procmailrc` varchar(128) NOT NULL default '', `spamassassinrc` varchar(128) NOT NULL default '', PRIMARY KEY (`id`), UNIQUE KEY `id` (`id`) ) ;
The last few fields in the users table are not required, but useful if you extend later. # To visualise the tables created: describe aliases; describe domains; describe users; # then quit mysql exit;
Next is to edit the MySQL's my.cnf file. In Ubuntu/debian this is created by default. In Mandrake I had to manually create a blank one in /etc. But we need to configure it, so: sudo vi /etc/mysql/my.cnf In previous version you needed to comment out this line #skip-networking However in todays file the default is to bind the address to localhost, which is fine. bind-address = 127.0.0.1 It is very useful at the start to log any SQL calls that makes it to MySQL. So enable these lines: general_log_file = /var/log/mysql/mysql.log general_log = 1 Then in a few weeks comment it out when everything is working, as it slows mysql down Restart MySQL to make sure its picking up the new settings. sudo /etc/init.d/mysql restart
Return to top.
Pop/IMAP
Courier IMAP Please refer to previous edition for more explanations. But below is the details of what you need to change. sudo vi /etc/courier/authdaemonrc Change to mysql mode. authmodulelist="authmysql" Further down enable logging. DEBUG_LOGIN=2
Changed user MYSQL_USERNAME mail Changed password to whichever you have chosen MYSQL_PASSWORD mailPASSWORD Changed database MYSQL_DATABASE maildb Changed users table MYSQL_USER_TABLE users Keep commented in crypt pw MYSQL_CRYPT_PWFIELD crypt Keep commented out clear pw # MYSQL_CLEAR_PWFIELD clear Added maildir MYSQL_MAILDIR_FIELD concat(home,'/',maildir) Added where clause MYSQL_WHERE_CLAUSE enabled=1 sudo vi /etc/courier/authmysqlrc
Lastly you can have a look at the imapd file, but no changes is needed. vi /etc/courier/imapd
Return to top.
Summary You now have a basic mail server! Before continuing to the advanced and secure mail server you must ensure the basic setup works. This will save you from loads of pain further on.
It is very easy to make typos, miss tiny steps, unclear steps or simple actual errors in this howto. • • • •
Insert stub data from data section Apply advice from test section judicously Ensure the mail server can receive email correctly first, then try sending. Once you are positive the mail has been received, the mail folders have been automatically created, only then you should test if you can actually read the emails before proceding
Ive created an EC2 bundle for this stage: flurdy-amis/ubuntu-mail-server-simple. Return to top.
Somewhere, something went terribly wrong
Advanced mail server Now lets extend this setup with more useful content checks , security and user interfaces.
Content Checks (Anti spam & anti virus) Amavisd-new Amavisd ties together all the different ways of checking email content for spam and viruses. The defaults are pretty good and also the ubuntu documentation is pretty clear, and recommended. Here is a tweaked version of it: Initially we will not enable spam or virus detection! This is so we can get amavis set up to receive, check and pass on emails before we go on and over-complicate it.
All of amavis' configuration files are in /etc/amavisd. They are now spread across several files in conf.d. Debian and Ubuntu defaults are now very sensible and spread into seperate files. cd /etc/amavis/conf.d 01-debian defaults are fine. Have a look at less 05-domain_id but dont change anything in it. Have a look at less 05-node_id but dont change anything in it. Have a look at less 15-av_scanners but dont change anything in it. Edit content check file sudo vi 15-content_filter_mode Comment out both virus and spam scans. (Default). # #@bypass_virus_checks_maps = ( # \ %bypass_virus_checks, \@bypass_virus_checks_acl, \ $bypass_virus_checks_re); # @bypass_spam_checks_maps = ( # \ %bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
Have a look at less 20-debian_defaults and less 21-ubuntu_defaults but dont change anything in them. 25-amavis_helpers defaults are fine. 30-template-localization defaults are fine. Edit user file sudo vi 50-user In the middle insert: @local_domains_acl =
qw(.); $log_level = 2; $syslog_priority = 'debug'; $sa_kill_level_deflt = 8.0; # triggers spam evasive actions $final_spam_destiny = D_PASS; # $final_spam_destiny = D_DISCARD;
We have now setup amavis to scan and pass along incomming email. Next we will setup postfix to talk to amavis. Append these lines to the end of the file (make sure they are not already present). (Note the -o lines have spaces in front of them. amavis vi /etc/postfix/master.cf
unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_che cks Also add the following two lines immediately below the "pickup" transport
service: -o content_filter= -o receive_override_options=no_header_body_checks
and then added to main.cf sudo vi /etc/postfix/main.cf content_filter = amavis:[127.0.0.1]:10024
Enable scanning by ClamAV of amavis' temporary files. sudo adduser clamav amavis
This should be it to get amavis working. If emails are picked up by amavis and passed back to postfix then it looks okay. Only when finished testing do you proced to uncomment the anti virus and anti spam lines in sudo vi 15-
content_filter_mode @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); But do that after the
next section (SpamAssassin). When things are working we will turn down logging level, and start bouncing/discarding spam. sudo vi /etc/amavis/conf.d/50-user @local_domains_acl = qw(.); $log_level = 1; $syslog_priority = 'info'; $sa_kill_level_deflt = 8.0; # triggers spam evasive actions #$final_spam_destiny = D_PASS; $final_spam_destiny = D_DISCARD;
Return to top.
Anti-Spam SpamAssassin The default config of spam assassin is okay. You could refer to previous edition for more configuration options. You do need to tell SpamAssassin to start smapd on boot. vi /etc/default/spamassassin ENABLED=1
One configuration option you could tweak is to enable Bayes and auto learning. vi /etc/spamassassin/local.cf
I read your email Return to top.
Anti Virus ClamAV
ClamAV does not need setting up. Configuration files are in /etc/clamav, but they are automatically generated, so do not edit. By default freshclam, the daemon that updates the virus definition database, is run 24 times a day. That seems a little excessive, so I tend to set that to once a day. sudo dpkg-reconfigure clamav-freshclam It will also ask if you want it to be daemon (yes) and which server is closest to you. If needed, the command below will redefine the configuration with a lot of questions. Not needed unless you need to configure. sudo dpkg-reconfigure clamav-base
Return to top.
Postgrey The default config of postgrey is okay. However you need to tell Postfix to use it. sudo vi /etc/postfix/main.cf And then edit the recipient restrictions:
smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, check_policy_service inet:127.0.0.1:10023, permit
You can tweak whitelisting in /etc/postgrey. You can tweak postgrey configuration by tweaking /etc/default/postgrey. E.g. delay, auto whitelisting, or reject message. POSTGREY_OPTS="--inet=10023 --max-age=365"
Return to top.
You know have an advanced mail server. You can use this, but Id recommend continuing. However this is a good point to test the set up so far and to insert some data in the db. Ive created an EC2 bundle for this stage: flurdy-amis/ubuntu-mail-server-spam. Return to top.
No, I will not fix your computer
Secure mail server Stopping hackers, phishers, spammers, your boss and your neighbour from accessing your server or the traffic in between is important, and easily done.
Authentication Normal email traffic between clients and servers are in open plain text. That includes passwords and content of emails.
SASL SASL secures the actual authentication (login), by encoding the passwords so that it can not be easily intercepted. The rest of the emails are however in clear plain text. SASL can be a royal pain to set up, especially as it does not support storing encrypted passwords by default in Ubuntu. Therefor my previous editions described how to configure SASL using plain text passwords in the database. Obviously this is not ideal, so there are ways to combine SASL and storing encrypted passwords. In the future the packages that comes with Ubuntu may support the password_format configuration option for SASL. But until then you can configure SASL to ask PAM to compare the passwords: Install packages if not all installed already: sudo aptitude install sasl2-bin libpam-mysql libsasl2-modules libsasl2modules-sql
Enable postfix to access SASL files: sudo adduser postfix sasl
Create sasl files accessibly even by chrooted Postfix: sudo mkdir -p /var/spool/postfix/var/run/saslauthd
Add SASL configurations to Postfix: sudo vi /etc/postfix/main.cf # SASL smtpd_sasl_auth_enable = yes # If your potential clients use Outlook Express or other older clients # this needs to be set to yes broken_sasl_auth_clients = no smtpd_sasl_security_options = noanonymous smtpd_sasl_local_domain =
Modify these existing configurations: # Add permit_sasl_authenticated to you existing smtpd_sender_restrictions smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, warn_if_reject reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unauth_pipelining, permit # Add permit_sasl_authenticated to you existing smtpd_recipient_restrictions smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, check_policy_service inet:127.0.0.1:10023, permit
Change how SASLAUTHD is run: sudo vi /etc/default/saslauthd # Toggle this to yes START=yes # Switch this to be under postfix's spool # And add -r so that the realm(domain) is part of the username OPTIONS="-r -c -m /var/spool/postfix/var/run/saslauthd"
Tell postfix how to interact with SASL: sudo vi /etc/postfix/sasl/smtpd.conf pwcheck_method: saslauthd mech_list: plain login cram-md5 digest-md5 log_level: 7 allow_plaintext: true auxprop_plugin: mysql sql_engine: mysql sql_hostnames: 127.0.0.1 sql_user: mail sql_passw: mailPASSWORD sql_database: maildb sql_select: select crypt from users where id='%u@ %r' and enabled = 1
(When SASL is working you can remove the log_level line.) Tell the pam how to to authenticate smtp via mysql: sudo vi /etc/pam.d/smtp
These must be on 2 lines only, but I have broken them up for easier to read. auth required pam_mysql.so user=mail passwd=aPASSWORD host=127.0.0.1 db=maildb table=users usercolumn=id passwdcolumn=crypt crypt=1 account sufficient pam_mysql.so user=mail passwd=aPASSWORD host=127.0.0.1 db=maildb table=users usercolumn=id passwdcolumn=crypt crypt=1
In addition to tailing var/log/mail.log and /var/log/mysql/mysql.log it is quite usefull to tail the auth.log as well when testing SASL. tail -f /var/log/auth.log
Restart postfix and saslauthd to enable SASL for sending emails. sudo /etc/init.d/saslauthd restart sudo /etc/init.d/postfix restart
Imap SASL / Courier I tend not to have SASL for my courier authentication, as I enforce TLS for all my clients. However if you have a more lenient access policy which is wise if you have many users, then you may want SASL in Courier as well: sudo vi /etc/courier/imapd
This may already be avaiable as a commented out line. If not replace the current line by adding UTH=CRAM-MD5 AUTH=CRAM-SHA1 so it resembles something like this: (Again on one line) IMAP_CAPABILITY="IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA AUTH=CRAM-MD5 AUTH=CRAM-SHA1 IDLE" sudo /etc/init.d/courier-authdaemon restart; sudo /etc/init.d/courier-imap restart
Return to top.
Encryption TLS Encrypting the traffic stops anyone else listening in on your email communications. And is very recommended. There are different types of communication to encrypt: The data traffic between your email applications and the server when you read emails or when you send emails, and communication between other email servers and your server. For the encryption of reading emails, it is Courier you need to configure. For sending, and beetwen server encryption it is Postfix.
TLS in Postfix To encrypt you need certificates. Ubuntu creates some for you for which you can use while setting up the server. However before you go live, it is recommended to create
your own with your proper domain name etc. Please refer to previous edition for more detail. There are already some TLS settings in the default debian/ubuntu version of this file. I moved these to the end, for clarity, but that is up to you. # TLS parameters #smtp_use_tls = no smtp_tls_security_level = may vi /etc/postfix/main.cf
#smtpd_use_tls=yes smtpd_tls_security_level = may #smtpd_tls_auth_only = no smtp_tls_note_starttls_offer = yes smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s tls_random_source = dev:/dev/urandom smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key #smtpd_tls_session_cache_database = btree:$ {data_directory}/smtpd_scache #smtp_tls_session_cache_database = btree: ${data_directory}/smtp_scache
Next we have a look at the master.cf file. vi /etc/postfix/master.cf By default only the normal smtp service is enabled, which is fine. But I prefer to enable submission (port 587), so that clients can use it, and I can restrict them to TLS only. Also enabled smtps service (port 465), for some compatebility with some older clients (outlook express etc). submission inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes # if you do not want to restrict it encryption only, comment out next line -o smtpd_tls_auth_only=yes # -o smtpd_tls_security_level=encrypt # -o header_checks= # -o body_checks= -o smtpd_client_restrictions=permit_sasl_authenticated,reject_unauth_desti nation,reject -o smtpd_sasl_security_options=noanonymous,noplaintext -o smtpd_sasl_tls_security_options=noanonymous # -o milter_macro_daemon_name=ORIGINATING smtps inet n - - - - smtpd -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_tls_auth_only=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_sasl_security_options=noanonymous,noplaintext -o smtpd_sasl_tls_security_options=noanonymous # -o milter_macro_daemon_name=ORIGINATING
TLS in Courier Again Ubuntu has created a certificate for you, but if you want to create your own, especially for a properly named server, then do this. cd /etc/courier openssl req -x509 -newkey rsa:1024 -keyout imapd.pem \ -out imapd.pem -nodes -days 999 For more details review last edition.
Then you need to edit vi /etc/courier/imapd-ssl By default Ubuntu already points to you certificate TLS_CERTFILE=/etc/courier/imapd.pem Modify this if needed. Also you if want to restrict IMAP users to SSL/TLS only toggle this setting to 1. IMAP_TLS_REQUIRED=1
For maximum compatability it is not wise to restrict to TLS only for the traffic between servers. As this means not all valid emails sent by others can reach your server. However enabling them the option to encrypt is a good idea. Be aware that the emails are not encrypted on your machine, nor on the server. For this type of client encryption, please refer to previous edition for more on GnuPG. In some situations SASL and TLS do not play well together. Those situations are in combinations of storing encrypted passwords, using MD5 authentication over encrypted traffic. I recommend, insisting on TLS traffic with your authenticating clients, which then negates the need for SASL. You know have an advanced secure mail server. Now is another good point to test the set up so far and to insert some data in the db. Ive created an EC2 bundle for this stage: flurdy-amis/ubuntu-mail-server-secure. Return to top.
Webmail Using among others the https://help.ubuntu.com/community/Squirrelmail as an updated reference.
Alternative If you prefer the prettier Roundcube, but less powerfull (at the moment), the follow the Roundcube section in the extention section.
Enable web access You may need to enable web access in the firewall. Check the firewall configuration if this neccessary. You need to copy a SquirrelMail configuration to apache. sudo cp
/etc/squirrelmail/apache.conf /etc/apache2/sites-available/squirrelmail And enable with this: sudo ln -s /etc/apache2/sitesavailable/squirrelmail /etc/apache2/sites-enabled/500-squirrelmail Or as Florent recommends, use: sudo a2ensite squirrelmail
You may accept the default apache configuration where squirrelmail is folder in all sites. But I prefer virtual hosting. But you dont need to do these next steps. sudo vi /etc/apache2/sites-available/squirrelmail Comment out the alias. # alias
/squirrelmail /usr/share/squirrelmail Uncomment the virtual settings., and insert your servers name. # users will prefer a simple URL like http://webmail.example.com DocumentRoot /usr/share/squirrelmail ServerName webmail.example.com If you have apache SSL enabled in apache, then
you can also uncomment the mod_rewrite section for further security. Reload apache to activate changes. First test if ok. sudo apache2ctl -t Then reload it. sudo /etc/init.d/apache2 reload You can now go toyourdomain.com/squirrelmail/ or mail.yourdomain.com if you chose virtual host. This should show a squirrel mail page. Log in wont work yet though.
Start configuring squirrel mail. sudo squirrelmail-configure
Initially change nothing. You can customize more afterwards. You can browse, and exit sub menues by typing R. Type 2 to edit server settings. Type A to edit IMAP settings. Type 8 to edit server software. Enter courier. courier Now they say using TLS over localhost is a waste of time. But I do anyway. Type 7 to edit secure IMAP. Type Y to enable it. Type 5 to edit IMAP port. Enter 993 Type S to save your changes. Hit Enter. Type Q to exit. You can now go to yourdomain.com/squirrelmail/ or mail.yourdomain.com if you chose virtual host. This should show a squirrel mail page. Log in will now work. (Except you may not have defined users, check data section. And they may not have received an email which also means you can not view any IMAP info.) Please refer to previous edition for more detail. E.g. creating address books and user preferences. Return to top.
Administration Enable web access
You may need to enable web access in the firewall. Check the firewall configuration if this neccessary. You may choose to restrict phpMyAdmin to a spefic virtual host. If so you need to, edit sudo vi /etc/apache2/conf.d/phpmyadmin and comment out the alias. #Alias /phpmyadmin /usr/share/phpmyadmin And insert the alias instead into a virtual host configuration in /etc/apache2/sites-available/. For this example we are not. Reload apache to activate changes. First test if ok. sudo apache2ctl -t Then reload it. sudo /etc/init.d/apache2 reload You can now go to http://yourdomain.com/phpmyadmin/, and login with the mail user. You can use it as it is, but I recommend securing it a bit more. One simple way is adding apache's .htaccess login requirement. Further restrictions can be restricting to a specific virtual host. Or renaming the folder. Purely ubfuscating, but simple. Or using the example in the webmail section, and adding SSL requirement to the connection. Or disabel mysql root's access via phpMyAdmin. Please refer to previous edition for example on htaccess, and mysql user restriction. You know have a finished mail server. This is as far as the main guide goes. Hope it was clear enough to follow. Now it is time to insert data, and to test how it works. Feel free to extend it with my suggestions further down. Ive created an EC2 bundle for this stage: flurdy-amis/ubuntu-mail-server-webmail. Return to top.
Data •
•
Add users and domains o Required domains and users o Example domains and users o Adding template Common SQL
Add users and domains So we got a fully set up mail server... Well no, there is no users, domains, no nothing! Okay, first you need add some default data, some which are required, some which make sense. Then we'll add your own users and domains.
Required domains and users First the required domains for local mail # Use phpMyAdmin or command line mysql INSERT INTO domains (domain) VALUES ('localhost'), ('localhost.localdomain');
Then some default aliases. Some people say these are not needed, but I'd include them. INSERT INTO aliases (mail,destination) VALUES ('postmaster@localhost','root@localhost'), ('sysadmin@localhost','root@localhost'), ('webmaster@localhost','root@localhost'), ('abuse@localhost','root@localhost'), ('root@localhost','root@localhost'), ('@localhost','root@localhost'), ('@localhost.localdomain','@localhost');
Then a root user. INSERT INTO users (id,name,maildir,crypt) VALUES ('root@localhost','root','root/', encrypt('apassword') );
Domains and users Now lets add some proper data. Say you want this machine to handle data for the fictional domains of "blobber.org", "whopper.nu" and "lala.com". Then say this machine's name is "mail.blobber.org". All email to lala.com is to be forwarded to whupper.nu. INSERT INTO domains (domain) VALUES ('blobber.org'), ('mail.blobber.org'), ('whopper.nu'), ('lala.com'); INSERT INTO aliases (mail,destination) VALUES ('@lala.com','@whupper.nu'), ('@mail.blobber.org','@blobber.org'),
('[email protected]','postmaster@localhost'), ('[email protected]','abuse@localhost'), ('[email protected]','postmaster@localhost'), ('[email protected]','abuse@localhost');
You also have two users called "Xandros" and "Vivita". INSERT INTO users (id,name,maildir,crypt) VALUES ('[email protected]','xandros','xandros/', encrypt('apassword') ), ('[email protected]','vivita','vivita/', encrypt('anotherpassword') ); INSERT INTO aliases (mail,destination) VALUES ('[email protected]','[email protected]'), ('[email protected]','[email protected]');
You want all mail for whooper.nu to go to xandros (catchall). INSERT INTO aliases (mail,destination) VALUES ('@whopper.nu','[email protected]');
There is also a "Karl" user, but he does want all mail forwarded to an external account. INSERT INTO aliases (mail,destination) VALUES ('[email protected]','[email protected]');
So what does each of these lines actually do? Well the domains are pretty straight forward. The users are as well, it requires four fields. ID is the email address of the user, and also its username when loggin in, described later on. NAME is optional description of the user. MAILDIR is the name of the folder inside /var/spool/mail/virtual. It must end in a /, otherwise it wont be used as a unix maildir format. CRYPT is the encrypted text password to use. The alises are the interesting part. Lets start from a top down view to see how emails get delivered: Say an email arrives addressed to "[email protected]". • • • • • •
Postfix looks up domains and say whopper.nu is an domain it listens to. Postfix then looks up aliases and searches for a row where the mail field matches "[email protected]". None does so it next searches for "@whopper.nu", which is the way to specify catch all others for that domain. It finds one row and its destination is "[email protected]". It then searches for "[email protected]" and finds one, which destination is the same as the mail, therefor it is the final destination. It then tries to deliver this mail. The look up says blobber.org is a local mail so it looks up users for a matching id and delivers it to its maildir.
Lets try "[email protected]". • • •
Postfix looks up domains and it is an domain it listens to. First lookup does not find this user, but the next finds the catchall "@lala.com". But its destination is another catchall, "@blobber.org". This means Postfix will look for "[email protected]". This address is not found either, nor is a catchall for blobber.org. Therefor this address is not valid and the message will be bounced.
Any mail arriving for "[email protected]" or "[email protected]", gets forward to an external address of "[email protected]". So forwarding is simple. I tend to use a subdomain for all my friends addresses as easily I forget what their real addresses are, and I use different email clients all the time. I also added the required aliases of postmaster and abuse to blobber.org and whopper.nu. The catchall for lala.com means they are not required for that domain. Another useful alias to add is root, as often you get admin mail from e.g cron jobs within those domains etc. Other often used aliases are info, sysadmin, support, sales, webmaster, mail, contact and all. But they are also honeypots for spam, so just include the ones you think you will need.
Adding template So to add a new domain to the system, You do this, replacing the italics with relevant data: INSERT INTO domains (domain) VALUES ('domain.tld'); INSERT INTO aliases (mail,destination) VALUES ('@domain.tld','email@address'), ('[email protected]','email@address'), ('[email protected]','email@address');
And to add a new user to the system, do this: INSERT INTO users (id,name,maildir,clear) VALUES ('email@address','short description','foldername/',encrypt('password')); INSERT INTO aliases (mail,destination) VALUES ('email@address','email@address');
Return to top.
Common SQL A selection of useful sql statements, if you are not using an admin/manager program to maintain your email domains and users. Find domains without a catchall
#Remember some might be disabled SELECT dom.domain FROM domains dom LEFT JOIN aliases al ON CONCAT( '@', dom.domain ) = al.mail WHERE al.mail is null OR al.enabled = 0 ORDER BY dom.domain ASC
Find aliases for an invalid domain SELECT al.* FROM aliases al LEFT JOIN domains dom ON dom.domain = SUBSTRING(al.mail,LOCATE('@',al.mail)+1) WHERE dom.domain is null OR dom.enabled = 0 ORDER BY al.mail ASC
Find all non local destination aliases SELECT al.* FROM aliases al LEFT JOIN domains dom ON dom.domain = SUBSTRING(al.destination,LOCATE('@',al.destination)+1) WHERE dom.domain is null ORDER BY al.enabled, al.destination ASC, al.mail ASC
Find all aliases for a certain domain SELECT al.* FROM aliases al WHERE SUBSTRING(al.mail,LOCATE('@',al.mail) +1) = 'domain.tld' ORDER BY al.enabled, al.mail ASC
Find all aliases for a certain domains, checking if enabled for both domain and alias select * from domains d join aliases a on a.mail like concat( '%','@',d.domain) and a.enabled = 1 where d.enabled = 1 and d.domain like '%foobar%' order by d.domain,a.mail
Return to top.
Test
Amazon.com Widgets • • • • • • • • •
Common problems Test strategy Tail, tail and tail again Switch off services Switch debug on Telnet is your friend Can postfix receive? Can postfix send? Can courier read?
Common problems Missed a step If you mistakenly or intentially skipped past sections, you may have missed an important step in your configuration, which my guide pressumes you have followed.
Typo 99% of all problems is spelling errors or typos you entered while following this guide. Sorry, but it just happens. Often it can be trivial, such as a space at the end of the configuration line which was not expected etc. Or not understanding my example where it is a multi line entry.
Typo by me Yes, I make a lot of mistakes. Nothing wrong in that, but I hope I have corrected most over time. Any new sections are however at risk... :)
Different application or configurations It is obviously entirely up to you how you set up your system. But the more you deviate from this guide, the more likely incompatibilties or confusion will arrise.
Distrobution/version differences If you run a different version or even distrobution to this guide, then some things will be different. Small issues, such as default values and significant things such as path differences etc. Some sections in this guide are not always thouroughly tested with every new release of Ubuntu, but these differences gets pointed out by people for me.
Walking before crawling Don't try the full blown mail server before the basics are working.
Gamma rays and little goblins Got to blame it on something or someone. Return to top.
Test strategy What steps to think of when testing.
Test early and frequently It is very helpfull to test early in this set up, to establish if the first sections are working as expected. So when you only have your very basic postfix and mysql up: Test it! That way you know that bit worked and can rule it out of any future problems. Don't wait till you complicated and mudded the water after amavis, courier etc is added. By constantly testing if you can send and receive you can tick off and black box each section as working, and immidietly spot issues.
Isolate the problem Testing how things work is often about isolating the problem first. So by using the steps of testing early above, you can see which step caused the problem. Also if you can't log into your webmail it is often nothing to do with the webmail section that is causing the problem. Often postfix itself is broken etc.
Test in order As part of the isolating the problem rule, you most of the time test in order, and test each section thus isolating the problem. This would then quickly isolate the problem when e.g. such as above issues of reading emails via the webmail. This would be in order: 1. 2. 3. 4. 5. 6.
Access: Can I get(ssh) to the box, and is there a firewall issue? Database: Is the database up, do my application reach it? Postfix: Can I send email by command line, do I receive emails via telnet? Content checks: Do they cause a problem? Courier: Can I read emails? Webmail: And last but not least does the web integration work?
Simplify the system Assisting in isolating the problem, you often have to disable options and applications. Such as turn of postgrey or content checks to make sure emails to get delivered. Previous editions do have some more detail on how to achieve this
Return to top.
Tail, tail and tail again Essential to monitor what actually happens, and tailing specifically the mail and mysql log. • • • •
/var/log/system.log /var/log/mail.log /var/log/mysql.log /var/log/apache2/access.log
In one window: tail -f /var/log/mail.log And in another window: tail -f /var/log/mysql.log In a third or more do your actual configuration or testing. Return to top.
Switch off services previous edition 1 previous edition 2 The previous editions has detail on switching services off untill time to test them. It also details locking down your server from spammers untill finished testing. Return to top.
Switch debug on Shorewall You can also switch on more messages for when the firewall is rejecting connections. Add info to all REJECT, BOUNCE and DROP policies. sudo vi /etc/shorewall/policy such as: net all DROP info
MySQL There is no point in tailing the mysql log if query debugging is not turned one. By default it is not. However in this guide I do switch it on, in case that was missed switch it on now: sudo vi /etc/mysql/my.cnf Make sure this is not commented out log = /var/log/mysql/mysql.log
Courier As mentioned in the setup , switching on debugging for Courier is easy: sudo vi /etc/courier/authdaemonrc DEBUG_LOGIN=2
Amavis You can also debug amavis: sudo vi /etc/amavis/conf.d/50-user And perhaps bump it up if already debugging: $log_level = 2; Return to top.
Telnet is your friend When testing a mail server, telnet is alpha & omega. You use it to simulate real mail servers to test responses by your mail server. 1. First you test it on the server to exclude firewall and network issues. 2. Then you test it from another machine to simulate an actual other mail server. 3. Once these are working you can use proper email clients, however 99% I just use mutt locally when I need to test if a server is working. Return to top.
Can postfix receive? Lets assume: • • • •
You have followed my guide up to basic configuration at least You have entered data into the database The services MySQL and Postfix are running. If testing a fuller stack, then amavis, postgrey, clamav-daemon, spamassassin etc must also be running.
Try this locally on the server first, then try from another machine once it is working locally. Lets try and send a message to [email protected] (replace with your own user in this setup, or use postmaster@localhost) from [email protected] (again replace with a real email address you use that is not associated with this server.) telnet localhost 25 # Open the hand shake with ehlo and the server name you are connecting from... # Change mail.example.com to something valid eg your servername EHLO mail.example.com # The mail server will then dump
out some details about its capabilities, e.g. >250-mail.flurdy.net >250PIPELINING >.... >.... # then say who is the sender of this email MAIL FROM: > 250 Ok # then say who the mail is for RCPT TO: > 250 Ok # then enter the keyword data data > 354 End data with . # enter message bodyand end with a line with only a full stop. blah blah blah more blah . > 250 Ok; queued as QWKJDKASAS # end the connection with quit > 221 BYE
If while you were doing this you were tailing the /var/log/mail.log you would see some activities and if any errors occured. (You should probably get some complaints about missing headers as we skipped most...) If while you were doing this you were tailing the /var/log/mysql.log as well you really should have seen some activity otherwise you have a problem. If you see any errors (or worse no activity) in these log files, this is what you need to fix! For common problems and solutions check the previous edition.
However if no errors popped up, and the folder /var/mail/virtual/xandros now exists then your server can receive emails! Return to top.
Can postfix send? • •
You need to make sure you can first receive emails as above The services MySQL and Postfix are running.
Basically you just tested that above, but we need double check if it can send out to other servers. Again we will first test locally, which should work, then remotely which introduces many possible problems. telnet localhost 25 # Open the hand shake with ehlo and the server name you are connecting from... # This time it has to be the name of your server EHLO mail.example.org # The mail server will then dump out some details about its capabilities, e.g. >250-mail.flurdy.net >250-PIPELINING >.... >.... # then say who is the sender of this email, which is a local user MAIL FROM: > 250 Ok # then say who the mail is for which is an external address e.g. gmail etc. RCPT TO: > 250 Ok # then enter the keyword data data > 354 End data with . # enter message bodyand end with a line with only a full stop. blah blah blah more blah . > 250 Ok; queued as QWKJDKASAS # end the connection with quit > 221 BYE
We have to assume receiving works above so no need to tail mysql's logs. However if any rejection errors occured in the mail.log then you have an error. However if no errors occured and you see in the log something like this: Dec 17 10:25:45 servername postfix/smtp[12345]: 12345678: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=15, delays=15/0.01/0.02/0.11, dsn=2.0.0, status=sent (250 2.0.0 Ok, id=12345-09, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 1234567) Then the sending emails work!
Return to top.
Can courier read emails • • • •
You need to make sure you can first receive emails as above You need to make sure you can send emails as above You need to make sure you have received an email and the folder /var/mail/virtual/xandros exists The services MySQL, courier-authdaemon and courier-imap are running.
There is not too much you can test via telnet for courier. But you can check if it is up and you can connect to it. telnet 127.0.0.1 10024 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. 220 [127.0.0.1] ESMTP amavisd-new service ready The rest you would have to test via a proper email IMAP client. Return to top.
Initialize Brief hints if you receive a ready setup machine (or EC2 AMI), and what then to check and to customize it to your setup. • • • • • • • • • • •
Stop services Restrict firewall Change passwords Check configurations Set machine name Certificates Start and test services Insert data Reload postfix Open firewall Test
Stop services First stop services so they wont accidentally do something. sudo /etc/init.d/postfix stop sudo /etc/init.d/courier-imap-ssl stop sudo /etc/init.d/courier-imap stop sudo /etc/init.d/courier-authdaemon stop sudo /etc/init.d/mysql stop sudo /etc/init.d/amavisd stop sudo /etc/init.d/spamassassin stop sudo /etc/init.d/clamav stop
Restrict firewall Check what the firewall rules are. vi /etc/shorewall/rules Refer to the firewall settings. Restrict to just SSH access for now.
Change passwords Next the passwords needs to be changed. For both the system and mysql.
System passwords Check which users are defined on the system. cat /etc/passwd Apart from all the system ones, there should probably be none (if EC2 AMI) or just your user if it is a standard Ubuntu install. If there are some users, you need to change their passwords.
SSH Access Next we check whom got SSH access. If there was any users defined, check their home folders for ssh keys. cat /home/username/.ssh/auth* Remove any you do not expect to be there. Next check if and which specific users has been defined for SSH access in vi /etc/ssh/sshd Usually this is fine.
MySQL passwords First you need to change the root mysql user. If none has been set do this mysqladmin -u root password new_password Otherwise do this and you will be prompted for the old password mysqladmin -u root password new_password -p Then the default mail user as well. If you know the old password mysqladmin -u mail password new_password -p Otherwise log into mysql as root: mysql -u root -p Enter new root password specified above, then: update mysql.user set password=password('apassword') where user='mail'; flush privileges; You may need to revisit the top of MySQL section to re-grant the mail use rights on the database. If you do not know the old root password, you have to restart mysql without grant rights. Google it... :)
Update postfix mysql configuration files with the new password. sudo vi /etc/postfix/mysql* password=apassword Update courier's authmysql file with the new password as well. sudo vi /etc/courier/authmysqlrc MYSQL_PASSWORD apassword
If SASL is set up, then you need to update its passwords. First in postfix SASL file: sudo vi /etc/postfix/sasl/smtpd.conf sql_passw: aPASSWORD Then on both lines in: sudo vi /etc/pam.d/smtp passwd=aPASSWORD
Check configurations You should scan the postfix, courier, etc. configurations to check if they match what you expect.
Set machine name Now you need to define your machine name, e.g. something like smtp.yourdomain.com. You need to define it in sudo vi /etc/mailname And then your domain name in sudo vi /etc/postfix/main.cf under the mydomain setting myorigin=yourdomain.com It could also be smart to check what the unix hostname is specified as hostname This can be reset by sudo hostname smtp.yourdomain.com. All though this does not have to be the same as your postfix mail server name. You may want to speficiy some hosts in hosts file as well, sudo vi /etc/hosts 127.0.0.1 localhost.localdomain localhost 127.0.0.1 smtp.yourdomain.com smtp
Certificates You could go along with the generated certificates (if they are there, default for Ubuntu). Or if you could create new ones with the correct machine name in them. Especially if this a mail server used by many, and authenticiy is important. Follow the TLS certificate instructions for Postfix and Courier.
Start and test services. Next you need to start your mail services and test them. sudo /etc/init.d/mysql
start sudo /etc/init.d/spamassassin start sudo /etc/init.d/clamav start sudo /etc/init.d/amavisd start sudo /etc/init.d/postfix start sudo /etc/init.d/courier-imap-ssl start sudo /etc/init.d/courier-imap start sudo /etc/init.d/courier-authdaemon start
So test tjenestene via testing section.
Insert data Insert your mail domains, aliases and users using the data section.
Some times there are test data already in the database. Remove them. E.g.; mysql -u mail -papassword maildb delete from domains where domain = 'bar.com'; delete from aliases where mail = '[email protected]';
Open firewall Then open up the firewall, follow the world access bit in the firewall configuration. Voila. Up and running. Well we hope. Return to top.
Extend Please refer to previous edition for how and why you can extend this mail server. By now you should have a fully working system. No point extending and complicating it untill then. What next? There are many ways to extend the server, to create your own powerfull customized version. • • • • • • • • • • • • • • • •
Remote MX mail backup o Relay recipient lookup Local file backup Sender ID & SPF Spam reporting White/Black lists PGP & S/MIME Relocation notice Pop-before-SMTP Admin Software Auto Reply Block Addresses Throttle Output Mail Lists Google Apps / GMail Roundcube webmail Sugestions?
Some of these sections can be brief as they are not core to this howto.
Remote MX mail backup With MX backup loosing emails are unlikely.
Normally if someone sends an email destined for you, their server will try and connect to your server. If it can't reach your server for whatever reason ( it is down, dns issues, there is network problems, or just too busy ), the other server will back off and try again in a bit. How many and for how long it will try again is determined by the sending server. Some of them are not very patience, and it will report undelivered after only a few attempts. So you would have lost that email. If you had specified a backup MX, this email may not have been lost. Upon first failure to connect to your server, the sender would see if there is any alternative server to send to. So it connects to your backup mx server. This server spools and queues your message and will try at intervals to send the message to you. It too will though eventually give up. What is the difference? Simple, you (or whoever controls the backup mx ) is in control how long and often to try connecting to your machine. So if you have a reasonable values and your server is not down for weeks, no mail is lost. How to implement it? First edit the DNS records again, and add a backup mx with a higher value. # your server details domain.tld IN MX 10 your.mailserver.name.tld # new backup server domain.tld IN MX 20 your.backupserver.name.tld
Now presuming the other backup mx is a postfix server identical to this, or you are backuing up someone else's server; Go into mysql and create this tables: CREATE TABLE `backups` ( `pkid` smallint(6) NOT NULL auto_increment, `domain` varchar(128) NOT NULL default '', `transport` varchar(128) NOT NULL default ':[]', `enabled` smallint(6) NOT NULL default '1', PRIMARY KEY (`pkid`), UNIQUE KEY `domain` (`domain`) );
Then still on the backup server, edit main.cf and add these: relay_domains = mysql:/etc/postfix/mysql_backups.cf transport_maps = mysql:/etc/postfix/mysql_transport.cf
You may choose to have this as the last line in the file, as you may use small cron jobs to modify this ip address, if you don't have a permanent static address. It should contain your IP addres, hence if you do not have a very static IP address, that you need to automatic editing if the postfix file. proxy_interfaces = 1.2.3.4
If someone comes with a better way, then let me know. Next create this file /etc/postfix/mysql_backups.cf
user=mail password=apassword dbname=maildb table=backups select_field=domain where_field=domain hosts=127.0.0.1 additional_conditions = and enabled = 1
Next create this file /etc/postfix/mysql_transport.cf user=mail password=apassword dbname=maildb table=backups select_field=transport where_field=domain hosts=127.0.0.1 additional_conditions = and enabled = 1
You noticed I added a transport lookup. This is a field in both the domain and the backup tables. In domains it is used to determine how to deliver the email, ie either virtual (correct) or local (not used in this howto). When backing up servers, your also need to specify in the transport field how to connect to the correct servers. Say you are backiup for a friends server, mail.friend.com, for the domains of friend1.com and friend2.com. So you should insert this into your backup table. INSERT INTO backups (domain,transport) VALUES ('friend1.com' , ': [mail.friend.com]' ), ('friend2.com' , ':[mail.friend.com]' );
The :[] tells to connect directly to this server, not doing any more look ups for valid MX servers. This shouls now work fine. Further tweaking of the queue values, review these and modify as appropiate. Shorter warning times are good for the sender, so that they realise the email has not arrived yet, but may also be annoying. Tradeoffs.. Look in the first main.cf configurations for ways to do so.
Relay recipient lookup Unfortunetly spammers are using backup mx as a way to saturate the networks with invalid emails, known as backscatter mail. They simply lookup a domain's MX servers and connect directly to one of the lower priority servers whom may be just a backup mx. This server if configured as above will not check for valid addresses aliases but will accept and queue all emails for the domain's it is configured as a backup mx for. These will then be delivered by the server later to the primary MX server, whom will then maybe reject them as the aliases are not valid. However the sender addresses are often invalid and a long trail of reject messages to and forth around the net follows... To avoid this you can enable relay recipient lookup in Postfix. Edit /etc/postfix/main.cf and add: relay_recipient_maps = mysql:/etc/postfix/mysql_relays.cf Then create a new file /etc/postfix/mysql_relays.cf user=mail password=apassword dbname=maildb table=relays select_field=recipient where_field=recipient
hosts=127.0.0.1 additional_conditions = and enabled = 1 Then add the following MySQL table: CREATE TABLE `relays` ( `pkid` smallint(6) NOT NULL auto_increment, `recipient` varchar(120) NOT NULL default '', `enabled` tinyint(1) NOT NULL default '1', `status` varchar(10) NOT NULL default 'OK', PRIMARY KEY (`pkid`), UNIQUE KEY `recipient` (`recipient`) );
If the relay_recipient_maps setting is set, then postfix will reject all email addresses not specificed in this table. As with many postfix lookups, it will recursively search for a match from the full address. In the following examples, emails to [email protected] are the only emails that will be accepted for the whole example.com domain. However for @example.org all emails will be accepted for backup, except any for [email protected] which will be rejected. insert into relays (recipient,status) values ('[email protected]','OK'), ('[email protected]','REJECT'), ('@example.org','OK');
Return to top.
Local file backup Here is rough backup script to backup your configurations and mail folders. You may want to backup the folders seperatly as they can quickly grow to GBs. Adding this to a cronjob automates this process. Be aware that you should stop postfix and courier while backing up the mail folders. And that if they have grown large, that this may take some time. tar czf mail-config.xxxxx.tgz /etc/postfic /etc/courier /etc/spamassassin /etc/clamav /etc/amavis /etc/mysql/my.cnf tar czf mail-fold.xxxx.tgz /var/spool/mail/virtual mysqldump -u mail -papassword -t maildb > data.sql mysqldump -u mail -papassword -d maildb > schema.sql tar czf mail-data.xxx.tgz schema.sql data.sql tar cf mail.xxxxx.tar mail-*.xxxxx.tgz
You may combine a full backup with a intermediate update of what has changed recently only. tar --newer-mtime "2005-01-01"
Return to top.
Sender ID & SPF Further security features is using Microsoft's Sender ID or Pobox's SPF. I'd use SPF as there is much argument over Sender ID. spf.pobox.com
www.microsoft.com/mscorp/safety/technologies/senderid/ SPF should limit who can send mail on behalf of your domains, and is an open design. I do recommend SPF, with some reservations, detailed below. While Microsoft is not always entirely evil, as sometimes they do nice things and make some useful software, I would prefer not to be locked into their Sender ID technology.
SPF configuration The pobox site has some nice SPF generation tools to setup your SPF configuration. Probably best to use theirs. But the way I have my setup, is generally one domain with detailed SPF, then all other domains just with an SPF alias to it. e.g: Main domain DNS TXT field: "v=spf1 a mx a:myserver.example.com include:aspmx.googlemail.com include:gmail.com ~all"
The important elements are: • • •
I list the mail servers and websites associated with this domain (the a and mx bit). I then specifically list the name of a server I may send mail from applications automatically using addresses within this domain. As you can see I also use Google Apps with this domain, thus tell SPF to also allow all mail servers associated with google mail.
Then for most of the other domains I would use this DNS TXT field: "v=spf1 a mx include:example.com ~all"
The important elements are: • •
I list the mail servers and websites associated with this domain Then I tell SPF to also allow all mail servers associated with my main domain (example.com).
And for all these I use ~all! Ps. Some domains I have added an even stricter SPF, as these are domains that will never send an email.
SPF problem It is worth noting about SPF, that you should leave the decision to whether to reject or allow the email to the mail servers. Therefor using -all instead of ~all is not a good choice. Leave it to the SPAM scoring by the receiving server, like SpamAssasin does it. You then minimise the risk of false positives. One of the reason I do discourage -all use, is that SPF has a distinct problem: It does not like email forwarding or use of backup MX! Consider this: Your address of [email protected] sends a joke email to a few friends. One of these is [email protected]. Trixie's email address is actually an alias and forwards the email to her private webmail account on hotmailnot.com. Now if your domain, hoopa.com, have a strict SPF set up, which only allows emails to be sent by its mail server. And you/the mail admin has added -all to the SPF, which tells other server to reject emails not from your server. This you think makes sense, spammers can not use your domain for spoof emails. So what happens: bellbell.org receives the email from lulu, and possible checks the SPF, which is OK, and forwards it on to hotmailnot.com. However if hotmailnot.com also checks SPF, it will receive the email from bellbell.org, check the SPF to see bellbell.org's mail server is allowed to send emails on behalf hoopa.com. SPF will say No!, and with the -all, hotmailnot.com email server will reject the email! 2nd scenario if lulu email trixie directly at hotmailnot.com, but hotmailnot.com main mail server was down, and email was sent to the backup mx server. When the main server came online again, and the backup spooled the email back to it, the SPF would again fail as the hoopa.com's SPF would not mention hotmailnot.com backup mx as an allowed mail server. Solution: Of course you can not list all possible forwarding / backup mx email server that your domain's users might at some point email! I simple just use the ~all option. Which simple say it is not the expected server, but probably ok. And if this is added to a scoring by the receiver, then the accumulated spam score might be enough to reject dodgy emails. Return to top.
Spam reporting todo
Reporting spam to Pyzor, Razor and SpamCop, for collaboration in spam fighting. More detail on SpamCop is here. pyzor.sourceforge.net razor.sourceforge.net Return to top.
White/Black Lists todo
You can implement white and black lists to explicitly allow or block domains and users. You have already visited the option of a blackhole list of known open relays in the postfix configuration. You can implement further lists inside Postfix or SpamAssassin. Amavisd-new already has a few well known white/black listed items in its config files. SpamAssissin also as a feture to automaticly learn white lists. Return to top.
PGP & S/MIME Adding support for GnuPG and S/MIME increases indiviual security. This is not implemented on the postfix server side, as this totally a client side option. However SquirrelMail has a GnuPG option. It is a plugin that can be downloaded from their website. Which can then be enabled via SquirrelMail's config script. Here is how to create a GnuPG key pair. # check you have not already got a key gpg --list-keys # then create one gpg --gen-key
To import GnuPG into Evolution; in your settings/preferences edit your account settings and add you private key under the security tab. The private key is found via listing the GnuPG keys as above, then it is the 8 characters after the "sub 1024g/" bit of you key. To use GnuPG with Thunderbird you need to install EnigMail.
S/MIME is another way to encrypt and/or sign messages. You can create you own certificate or use known organizations like Thawte. (Thawte was originally set up by the Ubuntu founder) Return to top.
Relocation notice If people change addresses, a bounced message stating so if people send email to the old address is quite useful. To implement this in postfix, frst create a lookup table in the database. CREATE TABLE `relocated` ( `pkid` smallint(6) NOT NULL auto_increment, `oldadr` varchar(128) NOT NULL default '', `newadr` varchar(128) NOT NULL default '', `enabled` tinyint(1) NOT NULL default '1', PRIMARY KEY (`pkid`), UNIQUE KEY `oldadr` (`oldadr`) ) ;
Then add this to /etc/postfix/main.cf relocated_maps = mysql:/etc/postfix/mysql_relocated.cf
The create this file /etc/postfix/mysql_relocated.cf user=mail password=apassword dbname=maildb table=relocated select_field=newadr where_field=oldadr hosts=127.0.0.1
Then if [email protected] has changed address to [email protected]: INSERT INTO relocated (oldadr,newadr)VALUES ('[email protected]','[email protected]');
If anyone sends an email to [email protected], they will get a message back stating he has changed address to [email protected]. Return to top.
Pop-before-SMTP If SASL didn't work, or you are using clients which dont support it, the Pop-BeforeSMTP is an easy way around that issue, so that people externally can still securly send mail via your server. Refer to my 2nd edition on Pop-before-SMTP setup. Return to top.
Admin software todo
Trying out a few admin software might make you life easier, if phpMyAdmin gets to crude. Quick search More to come later. Return to top.
Auto Reply todo
Postfix have now features to auto reply to an email, while still delivering it to its alias. Return to top.
Block Addresses If you use catch alls, which are useful for some domains, then eventually some addresses will be target for spam. You can then either stop the catch all, or stop indivdual addresses. By implementing a lookup and adding this restriction to smtpd_recipient_restrictions accomplises this. check_recipient_access mysql:/etc/postfix/mysql_block_recip.cf, smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, \ check_recipient_access mysql:/etc/postfix/mysql_block_recip.cf, \ reject_non_fqdn_recipient, reject_unauth_destination, \ check_relay_domains
Beware of the order is important here, if any options says ok before check_recipient_access it will ignore it. Next create mysql_block_recip.cf to lookup addresses. Either create a another table, or add a blocked field to aliases table. Return to top.
Throttle Output todo
For some users with restrictions on bandwidth, you may wish to control how much mail is sendt out. Postfix has long refused to implement these features, out of ideolocial beliefs that mail servers should not be restricted. However there are some ways around this. More to come later. Return to top.
Mail Lists Rich Brown has written a howto on adding Mailman, a mail list program, to my howto. Click here to read it. Do note it is not part of my howto, so do not contact me regarding it. And although I think it is fine, I can't guarantee it will work. If you do need assistance or need to talk about it, contact Rich via his howto or use the forums for this howto. If you want a simple mailling list, it can be implemented by simply seperating aliases in the destination field in the aliases table with a comma. INSERT INTO aliases (mail,destination) VALUES ( '[email protected]' , '[email protected],[email protected],[email protected]' );
Return to top.
Google Apps / GMail I have for various reasons integrated some Google Apps hosted domains into my mail server. And you can still have good control over the addresses by using your server with Google Apps. More information on Google Apps.
Why • • • •
Some already have their domain's email hosted with Google. Some people prefer Google's web based interface. Temporary Migrations. Include Google's security features on top of yours.
How Options
The easiest and simples solution is not to have a domain MXed to your server, and simply alias email to those domains. eg All email to joeblogs.co.uk hosted on your server are forwarded to joeblogs.com hosted with google. You may set up your own server to simple be a mail server backup (mx) for a domain hosted with google. If you are the first priority in the MX details of the DNS, you still have some control, but not all will obey the priority listing. E.g. spammers, but some valid senders as well. However the one I use and the option where you are most in control is to keep you server as the only MX server in the DNS. And only forward certain aliases onto Google after all your servers checks. Other aliases and user can just use your mail server if you prefer. I will explain how to do this in the next steps.
DNS You only put your mail server as the mx for the domain in question. Google will complain about this, as it will not be able to verify that email is setup correctly. Ignore this as it will still accept emails.
MySQL tables You setup you aliases as normal. However you domain table needs tweaking. This is because otherwise your server will just forward the email to itself. You can actually specify aliases in the domain table.
Example If for example:[email protected] wants to use gmail. [email protected] does not. If not already configured as a backup mx: Add a transport lookup to your /etc/postfix/main.cf file: transport_maps = mysql:/etc/postfix/mysql_transport.cf Then create /etc/postfix/mysql_transport.cf file: user=mail password=apassword dbname=maildb table=backups select_field=transport where_field=domain hosts=127.0.0.1 additional_conditions = and enabled = 1
Assuming there are no bloggs.com data in any tables (domain,alias,users,relays,backups): insert into domains (domain,transport) values ('[email protected]','smtp:[aspmx.l.google.com]:587'), ('bloggs.com','virtual:'); insert into aliases (mail,destination) values ('[email protected]','[email protected]'), ('[email protected]','[email protected]'); insert into users (id,name,maildir,crypt) values ('[email protected]','mary','bloggs.com/mary',encrypt('maryspassword'));
The domains insert is the interesting one. The transport map lookup checks recursively for an alias match and will first look for user@domain before it looks at
the general bloggs.com for which transport to use. The square brackets around aspmx.l.google.com indicates that this server will not lookup for mx settings for this domains DNS, but instead connect directly. (This can avoid never ending recursive lookups/relays) Note if you have backup mx configured and chosen to enable relay recipient lookup to avoid backscatter mail spam, then you need to add your Google Apps users to the relays table: insert into relays (recipient,status) values ('[email protected]','OK'); Refer to backup mx section for creation of this table.
Issues There are some items you should consider when integrating Google Apps. Privacy First there is the privacy issue. This is the same as if you were using Google Apps only or GMail. Google can and will read your email. However probably not a person, but they will use it for commercial reasons, E.g. showing relevant ads. Some people really hate this part and refuse to use Google's mail products. However I trust them a little bit, and do use it. Spam If you forward spam, then consider your own servers reputation. Should be okay though. SPF If you use SPF for your domain, consider that both your server and google will receive and send mail on behalf of that domin. Adding include:_spf.google.com should cover it. Google internally Be aware Google think they host you domain. So if others inside google, or using google hosted apps or GMail, if they email you, the email may not go via your email server, but directly to the Google Apps for your domain. That could be an issue if not all aliases you have use Google Apps. This needs to be tested more though. Especially as it may only be an issue if Google's servers are part of you domains MXs. It may be worth adding aliases in your Google Apps admin for the non google apps adresses to some user whom can handle these? Return to top.
Maildrop, spam folder and vacation messaging Villu have documented swapping in Maildrop for virtual transport and automatically deliverin spam to a spam folder. (And links to a post about vacation messaging)
Please read his post here. Return to top.
Roundcube webmail client As an alternative to SquirrelMail, Roundcube has a more modern feel to it. It is however not got the long testing track record of SquirrelMail, and is not yet a 1.0 release version. However if you prefer its very much more appealing interface then follow these easy steps: You may optionally want to uninstall SquirrelMail: sudo aptitude remove squirrelmail squirrelmail-locales
Then install Roundcube sudo aptitude install roundcube roundcube-mysql
It will ask you if you want to configure its database access, answer yes, then select mysql. Then it will ask for the root mysql uses password, which it will create a roundcube mysql user and ask for its desired password. This will create a symblink in /etc/apache2/conf.d/ to /etc/roundcube/apache.conf. Edit this file. sudo vi /etc/roundcube/apache.conf
Depending on your setup you may want to move those Alias commands at the top to your virtual hosts configuration, or for this example enable them here for all hosts. # Uncomment them to use it or adapt them to your configuration Alias /roundcube/program/js/tiny_mce/ /usr/share/tinymce/www/ Alias /roundcube /var/lib/roundcube
Next edit the configuration file sudo vi /etc/roundcube/main.inc.php
Modify these lines for added security and ease of log in: $rcmail_config['default_host'] = 'ssl://localhost:993'; $rcmail_config['smtp_server'] = 'ssl://localhost'; $rcmail_config['smtp_port'] = 465; # keep as default or change to your mail server name $rcmail_config['smtp_helo_host'] = 'mail.example.com'; $rcmail_config['create_default_folders'] = TRUE;
There are other tweaks and security features you can enable such as:
$rcmail_config['sendmail_delay'] = 1;
But perhaps concentrate on getting the basics working firtst... Save, exit and reload Apache to enable these aliases for Roundcube to work sudo /etc/init.d/apache2 reload
Then go to your roundcube installation depending where and how you modified those Aliases, e.g. at http://mail.example.com/roundcube. That should be it You can obviously modify and tweak further. One thing that may be usefull is to have the Roundcube Apache Alias on different virtual hosts, and configure username_domain in main.inc.php to append different email addresses, or configure the default_host to different mail server depending on virtual host... More details on the Roundcube Wiki. Return to top.
Suggestions? If you have any suggestions to other ways of extending a postfix server, then fire off a mail to me via the contact form further down. (Or rather, Id prefer that you write down the extension, and let me know the link! :)) Return to top.
Elastic Compute Cloud Recommended ec2 books
Amazon Web Services Made Simple: Learn how Amazon E… Donald Robinson … $17.99
Host Your Web Site On The Cloud: Amazon Web Services … Jeff Barr (Paperb… $26.96
Cloud Application Architectures: Building Applic… George Reese (P… $19.79
Programming Amazon Web Services: S3, EC2, SQS, FPS… James Murty (Pa… $31.49 12> Privacy Get Widget
Amazon.com Widgets • • • • •
Impressions ec2 introduction, tips and hwotos Using EC2 with this howto Amazon EC2 Images: AMIs EC2 Links
Impressions Easy to use. Anyone can use, not just big companies. Very useful. Tools are command line but simple. Firefox extensions work well. Recommended. I find it very usefull. Basically it is a colo hosting environment. Some may use it as for Saas, ie single scalable application in the cloud, but I use it as a hosting environment for complete servers.
ec2 introduction, tips and howtos I have made a separate tips and howto on the use of ec2 for general server needs. Hope it will be useful for people.
How I use it for my mail servers Different images to launch for different needs. Good way to scale backup MXs if needed. Can script backup to S3 of mail dirs etc.
Using EC2 with this howto If you plan to use EC2 to follow this howto, then familiarise yourself with EC2 first. Check the links further down, e.g. my tips.
Once competent enough on EC2, launch the latest official Ubuntu ec2 image or one of Eric Hammond images. You can cheat by using my other images, but you should really know how the whole server was built by starting from the bottom. When using EC2 images, be aware of security groups as they restricts access to your server on top of the firewall. Initially you will need SSH (22) access, quite soon you will need SMTP and IMAP ports opened, 25,143,465,587 and 993, and eventually webserver ports of 80 and 443. Read here for tips on securing AMIs. Also do not terminate your instances without backing up your machine. This you can do by either create your own image. Or backup certain data if you got an image to instantiate from. Back up to S3 or your local machine. Create images only now and then. Backup configurations, database, maildirs more regularily. Once launched, follow my Initialize section. 1st note: Spamhaus.org lists amazons ec2 ip ranges as dynamic, thus many mail servers will reject emails from it. (Including other people using this howto.) But Spamhaus has a simple web page to remove ips, which they link to in rejection messages. Simple look in your logs, click on the link on follow the instrucions: basically fill in your ip, email and state its for a mail server. Then Spamhaus will remove your IP from their database. 2nd Note: Amazon has extended this spam limitations, so if you have a busy mail server, follow their FAQ entry for removing mail throttling. 3rd Note: This fix needs to applied to the instances buildt on an early 8.04 base. This is not a problem with the later Hammond or any Canonical based images. 4th Note: Check I have not left my SSH key in the root or ubuntu user. 5th Note: Please let me know if I've been silly enough to leave a bash history or anything like that?
Amazon EC2 Images: AMIs Public AMIs to use as base:
Current AMIs:
AMI
Description
ami2d4aa444 US amiW: c997c68c amiEU: cf4d67bb amiAsia: 57f28d05 amiUS E: f8a64e91
S3 Name
Ubuntu version
Extended from
Status
US E:
EU:
ami91341ee5
US E:
ami9c947cf5
US E
ami0a957d63
US E
amic0ee06a9
ubuntuUbuntu's images/ubuntu- 10.04 official server lucid-10.04LTS AMI i386-serverLucid 20100427.1
My base flurdyUbuntu server 10.04 amis/ubuntuAMI. Postfix LTS lucid-basenot Lucid 20100604-02 configured. flurdyBasic Postfix amis/ubuntu- 10.04 server, with lucid-mailLTS MySql, server-baseLucid amavis, TLS. 20100610-01 flurdyamis/ubuntu- 10.04 With Courier lucid-mailLTS IMAP, SASL imap-20100610- Lucid 01 With flurdy10.04 RoundCube amis/ubuntuLTS Webmail, lucid-mail-webLucid PhpMyAdmin 20100611-01
OK
ami2d4aa444 Ubuntu's official OK amibase cf4d67bb
amiflurdy's f8a64e91 base
Testing
amiBasic 9c947cf5 Postfix
Testing
amiWith Testing 0a957d63 Courier
Older AMIs: ( Images with strike through them are no longer recommended. Their are fine for experimenting and testing, but should not be used for permanent "live" servers ) AMI ami1515f67c ami-
Description Base install: Canonical's Official Ubuntu 9.10 Karmic 32bit US Base install:
S3 Name canonical-cloudus/ubuntu-karmic9.10-i386-server20091027.1 canonical-cloud-
Ubuntu Extended from version 9.10 Karmic 8.10
5059be39 amic4f615ad amice44a1a7 ami4132d428 ami0f41a466 amieb39df82 ami8541a4ec ami9941a4f0 ami395fba50 ami275fba4e ami-xxx
ami-xxx
Canonical's Official Ubuntu 8.10 Intrepid 32bit US Base install: Alestic Ubuntu 8.04 LTS Hardy 32bit US Base install: Alestic Ubuntu 8.04 LTS Hardy 32bit US Clean with all packages installed but no configuration Clean with all packages installed but no configuration
us/ubuntu-intrepid20090422-i386
Intrepid
alestic/ubuntu-8.04hardy-base-20091011
8.04 LTS Hardy
alestic/ubuntu-8.04hardy-base-20080430
8.04 LTS Hardy
flurdy-amis/ubuntumail-server-clean20090529-2 flurdy-amis/ubuntumail-server-clean080502-1 flurdy-amis/ubuntuJust mysql, postfix and mail-server-basiccourier configured 20090604-1 flurdy-amis/ubuntuJust mysql, postfix and mail-server-simplecourier configured 080504-1 flurdy-amis/ubuntuIncluding anti spam mail-server-spamand anti virus 080504-1 Including TLS and flurdy-amis/ubuntuSASL encryption and mail-server-secureauthentication 080527-2 flurdy-amis/ubuntuWith webmail and mail-server-webmailadmin enabled 080527-1 flurdy-amis/ubuntuWith back up mx mail-server-backupxxx flurdy-amis/ubuntuWith back up mx only mail-server-backuponly-xxx
8.10 ami-5059be39 Intrepid (Canonical) 8.04 LTS ami-ce44a1a7 Hardy (Old Alestic) ami-4132d428 8.10 (Clean Intrepid Canonical) 8.04 LTS ami-0f41a466 Hardy (Clean Alestic) 8.04 LTS ami-8541a4ec Hardy (Simple) 8.04 LTS ami-9941a4f0 Hardy (Spam) 8.04 LTS ami-395fba50 Hardy (Secure) 8.04 LTS ami-275fba4e Hardy (Webmail) 8.04 LTS ami-395fba50 Hardy (Secure)
If you have a comment or question about the ec2 images, please discuss it in the forums? If you notice a security problem, or I have not cleaned the images properly please let me know?
EC2 Links • • • •
Amazon web services (AWS) Elastic Computing Cloud (EC2) Simple Storage Service (S3) AWS Cost Calculator
•
EC2 Resource Centre
•
EC2 Starter Guide
•
EC2 Firefox extension: Elasticfox
• • • • • • • • •
Elasticfox for Firefox 3 S3 Firefox extension: S3Fox EC2 to S3 Admin Scripts Eric Hammond 8.04 AMI alestic, ubuntu ec2 images Ubuntu ec2 Ubuntu Cloud Ubuntu ec2 Starter Guide My ec2 Tips and Howtos
Return to top.
Appendix • • • • • • • • •
About author Contact Why References Software Links Difference between Ubuntu versions Download Todo Change Log
About author Ivar Abrahamsen, an IT Senior Consultant from Norway. Specialising in developing and integrating middleware application systems. Mainly open source and Java based technology stack. Recently moved back to Oslo, Norway after 15 years in Manchester. Return to top.
Contact Remember I have stood on the shoulders of giants. I just ended up with a system that worked for me, and decided to document its evolution. Before contacting, have you?: •
• • • • • •
Read this document properly? Followed it step by step? (While we can not insist on the same setup for everyone, assistance is easier and more likely if less customised) Applied the solutions provided in the test section? Read the forums for solutions already posted? Read the FAQ? Tailed the mail.log? It usually tells you what the problem is! Tailed the mysql.log? If nothing happens there it should indicate something... Include a short dump of the mail.log with your post. (Remember to anonymise the servernames, email adresses & definetly passwords!)
Forums Use the Ubuntu forums! :) • •
Here is a thread on this specific mail server howto. And another one by me which is also used..
Please participate in the forums. • • •
If you see an issue you also have, contribute with more information. And even better if it something you may know how to solve, please let people know. And especially, if you post a problem, then solve it, let people know what the solution was! (and not just that you solved it...)
I am rubbish in replying to emails, and the forums are read and answered by people whom know a lot more about Postfix than me. Questions sent to me directly may not be answered for a while or at all unfortunetly.
Return to top.
Consultancy and advice Not to be rude, but in attempt to reduce the volume of emails I receive please consider the following: • • • •
•
• • • • • •
If you simply want advice please use the forum. My references and the people whom follow the forums and the postfix mailling lists are much more knowledgable than me. People whom follow the forums and the postfix mailling lists are much more likely to reply than me. I am a firm believer in: Give a man a fish; you have fed him for today. Teach a man to fish; and you have fed him for a lifetime. (Playing far too much Civ was not all wasted..) So if you have any questions, problems with using this guide, or any other issues with this guide, please use the forum. Then it is also an available archive for others to find solutions in the future. Any technical difference of opinion, please use the forum. Interested in Postfix, or got a technical query about it? Considered the postfix mailling lists. If you made / found an extensions to this tutorial, Fantastic! Please let me know, and Ill link to it. And shout at me, if I am slow in doing so. :) Any clear technical mistakes by me in this guide, then let me know, but perhaps discuss it first in the forums? If you find any spelling mistakes or broken links, please let me know. Thank you messages are very appreciated however! Actually it makes my day. :) (People whom buy a T-shirt of me makes my week.. ;p )
If you still want my advice or hire me: • • •
You may hire me as a consultant via www.eray.co.uk. You may hire me to set up a complete server for you via www.eray.co.uk. In the end I am contactable via flurdy.com/contact :) However : o Do remember my fishing analogy. o I do not appreciate people whom wants shortcuts and ask me to set their server up for them, especially for free. o Nor do I find it amusing when people have not even followed my suggested steps, and/or not tested properly and ignored the logs etc. o Email servers are important and potentially dangerous, so you really need to learn how it works, and how to tweak it.
Return to top.
Why
Why your own mail server Main reason: Because you can. Other good reasons: Basically it leaves you in complete control, to expand, customize and tweak your mail server to your needs. You are not dependant on 3rd party providers, limited by their technology contraints or your budgets. With your own mail server you can add as many aliases, users and domain as you'd like, be as restrictive or open about security, virus, spam, file sizes etc as you prefer. And is it is well known, frequently updated, open source application stack, you can also trust the software you use.
Why I wrote this howto When I set up my first email server I used a mix of other howtos on the net. And they were so helpfull that I though I would contribute back with my experience. And it has been useful as a recipe script for myself every time I need to install/update a server. A less angelic reason is that back in 2003 I was setting up mail server for a few friends and collegues. Soon I was getting more request, and being a lazy programmer, I thought.. "Why don't I write a howto and let them do it themselves..." Soon it was listed on postfix.org and I was getting thousends of hits and lots of emails. (blessing in disguise)
Why I wrote this edition Or rather why no new edition or updates for two years? Well basically no time or need to do so, so basically lazyness... My last edition was written two years ago, and was pretty complete and thorough so my inclination to write a new one has been low, especially as my own mail server had not changed since then either. But then my server started crashing so I upgraded it to Ubuntu 8.04 which went pretty smooth, but with a few tweaks. So time for another edition. This time I expanded reliability to include the possibility of running backup mx servers using Amazon's Elastic Computing Cloud. Note, however this is an optional extra at the end. Return to top.
References •
Postfix howtos
• • • • • • • • • • • • •
Kyle's book John Locke on TechRepublic Hildebrandt's book Hildebrandt's website List-Petersen Genco Yilmaz Christop Haas Nenzel & Peet Peters Matthews Stepanov Andy "Besy" Meta Consultancy
New references • • • • •
Postfix TLS Postfix main.cf doc saslauthd Bypassing amavisd Ubuntu Help: Squirrelmail
Return to top.
Software Links Please refer to the previous edition for a list of urls and suiteable downloads. However most are unneccessary with decent package manager.
Difference between Ubuntu versions I used to distinguish differences, which were available in a previous edition.
Download Please refer to the previous edition for a complete lists of downloads that are available.
Change log Brief list of latest changes. • •
2010-06-09: Improved Google Apps integration. Added backup relay recipient lookup. Update phpmyadmin section. 2010-06-07: Updated for Ubuntu 10.04 LTS Lucid Lynx Modified mysql log option. Removed dynamic uid & gid in postfix.
• • • • • • • •
2010-02-15: Redid SASL secure authentication section. 2009-12-16: Expanded test section with text from older editions and new babble. 2009-11-25: Bumped to edition 9! And added Roundcube as webmail client. 2009-11-11: Updated to work with 9.10 Karmic Koala. 2009-06-04: made basic server image available on ec2. based canonical's official ec2 ami. 2009-06-02: made clean server image available on ec2. based canonical's official ec2 ami. 2009-05-29: changed contact section. 2009-05-29: started 8th edition
Used to refer to all changes, but got too long. A previous edition contains such a list. Return to top.
Todo • • •
Spell check! Remove uid and guid Create backup mx AMI
Please refer to the previous edition for some old todos....
FAQ There is not yet an extensive FAQ. But please, most of the frequent questions have been asked and answered in the forums. Most are also unneccessary as following the test section will have solved them. Some question that frequently get sent to me, which first of all should have been asked in the forums and has been answered there many times, which then I tend to ignore are:
Squirrelmail does not allow me to log in This is due to many things. Most are due to skipping too fast forward, ignoring test sections etc. Answers: o
Does postfix work? No point trying to run before you can crawl. Send emails to recipients
o
o o
on your server, tail mail.log to see if everything is okay. Often mysql is not configured properly, check the mysql logs for activity. Have they ever received an email? If not they can not log into squirrelmail as the email folders will not yet exist. Does Courier work? If it doesn't then you have still got some more setup to do. If all above is okay, then it may be a problem with your Squirrelmail setup. Check empty spaces in squirrelmail mysql setup. More details in test section.
Email folders do not exist Mentioned many times in this guide and forums. Answers: o
o
Have they received an email? If not they you can not log into squirrelmail as the email folders will not yet exist. When receiving their first email, postfix will create all the neccessary folders. If it does not your postfix setup is broken. There is a program that creates the folders for you. I do not recommend it, as basically your postfix setup is broken if no folders are created, and you better fix it instead.
Creating a Linux Mail Server (Postfix, Procmail, Fetchmail, SpamBayes, Courierimap, Mutt, SquirrelMail)/III
What this Document is About Software Used Installing the Software • • • • •
Installing Postfix Installing Fetchmail Installing Procmail Installing SpamBayes Installing Mutt
• • •
Installing courier-imap courier-authlib Installing SquirrelMail Installing pop-before-smtp
Postfix Configuration •
main.cf
Fetchmail Configuration Spambayes and Procmail Configuration • •
Spam System Initial Setup Procmail Configuration for Spambayes
Mutt Configuration • •
.muttrc Configuration New Mutt Commands
SquirrelMail Configuration Courier Authentication Pop-before-smtp Configuration Additional Simple Procmail Recipes and Mailbox Config Conclusion Resources • •
Example Files Links
Contributions
What this Document is About This document covers how I have set up my Linux box to work as a Mail Server. The details will be specific to Arch Linux, but it will be be general enough for any *nix distribution if you have a knowledge of installing software packages, finding the location of configuration files, and starting/stoping services..
My goals in creating this mail system are to have all of my mail, from various email accounts, stored locally on my machine so I can access it anytime via SSH, the web, or imap. I wanted all of my mail delivered to my local user account and I wanted the spam to be sucked out and launched into space. The backbone of the mail system is the Mail Transport Agent (MTA). It will handle receiving mail addressed directly to your machine, sending it to the correct users mail box, and sending mail out from the machine. I determined Postfix to be the best choice for my MTA and after some reading found Procmail for sorting mail, SpamBayes for detecting spam, and Fetchmail for grabbing mail from my pop mailboxes. Bellow is a flow diagram for mail moving through the system:
The SpamAssassin and sa-learn can be ignored. I will describe how to use it with SpamBayes since I was not happy with Spam Assassin's performance and had made the diagram when I was still using it.
Software Used • •
Arch Linux Postfix 2.1.5
Secure, fast, easy to administer drop in replacement for Sendmail. (MTA) Provides smtp-server. (Requires sasl installed and support for sasl compiled into Postfix if sending mail through an intermediary as you may need to do if you have a domain name pointing to a dynamic IP address.) •
Cyrus SASL 2.1.21 SASL authentication daemon.
•
Procmail 3.22 Highly configurable auto mail processing.
•
Fetchmail 6.25 A remote-mail retrieval utility.
•
SpamBayes 1.1a1 a Bayesian anti-spam filter, initially based on the work of Paul Graham. The major difference between this and other, similar projects is the emphasis on testing newer approaches to scoring messages.
•
Mutt 1.4.2.1 A small but very powerful text-based mail client
•
Courier-imap 4.0.4 IMAP / POP3 Server
•
Courier-authlib 0.57 Authentication library for the courier mailserver.
•
SquirrelMail cvs 1.5.1 For checking mail via the web. (requires imap-server, perl, and php)
•
Pop-before-smtp 1.38 A simple daemon written in Perl, to allow email relay control based on successful POP or IMAP logins.
Installing the Software
Installing Postfix A note for those with dynamic IP addresses: Before installing Postfix we must first consider if it will need to make use of Simple Authentication and Security Layer (SASL). If you have a dynamic IP address and are using a service like dyndns.org you will need to have Postfix send your mail through your ISP's mail server (with authentication) making use of SASL for the connection. This is because many domains that you will send email to will recognize your hostname as pointing to a dynamic IP address and send the mail back. Check if your distributions Postfix package was built with SASL support or if you are compiling Postfix from scratch add SASL with a ./configure option. If you've determined you do not need SASL support in postfix: $ pacman -Sy postfix If you do need SASL support we need to install cyrus-sasl and use the Arch Build System (abs) utility for building Postfix. Follow the instructions bellow or follow these Arch Wiki instructions: http://wiki.archlinux.org/index.php/PostFix_Howto_With_SASL $ pacman -Sy cyrus-sasl $ abs $ cd /var/abs $ mkdir -p local/postfix $ cp -v extra/daemons/postfix/* local/postfix Now edit /var/abs/local/postfix/PKGBUILD. Under the build() section change: make OPT="${CFLAGS}" || return 1 to: make OPT="${CFLAGS}" \ CCARGS="-DUSE_SASL_AUTH -I/usr/include/sasl/" \ AUXLIBS="-L/usr/lib/sasl2 -lsasl2" || return 1
Now run: $ makepkg $ pacman -A postfix*.pkg.tar.gz I like to then copy the package into a directory under /var/lib/pacman so I have access to all of my abs built packages later: $ mkdir /var/lib/pacman/abs_built $ cp postfix-*.pkg.tar.gz /var/lib/pacman/abs_built
Installing Fetchmail $ pacman -S fetchmail
Installing Procmail $ pacman -S procmail
Installing SpamBayes SpamBayes is not yet in the pacman repository. Download it, extract, and install (substitute version numbers with most current): $ wget http://dl.sourceforge.net/sourceforge\ /spambayes/spambayes-1.1a1.tar.gz $ tar zxvf spambayes-1.1a1.tar.gz $ cd spambayes-1.1a1 $ python setup.py install
Installing Mutt $ pacman -S mutt
Installing courier-imap and courier-authlib SquirrelMail as well as other mail clients (evolution, outlook, cell phone, etc.) will use this to connect to the mail box. Courier-imap also depends on courier-authlib, so this command will install that as well.
$ pacman -S courier-imap
Installing SquirrelMail $ pacman -S squirrelmail
Installing pop-before-smtp You are going to want to consider using Postfix/TLS or pop-before-smtp if you are interested in users connected via IMAP being able to send mail (using your mail server as a SMTP relay). I recommend using Postfix/TLS but will not cover how to install/configure it in this document. Pop-before-smtp is not in pacman repositories. Substitute version numbers with the most current. $ wget http://dl.sourceforge.net/sourceforge/popbsmtp/pop-beforesmtp-1.38.tar.gz $ tar zxvf pop-before-smtp-*.tar.gz $ cd pop-before-smtp-* Read the README in this directory and then read README.QUICKSTART in the contrib directory for where to copy the necessary files to.
Postfix Configuration
main.cf Now change to the '/etc/postfix' directory and open up main.cf for editing. The domain name for my machine is through dyndns.org. Set your myhostname and my domain to something like this: myhostname = example.com mydomain = example.com Next we are going to want to set Postfix up for using qmail-style delivery with Maildir (where each email is stored in its own file). This is for courier-imap which SquirrelMail will be using to access the mail and will have other advantages. Don't forget the ending forward slash.
home_mailbox = Maildir/ Set it up so that incoming mail is sent through procmail and so we can alias names/users to mailboxes. mailbox_command = /usr/bin/procmail -a "$DOMAIN" alias_maps = hash:/etc/postfix/aliases If you have a dynamic IP or have trouble with mail being returned from large ISP's or mail systems you are going to have to take a few extra steps. We will set it up so that mail will go through a relay host. In this example I will use my ISP's (SBC) smtp server as an example. Make these additional changes to main.cf relayhost = [smtp.sbcglobal.yahoo.com] # authentication for sbc yahoo smtp_sasl_auth_enable = yes smtp_sasl_security_options = noanonymous smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd That wraps up our changes to main.cf. We just need to create the hash aliases databases that we defined above. Add these lines to the top of /etc/postfix/aliases (replacing hypexr with your user account of course): root: hypexr spam: hypexr ham: hypexr
Generate the database. From /etc/postfix dir: $ postalias aliases Set up smtp server and authentication for SBC mail relay. sasl_passwd: smtp.sbc.mail.yahoo4.akadns.net [email protected]:my_passw
Note: Server name can change from time to time. There is a CNAME that points to this name but CNAMEs do not work here. Generate the database: $ ./postmap sasl_passwd Start the postfix server and we are done with postfix!!!! $ /etc/rc.d/postfix start Having problems getting it to start? Check out the Postfix log file usually in /var/log/mail or /var/log/maillog.
Fetchmail Configuration We are going to run our fetchmail daemon from user root. So create the file /root/.fetchmailrc: set postmaster "postmaster" set bouncemail set no spambounce set properties "" set daemon 60 poll pop.example.com with proto POP3 user 'user_name' there with password 'my_password' is 'local_user_to_deliver_to' here options flush poll pop.gmail.com with proto POP3 port 995 user '[email protected]' there with ssl with password 'my_password' is 'hypexr' here options keep This tells fetchmail to check every minute for mail at pop.mypop.com and also at my gmail account. The flush option at the end of the mypop entry tells it to remove the mail after it is downloaded. I do not want the mail removed from my Gmail account so I use the keep option. Google also uses ssl, hence the "with ssl". Fetchmail comes with its configuration utility fetchmailconf which makes setting up your .fetchmailrc file easy. I don't believe that Arch Linux has a rc script for fetchmail. The one that I made (/etc/rc.d/fetchmail) looks like this (stat_busy, stat_fail, stat_done are specific to Arch Linux. You could take these out):
#!/bin/sh . /etc/rc.conf . /etc/rc.d/functions DAEMON_NAME="fetchmail" DAEMON_CONF="/root/.fetchmailrc" DAEMON_PATH="/usr/bin/fetchmail" PID=`pidof -o %PPID ${DAEMON_NAME}` case "$1" in start) stat_busy "Starting ${DAEMON_NAME}" [ -z "$PID" ] && ${DAEMON_PATH} -f ${DAEMON_CONF} -d 60 &> /dev/null if [ $? -gt 0 ]; then stat_fail else add_daemon ${DAEMON_NAME} stat_done fi ;; stop) stat_busy "Stopping ${DAEMON_NAME}" [ ! -z "$PID" ] && kill "$PID" &> /dev/null if [ $? -gt 0 ]; then stat_fail else rm_daemon ${DAEMON_NAME} stat_done fi ;; restart) # calling 'stop' and 'start' without the $0 fails... $0 stop sleep 3 $0 start ;; *) echo "usage: $0 {start|stop|restart}" esac exit 0
And start the fetchmail daemon: $ /etc/rc.d/fetchmail start Or if you don't want to create/modify an rc script you can start the fetchmail daemon like:
$ /usr/bin/fetchmail -d 60
Spambayes and Procmail Configuration We are going to be using Spambayes to train and determine if mail is spam and Procmail to sort the mail into folders when they enter our mail system. These two go together as far as Procmail is going to use Spambayes to help sort the mail. I had used SpamAssassin for a year and had not been very happy with its performance. I have only been using Spambayes for a few weeks and have been pleasantly surprised by how well it has worked and how quickly it adapts. In fairness I had been training SpamAssassin on all of my spam, the spam it had caught and the spam it had missed as well. It was also an older version and I was not using all of the features. With that being said if you are curious give it a try. Many other excellent programs exist. A quick search on Google will turn them up. When Spambayes is installed it is going to move several files that start with sb_ into your /usr/bin directory. We are going to be interested in sb_mboxtrain.py and sb_filter.py. The former trains the spam filter and the later classifies incoming spam. There are two methods I tried for the initial training: 1. Since I already had thousand of spam and ham (non spam) messages saved up I first tried initially training on this older mail. The results were okay but it was miss sorting mail. These problems would have sorted them selves out with the continued training of the misclassified mail but it would have also took a little while because the database had already been trained so much. 2. The second method I used was extremely effective. I started out with no training on initial messages. I simply let Spambayes work with no information. I then sent the spam and ham messages that ended up in the wrong mailbox to a designated folder. I then run the Spambayes training program on these miss classified messages. The results from the second method have been amazing. Without even having to manually classify very many messages Spambayes has been performing at superior levels. Check out Spambayes Wiki for many other training methods. I will discuss easy methods of sorting miss classifications in the Mutt and Squirrel Mail sections.
Spam System Initial Setup We are going to want Maildir folders to hold our regular mail, mail classified as spam, mail classified as unsure, missed spam, and missed ham.
If the standard Maildir directory (~/Maildir) doesn't already exist lets create it with the necessary sub directories to make it a mail box. Run these commands from your user account who will be receiving the mail. $ mkdir ~/Maildir $ mkdir ~/Maildir/{new,cur,tmp} And now the other 4 directories that we are going to use for dealing with spam (Making the directories hidden is going to be done for integration with Squirrelmail): $ mkdir ~/Maildir/{.MissedSpam,.MissedHam,.CaughtSpam,.Unsure} $ mkdir ~/Maildir/.MissedSpam/{new,cur,tmp} $ mkdir ~/Maildir/.MissedHam/{new,cur,tmp} $ mkdir ~/Maildir/.CaughtSpam/{new,cur,tmp} $ mkdir ~/Maildir/.Unsure/{new,cur,tmp} The messages that we are going to be training as spam are going to be in ~/Maildir/.MissedSpam and ham will be in ~/Maildir/.MissedHam. Lets create the script and cron job that will take care of training the Spambayes db. Create a file ~/Maildir/train_spambayes and add this: #!/bin/sh # Script to copy mail missed spam and ham into correct folders # and run sb_mboxtrain.py to train spambayes # Training will be done only on missed spam and ham # Files we saved or bounced may be in the new directory # We want them in cur for training mv /home/hypexr/Maildir/.MissedSpam/new/* \ /home/hypexr/Maildir/.MissedSpam/cur 2>/dev/null mv /home/hypexr/Maildir/.MissedHam/new/* \ /home/hypexr/Maildir/.MissedHam/cur 2>/dev/null /usr/bin/sb_mboxtrain.py -d /home/hypexr/.hammie.db \ -g /home/hypexr/Maildir/.MissedHam \ -s /home/hypexr/Maildir/.MissedSpam
Make the script executable: $ chmod +x ~/Maildir/train_spambayes Set up our database that will be consulted for new email and trained on new spam/ham:
$ /usr/bin/sb_filter.py -d $HOME/.hammie.db -n You will want cron installed so that it can launch the script above, which trains Spambayes, every night at 3:21 am. Edit your cron jobs with the command crontab -e and add: 21 3 * * * ~/Maildir/train_spambayes
Procmail Configuration for Spambayes These are basic procmail recipes that will sort the spam from ham. From Spambayes on Unix or Linux. Add these lines to your mail recipient's ~/.procmailrc: SHELL=/bin/sh MAILDIR=$HOME/Maildir DEFAULT=$HOME/Maildir/ CAUGHT_SPAM=$MAILDIR/CaughtSpam/ UNSURE=$MAILDIR/Unsure/ #Spambayes process :0fw:hamlock | /usr/bin/sb_filter.py -d /home/hypexr/.hammie.db :0 * ^X-Spambayes-Classification: spam ${CAUGHT_SPAM} :0 * ^X-Spambayes-Classification: unsure ${UNSURE} # Catches everything else. # Anything left over goes into the DEFAULT folder :0: ${DEFAULT}
So incoming mail is not filtered through procmail's rules where Spambayes's sb_filter decides whether it is spam or ham. The mail is now sitting in the correct directories and ready to be accessed by whatever reader/front-end you would like. Next I going to describe setting up Mutt and SquirrelMail for viewing your mail. Mutt will be the most powerful option combining speed and sleekness but SquirrelMail will be able to match functionality.
Mutt Configuration First we need to set the MAIL variable so that Mutt will know where to look for your mail. In your ~/.bashrc (create if does not exist) add the line: MAIL='~/Maildir' Exit out of your shell and log back in so that the variable will be set or run export MAIL='~/Maildir'. On some systems mutt will not use $MAIL to determine the location of your mail box. When trying to start mutt if it reports a message that no mail directory exists you can try setting the MAILDIR variable or just create the directory that it wants to use and make it a valid Maildir: mkdir -p ~/.maildir/ {cur,new,tmp} where .maildir is the folder that mutt wants to use. If you use a directory other than ~/Maildir remember to substitute that wherever I reference Maildir in this document. Now for the fun part. Lets make everything look pretty and integrated in Mutt. If you start up Mutt right now you will see any mail that your system has received since you have completed the steps above. If no mail shows up in Mutt and there should be mail you need to first check if Spambayes has filtered it into your .Unsure or .CaughtSpam directories. Type 'c' to change directories and enter ~/Maildir/Unsure and check out the mail thats been classified as spam: 'c' ~/Maildir/CaughtSpam. If you still have not found the mail that you are expecting to have received make sure that you have configured Postfix, Fetchmail, and Procmail correctly. Also, try sending mail directly to [email protected] and see if it shows up in one of the directories above. Now mail is flowing into your system and we can check how Spambayes has classified it by checking the different mail folders. The miss classified mail just needs to be moved into the correct folders for the training. To manually move the misclassified spam mail: 1. From you Maildir inbox tag each message that has arrived that is spam. Do this by pressing 't' and an asterisk will appear by the messages. After all of the messages have been tagged, press '; m' and enter ~/Maildir/MissedSpam and the spam messages in your inbox will be moved to the MissedSpam mailbox for training by spambayes. 2. When looking at the mail that it has classified as spam in '~/Maildir/CaughtSpam', tag all of the messages it considered spam that were ham and press '; C' and enter ~/Maildir/MissedHam. Now tag them again and press '; m' and enter ~/Maildir to move them into your inbox, where they should have been placed in the first place. Of course all of this manual stuff is a huge pain so lets automate all of it in .muttrc.
.muttrc Configuration This .muttrc file is going to save you a quazillion.. thats right a quiazalliona hours in productivity in dealing with our current setup. Here it is: # Emacs is my editor of choice. I have mostly read that people like # using emacs-client here and using an emacs server so that the # editing opens up in an existing emacs session but I prefer # loading up a fresh instance. The default editor is some 'vi' set editor="emacs" # Setting these macros is going to save us all of that time I was # talking about. Now instead of having to type all of those # characters to move the mail around we will can just tag the messages # and, for the instance of spam in your inbox, hit 'S'. # Move mail to correct directories macro index S ";s~/Maildir/.MissedSpam\r\r$\r" macro pager S ";s~/Maildir/.MissedSpam\r\r$\r" macro index H ";C~/Maildir/.MissedHam\r\r;s~/Maildir\r\r$\r" macro pager H ";C~/Maildir/.MissedHam\r\r;s~/Maildir\r\r$\r" macro index M ";s~/Maildir/\r" macro pager M ";s~/Maildir/\r" # This one lets me quickly move job search related emails to the # correct directory. Want to give me a job? :) macro index B ";s~/Maildir/.Job\r\r$\r" macro pager B ";s~/Maildir/.Job\r\r$\r # This is how we are going to move around to the different # mailboxes. Hitting 'alt-1' will take us to our inbox # 'alt-2' and we are looking at the mail Spambayes classified # as spam. etc. # The extra mailboxes I have in here are for mailing lists I am on. # I will show the procmail recipe that automatically puts # the mailing list mail into its correct mailbox further down. # Move to mailboxes quickly macro index 1 "c~/Maildir\r" macro pager 1 "c~/Maildir\r" macro index 2 "c~/Maildir/.CaughtSpam\r"
macro pager 2 macro index 3 macro pager 3 macro index 4 macro pager 4 macro index 5 macro pager 5 macro index 6 macro pager 6 # Training Maildirs for macro index 8 macro pager 8 macro index 9 macro pager 9 # Job E-mails macro index 0 macro pager 0
"c~/Maildir/.CaughtSpam\r" "c~/Maildir/.Unsure\r" "c~/Maildir/.Unsure\r" "c~/Maildir/.Tur\r" "c~/Maildir/.Tur\r" "c~/Maildir/.Sbayes\r" "c~/Maildir/.Sbayes\r" "c~/Maildir/.Wikka\r" "c~/Maildir/.Wikka\r" Spam and Ham "c~/Maildir/.MissedSpam\r" "c~/Maildir/.MissedSpam\r" "c~/Maildir/.MissedHam\r" "c~/Maildir/.MissedHam\r" "c~/Maildir/.Job\r" "c~/Maildir/.Job\r"
# Setting these colors makes it easy to tell which emails # Spambayes has missed. # Turn spam red and unsure green color index red default "~h '^X-Spambayes-Classification: spam'" color index green default "~h '^X-Spambayes-Classification: unsure'" # Turn gmail mail brightblue # (My regular expression is not correct here. I think that it # is looking anywhere in the header for my gmail address # instead of what I wanted.. Just in the 'To:'. # It still works for the most part) color index brightblue default "~h '(^)*[email protected]' # Lets set more colors to make things look beautiful. Judging by # my title for this section I must have had bad feelings towards # colors when I added this. #Color crap color index brightwhite default ~N # color for new messages color status black yellow color attachment brightyellow default # file attachments color search brightred default # search matches color quoted brightyellow default # quoted text in replies color quoted1 magenta default # quoted text in replies color body cyan default "((ftp|http|https)://|news:)[^ >)\"\t]+" # URLs color body cyan default "[-a-z_0-9.+]+@[-a-z_0-9.]+" # email # The rest of this stuff is pretty important for functionality and
#
readability.
# Don't ask to move read message set move=no # My signature is in this file set signature="~/.signature" # Change mail to look like from [email protected] # Stick your email address here. my_hdr From: Scott Rippee # Header control h displays header when in pager # I want to only see the unignored by default ignore * # weed out all headers unignored date from: to cc subject organization # now show me these... # Thanks to many people whom have written the documents and posts that I # have gotten these settings from.
New Mutt Commands Here are the most useful (as far as dealing with spam) commands that we can now use in mutt: t S H alt-1 alt-2 alt-3 alt-8 alt-9
Tag messages to be manipulated Send message/s to spam training folder Send message/s to ham training folder and to inbox Change mailbox to Maildir/ (inbox) Change mailbox to .CaughtSpam Change mailbox to .Unsure Change mailbox to .MissedSpam Change mailbox to .MissedHam
So for example, if a couple of spam messages show up in your inbox you can now tag them and hit 'S' (remember its uppercase s) and have them vanished into .MissedSpam for training.
SquirrelMail Configuration To configure SquirrelMail go into your html_base_directory/squirrelmail/config and run ./conf.pl. Edit the options to fit your needs. If you want to test your config cruise over to http://localhost/squirrelmail/src/configtest.php.
Now go to http://localhost/squirrelmail/ and login using your user account. If you have any email in your in box you should see it after you log in. Now click on the Folders option that is along the top, select all of the mailbox folders that we created earlier (they are listed here because we prefixed them with a period), and click on subscribe. Hit refresh and the mailboxes should be listed in the side panel. Now you can use SquirrelMail for almost all of you mail needs including tagging messages and moving them to the correct folders for Spambayes training and checking on the mail that Spambayes has classified as spam and unsure. The only functionality from our mutt setup that can't be duplicated in SquirrelMail is copying missed ham into its training folder and then moving it into your inbox. This is due to the lack of copying ability. So you will have to go into the ham training folder to see some of the mail that should be in your inbox. !!! IMPORTANT SECURITY NOTE!!! - Using SquirrelMail like this is not secure! Sending the plain text passwords that you use for your Unix account can be accessed by by a 3rd party, which would them access to your account. You should set up a secure server with SSL and have Courier-imap use passwords that differ from the users system password using Courier Authlib, which I will demonstrate in the next session. Excellent documents exist all over the net for setting up your web server with ssl.
Courier Authentication Now is a good time to address a bit of essential security. As of now when a user connects via the SquirrelMail or IMAP interface the passwords are being sent in plain text and if these are sniffed that users system account is compromised. Without adding secure connections we can minimize the threat by using alternate
passwords for imap connections. The snoop would then have access to your mail, but not be able to log in to the system. Edit your authdaemonrc file. If your on Arch Linux this will be in /etc/authlib. Under authmodulelist I commented out: authmodulelist="authpam authpwd authuserdb authshadow ..." And replaced it with: authmodulelist="authuserdb" I believe that adding authuserdb to the front of the list may work, but I am only using this method of authentication so it is all I need in my list. Lets create the user account entry:
$ userdb "some_user_name" set home=/home/user_dir \ mail=/home/user_dir/Maildir \ uid=users_userid gid=users_groupid $ userdbpw | userdb "some_user_name" set systempw
You will now be prompted for this pseudo users new password. Next make sure that /etc/authlib/userdb has read/write/execute access for the owner. If not chmod 700 /usr/authlib/userdb. Now create the user database that courier is going to look to for authentication: $ makeuserdb Again look to using Postfix/TLS for a tunneling the connection through a secure connection.
Pop-before-smtp Configuration To successfully configure pop-before-smtp follow the straight forward instructions in contrib/README.QUICKINSTALL. When you are done and everything is working it is important to make sure that you have not some how made your mail
server an open relay! This would cause your mail server to be flooded with traffic and be put on the open relay ban lists. !!! IMPORTANT SECURITY NOTE!!! - I will not use the pop-before-smtp method because it involves sending a password over an insecure connection. Use Postfix through a SSL connection, Postfix/TLS, to accomplish this. Since your local machines are likely already allowed to use your Postfix as a mail relay the only time that you would be using this method is from a non-trusted server where you password can be sniffed. If you still choose to use this method make sure that you have changed the users courier password as described in Courier Authentication.
Additional Simple Procmail Recipes and Mailbox Config If you are a member of any news letters/mailing lists you can have them automatically sorted to their own mailbox using Procmail. First create the new mail directory in your ~/Maildir directory. Lets call it .Sbayes and pretend that we are on the Spambayes mailing list. So exactly as before with the spam related mailboxes: mkdir -p ~/Maildir/.Sbayes/{cur,new,tmp} Now add this new rule to your ~/.procmailrc: # [email protected] mailing list :0 * ^X-BeenThere: [email protected] ${MAILDIR}/.Sbayes/ In the .mutrc example above it shows an example of setting alt-5 as the macro to see your Spambayes emails.
A slightly different method can be used to send spam and ham for training. Adding the following lines will allow you to bounce spam messages to spam@localhost (localhost or your mail_domain.com) and ham messages to ham@localhost: # Mail has been bounced to spam@localhost :0 * ^Resent-To:.*spam ${MAILDIR}/.MissedSpam # Mail has been bounced to ham@localhost
:0 * ^Resent-To:.*ham ${MAILDIR}/.MissedHam
If you want to bounce messages to the spam and ham addresses like this, you will need to make sure that the headers stay intact. Do not try using this or a similar technique for forwarding mails to an address because the email's headers would not be intact and this is a large part of the spam classification. This method works because above we specified spam and ham as aliases for hypexr in the Postfix aliases file.
Conclusion For Arch Linux users, Add saslauthd authdaemond courier-imap postfix fetchmail to DAEMONS in your /etc/rc.conf file and they will be started automatically when your machine boots. If you are running a web server have the httpd service listed after the services we are adding. /etc/rc.conf DAEMONS=(... saslauthd authdaemond postfix courier-imap fetchmail ...) Enjoy your new mail system and centralized mailbox. If you are handling a large number of users for you email system this is just the tip of the iceberg and you are going to mostly be interested in virtual users, security, and a system wide spam classifier. There are many good documents on various setups to accomplish this, see the links section. If you have any suggestions, questions, comments, or corrections send me an email or drop a note in the forum. If this document has been of any use to you, cheers. :)
Resources
Example Files o o o o
.fetchmailrc .procmailrc .muttrc train_spambayes
Links o o o o o o o o o
Arch Linux Postfix Procmail Fetchmail SpamBayes Mutt SquirrelMail Courier-imap Courier-authlib
Virtual users and mailboxes using MySQL and setting up SSL for SquirrelMail o o
Arch Wiki - Postfix Howto Gentoo - Virtual Mailhosting System with Postfix Guide
Other Useful links o o o o o o o
Arch Linux Postfix HowTo Jkx@Home's Courier Authentication Postfix with TLS and SASL - geared towards FreeBSD Setting up Postfix + MySQL + Courier-IMAP + Mailman + vmail + Procmail + SpamAssassin - geared towards Fedora/Red Hat Postfix Howtos and FAQs The Mutt E-Mail Client Manual Timo's procmail tips and recipes
Contributions Thanks to Constantinos Laitsas for reworking the fetchmail rc file so that it will start as a daemon correctly Thanks to Robert Schweikert for corrections to the train_spambayes script and the path to sb_filter.py Thanks to Alex Satrapa for tips on example domain naming conventions Thanks to Austin Duncan for tips on postfix configuration and mutts workings on other distros
