Installing and configuring a relay server

Introduction
Basic postfix installation
Basic configuration
- Configuring a relay server
  - Analysis of errors and active configurations
  - Attempt to send e-mail through the gateway and analyze errors
  - Analysis of the Postfix queue and the relay problem
  - Visualization of a complete communication

Introduction

Following the theory seen previously we will move to the practice of setting up an MTA service with the postfix application. Honestly after using several system such as sendmail, qmail or exim my choice is clearly for postfix. In fact the only one in the list mentioned above that I like is exim, but being very comfortable with postfix I never took the time to learn it in depth.

For the others :

sendmail: first system (to my knowledge) free allowing the transfer of email, unfortunately the writing of the configuration is painful
qmail: A hell to install, the main developer has firm positions on the operation and have used patches to add functionality … I put it a few times in place, with more maturity now, I refuse to work with P.

Here is what we will cover:

Basic installation of postfix as email relay server, in other words the server will not deliver messages in local boxes but will gateway to another server. This will allow us to see the simplest configuration !!
Configuring the postfix service to deliver e-mail locally to the system user for a named domain
We will set up the dovecot service to allow e-mail retrieval via the pop3 and imap service from a remote machine.

Basic postfix installation

Of course we will use the service of pasting of the distribution to realize the installation, to be sure to have the regular updates of the service, especially that it is exposed on the Internet.

There is a large number of packages available with Ubuntu 16.04:

$ apt-cache search postfix | grep ^postfix postfix - High-performance mail transport agent postfix-cdb - CDB map support for Postfix postfix-dev - Loadable modules development environment for Postfix postfix-doc - Documentation for Postfix postfix-ldap - LDAP map support for Postfix postfix-mysql - MySQL map support for Postfix postfix-pcre - PCRE map support for Postfix postfix-pgsql - PostgreSQL map support for Postfix postfix-cluebringer - anti-spam plugin for Postfix postfix-cluebringer-mysql - metapackage for mysql support in postfix-cluebringer postfix-cluebringer-pgsql - metapackage for postgresql support in postfix-cluebringer postfix-cluebringer-sqlite3 - metapackage for sqlite3 support in postfix-cluebringer postfix-cluebringer-webui - anti-spam plugin for Postfix postfix-gld - greylisting daemon for postfix, written in C, uses MySQL postfix-policyd-spf-perl - Simple Postfix policy server for RFC 4408/7208 SPF checking postfix-policyd-spf-python - Postfix policy server for SPF checking postfixadmin - Virtual mail hosting interface for Postfix

Before getting excited with all the possibilities we will start with the basic one: D.

$ sudo apt-get install postfix

When installing the pasting we have the following question:

Postfix Configuration --------------------- Please select the mail server configuration type that best meets your needs. No configuration: Should be chosen to leave the current configuration unchanged. Internet site: Mail is sent and received directly using SMTP. Internet with smarthost: Mail is received directly using SMTP or by running a utility such as fetchmail. Outgoing mail is sent using a smarthost. Satellite system: All mail is sent to another machine, called a 'smarthost', for delivery. Local only: The only delivered mail is the mail for local users. There is no network. 1. No configuration 2. Internet Site 3. Internet with smarthost 4. Satellite system 5. Local only General type of mail configuration: 1

I opted not to have a configuration, because we are in the process of climbing in proficiency as well take the opportunity to shit a little more: P. On the other hand, if I had to install in “Satellite system” mode in my network, I will select this option because it is valid and will save me time.

Good the installation is completed as usual the system of pumping works perfectly.

Basic configuration

The configuration of postfix is of course in the directory / etc, the configuration of the service is mainly in 2 files:

master.cf: This configuration file defines the configuration of postfix component processes, we will see when analyzing the service architecture that postfix is composed of a multitude of small program. It is rare that we have to modify this file, but it happens.
main.cf: The main configuration file of the postfix service, the latter will define the domain names of which we accept emails, the list of machines that can use the service, the definition of the configuration of the database (if there is storage in the latter, …). We will work mainly this file!

Let’s look at the main.cf file:

$ cat /etc/postfix/main.cf cat: /etc/postfix/main.cf: No such file or directory

Of course we said no configuration: D, good little criticism to look at the boot file if you try to start it here is the result

$ sudo /etc/init.d/postfix start $ echo $? 0

No error saying that there is no configuration file and the error code is OK either 0: – /. If we look at the init script file this is from line 31:

31 test -x $DAEMON && test -f /etc/postfix/main.cf || exit 0

Configuring a relay server

I think we do not have the choice to put ourselves in, so write the configuration for sending email only, this configuration will not be able to receive emails. This configuration is widely used in the industry because you probably do not want all servers to send emails directly because:

If your servers are in a small network, communications to the outside world is limited
Your posts do not need to transmit emails over the internet, if this is a required then you want to be able to identify a machine that is infected with malware. By using a relay server you have a place to validate and not all the workstations.
If you realize massive sendings for customers, legitimate, your relay server can realize a management of the sending to not clog the whole.
By using a relay server this will allow you to have a visibility of the sendings (statistic, limitation, posting, …)

There is probably a lot of other valid reason for you: D.

Let’s make a simple configuration file! Here is a start that we will be able to feed / enrich later:

1 smtpd_banner = $myhostname ESMTP $mail_name 2 biff = no 3 4 5 smtpd_relay_restrictions = permit_mynetworks defer_unauth_destination 6 myhostname = $myorigin 7 myorigin = mail.example.com 9 mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 10 inet_interfaces = all 11 inet_protocols = ipv4 12 13 relayhost = relay_mail.fai.com

Well it’s part, I invite you to consult the documentation of postfix which is really good for the set of parameters, it is in English, but really complete.

smtpd_banner: This indicates the banner that will be passed to the client when establishing connection
smtpdrelayrestrictions: Set the restriction on who can send an email through your server. This setting is ULTRA important, if you leave your postfix server open on the internet the spammers will use it quickly and your IP will be black listed. In the present configuration we allow authorized networks (permit_mynetworks) and / or if the client does not list in authorized IPs if the delivery domain is our server or if the domain is specifically listed to be relayed.
myhostname: The hostname of the server where postfix is running
myorigin: the name that will be displayed in the email header, sometimes it is different from the hostname
mynetworks: definition of IPs and networks that can use the server to send e-mails freely. In the present situation only the server itself can send e-mails to the outside. Not very practical but this will allow me to demonstrate the error of mailings we will change later: D.
inet_interfaces: Definition on which interface the postfix server will listen, for example eth0, eth1, eth2 in the present situation I listen on all the interfaces.
inet_protocols: Protocol used IPv4, IPv6.
relayhost: To which email server we will send ALL emails.

It looks good, we will be able to start the service:

$ /etc/init.d/postfix start

The postfix system uses syslog to write the logs so you have to have the service present to be able to consult the logs … Consult the logs to validate that all works well, under Ubuntu we have 2 file /var/log/mail.err and / var / log / mail.log.

Let’s consult the error logs to get started:

$ tail /var/log/mail.err Jan 12 22:16:29 mail01 postfix/smtpd[1058]: error: open database /etc/aliases.db: No such file or directory $ tail /var/log/mail.log Jan 12 22:16:24 mail01 postfix/master[1054]: daemon started -- version 3.1.0, configuration /etc/postfix Jan 12 22:16:29 mail01 postfix/smtpd[1058]: error: open database /etc/aliases.db: No such file or directory Jan 12 22:16:29 mail01 postfix/smtpd[1058]: warning: dict_nis_init: NIS domain name not set - NIS lookups disabled

As always, I like to leave configuration errors in my examples to demonstrate the analysis of the resolution process: D

Analysis of errors and active configurations

So first reflex, we copy / paste the error in google: D, when we do not know we are looking for information. Although I knew the answer I did the exercise for fun, here is the first link found as of January 12, 2017:

http://serverfault.com/questions/683243/postfix-aliases-db-no-such-file-or-directory

Okay a great solution, but we did not have the entries mentioned:

alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases, nis:mail.aliases

In fact even if our file /etc/postfix/main.cf contains only 13 lines in reality we have 979 properties of defined configuration: D. Yep postfix has full configuration available and FULL default configuration: D. Now how to consult them? Thanks to the postconf command

$ postconf | wc -l 979 $ postconf | head 2bounce_notice_recipient = postmaster access_map_defer_code = 450 access_map_reject_code = 554 address_verify_cache_cleanup_interval = 12h address_verify_default_transport = $default_transport address_verify_local_transport = $local_transport address_verify_map = btree:$data_directory/verify_cache address_verify_negative_cache = yes address_verify_negative_expire_time = 3d address_verify_negative_refresh_time = 3h

That’s all that, let’s look for the alias instruction to see:

$ postconf | grep alias_ alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases, nis:mail.aliases [... OUTPUT COUPÉ ...]

The Internets was therefore right: P, we have this entry defined, the important thing here is really the command postconf: D, for information this file allows to manage email aliases. Well we have 2 choices correct the problem or disabled the feature, to make the choice what is the role of the machine? Allow the transfer of email in gateway mode so no need for aliases: D, do not panic, we will see it during local delivery: D.

So I added the following lines in the /etc/postfix/main.cf configuration file:

alias_database = alias_maps =

We restart and more error: D.

Validate that the service is available:

$ ps aux | grep post root 165 0.0 0.0 65408 4612 ? Ss 13:24 0:00 /usr/lib/postfix/sbin/master $ ss -lntp State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 100 *:25 *:*

It looks good: D, we do a test: D. Ha ouin is still a problem: P, so do not panic yet it’s normal. I believe that we learn more with errors than when all works: D. This also gives you the opportunity to analyze the diagnostic process.

Attempt to send e-mail through the gateway and analyze errors

It’s time for a test, of course we could set up a free client: P, but if you’re remote on servers in a limited network segment it can be difficult. As the SMTP protocol uses text instructions to communicate, we will be able to simulate it.

My server currently at IP 172.17.0.2, we will use another machine on the network to transmit the email with the TELNET application.

$ telnet 172.17.0.2 25 Trying 172.17.0.2... Connected to 172.17.0.2. Escape character is '^]'. 220 mail.example.com ESMTP Postfix

As you can see from the connection we have the banner we defined: 220 mail.example.com ESMTP Postfix

I introduce myself, in other words my machine presents itself:

ehlo hostname_de_ma_workstion.lan.com 250-mail.example.com 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN

The server answers me the protocol and information on its system.

I indicate the email comes from who:

MAIL FROM: super_dude@shibarecords.com 250 2.1.0 Ok

It’s time to tell who the email will be sent to:

RCPT TO: mon_ami_super_geek@shibarecords.com 454 4.7.1 <mon_ami_super_geek@shibarecords.com>: Relay access denied

And BOOM, message: Relay access denied so the server refuses to send my email to relay over the internet. Well, I’m leaving cleanly:

quit 221 2.0.0 Bye Connection closed by foreign host.

If we look at the log /var/log/mail.log, same message:

Jan 13 13:37:59 mail01 postfix/smtpd[338]: connect from unknown[172.17.0.1] Jan 13 13:38:16 mail01 postfix/smtpd[338]: NOQUEUE: reject: RCPT from unknown[172.17.0.1]: 454 4.7.1 <mon_ami_super_geek@shibarecords.com>: Relay access denied; from=<super_dude@shibarecords.com> to=<mon_ami_super_geek@shibarecords.com> proto=ESMTP helo= Jan 13 13:38:47 mail01 postfix/smtpd[338]: disconnect from unknown[172.17.0.1] ehlo=1 mail=1 rcpt=0/1 quit=1 commands=3/4

At the reading of the logs we see that the connection comes from reject: RCPT from unknown [172.17.0.1], look at the configuration of the email service, in the file /etc/postfix/main.cf, I will bring your attention to 2 settings :

smtpd_relay_restrictions = permit_mynetworks defer_unauth_destination mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

smtpdrelayrestrictions: In the present configuration we allow authorized networks (permit_mynetworks) and / or if the client does not list in IP allowed if the delivery domain is our server or if the domain is specifically listed to be relayed.
- Since we do NOT deliver anything locally and do not specify a domain name for the relay, we will allow relay based on the IP address.
mynetworks: Currently we only allow relay if the email comes from the client 127.0.0.1, so the machine itself.

Now that we have seen the error we will add the appropriate network, I add the 172.17.0.0/16 network:

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 172.17.0.0/16

We restart postfix and we redo the test.

$ telnet 172.17.0.2 25 Trying 172.17.0.2... Connected to 172.17.0.2. Escape character is '^]'. 220 mail.example.com ESMTP Postfix ehlo host_workstation.example.com 250-mail.example.com 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN MAIL FROM: toto@gmail.com 250 2.1.0 Ok RCPT TO: demo@shibarecords.com 250 2.1.5 Ok DATA 354 End data with . le corps du message qui sera dans le courriel . 250 2.0.0 Ok: queued as DDC03A029B1 QUIT 221 2.0.0 Bye Connection closed by foreign host.

As you can see the RCPT TO instruction: demo@shibarecords.com has worked well now! So I was able to move to the other step either to transmit the content of the message or the body body of the email with the help of the DATA statement, which ends with a single point on a line. Once finished the postfix server gives me the message number in the DDC03A029B1 processing queue. The quit instruction allows me to close the communication.

Here is the result in the logs:

Jan 13 21:39:20 mail01 postfix/smtpd[167]: connect from unknown[172.17.0.1] Jan 13 21:39:54 mail01 postfix/smtpd[167]: DDC03A029B1: client=unknown[172.17.0.1] Jan 13 21:40:12 mail01 postfix/cleanup[170]: DDC03A029B1: message-id=<> Jan 13 21:40:12 mail01 postfix/qmgr[166]: DDC03A029B1: from=<toto@gmail.com>, size=237, nrcpt=1 (queue active) Jan 13 21:40:12 mail01 postfix/smtp[171]: warning: relayhost configuration problem Jan 13 21:40:12 mail01 postfix/smtp[171]: DDC03A029B1: to=<demo@shibarecords.com>, relay=none, delay=23, delays=23/0.01/0.3/0, dsn=4.3.5, status=deferred (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) Jan 13 21:40:14 mail01 postfix/smtpd[167]: disconnect from unknown[172.17.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5

Yes, yes, I know there is still a problem: I have to show you the problem analysis process: D.

Analysis of the Postfix queue and the relay problem

So if we look at the error:

Jan 13 21:40:12 mail01 postfix/smtp[171]: DDC03A029B1: to=<demo@shibarecords.com>, relay=none, delay=23, delays=23/0.01/0.3/0, dsn=4.3.5, status=deferred (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found)

I think that the error is explicit, the configuration in the relayhost of postfix is not good so it is not able to realize the passage of the witness to the other machine. Since he does not send e-mail on the Internet freely, he can not transmit the e-mail. Of course the solution of the problem is simple to put a suitable server: D, good but take the opportunity to analyze the postfix queue.

During the presentation of the theory of the email service I mentioned that the email system keeps the messages if it is not able to deliver it. Yeah that’s exactly the situation, what luck: P !!!

You can see the current queue of postfix this last contains contains the emails being processed as well as the emails in error, let’s see the state of the latter:

$ mailq -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- DDC03A029B1 237 Fri Jan 13 21:39:49 toto@gmail.com (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) demo@shibarecords.com -- 0 Kbytes in 1 Request.

Here we find the queue ID, the date, the sender, the error message or at least the status of the processing and the destination!

Where are his messages stored on the system ?? !!! In a binary file in a private format, no I laugh: P! The spool, the processing directory, where is the files being processed is in the / var / spool / postfix directory

$ ls /var/spool/postfix active bounce corrupt defer deferred dev etc flush hold incoming lib maildrop pid private public saved trace usr

In our case, the emails are in the defer and deferred directory, we will see in more detail the different tail of postfix in / if we realize a section performance. Well, we’ll get along in 3 months (I’m generous) if you have a problem you will not remember that it’s in the defer directory so just remember the directory / var / spool / postfix and made like me a little find with the message ID ;-).

$ find /var/spool/postfix -name "*DDC03A029B1*" /var/spool/postfix/defer/D/DDC03A029B1 /var/spool/postfix/deferred/D/DDC03A029B1

Magic: D !!

/ var / spool / postfix / defer / D / DDC03A029B1: This file contains email status information for postfix

<demo@shibarecords.com>: Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found recipient=demo@shibarecords.com offset=624 dsn_orig_rcpt=rfc822;demo@shibarecords.com status=4.3.5 action=delayed reason=Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found

/ var / spool / postfix / deferred / D / DDC03A029B1: This file contains the message itself:

C_ 237 642 1 0 237 0T1484343589 154488Acreate_time=1484343594Alog_ident=DDC03A029B1Arewrite_context=remoteStoto@gmail.comAlog_client_name=unknownAlog_client_address=172.17.0.1Alog_client_port=37250A&log_message_origin=unknown[172.17.0.1]A*log_helo_name=host_workstation.example.comAlog_protocol_name=ESMTPAclient_name=unknownAeverse_client_name=unknownAclient_address=172.17.0.1Aclient_port=37250A&helo_name=host_workstation.example.comAprotocol_name=ESMTPAclient_address_type=2A#dsn_orig_rcpt=rfc822;demo@shibarecords.comOdemo@shibarecords.comRdemo@shibarecords.comMNBReceived: from host_workstation.example.com (unknown [172.17.0.1])N8 by mail.example.com (Postfix) with ESMTP id DDC03A029B1N< for <demo@shibarecords.com>; Fri, 13 Jan 2017 21:39:49 +0000 (UTC)NN-le corps du message qui sera dans le courrielXE

It’s file are use as reference point, if we delete the files the queue will be “empty” because no more file:

$ sudo mailq -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- DDC03A029B1 237 Fri Jan 13 21:39:49 toto@gmail.com (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) demo@shibarecords.com -- 0 Kbytes in 1 Request. $ sudo rm /var/spool/postfix/defer/D/DDC03A029B1 /var/spool/postfix/deferred/D/DDC03A029B1 $ sudo mailq Mail queue is empty

Of course, it’s very haxor and it’s super convenient, but unfortunately it’s not very optimal, there is a risk of error and if you want to delete messages in the queue you are obliged to make a script with a criterion , moreover they are not necessarily all in the queue differed.

We can use another more beautiful method, I will put another email in the queue …

$ telnet 172.17.0.2 25 Trying 172.17.0.2... Connected to 172.17.0.2. Escape character is '^]'. 220 mail.example.com ESMTP Postfix ehlo ma_machine 250-mail.example.com 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN MAIL FROM: un_haxor@gmail.com 250 2.1.0 Ok RCPT TO: un_autre_geek@gmail.com 250 2.1.5 Ok data 354 End data with . bon moi aussi je veux supprimer des messages comme un haxor . 250 2.0.0 Ok: queued as 5011DA029B1

So on the server

$ mailq -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- 5011DA029B1 242 Fri Jan 13 22:11:32 un_haxor@gmail.com (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) un_autre_geek@gmail.com -- 0 Kbytes in 1 Request. $ sudo find /var/spool/postfix -name "*5011DA029B1*" /var/spool/postfix/defer/5/5011DA029B1 /var/spool/postfix/deferred/5/5011DA029B1

You have the postsuper documentation utility: postsuper man, this utility allows you to manipulate the messages in the queue. I leave you the pleasure to consult the man page. We will be able to delete emails, pause emails, …

So if we perform the operation to delete the message the result:

$ sudo postsuper -d 5011DA029B1 postsuper: 5011DA029B1: removed postsuper: Deleted: 1 message $ sudo mailq Mail queue is empty

For those who could say, yeah but it’s less Hax0r (Hacker), do not panic, if you look at the man page, it is possible to look like a demi-god very easily: P …

$ mailq | tail -n +2 | grep -v '^ *(' | awk 'BEGIN { RS = "" } # $7=sender, $8=recipient1, $9=recipient2 { if ($8 == "user@example.com" && $9 == "") print $1 } ' | tr -d '*!' | postsuper -d -

The above statement allows you to delete emails whose recipient is only user@example.com, open the man page when the person above your shoulder to the head turned a small copy / pasted and he will be really impressed: P .

Visualization of a complete communication

Good fun with errors and problem analysis but it would be nice if you have a point of reference when it also works: P. So I will changed the relay server to a valid one for the demonstration.

In order to allow the sending of email, I will modify my server to allow my current IP address in the mynetwork configuration of my server.

I am going to change the postfix configuration to use my email server which will accept all emails from my IP address!

But first we will put an email in the queue, to demonstrate how postfix this will be in the fix!

So I reused the telnet command to send it here is the result:

$ sudo mailq -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- 0F04FA029B1 216 Tue Jan 17 13:11:00 bob@shibarecords.com (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) bob@shibarecords.com -- 0 Kbytes in 1 Request.

Now modify the configuration, here is the result of the configuration:

$ cat /etc/postfix/main.cf | grep relay smtpd_relay_restrictions = permit_mynetworks defer_unauth_destination relayhost = mail.

shibarecords

.com

We restart postfix

Following the change the email is still in the queue:

$ mailq -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- 0F04FA029B1 216 Tue Jan 17 13:11:00 bob@shibarecords.com (Host or domain name not found. Name service error for name=relay_mail.fai.com type=A: Host not found) bob@shibarecords.com -- 0 Kbytes in 1 Request.

Postfix will eventually redo the send, a given moment: P, according to his mailing planner it is also possible to force a send.

The postqueue command allows us to force the processing of all emails:

$ sudo postqueue -f

Here is the result in the logs:

Jan 17 13:16:02 mail01 postfix/pickup[313]: 65722A03803: uid=105 from=<bob@shibarecords.com> orig_id=0F04FA029B1 Jan 17 13:16:02 mail01 postfix/cleanup[329]: 65722A03803: message-id=<> Jan 17 13:16:02 mail01 postfix/qmgr[314]: 65722A03803: from=<bob@shibarecords.com>, size=330, nrcpt=1 (queue active) Jan 17 13:16:04 mail01 postfix/smtp[331]: 65722A03803: to=<shibarecords@shibarecords.com>, relay=mail.shibarecords.com[192.99.13.211]:25, delay=303, delays=302/0.01/0.77/0.91, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 01DC75A44D6D) Jan 17 13:16:04 mail01 postfix/qmgr[314]: 65722A03803: removed

We now see that the server relay = mail.shibarecords.com [192.99.13.211] has taken the email and the queué, we have the ID for the follow, but we lose all visibility now: D.

Well it’s less fun, but it’s part of the game: D