Category Archives: Docker

Discourse on Debian Wheezy via Docker

Introduction

The makers of the StackExchange network have been working on Discourse for quite a while now. It is a modern communication platform, with the goal to be a nice mixture of classical internet forums, mailing lists, and established social media features:

Discourse is a from-scratch reboot, an attempt to reimagine what a modern, sustainable, fully open-source Internet discussion platform should be today – both from a technology standpoint and a sociology standpoint.

Technically, Discourse is based on various single components with Ruby on Rails at its core, thin as a leightweight web server, Redis for caching and job control, Sidekiq for micro job management, and PostgreSQL as a persistent database backend. Likewise, deploying Discourse right from the git repository can be a time-consuming task, not to mention the implications of maintaining a Discourse instance or running multiple Discourse instances on the same host. Discourse realized that the complicated deployment certainly prevents a couple of people from using it. Fortunately, the awesome “leightweight VM” container system Docker became production-ready in June 2014:

This release’s “1.0” label signifies a level of quality, feature completeness, backward compatibility and API stability to meet enterprise IT standards. In addition, to provide a full solution for using Docker in production we’re also delivering complete documentation, training programs, professional services, and enterprise support.

… and Discourse consequently decided to work on a Docker-based deployment system, which is the default by now, and even the only supported method (as you can see, Discourse evolves quite quickly). However, Docker is officially supported only on very modern Linux distributions, because Docker’s libcontainer as well as AuFS require kernel features that were only recently introduced. Consequently, Docker (and with that Discourse) is not supported on e.g. Debian 7 (Wheezy), which is the current “stable” Debian as sysadmins love it. But there is something we can do about it.

Although not officially supported or recommended, Docker and Discourse can be run on Debian 7 by backporting a more modern Linux kernel. Currently, Wheezy comes with a kernel from the 3.2 branch. The wheezy-backports (or Debian 8) kernel currently is from the 3.14 branch, which is new enough. Is using a kernel from backports safe? I can only tell that my system runs without quirks, and as you will find on the web, a couple of other people are also running successfully with a kernel from backports.

Hence, if you want to deploy Discourse on Wheezy, you can do it, and it will work perfectly fine. I will quickly go through the required steps in the next sections.

Using a kernel from backports

Follow the official instructions for using the backports repository:

# echo "deb http://ftp.de.debian.org/debian wheezy-backports main" > \
    /etc/apt/sources.list.d/wheezy-backports.list
# apt-get update

Get the newer kernel:

# apt-get -t wheezy-backports install linux-image-amd64 linux-headers-amd64

Reboot:

# reboot

Verify:

$ uname -a
Linux gurke2 3.14-0.bpo.1-amd64 #1 SMP Debian 3.14.12-1~bpo70+1 (2014-07-13) x86_64 GNU/Linux

Install Docker

Following the official installation instructions for Ubuntu (which Debian is very close to):

# apt-get install apt-transport-https
# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
# sh -c "echo deb https://get.docker.io/ubuntu docker main \
   > /etc/apt/sources.list.d/docker.list"
# apt-get update
# apt-get install lxc-docker

Install Discourse ecosystem

Following the instructions from here and here:

# install -g docker -m 2775 -d /var/docker
# adduser discourseuser
# usermod -a -G docker discourseuser

Now, as user discourseuser:

$ git clone https://github.com/discourse/discourse_docker.git /var/docker

Configure Discourse app

As user discourseuser:

$ cd /var/docker
$ cp samples/standalone.yml containers/app.yml

Edit app.yml to your needs. In my case, I made the Discourse container’s web server and SSH server not listen on an external interface, by changing the expose section to:

expose:
  - "127.0.0.1:8200:80"
  - "127.0.0.1:2222:22"

That means that web and SSH servers of the container are reachable through localhost, but not from outside. This has important security advantages:

  • We can proxy the HTTP traffic of the Discourse web application through a web server running on the host (e.g. nginx), and encrypt the connection (the Discourse Docker container does not have TLS support built-in, duh).
  • Why would you want to expose the Discourse container via SSH to the internet, anyway? That’s a severe mistake, in my opinion, and you should disable that unless you have a very good reason not to.

I have set DISCOURSE_SMTP_ADDRESS: 172.17.42.1, as 172.17.42.1 is the IP address of the host in the private virtual Docker network in my case. On my Debian host, I am running an Exim4 MTA, which is configured with dc_local_interfaces='172.17.42.1' and dc_relay_nets='172.17.0.0/16', i.e. it listens on the local Docker network and relays mails incoming from that network. That way, the Discourse instance running within a Docker container can send mail through the Exim MTA running on the Debian host. I have described that in more detail in another blog post.

Each change of the configuration file app.yml requires a re-build via ./launcher rebuild app. A re-build implicates stopping, destructing, bootstrapping, and launching the container. What I have learned just recently: don’t worry too much about the word “destruction”. It does not implicate data loss. You can re-build your Discourse container at any time. Persistent data is stored in the PostgreSQL database and in the shared volume, both of which are not affected by a re-build.

Hope that helps!

Discourse Docker container: send mail through Exim

Introduction

The Discourse deployment was greatly simplified by introducing Docker support (as I have written about before). Discourse heavily depends on e-mail, and its ability to send mail to arbitrary recipients is essential. While the recommended way is to use an external service like Mandrill, it is also possible to use a local MTA, such as Exim. However, when you set up the vanilla Discourse Docker container, it does not contain an pre-configured MTA, which is fine, since many have a well-configured MTA running on the host already. The question is how to use that MTA for letting Discourse send mail.

Usually, MTAs on smaller machines are configured to listen on localhost only, to not be exposed to the Internet and to not be mis-used for spam. localhost on the host itself, however, is different from localhost within a Docker container. The network within the container is a virtual one, and it is cleanly separated from the host. That is, when Discourse running in a container tries to reach an SMTP server on localhost, it cannot reach an MTA listening on localhost outside of the container. There is a straight-forward solution: Docker comes along with a network bridge. In fact, it provides a private network (in the 172.17.x.x range) that connects single containers with the host. This network can be used for establishing connectivity between a network application within a Docker container and the host.

Exim’s network configuration

Likewise, I have set up Exim4 on the Debian host for relaying mails that are incoming from localhost or from the local virtual Docker network. First I looked up the IP address of the docker bridge on the host, being 172.17.42.1 in my case (got that from /sbin/ifconfig). I then instructed Exim to treat this as local interface and listen on it. Also, Exim was explicitly told to relay mail incoming from the subnet 172.17.0.0/16, otherwise it would reject incoming mails from that network. These are the relevant keys in /etc/exim4/update-exim4.conf.conf:

dc_local_interfaces='127.0.0.1;172.17.42.1'
dc_relay_nets='172.17.0.0/16'

The config update is in place after calling update-exim4.conf and restarting Exim via service exim4 restart.

Testing SMTP access from within container

I tested if Exim’s SMTP server can be reached from within the container. I used the bare-bones SMTP implementation of Python’s smtplib for that. First of all, I SSHd into the container by calling launcher ssh app. I then called python. The following Python session demonstrates how I attempted to establish an SMTP connection right to the host via its IP address in Docker’s private network:

>>> import smtplib
>>> server = smtplib.SMTP('172.17.42.1')
>>> server.set_debuglevel(1)
>>> server.sendmail("rofl@gehrcke.de", "jgehrcke@googlemail.com", "test")
send: 'ehlo [172.17.0.54]\r\n'
reply: '250-localhost Hello [172.17.0.54] [172.17.0.54]\r\n'
reply: '250-SIZE 52428800\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-PIPELINING\r\n'
reply: '250 HELP\r\n'
reply: retcode (250); Msg: localhost Hello [172.17.0.54] [172.17.0.54]
SIZE 52428800
8BITMIME
PIPELINING
HELP
send: 'mail FROM:<rofl@gehrcke.de> size=4\r\n'
reply: '250 OK\r\n'
reply: retcode (250); Msg: OK
send: 'rcpt TO:<jgehrcke@googlemail.com>\r\n'
reply: '250 Accepted\r\n'
reply: retcode (250); Msg: Accepted
send: 'data\r\n'
reply: '354 Enter message, ending with "." on a line by itself\r\n'
reply: retcode (354); Msg: Enter message, ending with "." on a line by itself
data: (354, 'Enter message, ending with "." on a line by itself')
send: 'test\r\n.\r\n'
reply: '250 OK id=1X9bpF-0000st-Od\r\n'
reply: retcode (250); Msg: OK id=1X9bpF-0000st-Od
data: (250, 'OK id=1X9bpF-0000st-Od')
{}

Indeed, the mail arrived at my Google Mail account. This test shows that the Exim4 server running on the host is reachable via SMTP from within the Discourse Docker instance. Until I got the configuration right, I observed essentially two different classes of errors:

  • socket.error: [Errno 111] Connection refused in case there is no proper network routing or connectivity established.
  • smtplib.SMTPRecipientsRefused: {'jgehrcke@googlemail.com': (550, 'relay not permitted')} in case the Exim4 SMTP server is reachable, but rejecting your mail (for this to solve I had to add the dc_relay_nets='172.17.0.0/16' to the config shown above).

Obviously, in order to make Discourse use that SMTP server, it needs to be configured with DISCOURSE_SMTP_ADDRESS being set to the IP address of the host in the Docker network, i.e. 172.17.42.1 in my case.

Hope that helps!