Building an Ubuntu Updates Mirror with Docker and Docker Compose

Looking for a new project for the home network? Or maybe you’ve got a few Ubuntu servers and you don’t want to download the same apt packages for each server. You can use Docker, Docker Compose and open source tools to make your own functioning apt repository mirror. Let’s find out how.

Building an apt-mirror Docker image

When I set up new services on my Linux servers, I prefer to configure them to run in Docker containers. This minimizes the number of packages I have to install on the server and encapsulates all of the configuration I need to run the service.

Let’s start by setting up a directory for the project:

1
2

$ mkdir aptmirror-docker
$ cd aptmirror-docker

Next, create a script called entrypoint.sh. This will run apt-mirror inside of the Docker container to keep it up to date. Adjust the sleep interval as desired, but I wouldn’t go lower than 4 hours.

1
2
3
4
5
6
7
8

#!/bin/bash

while true;
do
  apt-mirror
  echo "Start sleep 6h..."
  sleep 6h
done

We’re going to use apt-mirror to do the heavy lifting of cloning the official repos for us. We just need to make a container for it to run in. We’ll use Alpine Linux as a base image to reduce image size. This means we won’t be able to use apt-get to install apt-mirror, but we can clone the Github repo instead.

Here’s what my Dockerfile looks like:

1
2
3
4
5
6
7
8

FROM alpine:3.10
RUN apk --no-cache add perl wget git gzip && \
    git clone https://github.com/apt-mirror/apt-mirror.git /tmp && \
    cp /tmp/apt-mirror /usr/bin
WORKDIR /aptmirror
COPY . .
VOLUME ["/aptmirror"]
CMD ["/bin/sh", "entrypoint.sh"]

This Dockerfile will use apk, the Alpine package manager, to download apt-mirror’s dependencies and git. Next it clones the apt-mirror repository into the /tmp folder and copies the apt-mirror script to /usr/bin for later use. We set a working directory and copy the contents of the project folder . into the working directory (also .). Finally we define a Docker volume to persist the apt-mirror repository and then call our entrypoint script.

Now let’s build the image and see what happens.

1
2
3
4
5

$ docker build -t aptmirror .
$ docker run --rm -it aptmirror

apt-mirror: can't open config file (/etc/apt/mirror.list) at /usr/bin/apt-mirror line 318.
Start sleep 6h...

Well, there is no /etc/apt directory in the container and there certainly isn’t a mirror.list file in it. The apt-mirror repo contains a sample mirror.list so we can start from there.

Make your own copy of the sample mirror.list in the project directory and edit it to suit your preferences. The syntax for specifying repos is the same as what you’d find in sources.list on an Ubuntu or Debian system, so if you get stuck you can reference this. It may also be useful for you to reference what you already have defined in /etc/apt/sources.list on your Ubuntu servers.

Here’s what mine looks like for Ubuntu 20.04:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

set base_path    /aptmirror
set cleanscript $var_path/clean.sh
set defaultarch  amd64
set nthreads     20
set _tilde 0

deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse

clean http://archive.ubuntu.com/ubuntu

If you support more than one release of Ubuntu feel free to add more deb lines but be aware that the more repositories you add to your mirror the more disk space it’s going to take up. At the time of this writing, the mirror that I set up using this procedure uses about 170GB of space.

Setting up the Compose Stack

Instead of running the docker run command ourselves every time, let’s use Docker Compose.

First, we need to decide how we’re going to store our mirror’s files.

Volumes or Bind Mounts

If we allowed our apt-mirror Docker container to run without a volume, it would spend a long time downloading the mirror files and then the second you destroyed the container all of those files would be lost.

Contrary to intuition, you want to be able to destroy that container. You may want to update it in the future, and you wouldn’t want to have to re-download everything to do it.

To solve this problem, we can use either a Docker volume or a bind mount.

Briefly, volumes are a Docker-managed storage solution that create volumes that can be shared between Docker containers and managed independently from the filesystem. They’re usually what you want to try first since they don’t rely on any specific configuration on the host. This way, all you need to do to set this up again on a new server in the future is clone the project directory, cd into it and run docker-compose up to rebuild it and start downloading a fresh repository into a new volume (or a backup of the original one).

Bind mounts are more of a mapping between directories on the host and ones in the container. You’d use this if you prefer your mirror repository files to be in a directory on the host. This is probably what you want if you use ZFS or different physical volumes and want to be able to say where the mirror data goes.

For a volumes-based setup:

docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

version: '3'

services:
  aptmirror:
    build: .
    image: aptmirror
    restart: unless-stopped
    volumes:
      - aptmirror:/aptmirror
      - ./mirror.list:/etc/apt/mirror.list

volumes:
  aptmirror:

Note the volumes: section at the bottom. This defines the volume name and makes it available to any container in the stack. Under the aptmirror service definition, we see:

1
2

  - aptmirror:/aptmirror
  - ./mirror.list:/etc/apt/mirror.list

The first line maps the volume aptmirror to the directory inside the container /aptmirror (<source>:<destination>).

The second line maps the local project file ./mirror.list to the file inside the container /etc/apt/mirror.list. This makes it so changes we make to the configuration file we created earlier reflected in the container without having to rebuild it every time.

Now let’s look at an example with bind mounts:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

version: '3'

services:
  aptmirror:
    build: .
    image: aptmirror
    restart: unless-stopped
    volumes:
      - /mnt/aptmirror:/aptmirror
      - ./mirror.list:/etc/apt/mirror.list

Similar syntax, but this maps the directory /mnt/aptmirror to /aptmirror inside the container instead. The line for mirror.list remains the same.

The remaining examples will assume you’re using volumes and not bind mounts.

Now let’s start the stack and begin building our mirror by running docker-compose up.

You should now see your aptmirror container constructing your mirror for you. Let the container run to completion (mine took about 2 hours) while we move on to the next step.

Serving the mirror

Our mirror isn’t much good to us in its current state. We need to serve it over HTTP so our servers can get their updates from it. I’ll use nginx for this.

Here’s my nginx.conf file:

1
2
3
4
5
6
7
8
9

server {
    listen       80;
    server_name  localhost;

    location / {
        root   /usr/share/nginx/html/mirror/archive.ubuntu.com;
        autoindex on;
    }
}

We can add a new web service to the Docker Compose stack:

1
2
3
4
5
6
7
8

web:
    image: nginx:1.17-alpine
    restart: unless-stopped
    volumes:
      - aptmirror:/usr/share/nginx/html:ro
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 80:80

Note the use of volumes above: we want to map the aptmirror volume to the nginx default html directory in the container. We’re also mapping it in ro (read-only) mode as a security precaution. And of course we want the nginx.conf file we’ve created to be bound somewhere where nginx can see it.

Your final docker-compose.yml should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

version: '3'

services:
  aptmirror:
    build: .
    image: aptmirror
    restart: unless-stopped
    volumes:
      - aptmirror:/aptmirror
      - ./mirror.list:/etc/apt/mirror.list
  
  web:
    image: nginx:1.17-alpine
    restart: unless-stopped
    volumes:
      - aptmirror:/usr/share/nginx/html:ro
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 80:80

volumes:
  aptmirror:

Let’s start the new service:

1

$ docker-compose up -d

If you punch your docker host’s IP address into a browser you should you should see a directory index with a single folder in it like this:

Congratulations! You’re now the proud owner of an Ubuntu repository mirror.

Configuring clients

Our update mirror isn’t much use to us unless we configure our Ubuntu machines to use it. To do this, we need to edit /etc/apt/sources.list on the machine, add our mirror and comment out the official ones. You’ll probably need to update this if you ever do a dist-upgrade since sources.list will likely be overwritten during that process.

I added the following two lines to my sources.list:

1
2

deb http://<your_ip_here>/ubuntu focal main restricted universe multiverse
deb http://<your_ip_here>/ubuntu focal-updates main restricted universe multiverse

I also went through and commented out all of the official deb lines. Make sure the release (focal in this case) matches what you see in your sources.list!

Now we can run an index update on the machine we’re setting up and see if it worked:

1

$ apt-get update

You should see successful updates from your new mirror. If you don’t, make sure you’ve got the correct IP address. Also make sure there isn’t a firewall running on your Docker host.

I would definitely test this out with a VM before you mess with any production servers, but this is working pretty okay for me and I enjoyed building it.

You can find the code and configuration for this setup here.