The project you're currently working on is configured with docker-compose. Everything is working perfectly. Then suddenly you need to do an urgent fix in the project you finished six months ago. The old project was perfectly configured with docker-compose as well and was working smoothly, but now isn't. What happened? Well, both projects have services that are using the same ports. That story sounds familiar to you? If you're working on multiple projects is very common. You need to configure services to use different ports if you want to keep all of them running, or if not stop one before start the other. I find that kind of annoying. Searching for a way to solve this issue I found Traefik.
Traefik is modern reverse proxy and load balancer that configures itself automatically and dynamically. It has a lot of nice features like:
For the sake of this article we're going to focus in Traefik's capability to uses service discovery to configure itself dynamically from the services themselves. That is the one that allow us to have multiple services running even if they listen in the same port. When configuring the services behind Traefik we don't need to specify in which port they're going to be listening, Traefik will configure them dynamically!
However the images that uses the services need to have exposed the ports, to Traefik be able to effectively communicate with them. Check the Troubleshooting section for more details.
We can follow the basic example of Traefik with Docker to have it running in our dev env.
version: "3.3"
services:
traefik:
image: "traefik"
container_name: "traefik"
command:
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
ports:
- "80:80"
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
- traefik
whoami:
image: "containous/whoami"
container_name: "simple-service"
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.docker.network=traefik"
networks:
- traefik
networks:
traefik:
external: true
In the basic example we can observe that the service whoami uses the labels to configure how it'll communicate with
Traefik:
whoami:
...
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.docker.network=traefik"
...
After executing docker-compose up -d you will be able to access whoami.localhost in your browser.
In the custom basic example I created, I added the external network traefik,
to allow services defined in other docker-compose files to communicate effectively with Traefik. It's possible define
all the services that we use in a single docker-compose file, but I personally prefer to define the services needed for
each of my projects in a dedicated docker-compose file.
I used
traefikas the network's name, but it can have any name.
This is an example of a docker-compose file I'm using in my project Habits-Inventivery
version: "3.8"
services:
php:
image: webcu/laravel-installer
restart: always
working_dir: /laravel
volumes:
- .:/laravel
- ./docker/xdebug.ini:/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
labels:
- "traefik.enable=true"
- "traefik.http.routers.habits.rule=Host(`habits.localhost`)"
- "traefik.http.routers.habits.entrypoints=web"
- "traefik.docker.network=traefik"
networks:
- traefik
- mysql
entrypoint: ["php", "-S", "0.0.0.0:80", "-t", "public", "server.php"]
networks:
traefik:
external: true
mysql:
external: true
After having Traefik up and running start a new project from the point of view of dev env is really quick. We create a docker-compose file inside of the project, define the services we need, linked the ones that need to be linked to Traefik using the external network, and we can start to focus in delivering the product.
Traefik can't connect to others services created in a different docker-compose.yml file.
Are those services using the same external network that Traefik?
Traefik to be able to communicate with other services created using another docker-composer.yml file needs to share the same network with them.
Does the container have ports opened?
If you're using a standard image this problem is not likely to happens to you, but if you're experimenting in your own, like I was doing it can happen 🙂.
I double-checked that the services shared the same external network, still I wasn't able to see the other containers from the Traefik's dashboard. The problem was that the image from which was created the container needs to export a port to other services like Traefik be able to communicate with them.
How do I realized what was the problem? I had the image containous/whoami that worked correctly. First I duplicated it in the same docker-compose.yml file that was the Traefik image. Everything worked OK. I moved this new service from that docker-compose.yml file to the other one that had the other services, everything worked perfectly again, this helped me to realize that the problem wasn't related to the shared network between the services. I changed the image of the new service to the one of Node and Traefik stopped seeing it. I changed it to the one of PHP and I got the same result. Then I decided to go to check the containous/whoami source code to see what it had different to the other images. I saw that it had exposed the port 80 that is the one that I had configured in Traefik like entrypoint for the web address (this is configurable).
traefik:
image: "traefik"
container_name: "traefik"
command:
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
I changed the Dockerfile of my custom PHP image to expose the port 80, and still wasn't working. What was the problem? I forgot rebuild the image when recreating the container then I was still using the image that didn't have any open port. I rebuilt the image, recreated the container, and I saw the services in the Traefik's dashboard.
The fastest way to solve this issue is to check the running containers and see the difference between them. I
used docker-compose ps and I saw that the whoami container had exposed the port 80, but the others
services I wanted to connect with Traefik, no.
You can't access local sites ending in .localhots from Firefox
This problem was driving me crazy. I was thinking that was something wrong with the web servers, but after checking in Chrome I saw that I can access them without problems. Looking at the requests in Mozilla I deduced that the problem was caused by: Referrer Policy
Going deeper the rabbit hole I found that Mozilla doesn't loopback the *.localhost domains to 127.0.0.1. Here is the track of a bug reported:
1220810 - (let-localhost-be-localhost) Consider hardcoding localhost names to the loopback address
The easiest solution is to add manually to /etc/hosts entries for the url that we want to handle locally but this process is very repetitive and tedious.
Could we do better?
We could create a script that giving one url map it to localhost. This sound kind of better but still requires some manual intervention.
Proxy auto-config (PAC)
This is a much more generic solution and doesn't involve any manual intervention after the initial setup. I got the idea from this issue: DNS with traefik not working? · Issue #3 · jtreminio/dashtainer
Looking at typicode/hotel. I copied the function change the conf values for hardcode values and create the file /etc/proxy.pac.
Then I opened the proxy config of mozilla, and I added the file to it:
Et voilà , problem solved!
Traefik allows us to create dynamic dev environments, thanks to its ability to configure itself automatically and dynamically. It is a really powerful tool, with an incredible set of features that expand much more than our local dev env.