this post was submitted on 03 Nov 2024
25 points (100.0% liked)

Selfhosted

40113 readers
799 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I have some of my Docker containers bind to Tailscale IPs on the host. Even though I edited the Docker service to depend on tailscaled.service and it starts about 10 seconds after it, every once in a while Tailscale interface won't be ready by the time the containers try to spin up after a reboot.

Since restart policies do not apply to containers that has never started and ran at least for a while, Docker gives me no choice but to restart either those containers or the Docker daemon manually.

Is there a way to have Docker daemon try to restart containers even though they failed on the first try?

A restart policy only takes effect after a container starts successfully. In this case, starting successfully means that the container is up for at least 10 seconds and Docker has started monitoring it. This prevents a container which doesn't start at all from going into a restart loop.

https://docs.docker.com/engine/containers/start-containers-automatically

all 8 comments
sorted by: hot top controversial new old
[–] danielquinn@lemmy.ca 12 points 1 week ago (3 children)

You probably want to look into Health Checks. I believe you can tell Docker to "start service B when service A is healthy", so you can define your health check with a script that depends on Tailscale functioning.

[–] just_another_person@lemmy.world 4 points 1 week ago (1 children)

This is the correct answer, but you need a few things to clarify:

  1. The issue isn't the Docker system service. Don't make that depend on Tailscale
  2. Add a healthcheck and restart policy to the container to make it fail when conditions aren't met, and restart until they are successful
  3. Build in some time tolerance at the service level inside the container to prevent it from flailing if your Tailscale healthchecks don't pass after they initially start. Don't rely solely on container health checks to ensure it works properly as that might not always be possible.

Considering it's tailscale, one may want to have the service fail though as tailscale is sometimes not used for convenience but security concerns instead.

[–] Bakkoda@sh.itjust.works 2 points 1 week ago

I need to figure this out on my immich stack. VM reboot only starts two out of the 4 services. Ty for the info.

[–] horse_battery_staple@lemmy.world 8 points 1 week ago* (last edited 1 week ago) (1 children)

Delay the start of your containers with the tailscale dependency. Are you using required or depends_on in your docker-compose.yml

https://hatchjs.com/docker-compose-conditionally-start-service/

If you're using kubernetes you can make the requirements at the pod level

https://stackoverflow.com/questions/69423932/container-initialization-order-in-pod-on-k8s

Edit: If using docker-compose.yml you can set a condition on a healthcheck

You can also specify a condition that must be met before the service is started. For example, the following configuration will start the web service only if the db service is healthy:

version: ‘3.7’
services:
web:
image: nginx
depends_on:
– db
condition: service_healthy

The service_healthy condition checks the health of the db service using the docker-compose ps command. If the db service is healthy, the web service will be started.

[–] lemba@discuss.tchncs.de 3 points 1 week ago