1w ago

Using Jenkins to deploy Docker containers?

I am wanting to automate some homelab things. Specifically deploying new and updating existing docker containers.

I would like to publish my entire docker compose stacks (minus env vars) onto a public Git repo, and then using something to select a specific compose from that, on a specific branch (so I can have a physical seperate server for testing) automatically deploy a container.

I thought of Jenkins, as it is quite flexable, and I am very willing to code it together, but are there any tools like this that I should look into instead? I've heard Ansible is not ideal for docker compose.

15 comments

I'm gonna be real: You want kubernetes + gitops (either fluxcd or argocd or the rancher one).
I mean sure, jenkins works, but nothing is going to be as smooth as kubernetes. I originally attempted to use ansible as many people suggested, but I got frustrated becuase it struggled to manage state in a truly declarative way (e.g. when I would change the ports in the ansible files the podman containers wouldn't update, I had to add tasks for destroying and recreating the containers).
I eventually just switched to kubernetes + fluxcd. I push to the git repo. The state of the kubernetes cluster changes according. Beautiful. Simple. Encrypted secrets via sops. It supports the helm package manager as well. Complex af to set up though. But it's a huge time saver in the long run, which is why so many companies use it.

as @damnthefilibuster@lemmy.world already mentioned: GitLab CI
Jenkins is a CI application from before CI was cool. GitLab CI is integrated and can trigger on certain events. Additionally you mentioned, that you want to publish on a public repo anyway.
You are probably are comfortable with containers. So GitLab CI should be easy for you to learn - as it pretty much starts up a container to do certain tasks. I've seen suggestions for Kubernetes, which for sure is the more mature solution. But i would question, whether you need the added functionality and complexity of K8s for a home setup.
To gain access to your local network, you can use the runner for a secure connection (as described by damnthefilibuster). or you could SSH into the machine, as long as you have it in a DMZ. Drawback is that you have to be more sure about your network infrastructure. Benefit is that it is a more general approach. Obviously you need to store all certs, keys and preferably even addresses in secrets, not the .gitlab-ci.yml.
As you can see from this thread, there are many ways which lead to rome. My advice is to start with something simple and lightweight, which you understand. adding complexity down the road is easier, than removing it.

I've heard Ansible is not ideal for docker compose.
Not sure what you heard, but I use ansible to push docker compose files to VMs and publish containers without issue. Works nicely.
I usually create systemd service files to start/stop the compose jobs and have ansible set it all up.
- I don't think it necessarily needs to be either or. Organizing the playbooks and folders myself can be stressful so an extra layer of organization might work best for you. There are other tools like Semaphore that are specifically built for Ansible executions though. Might need a lot of duct tape for Jenkins to run Ansible.
  And if you're not a fan of yaml you can always nope out and embed shell scripts into your Playbooks. You can even put Docker compose yaml inside a playbook but it's a bit inception-y and I don't really recommend that.

I did Jenkins for a while. It works, but it was built before the world of containers and is now fairly antiquated. Most extensions are no longer maintained.
Look into forgejo, and then use actions to auto deploy

Isn't Jenkins a continuous integration system? I'd think that you'd want a configuration management system (like ansible) if you're not trying to set up CI infrastructure.

I use Ansible to deploy a bunch of containers with intradependencies (shared volumes, networks and settings). One of the containers is homemade with the source pulled from codeberg. Variables are kept in a separate file and passwords in an encrypted one and the whole thing is in a private repo. It is quite flexible.
When I started out converting from compose, I literally asked Copilot for "this, but in Ansible", which got me pretty far.

You wanna know a fun way to do this?
GitHub (and I think Gitlab too) supports you running their runner within your own infra. It’s literally a binary that needs permissions and space. Then, you can tell your git repo to use that runner to run docker compose and as part of the “build” process, deploy you container to the same or an in-network machine.
This is not secure, it’s probably going to involve a lot of hard coding of local IPs or server names etc. But you can make it work.
I use this way to get a Win11 PC to run some regular containers on itself. Works like a charm.
- I did this and the fun thing about it is that your runner can access things inside your network that a regular GitLab runner can't. I've used it to manage a k8s cluster that isn't exposed to the Internet at all.

I've used Ansible to deploy docker compose and it worked pretty well. You will have to do some learning if you aren't familiar with it, but I'd say it's worth it.
Like others, I would not recommend Jenkins.

Why? Tools for this exist. Jenkins is not that tool.
- Like what?
  
  People are naming them all over this thread. Use the right tool for the right job, don't try to shoehorn a function for a dead-ass and dying tool into a different use-case just because you know it.
- Exactly. Look into Komodo instead

15 comments