this post was submitted on 02 Feb 2024
126 points (93.8% liked)

Selfhosted

40394 readers
391 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

Edit: wow, this is a never ending comment section!

you are viewing a single comment's thread
view the rest of the comments
[–] farcaller@fstab.sh 6 points 9 months ago

I went for a much simpler approach lately as I downscaled my hardware for efficiency.

I run NixOS on the bare metal. It gives the system management a declarative approach, just like kubernetes would. On top of that, I run libvirt as a hypervisor. In other scenarios I'd use tinyvmm and cloud-hypervisor, but I found qemu way better for the variety of homelab workloads and libvirt is pretty straightforward.

Some vms have pci passthrough, e.g. my routeros vm gets a bunch of NICs directly, some have various funny network topology. Libvirt used to be a pain in that regard, but it's actually fine with NixOS because you manage both sides of the networking stack in declarative configuration.

I run NixOS on the vms too (now for the sake of easy upgrades), and I have a bit of a split between running services natively (systemd is very good about “containerizing” things nowadays) and using docker (mostly because of laziness, e.g. Elastiflow was easier to deploy this way). Finally, I have a single dokerized Ubuntu that's more like a VM (as in, I never had a dockerfile for it, it's fully stateful) running the matter home automaton bits because I gave up on properly containing the matter python stack and went for an easy way out.

Now, a word about alternatives.

I used to run Ubuntu. No more. Upgrading the OS is always a huge pain even if everything is in docker. I want my OS to be managed in a config file and be able to easily roll back to the previous state. I used to run k3s, but even though it is much thinner than k8s, it is still very much ram hungry and I just don’t want to pay for that. Besides, complex networking is often non-trivial due to how its networking works, and multus is a world of pain. I used to run different hypervisors for the VMs (kubevirt, tinyvmm, a bunch others). I went way back to libvirt mostly because it’s straightforward in tuning very specific qemu bits I cared for in the homelab. I have some cpu overprovisioning, so I want to make my quotas set up extremely precisely, sacrificing the right workloads.