🖥️ Self-Hosted Infrastructure

The ebook pipeline and the media stack each started as one workflow I wanted to own instead of rent. The homelab is the same instinct applied to everything else. Photos, recipes, budgets, home automation, game servers, and the monitoring that watches all of it run on hardware in my house, with no subscription that can raise its price and no company that can sunset the product out from under me.

Owning the infrastructure changes how I treat it. When a service misbehaves I can read its logs, and when it disappoints me I can replace it. When nothing existing does what I want, I write my own and run it next to everything else. That last part turned out to matter more than I expected when this started.

One mini PC running Unraid, with 59 Docker containers on it as of this writing. 47 of those are software other people wrote: the media servers, the book pipeline, photo backup, recipe management, budget tracking, home automation, DNS filtering, uptime monitoring, and the reverse proxy that fronts all of it. The other 12 are apps and tools I built myself, which get their own section below.

The homelab page keeps the current inventory: network layout, hardware, and the full services table. I won't repeat it here. This page covers how the system got to where it is and the decisions made along the way, because the container count is the least interesting number on it.

The newest layer of the homelab exists so an AI assistant can operate it. A set of MCP servers sits behind one OAuth gateway, and through that gateway an assistant can query AdGuard's DNS logs, check array health on Unraid, read Uptime Kuma monitors, manage recipes in Mealie, post to Discord, and run Docker operations through a custom server I wrote called p6ops. This is write access, not just read: the assistant can restart containers, edit the Caddyfile, and pull image updates. The gateway handles authentication once, for every server behind it.

p6ops exists because of a lie. The Unraid MCP server has a container-update action, and it reports success: true every time. During one update I checked the container ID, image ID, and uptime before and after the call instead of trusting the response. Nothing had changed. The container sat on the old image with the old uptime while the tool reported success anyway. So I built p6ops around the opposite contract. Every mutating tool captures state before the change, re-reads it after with an independent call, and reports whether anything actually moved. The response says whatever it says. A changed container ID means the update actually ran.

Every internal service started on plain HTTP at a .lan hostname, with DNS rewrites in AdGuard pointing the names at the server. It worked, and it aged badly. Browsers warn on every page, some web APIs refuse to run outside a secure context, and every login on my own network crossed the wire in cleartext.

Internal services can't get certificates through a normal HTTP challenge, since Let's Encrypt can't reach into a private network to verify anything. The DNS-01 challenge gets around that. You prove control of the domain by publishing a DNS record, and the certificate authority never needs to connect to the server at all. Setting it up meant rebuilding Caddy with the Cloudflare DNS plugin, which the stock image doesn't include, then issuing a real certificate for every service and moving each hostname onto the new naming scheme.

Everything internal now answers over HTTPS with a valid certificate and no browser warnings. Getting there produced some of the best debugging stories on this page, covered below.

The homelab doubles as a dev platform. YTPlayer, LedgerView, RoadTrip, Rust Rush, Roundup, Stash, an ESPN Fantasy MCP server, and an AI Trading Monitor all run in the same Docker environment as the hosted services, deployed the same way, proxied by the same Caddy, and watched by the same monitoring. The trading monitor is a dry-run algorithmic trading system with a circuit breaker that halts it when losses cross a threshold, and it has never touched a real dollar. The Movie Analytics Dashboard lives here too and has its own write-up. Going from an idea to an app running behind HTTPS with monitoring in an afternoon, on hardware I already own, removed most of the friction between wanting a tool and having one.

Deletion is the other half of the discipline. Anything that stops earning its place gets removed. Hermes, a local-model AI agent project, went first, after the model fabricated data during testing and the project then hit an API credits wall. Syncthing never found a lasting use case after two pivots. MusicGrabber duplicated what Apple Music already covers. SmarTunarr and an older Obsidian MCP fork went the same way, the fork replaced by a cleaner vault server. The standing rule is no complexity without demonstrated value. 59 containers is already a lot, and it would be far more without the teardowns.

Laid end to end, the phases look like this.

One Caddy directive silently turned off certificate issuance. Caddy's auto_https off and auto_https disable_redirects read like neighbors and behave nothing alike. I wanted the second, to stop Caddy from forcing redirects during the migration, and wrote the first, which shuts down the entire automatic certificate engine. Nothing errored. Caddy started clean, the TLS policies looked correctly registered in the loaded config, and certificates simply never got issued. It took real debugging to find, because every place I checked said the setup was right. The fix was one word.

An in-place DNS edit deleted around 28 rewrites. Partway through the HTTPS rollout, an edit meant to add new hostnames alongside the existing .lan entries removed roughly 28 of them instead. I looked at the restore job in front of me and decided against it. The .lan scheme was on its way out anyway, so instead of rebuilding the old entries I retired the naming scheme on the spot and finished moving everything to the HTTPS-only names. The mistake mostly changed the schedule.

Palworld's map came back blank after moving off a paid host. Our Palworld server used to run on a paid host, until I brought it home. The migration was plain enough: export the save files over FTP, copy them into a fresh container's own save folder, start it up. Characters, inventory, and bases all survived. Map exploration didn't, and the fog of war covered the whole world as if nobody had ever left spawn. That looks like a corrupted save and isn't one. Palworld tracks exploration state separately from the rest of the save data, so a migrated world keeps everything you built and forgets where you've walked. Once you know that, the blank map reads as expected behavior instead of a failed migration.

The Tech Queue, my backlog of homelab ideas, holds 37 entries right now. Some will ship as containers, some will get cut once a better option shows up, and the queue refills faster than it drains either way. There is no finished state this is heading toward. The system keeps growing, I keep refining it, and that suits me fine.

I'd build HTTPS in from the start instead of treating it as a later upgrade. Launching everything on plain .lan HTTP felt simpler at the time and turned into a full network migration later: a rebuilt Caddy image, certificate issuance for every service, and roughly 28 DNS rewrites tangled up in the transition. Starting encrypted would have cost one extra afternoon of setup and saved the whole project.

I'd also stop trusting written docs for anything that changes often. During one deployment I picked a port by checking live container bindings directly, and it was the right call, because the reference doc's port table was stale and missing several ports already in use. Ports, running containers, and anything else that shifts weekly need verification against the live system. A doc records what was true when someone last edited it, and nothing more.

And I'd treat a decommission as its own checklist rather than a single delete. Tearing down a service means more than removing the container and its compose block. The dashboard still has a widget pointing at a dead URL, and home automation still lists an auto-discovered device the integration left behind. A few of my teardowns left that kind of debris, because "remove the service" only meant "remove the container" in my head, and the leftovers surfaced weeks later as mystery entries in other tools.