The Lab
A self-hosted infrastructure and AI platform, designed, built, and operated as a working demonstration of full-stack systems architecture: virtualization, infrastructure-as-code, zero-trust identity, verified backup/DR, and a custom multi-agent AI operations system.
What this demonstrates
- Virtualization & compute: a multi-node Proxmox cluster with a dedicated bare-metal AI host and tens of terabytes of RAID-protected shared storage over iSCSI and NFS.
- Infrastructure as code: networking and configuration managed as OpenTofu and Ansible, run through a single orchestration plane, with GitOps container deployment from a self-hosted git server.
- Zero-trust identity & ingress: one hardened entry point, federated to Microsoft Entra ID, so every self-hosted service inherits enterprise-grade Conditional Access rather than standing on its own.
- Resilience: layered, dual-targeted, verified backups. Restores get tested, not assumed.
- Operational discipline: scheduled self-healing, a designed patching cadence, secrets-as-code, and a documented decision log (below).
- A custom multi-agent AI platform: three role-tuned agents with governed cloud identities, a security-gated tool broker, git-versioned memory, and a task bus.
The multi-agent AI platform
The centerpiece. A small distributed system with an explicit governance model: which agent may think how hard, which may touch what, at what cost, and with what human oversight.
One designed persona, instantiated as three role-tuned agents, routed on two independent axes: reasoning depth and action reach.
| Agent | Role | Model tier | Compute basis |
|---|---|---|---|
| Athena | Strategist / architect. Design, deep reasoning, research | Cloud, frontier-class | Metered |
| Vulcan | Privileged operator. Infrastructure, IaC, in-repo edits | Mid-tier, usage-capped | Usage-capped plan |
| Oracle | First responder. Triage, scripting, retrieval, always-on monitoring | Local, on-prem | Electricity only |
Capability scales with proven trust: the least-proven agent gets the least-dangerous reach, and the most privileged credentials are isolated to a single agent. An agent that hits its ceiling escalates with a structured hand-off instead of silently overreaching.
Governed identity. Each agent holds a directory-governed cloud identity under Microsoft's emerging agent-identity model, gated by a certificate credential and Conditional Access for agents, rather than a shared service-principal secret or a human's admin role stretched to cover a bot.
A security-gated broker, not raw access. The agents never get direct shell or cloud credentials. Every action, from a container restart to an OpenTofu run, passes through a single broker that authenticates the agent's identity token, checks it against a narrow allowlist of granted capabilities, rate-limits, and logs. There is no general "run anything." Privileged verbs run as an unprivileged, scoped service account.
A four-tier autonomy model governs how freely an agent acts, from "just do it" for greenfield, low-stakes work up to a hard stop, meaning a loud alarm and an explicit human go-ahead, for anything touching domain controllers, storage, network core, or credential stores. A dedicated safety gate fires on credential handling, destructive commands, and recognized prompt-injection attempts, and it cannot be talked out of firing by anything inside a prompt.
Persistent, versioned memory. All three agents share one git-versioned memory, split into a lossless canonical corpus, an append-only episodic log, and a small always-loaded "what's hot" working view. A nightly consolidation pass reconciles what's been learned against what's actually still true, cross-links related episodes, and forgets low-value noise. It only proposes changes to the canonical record. A human ratifies them.
Work moves as tickets, not chat. Cross-agent work runs through a Kanban-style task bus, with structured cards carrying an owner, a priority, a definition of done, and a completion receipt, rather than ad-hoc conversation between agents.
Agentic infrastructure: the access model behind the platform
The platform above only works because the agents can safely hold real privilege. Everyone is racing to let agents write code; far fewer are letting agents operate the infrastructure underneath it. The full architecture, the use-time authorization broker, the Verified ID two-key approval, the tiered least-privilege accounts that sit disabled until an approved window, and the same model extended from on-premises systems to the cloud tenant, is written up as a standalone brief.
How I make the tradeoffs
Architecture is a series of decisions made under constraint and uncertainty. I keep a running decision log with the context, the call, the reasoning, and the cost, so the reasoning survives the decision. A few, unedited:
Accepting a single point of failure, on purpose
Context. Every VM disk, including both domain controllers, lives on shared storage from one device. An outage there cascades to DHCP, DNS, and authentication. Redundant compute nodes protect against a hypervisor failing; they do nothing for a storage outage.
Decision. Acknowledge the single point of failure, keep it in the highest-protection tier with a human in the loop before any live interruption, and deliberately not buy redundant hardware. Spend the limited budget on a local AI inference node instead.
Rationale. This is a homelab, not an enterprise SLA. A storage outage costs no revenue; worst case is falling back to hosted services for a few hours. Data durability is covered separately through RAID, verified backup images, and planned offsite copies. Durability isn't the same problem as availability, and only availability is at stake here. Sometimes the cure costs more than the disease.
Consequences. A known, priced, owned risk instead of an unknown one. The discipline this encodes matters more than the lab: surface new or unaccepted risk loudly, and stop relitigating a risk you've already weighed and accepted.
Abandoning a design the moment the evidence said it was wrong
Context. Onboarding the agent federation to a governed cloud identity model. The first instinct, carried over from how you'd provision a human admin, was directory roles, role-assignable groups, access packages, and just-in-time elevation.
Decision. Grant access as narrow, per-identity application permissions instead. Abandon the entire group/package/elevation scaffolding built for humans.
Rationale. Tested against the live tenant, the human pattern was structurally wrong for a non-human identity: agents can't join the groups that pattern depends on, and the elevation tooling doesn't see them. Agents are shaped like applications, not like users, and the runtime gate for an application is a certificate and a policy, not a human's step-up authentication.
Consequences. The read tier went live cleanly on the corrected model. The lesson worth more than the result: abandon a half-built design the moment the evidence says it's wrong, sunk cost notwithstanding.
One hardened door, not twenty open ones
Context. Roughly twenty public-facing services behind one dynamic IP address.
Decision. A single hardened entry point terminates every public hostname. No service gets its own direct exposure.
Rationale. Centralizing ingress centralizes the TLS story, the authentication story, and the attack surface into one place to harden, monitor, and reason about. Exposing each service individually doesn't scale, and it fragments authentication into as many weak points as there are services.
Consequences. That single entry point becomes a critical dependency. If it's down, nothing external works. Accepted in exchange for every service inheriting the same enterprise-grade access policy instead of standing on its own, which is the same "one hardened boundary, not per-door locks" logic behind the identity architecture in the Stryker Lessons Learned brief.
Git is the only source of truth
Context. Managing a fleet of containerized services across multiple hosts.
Decision. Every service deploys from a Git repository. Direct, undeclared changes on a server are banned. A change ships only as commit, then redeploy.
Rationale. The host becomes disposable; the repository is the single truth. Every change is reviewable, reversible, and auditable, and there's never a gap between what's declared and what's actually running.
Consequences. A little ceremony, since you commit to change anything, in exchange for full traceability and a fleet that can be rebuilt from the repo alone.
Why publish this at all
Because the good part was never the specific IP scheme or the exact list of containers. It's the pattern underneath: least privilege applied to non-human identities while the industry is still deciding what to call them, a decision log that makes the reasoning auditable instead of tribal knowledge, and the same identity-architecture instincts from the Stryker brief applied at home, where the only stakeholder is me and the only excuse for cutting a corner would have been that nobody's watching.