Self-hosting a GoModel LLM gateway on NixOS

I wanted a central place to manage my LLM provider connections, per-app keys, models, and budgets. GoModel is an OpenAI-compatible gateway, and this post walks through deploying it on NixOS with rootless Podman quadlets.

Architecture

LLM clients hit an NGINX reverse proxy, which terminates TLS and forwards to GoModel on localhost. GoModel keeps state in Postgres instead of the default SQLite, so usage data survives container rebuilds, and caches responses in Redis. Everything is a rootless Podman quadlet managed by NixOS, with secrets from sops.

Quadlet containers

The containers are declared with quadlet-nix. GoModel waits for its database and cache, keeps itself up to date, and restarts when the config or secrets change (X-RestartTriggers, more on that below):

let
  gomodelConfig = pkgs.writeText "gomodel-config.yaml"
    (builtins.readFile ./gomodel-config.yaml);
in {
  home-manager.users.podman.virtualisation.quadlet.containers.gomodel = {
    serviceConfig = {
      Restart = "always";
      TimeoutStartSec = 900;
    };
    unitConfig = {
      After = [ "gomodel-db.service" "gomodel-redis.service" ];
      Wants = [ "gomodel-db.service" "gomodel-redis.service" ];
      X-RestartTriggers =
        "${config.sops.secrets."gomodel/env".sopsFileHash} ${gomodelConfig}";
    };
    containerConfig = {
      image = "docker.io/enterpilot/gomodel:latest";
      autoUpdate = "registry";
      environmentFiles = [ config.sops.secrets."gomodel/env".path ];
      environments = {
        STORAGE_TYPE = "postgresql";
        REDIS_URL = "redis://gomodel-redis:6379";
        RESPONSE_CACHE_SIMPLE_ENABLED = "true";
        METRICS_ENABLED = "true";
        ADMIN_UI_ENABLED = "true";
      };
      networks = [ "internal" ];
      publishPorts = [ "127.0.0.1:<port>:8080" ];
      volumes = [ "${gomodelConfig}:/app/config.yaml:ro" ];
    };
  };
}

Postgres and Redis are plain quadlets with named volumes. All three containers share a Podman network and reach each other by container name:

containers.gomodel-db = {
  containerConfig = {
    image = "docker.io/postgres:alpine";
    autoUpdate = "registry";
    environmentFiles = [ config.sops.secrets."gomodel/postgresql".path ];
    environments = {
      POSTGRES_DB = "gomodel";
      POSTGRES_USER = "gomodel";
    };
    networks = [ "internal" ];
    volumes = [ "gomodel-db.volume:/var/lib/postgresql" ];
  };
};

containers.gomodel-redis = {
  containerConfig = {
    image = "docker.io/redis";
    autoUpdate = "registry";
    exec = "redis-server --appendonly yes";
    networks = [ "internal" ];
    volumes = [ "gomodel-redis.volume:/data" ];
  };
};

Budgets and rate limits

Every client gets its own user path: one app's key only works under /apps/openwebui, another's under /apps/opencode, so one app can't eat another's budget. Paths are hierarchical, so the same scheme can organize apps, teams, and individual users (for example /teams/research/openwebui). Budgets and rate limits attach to those paths, giving each one its own monthly dollar cap and hourly token limit:

budgets:
  enabled: true
  user_paths:
    - path: /apps/openwebui
      limits:
        - period: monthly
          amount: 1.00
    - path: /apps/opencode
      limits:
        - period: monthly
          amount: 50.00

rate_limits:
  enabled: true
  user_paths:
    - path: /apps/openwebui
      limits:
        - period: hour
          max_tokens: 5000000

virtual_models:
  - source: glm-5.3-flash
    target: zai/glm-5.3-flash
  - source: deepseek-flash
    target: deepseek/deepseek-flash

users:
  - path: /apps/openwebui
    description: Open WebUI
    allowed_models: ["zai/glm-5.3-flash", "deepseek/deepseek-flash"]

Virtual models give clients stable names while the upstream target stays free to change. Note that allowed_models has to list the original target names (zai/glm-5.3-flash), not the virtual aliases, because the allow-list is checked against the model the request resolves to after aliasing.

Wiring up a client

A client only needs the base URL https://llm.example.com/v1 and its key. The key is bound to a user path, so the URL stays the same for every app. Virtual keys are not kept in the config file; you create and manage them in the dashboard at /admin/dashboard, which you log into with the master key. Usage and costs show up there per path.

Secrets

The master key and database password are generated once and stored in sops, owned by the unprivileged podman user:

sops.secrets."gomodel/env" = { owner = "podman"; };
sops.secrets."gomodel/postgresql" = { owner = "podman"; };

Because X-RestartTriggers watches the secret hash, rotating a key or editing the config restarts the container automatically on colmena apply.

Providers are connected the same way: the z.ai and DeepSeek API keys are environment variables in gomodel/env, which is all GoModel needs to expose them as the zai and deepseek providers used by the virtual models above.

NGINX Reverse Proxy

TLS comes from the wildcard cert (DNS challenge), /metrics stays on localhost for scraping, and streaming needs the long timeouts and disabled buffering:

services.nginx.virtualHosts."llm.example.com" = {
  useACMEHost = "wildcard.example.com";
  acmeRoot = null;
  forceSSL = true;
  locations."/metrics" = { return = "403"; };
  locations."= /dashboard" = { return = "302 /admin/dashboard"; };
  locations."/" = {
    proxyPass = "http://127.0.0.1:<port>/";
    proxyWebsockets = true;
    recommendedProxySettings = true;
    extraConfig = ''
      proxy_connect_timeout 60s;
      proxy_send_timeout 1200s;
      proxy_read_timeout 1200s;
      proxy_buffering off;
    '';
  };
};