# Distributed services on Kubernetes

Markdown version of <https://sam-dumont.github.io/immich-memories/docs/run/reference/cluster-example/>.

Use separate services when you need independent GPU placement or already run a reader and music
server. Start with the [base Kubernetes install](https://sam-dumont.github.io/immich-memories/docs/run/kubernetes.md). For one GPU shared between
preparation and rendering, the [one GPU service](https://sam-dumont.github.io/immich-memories/docs/run/reference-setup.md) is simpler.

**Deployment: Distributed Kubernetes services**

Prepare a cut:

- **Your Immich** (Existing Immich server)
  - **Immich API**, Your configured URL: Provides metadata, previews and original media. Upload is optional; the app does not change the source originals.
  - Storage: Your photo and video library
- **Immich Memories** (App pod · one replica)
  - **App + CLI**, Choose the cut: Reads Immich metadata and previews, saves reusable facts, then chooses the cut. On GPU and Full tiers, Laya runs in the app, even with a remote preparation service.
  - Storage: Data/cache · output · model PVCs
- **Preparation services** (Pods + supplied reader endpoint)
  - **Inference**, GPU reservation · 1 allocation: Receives sampled frames for classification. The overlay reserves one advertised NVIDIA GPU allocation for this pod.
  - **Caption server**, GPU allocation must be configured: Receives picture tiles and frame strips. The shipped CUDA caption deployment has no GPU resource reservation. Configure allocation or advertised sharing before scheduling it.
  - **Text reader**, Existing API · not deployed here: The overlay expects an existing text-reader endpoint and a served model name. This service receives annotation text, not pictures. Configure it separately; it can run on your network or another permitted endpoint.
  - Storage: Service model caches

1. **Metadata + previews**: Preparation starts with your library; originals remain in Immich.
2. **Preview tiles + frames**: Classification and captions go to separate services. The text reader receives annotations through its separately configured endpoint.

Service boundaries, not node placement. Reserve GPUs explicitly; a namespace is not an authentication boundary. Reader and music endpoints are configured separately.

Render a film:

- **Your Immich** (Existing Immich server)
  - **Immich API**, Your configured URL: Provides metadata, previews and original media. Upload is optional; the app does not change the source originals.
  - Storage: Your photo and video library
- **Immich Memories** (App pod · one replica)
  - **App + CLI**, Assemble the film: Hands the chosen cut to the render worker. The UI and run history stay here.
  - Storage: Data/cache · output · model PVCs
- **Render sidecar** (Same pod as the app)
  - **Render worker**, App pod GPU reservation · 1 allocation: The app and render sidecar share their pod. The app calls the authenticated worker on 127.0.0.1:8093. The sidecar receives the Immich key, fetches originals and returns the film to the app. The diagram does not promise that all workloads fit on two GPU nodes.
  - Storage: Worker scratch/cache · ephemeral

1. **Cut + Immich credentials**: The authenticated render request includes the Immich URL and API key. Use HTTPS across an untrusted network.
2. **Worker → originals**: The worker needs its own network route to Immich. It returns the finished film to the app.
3. **Optional delivery**: After rendering, the app can upload the finished film if you enable delivery.

Service boundaries, not node placement. Reserve GPUs explicitly; a namespace is not an authentication boundary. Reader and music endpoints are configured separately.

Open the app:

- **Your Immich** (Existing Immich server)
  - **Immich API**, Your configured URL: Provides metadata, previews and original media. Upload is optional; the app does not change the source originals.
  - Storage: Your photo and video library
- **Immich Memories** (App pod · one replica)
  - **App + CLI**, App listener · port 8080: Serves the UI and proxies previews on the app listener. For Compose, open the published host port in your browser; the container still listens on 8080. The browser does not receive your Immich API key.
  - Storage: Data/cache · output · model PVCs
- **Your browser** (Desktop or phone)
  - **Web UI**, App URL · authenticated if exposed: Connects to the app, which serves cached media or fetches it from Immich. Configure authentication before opening access to other machines.
  - **HTTPS proxy + OIDC**, Operator-provided services: The example enables OIDC but does not deploy an ingress or identity provider. Supply the HTTPS proxy and identity registration; the app handles the login flow. This card groups the access dependencies, not one container.

1. **Preview requests**: The app keeps the Immich key on the server.
2. **Browser → app**: UI requests and media go through the app, behind the same login.

Service boundaries, not node placement. Reserve GPUs explicitly; a namespace is not an authentication boundary. Reader and music endpoints are configured separately.

[How I run it](https://sam-dumont.github.io/immich-memories/docs/run/how-i-run-it.md) describes the maintainer's own cluster built from these pieces, why each one is there, and where it departs from this overlay.

## What runs where

The shipped `deploy/kubernetes/overlays/maximalist` example composes these pieces. Use it as a
starting point and keep the services you need.

| Component | Deployed by the overlay | Needs |
| - | - | - |
| App and persistent store | Yes, one app replica | Immich access and the base PVCs |
| Render worker | Yes, sidecar in the app pod | NVIDIA GPU allocation; shared bearer token |
| Classifiers and Demucs stems | Yes, separate inference pod | NVIDIA GPU allocation; model cache |
| Captions | Yes, separate caption pod | NVIDIA runtime; reviewed GPU-sharing settings |
| Text reader | No | Existing compatible endpoint with 32k context |
| ACE-Step music | No | Existing ACE-Step API server, if music generation is enabled |
| HTTPS proxy and identity provider | No | Ingress/proxy and OIDC registration, if keeping the example's login settings |

Laya remains in the app. The reader receives selection text; inference and captions receive picture
previews. The render worker downloads originals from Immich with the API key supplied by the app.
[Privacy](https://sam-dumont.github.io/immich-memories/docs/run/privacy.md) lists the data sent to each service.

## Before applying

Keep the storage, namespace and NVIDIA prerequisites from the [base install](https://sam-dumont.github.io/immich-memories/docs/run/kubernetes.md#prerequisites).
The render sidecar and inference pod each claim one advertised GPU allocation. Putting both on
one physical card requires device-plugin sharing configured by the cluster operator; the overlay
does not set that up. Choose node selectors that match your available allocations.

The caption overlay intentionally has no `nvidia.com/gpu` request. It can share devices exposed
by the NVIDIA runtime, but this provides no scheduler reservation or memory guarantee.
For a dedicated card, add the [explicit GPU request](https://sam-dumont.github.io/immich-memories/docs/reference/caption-service.md#kubernetes)
and set its node selector. Confirm driver/runtime support and memory headroom for every service.

Use a release deployment bundle or a checkout of the release you will run. Keep the app and render
worker tags equal. Inspect image pins in the base and each add-on kustomization; the caption image
uses a separate llama.cpp build. [Caption image pinning](https://sam-dumont.github.io/immich-memories/docs/reference/caption-service.md#kubernetes)
explains that serving contract.

## Configure your endpoints and access

Like the base, the overlay stores no credentials. From the deployment bundle or release checkout,
create its three Secrets in the cluster first, each from its example, filled in outside the
checkout:

```bash
cd deploy/kubernetes
kubectl apply -f base/namespace.yaml
openssl rand -hex 32   # the render-worker token
for example in base/secret.yaml.example \
    overlays/render-sidecar/render-worker-secret.yaml.example \
    overlays/maximalist/maximalist-secret.yaml.example; do
  secret_file=$(mktemp)
  cat "$example" > "$secret_file"
  vim "$secret_file" && kubectl apply -f "$secret_file"
  rm "$secret_file"
done
```

What goes in each Secret, and what to change in `overlays/maximalist/config-map.yaml`:

| File or setting | Replace or confirm |
| - | - |
| Base Secret | Your Immich URL and API key |
| Render-worker Secret | Same Immich URL and the generated token |
| Extra Secret | Your OIDC credentials and any reader/music API keys |
| `advanced.llm` | Reader URL and the exact model name served there |
| `advanced.ace_step` | Music API URL, or `enabled: false` to use bundled/chosen music |
| `advanced.auth`, `advanced.server` | Public HTTPS URL, allowed accounts, trusted proxy addresses and secure cookies |
| `network` and cache limits | Outside map calls you permit and limits that fit the PVC |

The example enables OIDC and assumes HTTPS. Configure your own
[authentication and proxy](https://sam-dumont.github.io/immich-memories/docs/run/authentication.md#behind-a-reverse-proxy-with-tls), and register
its callback/logout URLs with the identity provider. The overlay does not create an Ingress.
For a private port-forward install, replace the OIDC settings with the access method you chose;
keep the app private until that method works.

The additional NetworkPolicy permits reader port 9999 and music port 8001. Change those ports
with your endpoint URLs. These are port rules, not destination allow-lists. The inference and
caption listeners have no built-in authentication; keep them private and restrict their clients
in your network policy. A ClusterIP does not authenticate requests.

## Render, apply and verify

```bash
kubectl kustomize overlays/maximalist
kubectl apply -k overlays/maximalist
kubectl rollout status -n immich-memories deploy/immich-memories
kubectl rollout status -n immich-memories deploy/immich-memories-inference
kubectl rollout status -n immich-memories deploy/immich-memories-captioner
kubectl get pods -n immich-memories -o wide
kubectl exec -n immich-memories deploy/immich-memories -c immich-memories -- immich-memories models fetch
kubectl exec -n immich-memories deploy/immich-memories -c immich-memories -- immich-memories config show tier
kubectl exec -n immich-memories deploy/immich-memories -c immich-memories -- immich-memories preflight -v
```

The overlay explicitly sets `IMMICH_MEMORIES_DEPLOYMENT_TIER=full`: the environment wins over the config
file. Seeing Full in `config show` confirms the setting; preflight checks its required services
and models. Run preparation on a small date window, review a first film and confirm its reported
render path before enabling daily automation.

If a pod stays Pending, check GPU allocations, node selectors and PVC events. If preflight fails,
check the relevant endpoint, model name, credentials and NetworkPolicy before changing tiers.
Keep the worker's loopback `exec` probes; kubelet HTTP probes cannot reach its loopback listener.
[Service diagnostics](https://sam-dumont.github.io/immich-memories/docs/run/maintenance/health-logs-cache.md) has the logs and health routes.

## Adapt the example

Remove unwanted services from your own composition rather than copying every option. Applying
sibling app overlays one after another replaces earlier patches; list the app components in one
kustomization instead. Preserve `enableServiceLinks: false` and one app replica.

For PostgreSQL, add the database Secret and egress port to that composition; the overlay does
not deploy PostgreSQL. [Database setup](https://sam-dumont.github.io/immich-memories/docs/run/database.md) and
[overlay composition](https://sam-dumont.github.io/immich-memories/docs/run/reference/kubernetes.md#database) cover the details.

Terraform users can start from `deploy/terraform/examples/maximalist`. That example creates the
app, render sidecar and captions, but **not inference**: deploy inference separately in the same
namespace and match the configured URL. Reader, music and identity services remain external.
[Terraform inputs](https://sam-dumont.github.io/immich-memories/docs/run/reference/terraform.md) describe those switches.
