# Storage and backups

Markdown version of <https://sam-dumont.github.io/immich-memories/docs/run/maintenance/storage-backups/>.

Back up the store before an upgrade. It holds prepared facts, people, settings, review decisions
and run history. Losing a preview is a download; losing your decisions is a different problem.

## What to keep

| Data | Default location | Keep it? |
| - | - | - |
| Store | `~/.immich-memories/store.db`, or your PostgreSQL schema | Yes |
| Config | `~/.immich-memories/config.yaml` and deployment `.env`/Secrets | Yes |
| Saved-credential encryption key | `IMMICH_MEMORIES_SECRET_KEY` | Yes, with the store backup |
| Session signing key | `IMMICH_MEMORIES_STORAGE_SECRET` or `.storage_secret` | Keep to preserve logins |
| Local films and run artifacts | `output.directory` | Keep films you have not delivered elsewhere |
| Model weights | Config/models volume | Keep to avoid downloading again |
| Thumbnails and downloaded clips | `cache.directory` | Disposable |

In Docker, the named config volume contains the store and config; `./output` contains films.
In Kubernetes, those are the cache and output PVCs. Keep SQLite on local disk, with one writer
host. [Database options](https://sam-dumont.github.io/immich-memories/docs/run/database.md) cover PostgreSQL and shared installations.

## Back up

Python:

```bash
immich-memories store backup
```

Docker:

```bash
docker compose exec immich-memories immich-memories store backup
docker compose cp immich-memories:/home/immich/.immich-memories/backups ./backups
```

Kubernetes:

```bash
kubectl exec -n immich-memories deploy/immich-memories -c immich-memories -- immich-memories store backup
kubectl cp -n immich-memories <pod>:/home/immich/.immich-memories/backups ./backups -c immich-memories
```

Backups can run while the app is up. Copy them off the host. Keep each `.db` or `.dump` together
with its `.manifest.json`; restore requires both. Save the encryption key separately and securely.
PostgreSQL needs `pg_dump` on `PATH`, at least as new as the server; the image includes it.

## Restore

Stop the UI, timers and workers first. `--force` replaces an existing store: use the backup you intend
and keep a copy of the current state. Restore with the release that made the backup when rolling back.
A corrupt or truncated backup is caught before anything is replaced: the existing store stays as
it was and the command exits with an error.

Python, after stopping the app:

```bash
immich-memories store restore --from /path/to/store.db --force
immich-memories preflight
```

Docker, with backup and manifest on the same mounted volume:

```bash
docker compose stop immich-memories
docker compose run --rm immich-memories immich-memories store restore \
  --from /home/immich/.immich-memories/backups/store.db --force
docker compose up -d
docker compose exec immich-memories immich-memories preflight
```

### Kubernetes restore job

Suspend installed CronJobs first and record which were active. Wait for any running scheduled
Job to finish, then scale the Deployment down and wait for its pod to release the PVC:

```bash
kubectl get -n immich-memories cronjob -l app.kubernetes.io/name=immich-memories
kubectl patch -n immich-memories cronjob immich-memories-auto -p '{"spec":{"suspend":true}}'
kubectl patch -n immich-memories cronjob immich-memories-monthly -p '{"spec":{"suspend":true}}'
kubectl scale -n immich-memories deploy/immich-memories --replicas=0
kubectl wait -n immich-memories --for=delete pod -l 'app.kubernetes.io/name=immich-memories,!job-name,!batch.kubernetes.io/job-name' --timeout=120s
```

Save this as `restore-job.yaml`. Replace `X.Y.Z` with the release and `store.db` with the actual
backup filename. For PostgreSQL, use its `.dump` and add the same database Secret as the Deployment.

```yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: immich-memories-restore
  namespace: immich-memories
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      enableServiceLinks: false
      securityContext:
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
      containers:
        - name: restore
          image: ghcr.io/sam-dumont/immich-memories:X.Y.Z
          command: [immich-memories, store, restore]
          args: [--from, /home/immich/.immich-memories/backups/store.db, --force]
          envFrom:
            - secretRef:
                name: immich-memories-secrets
          volumeMounts:
            - name: data
              mountPath: /home/immich/.immich-memories
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: immich-memories-cache
```

```bash
kubectl apply -f restore-job.yaml
kubectl wait -n immich-memories --for=condition=complete job/immich-memories-restore --timeout=300s
kubectl logs -n immich-memories job/immich-memories-restore
```

Only after a successful restore:

```bash
kubectl delete -n immich-memories job/immich-memories-restore
kubectl scale -n immich-memories deploy/immich-memories --replicas=1
kubectl exec -n immich-memories deploy/immich-memories -c immich-memories -- immich-memories preflight
```

Resume only CronJobs that were active before maintenance with `spec.suspend=false`.
The selector excludes both current `batch.kubernetes.io/job-name` and legacy `job-name`
labels added by the Job controller, so Job and CronJob
pods do not block the Deployment wait. It also works with the Terraform Deployment.

A SQLite backup restores to SQLite; a PostgreSQL backup to PostgreSQL. To change backend, restore
first, then use [store copy](https://sam-dumont.github.io/immich-memories/docs/run/database.md).

## Caches

The preview cache defaults to 30 GB, about 0.35 MB per picture a film's scope reaches, so a
library of 90,000 fits. Video clips default to 10 GB, and so do prepared sources: the cut segments
and photo clips a later render of the same selection reuses
(`advanced.analysis.prepared_source_cache_max_size_gb`). Local-only films and failed deliveries remain on
disk. After a confirmed upload to Immich, the local film and its run folder are removed, whether
the run uploaded it or you did later with `runs upload` or **Upload to Immich**. `runs storage`
lists what is kept; `runs delete` removes a run you no longer want.

```yaml
cache:
  video_cache_max_size_gb: 10
  video_cache_max_age_days: 7
  thumbnail_cache_max_size_mb: 30000
```

These are the defaults.

A preview averages about 315 KB: allow roughly **0.35 MB per candidate picture**. Preparing
60,000 pictures needs about 21 GB for previews alone. Use the number of pictures in the requested
period, including pictures preparation will later exclude from the film.

The thumbnail budget controls retention. Preparation protects its current previews until every
analysis stage has read them, so that working set can exceed the budget. Lowering the budget
does not make a large preparation fit on a small disk. Increasing it does not grow the volume.

Leave room for the store, job records, model runtime caches and downloaded video clips on the
same volume: a 100 GiB data volume with a 50,000 MB thumbnail budget, for example, leaves space
around a large preview set. Finished films are separate, at `~/Videos/Memories` on Python or `/app/output` in the image.

### Temporary working files

Keep `cache.directory` and the output directory on mounted storage with free space. The app
uses `<cache.directory>/scratch` for Python and child-process temporary files, including sampled
video frames. The inference service uses `<cache_dir>/scratch`; the render worker uses
`<directory>/scratch`. Startup sets Python's temporary directory and `TMPDIR`, `TEMP` and `TMP`
together. An unusable scratch directory stops startup instead of falling back to `/tmp`.

Scratch directories must belong to the process user and allow no group or other access (0700).
On Kubernetes, preserve that mode across PVC remounts with `fsGroupChangePolicy: OnRootMismatch`.
See [scratch permission recovery](https://sam-dumont.github.io/immich-memories/docs/run/maintenance/upgrading.md#kubernetes-scratch-permissions) for an existing
directory whose permissions were widened. Demucs and render jobs also keep their own working
directories under their configured storage, including on the combined GPU worker.

Video and Live Photo preparation can keep sampled frames for many source clips at once. The
size of a preparation batch does not cap that disk usage.

Scratch needs space beyond the preview and video cache budgets. Temporary files are removed
when their work finishes. Each process works in a folder of its own inside the scratch and holds
a lock on it while it runs, so a process that gets killed (out of memory, a pod restart) leaves
an unlocked folder. The next start of the app, a CLI command or a worker removes those folders
an hour after they last changed, and never touches one a live process still holds. You don't
need to clean scratch by hand.

Past cut attempts live in `<cache.directory>/editorial-runs` and stay under
`cache.editorial_runs_max_size_gb` (5 GB by default). The
[config reference](https://sam-dumont.github.io/immich-memories/docs/reference/config-reference.md#cache) lists what is trimmed first and what
always stays.

### Full disk during preparation

A full disk can prevent both the work and its job-status update from being saved. The browser's
progress may stay stale until space is available. After freeing space, reload the page; a job
whose process has disappeared is marked interrupted.

Expand the full volume, or stop work and [clear only disposable caches](#clearing). Check free
bytes and free inodes with `df -h` and `df -i` on the cache, output and temporary filesystems.
For Kubernetes, verify the mounted filesystem has grown as well as the PVC's requested size.
Keep the store and retry the same request: completed facts and usable cached previews are reused.
Deleting the whole data volume loses those facts, settings and run history.

### Clearing

While the app is idle, use **Settings > Caches > Clear**. From a Python shell, these three
folders alone are safe to delete:

```bash
rm -rf ~/.immich-memories/cache/video-cache
rm -rf ~/.immich-memories/cache/thumbnails
rm -rf ~/.immich-memories/cache/prepared-sources
```

In Docker, clear through Settings rather than deleting a similarly named host directory.
Do not delete `store.db` or the whole data volume. Attempt directories also contain review/render
inputs; removing the whole cache loses those artifacts even though banked facts remain in the store.

## Moving an install

Copy the config, encryption key and any films you need. Use `store backup`/`store restore` for
the store, then run `config test` and `preflight` on the new host.
[Source settings](https://sam-dumont.github.io/immich-memories/docs/run/config-file.md#where-a-setting-comes-from) may need new paths or addresses.
