# Add a text reader

Markdown version of <https://sam-dumont.github.io/immich-memories/docs/better/reader/>.

Installing Full from scratch? Select its tab in [Quick start](https://sam-dumont.github.io/immich-memories/docs/get-started/quick-start.md), or use [native Mac](https://sam-dumont.github.io/immich-memories/docs/run/uv-pip.md#apple-silicon). This page sets up the reader those routes need.

A reader picks the music mood on any tier. With GPU inference, captions and Laya ready, it also enables **Full**: refinement of the rules draft. The selection reader receives text, not pictures.

Model-written titles and summaries are experimental and off by default. Turn them on with `title_screens.llm_titles: true` (or `--llm-title` for one run) and the reader names special days and writes the film's short summary. Years, months, seasons, people, birthdays, holidays and "on this day" films always keep the app's own title.

Use a model with a 32k context and valid structured replies. The app can run its local default, or call an API server you already use.

## Let the app run the local model

On native Linux or macOS, install llama.cpp with `llama-server` on your `PATH`. On a Mac:

```bash
brew install llama.cpp
```

On Linux, install a CPU or CUDA build from the [llama.cpp releases](https://github.com/ggml-org/llama.cpp/releases), or follow its [build instructions](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md). Put `llama-server` on `PATH`. GPU offload follows the build defaults; the owned process does not accept custom GPU-layer flags.

The published Docker image does not include `llama-server`. For Docker or Kubernetes, use [an existing server](#use-an-existing-server) or a [hosted reader](#hosted) below. On a native install with `llama-server` ready, enable the local reader:

```yaml
advanced:
  llm:
    enabled: true
    base_url: ""
    model: gemma-4-E4B-it-Q4_0
```

Fetch its pinned files before the first request:

```bash
immich-memories models fetch
immich-memories preflight
```

The app starts the model when needed and stops its owned reader before local music, stem separation and rendering. Selection also releases its Laya scorer before rendering. Later requests load the models again. With `enabled: false`, the reader sends no requests and starts no model. Setting a model, URL or API key never enables it. Preflight warns when a reader is configured but disabled. A custom GGUF can replace the default; [local reader settings](https://sam-dumont.github.io/immich-memories/docs/reference/llm-providers.md) cover paths and context size.

## Use an existing server

For oMLX, vLLM, Ollama's compatible API or another `/v1/chat/completions` endpoint:

```yaml
advanced:
  llm:
    enabled: true
    provider: openai-compatible
    base_url: http://192.168.1.50:8000/v1
    model: your-server-model-name
```

The app-owned llama.cpp reader keeps episode JSON schemas enabled; the external-server workaround below does not apply to it.

Use the exact name the server advertises. From Docker Desktop, `host.docker.internal` reaches a native server on the host; `localhost` reaches the container itself. Add `api_key` when the server requires one. Local servers get conservative defaults, hosted-looking (dotted-name) servers get faster ones; tune `reader_concurrency` and `structured_output` if needed. See [local server defaults](https://sam-dumont.github.io/immich-memories/docs/reference/llm-providers.md#use-an-existing-server) for the exact values.

The server manages its own model memory. The app does not unload an external server's models, so account for that memory alongside captions and music.

## Hosted

A hosted reader receives descriptions, dates, people and place names from your cut. Use a local reader if that text should stay home.

See the [measured hosted-reader time and cost](https://sam-dumont.github.io/immich-memories/docs/better/measured.md#hosted-reader-time-and-cost) for complete
films, free-text requests and provider quality checks. The prices cover reader calls after
picture preparation; the local machine still does selection and rendering.

```yaml
advanced:
  llm:
    enabled: true
    provider: openai
    model: gpt-5.6-luna
    thinking: low
    api_key: ${OPENAI_API_KEY}
```

The provider preset supplies the API URL. Provider names, native Ollama and Anthropic examples, token settings and batching live in the [LLM reference](https://sam-dumont.github.io/immich-memories/docs/reference/llm-providers.md).

## Check the setup

In Docker, prefix `immich-memories` commands with `docker compose exec immich-memories`
from the app's installation folder.

```bash
immich-memories preflight
immich-memories capabilities
```

These report connections, installation and available features; they do not certify the quality of a film. With `tier: auto`, GPU inference plus an enabled reader selects Full. GPU capability includes a remote inference service, a working local CUDA ONNX runtime, or a Mac's Metal GPU. Without these, selection stays Basic and the reader still picks the music mood. GPU and Full also require captions and Laya.

Review the result. When the reader stops answering (host down, connection refused, timeouts), the run says so once and the film ships the rules draft. A rejected API key or a missing model stops the run instead. [Laya](https://sam-dumont.github.io/immich-memories/docs/reference/llm-providers.md#the-laya-audience-pre-screen) handles caption-based sharing separately; sharing never asks the prose reader.

One `advanced.llm` section configures the reader for every role: selection, music mood, titles and summaries, and optional LLM captions.

Image captioning with the reader is a separate [opt-in](https://sam-dumont.github.io/immich-memories/docs/better/captions.md#explicit-llm-captions). That role sends pictures and needs a vision-capable model.
