Sentence parsing and pool rules
The parser is developed against one real family library and synthetic households. Those examples do not establish its accuracy on another library. Inspect the translation and pool before trusting a film.
You type what you want to see ("the cars I drove", "at the park with kids", "the first picture of each person") and get a film of it. The sentence picks the pictures; the same editor as every other memory makes the film (From library to film).
immich-memories generate --ask "at the park with kids" --dry-run # the translation, the pool and its rules
immich-memories generate --ask "at the park with kids" # then the film
What it needs:
tier: full: GPU preparation, captions and a configured text reader (advanced.llm). Every step is built for a small local model: it was tuned on Gemma 4 E4B. See Add a reader.- A prepared library. It reads the captions, faces and place names
immich-memories preparebanked, plus the letters Immich's OCR found in your photos. - The WordNet dictionary (11 MB), downloaded once by
immich-memories models fetchand checked against a pinned digest. Nothing downloads it at run time.
Reading the sentence calls your configured reader and searches the prepared library facts. Making the film can also use your configured caption, inference, music and rendering services; Privacy lists what each receives. The flags are on the generate page. The web UI has the same thing as a box on the Memory page, with a preview before any film: The web UI.
How it works
The sentence becomes filters, the filters make a pool of pictures, and the editor chooses from the pool the same way it chooses from a month or a trip. The model reads your words. Code links them to what the library holds. No model looks at a picture to decide whether it belongs: the pool is built from captions and facts the library already has.
Before any of that, a caption-dependent request pays for its own window: the run measures caption
coverage over the request's dates and, if anything is missing, warns with a count and an estimate,
prepares it the way prepare does, then answers. When the place is one the metadata settles alone
(home, home at the time, near home), only the pictures inside it are prepared; named places, printed
text and trips keep the whole window. A dry run shows the same warning and prepares nothing.
- Reading. The model splits the sentence into who, when, where and what. It can only pick phrases your sentence contains, and it does it three times in three orders. A word counts where two answers agree. A part you said nothing about sets no filter: say nothing about where and the film is from anywhere. A reply has to match the requested fields to vote. An unreadable reply gets one retry; fewer than two valid readings stops the request.
- Who. Grammar and your people registry, no model:
- "I", "me" and "my" are you, used for your age and your homes, never as a face in the photo (you are usually the one holding the phone: "the cars I drove" has no face in it).
- "We" and "our" are you and the people whose registered role is partner, not the whole household: name the people when you mean a different group.
- A name or a role from your people registry means that face, as Immich recognised it. A role word reaches the role it belongs to: "my son" or "my daughter" is the person registered as your child.
- A plural word for people ("friends", "kids") asks for company: a caption naming people, or the specific kind when the word is specific (children, teens, performers). A word list covers this in all 14 supported languages, because a dictionary lookup misses irregular plurals like "humans".
- "No", "without" and each language's own negation word turn that company into an absence instead of a requirement: "landscapes, no humans" drops any picture with a face or a caption subject who is a person, "sans les enfants" drops children instead of requiring them. A negated name ("without Alex") drops that person's own pictures the same way. The audience words carry the case a negation puts them in: Russian "без публики", "без толпы", "без зрителей" and Polish "bez publiczności" match. Each clause of the sentence is read for its own negation, so "with the kids, no rain" keeps the kids filter and a double negation ("not without the kids") cancels back to a requirement.
- "Only" narrows company to the kind named and excludes everyone else of that company: "only the performers" keeps musicians, singers, dancers and the rest of that cast, and drops a caption that also names another specific kind. The audience is the one exception for performers: a stage shot whose caption names the crowd too stays, and a shot of the audience alone never matched the performers in the first place.
- When. Years you write are pattern work. An age ("in our 20s") is read by the model as numbers, and the calendar is arithmetic from the birth date in your people registry. If a phrase such as "foggy days" supplies no dates, its visible modifier still filters the captions. A date with no end ("the birth of my son", from your people registry) is asked two questions: one occasion or many, and whether that occasion lasted one day. One occasion of one day ends on its own day before anything is prepared, so only that day's pictures are captioned and read. Many occasions, or one that lasted longer ("our honeymoon"), keep every day since.
- Where. The model votes one place per phrase among the options code builds: anywhere, a home, near home, away on a trip. A phrase that only repeats the subject ("at the park") stays the subject, not a place. One particular place at home has to be proven by GPS.
- What. The head noun of each phrase ("bread" in "bread making", "park" in "at the park with
kids"). When that head only frames an occasion, the modifier carries the subject instead:
"rainy weekends" offers "rainy" and WordNet's "rain"; "snowy days" offers "snowy" and "snow".
Generic heads are WordNet time periods plus
day,walk,trip,momentandtime. Their plural forms follow the same rule. A picture phrase such as "rainy pictures" also keeps its modifier; measured qualities such as "best blurry pictures" still use the facts filter. "Best" does not become a subject just because the dictionary also lists it as a noun. Concrete heads such as "cat" in "black cat" keep their meaning. The trace names the modifier and the head it replaced. Captions are written in English, so a subject word no caption uses ("gâteaux", "plage") is first named in English by the model. WordNet adds the other everyday names and the thing's own kinds and parts, and the model votes which one is the main subject. For a thing, the model also picks the caption subjects that name it too ("kitten" for a cat); a place or an activity keeps WordNet's names only, since the model offers the setting around it ("landscape" for a beach). A thing has to be what the caption is about ("a loaf of bread on a rack" yes, "a toddler holding toast" no). A place, a scene or something people do counts anywhere in the caption, since captions put the people first ("kids playing in a park"). A name the model added only counts as what the caption is about: a scenery film takes "a sandy beach under a blue sky", not "a toy bucket on the beach". A word printed in the photos (a club's name on the jerseys, read by Immich's OCR) vouches for the whole episode it appears in. Only what follows "not", "no", "without" or "except" is left out. - What the library measures. Some words link straight to fields the library holds, no
caption involved:
- a country, region or city Immich names on your photos ("I love <a country>");
- the kind of picture ("screenshots", "documents", "maps", "videos");
- "blurry": below the editor's own sharpness line (the softest 10 % of your library);
- how often a face appears ("people with more than 35 pictures");
- "farthest from home": the trip whose middle photo is farthest from where you lived then;
- "first" or "last" picture of each person. The first is when they show up for good, never before their birth date, so a scanned childhood photo or a wrong face match stays out.
- The film. The pool goes whole to the editor, filmed like an album whose written subject is your sentence: stories, standing, the family-viewing check, duplicates and length all apply. Forwarded pictures are kept (a club's photos tend to arrive through a group chat). One occasion of one day ("the birth of my son") goes to the special-day film instead. A season, month, year or range you wrote is filmed whole. An empty pool makes no film, with one exception: an occasion your sentence dates to one day ("the birth of my son", dated by your people registry) is still that day's film when the day has pictures, since no caption has to say "birth". The trace's FILM line says so. What the sentence excluded travels with it as a hard rule of the editorial brief, so the run's report can name a shot that still showed it, not only filter the pool by it.
Reading the trace
Every run prints how it read you before anything else runs, one line per decision. A real dry run:
"At the park with kids"
READING the model split your words 3 times; a word counts where 2 answers agree -> who: kids; where: the park; what: at the park | kids
WHO your words "kids" -> children must be in the photos, no one in particular (a plural word for children; a caption naming children shows it)
WHEN nothing said -> any time (no time words, years or people to date)
WHERE your words "the park" -> anywhere (nothing beyond the subject's own nouns)
WHAT your words "at the park | kids" -> subject words park (...)
POOL library 76367 -> kind of picture 74466 -> subject 1197 -> company 457
VERDICT possible: 457 pictures in the pool; ...
FILM the engine films the pool as an album whose written subject is your words: 457 pictures (the pool is the film's whole reach)
The opening title's language follows title_screens.locale, the film's own setting, not the
language you typed the sentence in: an English sentence in a French-titled film still gets a
French title.
POOL is the funnel: how many pictures were left after each filter. When a film is wrong, this
line usually says where. A subject count of 3 means the captioner never writes your word. A count
that barely moves means the filter did nothing.
VERDICT is one of four:
| Verdict | Meaning | What happens |
|---|---|---|
| possible | 12 pictures or more in the pool | the film is made |
| thin | fewer than 12 | a short film is made, and the run says why |
| not possible | the pool is empty, on a library the request's window has already read | no film; the trace names the filter that emptied it |
| needs preparation | a caption-dependent request's window has not been read yet | the run warns, prepares the window, then answers with one of the other three |
A computed selection ("the first picture of each person") is never called thin, however short.
The run does not pad a request it cannot show with something else. needs preparation is never
reported as not possible: an unread window is not known to be empty, only unread.
Which rules would drop pictures
The pool is what your words mean. The editor then refuses some pool pictures on rules that protect
every film: a screenshot, a second file of a picture it already has, a picture held for review. A
dry run asks those rules about the pool before anything renders, and prints them as RULES right
after POOL:
RULES 451 of 457 pictures pass the rules checked before cutting
another file of the same picture: 3 (the full-size file of the same picture plays instead) id-6c1f0e2a9b4d7e31, id-...
screens and documents: 1 (a screenshot, a screen or a document is never a source) id-...
held for review: 2 (a detector or you marked it never_auto; only your clearance lifts it) id-...
decided while cutting: who sees it (...); look-alikes (...); capture spacing (...)
not applied to a film you asked for: provenance (...); standing (...)
These are the run's own checks, called on the same banked facts, so the preview and the film
agree. It reads the pool's pictures from Immich first, like the film run does: one search call
per 1,000 pictures between the pool's first and last date, never one call per picture. Each picture
counts once, under the first rule that drops it. The ids are hashed the way
report hashes them: the terminal never prints a real one.
What it checks, in the run's order:
| Rule | Drops |
|---|---|
| hidden in Immich, a film this app made, long video | archived, locked or deleted pictures, uploaded memories, recordings over the source cap |
| another file of the same picture | the smaller copy (a shared album's downscale, a chat's re-send) |
| screens and documents | screenshots, screens, documents, by the document head, the screen head, the caption or a phone's screen size |
| held for review | a never_auto flag on the picture or its Live Photo clip, until you clear it |
| close-up of a face part, medical care, sheet of identical portraits | pictures that need an explanation |
| video frames miss the subject | a video whose sampled frames mostly miss the subject, unless starred |
Three rules depend on the shots around a picture or on a model reading, so they are only known while cutting and are named without a count: who sees it (the reader checks each chosen shot's caption for your sharing level; a detector hold keeps a picture out of shareable films), look-alikes (a shot repeating one already in the cut) and capture spacing (two shots of one moment within five minutes). Two rules do not apply to a film you asked for, and the block says so: provenance (forwarded and saved pictures stay) and standing (a pool picture stands on your subject whatever its score).
A picture prepare has not read yet has nothing for the rules to read, so it is not counted as
passing: the first line then reads "17 of 18 prepared pictures pass ...; 12 not prepared yet". The
run reads those pictures before applying the rules. There is no switch to turn a rule
off. Held pictures come back one at a time, when you clear them on the pool page.
With --dry-run the command stops after the trace and the rule preview. The model's answers are
banked in the store, so asking the same question twice does not ask the model twice. Translation
timing is on Measure your setup.
What it can read
A request only finds what the library already says. Captions name subjects and activities, Immich's metadata names people, dates and places, and OCR names printed words. A concept none of them names can't be found, however you phrase it.
- Captions decide the subject. If the captioner never writes your word, the pool comes back thin or empty. Company ("with kids") is read from captions too, not from the faces' ages.
- Occasions need a date or a caption. A wedding the captions never call a wedding can't be found; a birth works, dated by the people registry.
- Words have several senses. "Races" finds running races AND track days. Say which one you mean.
- WordNet is the 2006 dictionary. It has no "screenshot" or "app", and some first senses are odd ("dashboard" is a carriage's mud panel).
- The model votes on some parts: the main subject, the place, the kind of film. Three votes soften a bad answer; they don't remove it.
- The editor still decides the film. A list of one picture per person is cut to the film's length, so not every person is guaranteed a shot, and blurry pictures still lose to sharp ones.
- English reads best. Other languages go through the model and can lose meaning.
Wrong results help: report them as below, and check the issue tracker for what's known.
When a result is bad
Mark it on the run, then paste the report into an issue:
immich-memories report <run-id> --wrong <asset-id> --wrong <asset-id> --missing "the red bike"
Each photo marked wrong gets the filter that let it into the pool and whether the editor picked it. The missing words are checked against the run: did the reading keep them, were they offered or picked as the subject, and does any caption in the pool say them.
In the web UI, open the film's run and use Copy report, or Download report for the ZIP with the whole log. Same report, same redaction. A film made from the web UI is cut first, then rendered as a run of its own: the report of either run carries your sentence.
The report carries the sentence, the trace, the translation as data (the same JSON --ask-trace
writes), the funnel, the rule preview, the editor's picks and the run's logs. Before anything is copied:
- IDs become hashes that only match inside that one report;
- the names of people the sentence linked become their roles ("the owner's son"), and a birth
date the trace dated from becomes
[private]; - place names become "area A", and words read by OCR become "text-1", in your sentence too;
- captions stay out unless you pass
--include-flagged-captions, and then only those of the photos you marked. Pictures never.
The rest of the report and its privacy rules: report and Privacy.