We translate articles up to fifty thousand characters long. Most language models we use for this work cannot hold an article that long in their attention with the same care they bring to the first thousand characters. So we cut the article into chunks of around five thousand characters and translate each chunk in sequence. This sounds like a solved problem. It is not.
The first chunk is fine. The second chunk is fine. The third chunk is where the article quietly starts becoming a different article.
What forgets what
By “the third chunk,” we mean any chunk far enough from the start that the model is operating without a clear memory of what it did earlier. We pass the source for the current chunk and instructions for the translation. We do not pass a copy of the previous chunks. There is a reason: we have tried, and it costs context, slows the translation, and yields surprisingly small consistency gains. The model can read its own earlier work and still pick a different Ukrainian word for the same English term, because nothing in the system makes the earlier choice authoritative.
The kinds of things that drift across chunks:
- A theological term rendered one way in chunk one and another way in chunk three. We use виправдання for justification in chunk one and виправдовування in chunk three, and a reader who notices will silently wonder whether the author meant two different things.
- A Scripture book name. We use Ів. for John in chunk one and Євангелія від Івана in chunk four, because the surrounding sentence happened to call for a longer noun phrase. Same book. Different signal to the reader.
- Tone. A pastoral register set up in chunk one quietly turns academic in chunk five because the source itself drifted at a section break, and the model dutifully followed.
- Discourse markers. “Furthermore” gets translated three different ways across an article, none of them wrong, but the cumulative effect is a translation that does not feel written by one person.
None of these are wrong in the way a mistranslation is wrong. They are not flagged by reviewers as errors. They are not caught by spot checks. They show up as a quiet drag on the reader, who finishes the article slightly more tired than they should be, and cannot say why.
What does not work
The first thing we tried was a bigger context window. We extended the chunk to ten thousand characters, then fifteen. The drift was smaller, not absent. The model still picked a fresh rendering of виправдання every few thousand characters of its own output. Long contexts attenuate the influence of any single earlier decision; they do not fix the problem.
The second thing we tried was stronger instructions. We told the model to be consistent. We told it that consistency mattered. We listed terms it had to render uniformly. These instructions live in the system prompt and apply across all chunks, but they do not survive contact with a sentence where two consistent renderings are both plausible. The model picks whichever fits the local sentence rhythm best, which is a smart thing to do for the local sentence and the wrong thing to do for the article.
The third thing we tried was a single huge prompt: the full source, the full instructions, all in one pass. This works on short articles. On long articles it produces a translation that is internally consistent and externally truncated. The model runs out of output budget halfway through and stops.
What works
We extract a glossary before we translate.
Before chunk one, we read the full source and pull out the theological terms, Scripture references, proper nouns, and any recurring metaphor we expect to appear more than once. We assign a Ukrainian rendering for each. This glossary becomes part of the translation prompt for every chunk. Justification is виправдання here, throughout this article, no matter how many chunks we run. John is Ів. We pick once, at the article level, and the chunk-level decisions inherit.
The glossary is small enough to fit in every chunk’s prompt without meaningfully slowing the work. It is also small enough to write by hand if the model produces something we disagree with. We can edit one line and re-run the translation from any chunk forward. The consistency we get is not a property of the model. It is a property of the structure we built around the model.
The chunks still drift on the things the glossary does not cover. Tone drift, discourse markers, and sentence rhythm are harder to constrain this way, and we mostly catch them in review. But the visible consistency problems, the ones that make a reader notice, are mostly handled.
What this generalizes to
We used to think of long-form generation as a memory problem. The model forgets. If only it remembered better, the work would be consistent.
We have come to think the framing is wrong. The model does not need to remember; it needs to be told. A glossary is not a memory aid. It is an externalization of the choices that should be stable, in a form that survives any chunk boundary. The model does not need to know what it said earlier. It needs to know what the article will say throughout.
This pattern shows up wherever a long-running process produces output in pieces. A documentation site written paragraph by paragraph gets the same problem at the level of voice. A code generator producing functions one at a time gets it at the level of naming conventions. The fix is the same shape: decide once, write it down, pass the decision forward as input, not as expected memory.
The article we ship at the end is consistent because the model never had to remember. We never asked it to.