Take a 120-page office lease. Cut it into paragraphs, shuffle them, and paste the pile into an AI model. Then ask where the break clause is. According to the most thorough study of long-document behaviour yet published, the model will find it more reliably in the shuffled pile than in the lease as drafted.
That is the most useful thing to know about the context window explained for lawyers: the number on the vendor’s slide (200,000 tokens, one million tokens) tells you how much you can upload. It says nothing about how much the model will read well. The two diverge early, and fastest on exactly the documents lawyers care about: long, coherent, cross-referenced ones. The question is not “can I upload the whole lease?” (usually yes) but “what happens to page 74 when I do?” (usually nothing good).
What a context window is, and what it is not
An LLM has two kinds of memory. Its parameters, set during training, hold what Andrej Karpathy calls “a lossy zip file of internet knowledge”. The context window is different. In Anthropic’s documentation it “refers to all the text a language model can reference when generating a response, including the response itself”; it “represents a ‘working memory’ for the model”, distinct from “the large corpus of data the language model was trained on”.
Everything counts: the system prompt, every earlier message, uploaded documents, tool results and, on reasoning models, the model’s own “thinking” tokens. The window is per conversation, which is why Ethan Mollick calls the AI “an infinitely patient new coworker who forgets everything you tell them each new conversation”. A document in the window is read; one in the training data is remembered, badly (how LLMs work explains why).
Pages to tokens: a conversion table for contracts, transcripts and bundles
Models count tokens, not pages: sub-word fragments made by splitting text into its most frequent chunks. There is no exact page-to-token rate, so the figures below are planning assumptions, not measurements: a dense legal page at about 500 words, and English legal text at roughly four tokens for every three words. Check anything that matters in a tokeniser.
| Document | Typical length | Words (approx.) | Tokens (approx., English) |
|---|---|---|---|
| Retail lease (Bryckel) | 40–80 pages | 20,000–40,000 | 27,000–53,000 |
| Office lease (Bryckel) | 60–120 pages | 30,000–60,000 | 40,000–80,000 |
| Ground lease (Bryckel) | 100+ pages | 50,000+ | 67,000+ |
| Bake-off document set (Vals) | 100–900 pages | 50,000–450,000 | 67,000–600,000 |
| Small data room (hypothetical: 50 contracts × 40 pages) | 2,000 pages | ~1,000,000 | ~1,330,000 |
A modest data room already exceeds the largest window on the market; the rows above it fit, but are past the point where accuracy slips.
Current limits in September 2026
| Tool | Limit (from the vendor’s own page or announcement) |
|---|---|
| Claude (Opus 5, Opus 4.6–4.8, Sonnet 4.6, Sonnet 5 and newer) | 1M-token context, up to 128k output tokens, up to 600 images or PDF pages per request |
| Claude 200k models (e.g. Sonnet 4.5) | 200k tokens, up to 100 images or PDF pages per request |
| ChatGPT | Varies by model and plan and changes often; check OpenAI’s current documentation for the model you use |
| Gemini Notebook (formerly NotebookLM) | 50 sources per notebook free, 300 on Google AI Pro, 500–600 on Google AI Ultra |
| Harvey Vault | Up to 100,000 documents per project |
| CoCounsel Legal Tabular Analysis | Up to 10,000 documents × 100 questions |
Sources: Anthropic’s documentation and Google’s help page. The platform figures are not context windows at all; Vault reads 100,000 documents one at a time, as the last section explains.
Context rot: why accuracy falls long before the limit
In July 2025 Chroma published “Context Rot”, a test of 18 models including Claude Opus 4, GPT-4.1, o3 and Gemini 2.5 Pro: hide a relevant passage in a growing pile of text and ask about it. Four findings matter.
- “Model performance varies significantly as input length changes, even on simple tasks.” Every model degraded as input grew.
- The less the hidden passage resembled the question, the faster accuracy fell.
- Even one distractor, a passage that looks relevant but is not, hurt. A data room is made of distractors.
- On a benchmark of 306 prompts averaging about 113,000 tokens, a focused context beat the full context.
Anthropic’s documentation now says the same: “more context isn’t automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot.”
Chroma also found Claude models had “the lowest hallucination rates” with distractors present and “tend to abstain when uncertain”, while GPT models showed “the highest rates of hallucination”. A model that says “not found” is worth more than one that guesses, which is why every prompt below permits it.
Why a coherent contract is harder to search than a shuffled one
Chroma’s strangest result is the one to remember: “models perform worse when the haystack preserves a logical flow of ideas”. Shuffling the surrounding text helped all 18 models.
A well-drafted agreement is a web: a cap in clause 12 depends on a definition in clause 1, an exclusion in schedule 3 and a carve-out in clause 12.4. Coherent text gives the model a plausible story, and a plausible story lets it stop reading before the exception. Farrell Fritz described the failure in diligence: an IP assignment that looks broad until the definition of “Developed IP”, three layers deep, limits it to first-year improvements. A shuffled document offers no story, so the model has to look.
You cannot shuffle a lease before review. You can ask about one concept at a time, demand the operative words, and run a second pass at known trouble spots; see prompting long contracts.
Lost in the middle: where the model forgets
Position compounds length. Anthropic’s prompting guidance for inputs above about 20,000 tokens is to put the long material at the top and the question at the end: “Queries at the end can improve response quality by up to 30 percent in tests.” The cheapest fix is a quote-first wrapper: it forces the model to locate before it concludes and gives you references to check instead of confident prose. One question per run; a prompt that asks four things of an 80-page transcript does, in Clio’s words, “none of them particularly well”.
<document>
[paste the document, or attach it]
</document>
Question: [e.g. On what conditions can the tenant exercise the break right?]
Before answering, extract into <quotes></quotes> every passage from the document that bears on the question, each with its clause number and page. If no relevant passage exists, write <quotes>NONE</quotes> and stop.
Then answer using only those quotes, referring to them by number. Label any statement not traceable to a quote as INFERENCE.German and other languages cost more tokens
Tokenisers are trained mostly on English. Morphologically rich languages such as German need “substantially more tokens per word”, as the explainer at Sean Trott’s newsletter sets out. A Mietvertrag fills the window faster than the equivalent lease, hits context rot sooner and costs more per query on metered plans. The remedy is the same as for English, applied earlier: smaller chunks, one question at a time. Our guide on AI for legal translation covers the language side.
Practical rules: what to upload, what to chunk, what to leave out
Nico Kuhlmann of Hogan Lovells calls the context window the “digitale Handakte”, the digital case file you hand the model. Then the warning, in LTO:
Mehr Kontext ist nicht automatisch besser. Eine überladene Akte mit vielen irrelevanten Dokumenten kann die Qualität der Ergebnisse sogar verschlechtern. — Nico Kuhlmann, Hogan Lovells, LTO, 17 March 2026
With Chroma’s findings, that yields five rules: upload what the task needs, not the folder; one document, one question, one chat; documents first, question last, tags around each file; chunk by structure (clause groups, not page counts) and demand clause references; sweep for consistency at the end, because chunks disagree.
This is part [2 of 5] of a [commercial lease], covering clauses [8 to 14] (alienation, repair, insurance, break). Earlier parts are not in this conversation; do not assume their content.
Extract into a table with exactly these columns: Field | Extracted value | Clause reference (number and page) | Confidence (High/Medium/Low) | Note.
Fields: assignment conditions; subletting conditions; repair standard; insurance obligations; break dates; break conditions; break notice period and method; consequences of break for deposit and rent.
Write NOT IN THIS PART where a field is not covered here, rather than inferring it. At the end, list every defined term used here whose definition is elsewhere, and every cross-reference outside this part.Here are the extraction tables from all [five] parts of the same agreement: <part_1>...</part_1> ... <part_5>...</part_5>.
Do not re-analyse the agreement; check the tables against each other only. List: (1) every defined term with two different meanings or values; (2) every cross-reference no part contains; (3) every number, date or notice period that appears inconsistently; (4) every field marked NOT IN THIS PART in every part, which may mean it is missing altogether. Quote the conflicting entries side by side. Do not resolve them; I will.How the legal platforms work around the limit: Vault, Tabular Review, Tabular Analysis
Harvey Vault takes up to 100,000 documents per project; Legora’s Tabular Review puts documents in rows and questions in columns, every cell linked to its source; CoCounsel Legal’s Tabular Analysis runs across 10,000 documents and 100 questions. Harvey describes 800 emails becoming “4,000 data points” in minutes.
None of these is a bigger context window. Each is a way of never needing one: the platform retrieves one document at a time, asks the question against that small context, records the answer with its source, and moves on. That sidesteps context rot by design and inherits the weaknesses of retrieval instead, set out in our guide to RAG in legal research: the wrong passage can be pulled, and the same rule is applied to every contract, including the ones it does not fit. Harvey’s own framing is that the value “isn’t finding issues that a senior lawyer might miss, but in applying the same extraction rules consistently across all contracts”. Gemini Notebook does the same at small scale, answering only from loaded sources: “closed-universe analysis”, in Attorney at Work’s phrase.
The rule for choosing: under a few hundred pages, chunk it yourself in a general model on a no-training tier; a data room belongs in a table-based platform, following the due diligence data-room workflow. Either way, the table is a list of things to verify, not answers.
Where to go next: the AI glossary for lawyers defines token, context window and context rot in a line each, the fundamentals hub holds the rest, and every prompt above is in the prompt library. Chunking a real data-room folder, sweep included, is an exercise we do together in AI Lab for Lawyers, because knowing about context rot and having watched page 74 vanish are different things.
Frequently asked questions
How many pages can ChatGPT read at once?
It depends on the model and plan, and OpenAI changes the figures often, so check the current documentation rather than a blog. The more useful question is how much it reads well. Chroma's 2025 study found accuracy falls as input grows on every model tested, and a team building a lease tool found the GPT model it used could not process more than about 8,000 words. Treat a long upload as several shorter jobs.
Can I upload a 300-page contract to Claude?
On a current 1M-token Claude model, yes: Anthropic's documentation allows up to 600 PDF pages per request. Whether you should is a different question. The research shows recall degrades with length and that one irrelevant passage is enough to hurt accuracy. Upload the whole agreement for a first map, then work clause by clause with a quote-first instruction, and never paste client material into a consumer tier.
What is context rot?
Context rot is the observed decline in a model's accuracy and recall as the amount of text in its context window grows. Anthropic's own documentation now uses the term: more context is not automatically better. Chroma's July 2025 study of 18 models found performance varied significantly with input length even on simple tasks, that distractors made it worse, and that coherent text was harder to search than shuffled text.
Why does the AI miss things in the middle of a long document?
Two effects combine. Position matters: Anthropic's prompting guidance says that, for long inputs, putting the material at the top and the query at the end can improve response quality by up to 30 per cent in its tests. And length matters: Chroma found that the more text surrounds the relevant passage, and the less that passage resembles your question, the more often the model misses it. A quote-before-you-answer instruction and smaller chunks both help.
Is a bigger context window always better?
No. A bigger window lets you upload more; it does not make the model read more carefully. Chroma's study found focused context beat full context on a long-memory benchmark, and Hogan Lovells' Nico Kuhlmann warns that an overloaded digital file with many irrelevant documents can make results worse. Use the window for the documents the task needs and leave the rest out, or use a platform that reads documents one at a time.