Context Window

Webisoft AI team Updated 2 min read

A context window is the maximum amount of text an AI model can consider at one time, your instructions, the conversation history, any pasted or attached documents, and the response it is generating, measured in tokens. It works like the model’s working memory: everything inside the window is visible to the model when it answers; everything outside it simply doesn’t exist for that request.

Context windows vary widely by model, from a few thousand tokens to millions. A larger window lets you feed in whole contracts, codebases, or meeting archives in one request. But bigger isn’t automatically better: long inputs cost more (pricing is per token), respond more slowly, and models can pay less attention to material buried in the middle of a very long prompt, so relevance still beats raw volume.

The context window also explains a familiar chat behavior: in a long conversation, the earliest messages eventually fall outside the window, and the model “forgets” what you agreed on an hour ago. It isn’t being careless, that text is no longer in front of it.

Why it matters at work

The context window determines what an AI tool can actually do with your material in a single pass. It decides whether a 200-page contract can be reviewed whole or must be chunked, why a long support thread loses the plot, and why tools built on retrieval-augmented generation exist at all, RAG is largely a way to fit only the relevant slice of a huge knowledge base into a limited window. Anyone scoping an AI workflow should check the window size against their real documents before assuming it will fit.

A work example

A legal-ops team evaluating contract-review tools measures a typical MSA at roughly 40,000 tokens, then shortlists only models whose context window fits the full document plus instructions, avoiding a tool that would silently truncate the indemnification clauses at the end.

Get one practical AI-at-work idea in your inbox each week, subscribe to the AI Work+ newsletter.

FAQ

What happens when a conversation exceeds the context window? The model can no longer consider all of it at once. Depending on the app, older content is dropped or summarized, which is why long chats seem to forget earlier details.

Is a bigger context window always better? Not automatically. Larger windows let you include more material, but they cost more per request, and models can pay less attention to details buried in the middle of very long inputs. Relevance still beats volume.