Chain-of-Thought

Chain-of-thought is the practice of having an AI model work through a problem step by step, writing out intermediate reasoning before committing to a final answer, rather than jumping straight to a conclusion. Originally a prompt-engineering trick (“think step by step”), it improves accuracy on math, logic, planning, and other multi-step problems because a large language model generates one token at a time: writing the steps gives it room to compute, where a one-shot answer forces a guess.

The idea has since moved from prompt to product. Modern “reasoning models” (OpenAI’s o-series, Anthropic’s extended thinking, Google’s Gemini thinking modes, DeepSeek-R1, and others) are trained to produce a long internal chain of thought before answering, you often see it summarized as the model “thinking.” The trade-off is real: reasoning takes more tokens, which means more cost and slower answers, so it’s worth paying for on hard problems and wasteful on simple ones.

One caution: a chain of thought reads like an explanation, but it isn’t a guaranteed window into what the model actually computed. Models can produce plausible-looking steps that arrive at a wrong answer, or correct answers with post-hoc rationalizations, so visible reasoning helps you check the work, not blindly trust it.

Why it matters at work

Chain-of-thought is the difference between an AI that can eyeball a problem and one that can work a problem. Anything with multiple dependent steps, reconciling figures across reports, checking a request against layered policy rules, debugging a workflow, comparing vendor proposals on many criteria, benefits. It also makes review practical: an analyst can scan the model’s steps and catch the exact point where it misread a number, instead of staring at a bare wrong total.

A work example

An analyst asks a model to determine whether a proposed discount complies with the pricing policy; with step-by-step reasoning requested, the model walks through each policy condition in turn and flags the one clause the discount violates, a check the one-line answer had gotten wrong.

Get one practical AI-at-work idea in your inbox each week, subscribe to the AI Work+ newsletter.

FAQ

Does chain-of-thought actually make AI more accurate? It often helps on multi-step reasoning such as math, logic, and planning, because the model works through intermediate steps instead of jumping to an answer. It adds little on simple lookups and makes responses longer.

Is the chain of thought a true record of how the model reasoned? Not reliably. The written steps are generated text, and a model can produce a plausible-looking chain that does not reflect what actually drove its answer, so treat it as a useful aid rather than proof.