How ChatGPT Search Actually Works: The Pipeline Behind Every Answer

Most explanations of AI search stop at "it reads the web and summarizes it." That description is close enough to feel true and wrong enough to make you optimize for the wrong things. When a user types a question into ChatGPT and gets back three paragraphs with four little citation chips, a specific sequence of machinery ran between the keystroke and the answer. Each stage in that sequence throws some content away. If you understand which stage discards you, you know what to fix.

This is a reference for that pipeline. It covers ChatGPT's search path in detail, contrasts it with how Perplexity and vector-store systems differ, and maps every stage to the four points where visibility is won or lost: Retrievability, Candidacy, Citation, and Attribution. Implementations change, and no two vendors do this identically. The stages themselves have stayed stable across every retrieval-augmented system built since 2023, so the map outlasts the specifics.

Stage one: the model decides whether to search at all

The first thing that happens is a judgment call the model makes on its own. Not every prompt triggers a web search. Ask ChatGPT to rewrite a paragraph or explain a sorting algorithm and it answers from parameters, no retrieval involved. Ask it about a company's current pricing, a recent release, or anything time-sensitive, and it routes the query to the search tool.

This decision matters more than it looks. If the model answers a brand-relevant question from memory instead of searching, your carefully optimized pages never enter the running. Whatever the model absorbed during training is the whole answer, and that snapshot is months old and impossible to edit after the fact. The practical implication for anyone tracking visibility: a share of the queries you care about will be settled without any retrieval step, and no amount of on-page work touches those. You can only influence the answers where a search actually fires.

Stage two: fan-out, or the searches you never see

Here is where the real surprise lives. A single user question rarely becomes a single search. The model rewrites the prompt into several sub-queries and runs them in parallel. This is called query fan-out.

Suppose someone asks, "what's the best CRM for a small agency." The model might expand that into separate searches for best CRM small business, CRM for agencies pricing, and a few phrasings you would never predict from the original wording. Each sub-query hits the search backend independently and pulls its own set of results. The answer the user reads is stitched together from across all of them.

The consequence for visibility is direct and often missed. You are not competing for one query, you are competing across a spray of reformulations you never get to see. A page optimized to rank for the user's exact phrasing can miss every sub-query the model actually issued. This is why keyword-level thinking breaks down here. The unit of competition moved from the visible query to the invisible set of machine-generated ones, and covering that set is what shareof.ai calls Prompt-Space Coverage: the real space of prompts customers trigger, most of which never appear in any keyword tool. Fan-out is the first place a strong page quietly loses. It answered the question the user typed and stayed silent on the three the machine asked.

Stage three: retrieval, the gate everything passes through

Each sub-query now goes to a retrieval backend, and this is the single most important stage for visibility. ChatGPT uses a provider-managed web retrieval system whose implementation can change over time. Your page has to be in that index, and it has to surface in the top results for at least one sub-query, or nothing downstream can happen. The rest of the pipeline can only work with what retrieval hands it.

This is the gate. Everything about your content's brilliance is irrelevant if it does not clear this step. Retrieval is where being crawlable, indexed, and topically matched to the fanned-out queries converts into a seat at the table. Miss it and you are invisible for that answer, no matter how authoritative the page.

Vendors differ sharply on how they fill the index behind this gate. ChatGPT uses provider-managed web retrieval, so classic index coverage carries over. Perplexity runs its own crawler (PerplexityBot) alongside third-party indexes, which means its candidate pool can include and exclude different pages than Bing's. Some enterprise and vertical systems skip the open web entirely and retrieve from a vector store: a private, pre-embedded corpus where "retrieval" means nearest-neighbor search over document embeddings rather than a web query. Those three backends reward different work. A vector-store system rewards how you chunk and embed a known document set; a web-index system rewards being crawlable and linked across the open web.

shareof.ai measures the outcome of this stage as Share of Retrieval: the fraction of candidate sets your content appears in, upstream of whether you are ever cited. It is the leading indicator, because it moves before citations do and it explains them. Two pages can be equally good, and the one that shows up in more candidate sets wins more answers for reasons that have nothing to do with quality. The gap between the pages that could have answered a query and the smaller set retrieval actually pulled is the Retrieval Gap, and closing it is upstream, technical work: crawl access, index coverage, and matching the language of the fanned-out queries rather than the language of your marketing.

Stage four: reranking scores chunks, not pages

Retrieval is tuned for recall. It casts wide and pulls back more candidates than the model can possibly read, so a second, sharper model reorders them. This is the cross-encoder reranker, and it introduces a distinction that changes how you should write.

A first-stage retriever typically scores your query and a document separately, then compares the two vectors. Fast, cheap, approximate. A cross-encoder does something more expensive: it reads the query and a passage together, in one pass, and outputs a relevance score for that specific pairing. Because it sees both at once, it catches nuance the first stage missed, and because it is slow, it only runs on the shortlist retrieval already produced.

The word to sit with is passage. The reranker does not score your page. It scores chunks of it. Your content gets split into segments, often a few hundred tokens each, and each segment is judged on its own merit against the sub-query. A page can be excellent as a whole and still lose, because the reranker only cares whether one self-contained passage answers the exact question. Sprawling pages where the answer is smeared across ten scrolls of context underperform tight ones where a single 50-to-150-word block stands on its own. This is Candidacy in the visibility funnel. The question is no longer whether your page is good. It is whether a specific chunk of it survives the rerank. Write so that individual passages can be lifted out and still make sense, because that is literally the operation being performed on them.

Stage five: context assembly, where the budget bites

The reranked passages now get packed into the model's context window, and the constraint here is brutal and underappreciated. The model does not read everything retrieval found. It reads what fits.

Only a few thousand tokens of retrieved material typically make it into context. Retrieval might surface dozens of candidate pages across all the sub-queries combined, and the overwhelming majority never reach the model at all. Assembly picks the top-scoring chunks until the budget is spent, then stops. Everything below that cutoff was retrieved, reranked, and discarded before the model saw a word of it.

This is the compounding filter. To influence the answer, your chunk has to clear retrieval, then survive reranking, then rank high enough to fit inside a budget that discards most of what got that far. Each stage narrows the field, and the narrowing multiplies. A page can win the first two rounds and still lose here simply because six other chunks scored higher and there was no room left. It is worth internalizing how small the surviving set is. The whole open web goes in one end, and a handful of passages totaling a few thousand tokens come out the other.

Stage six: synthesis, where the model writes the answer

With its context assembled, the model finally does the thing everyone pictures when they imagine AI search: it writes. It reads the passages it was handed and generates a fluent answer grounded in them.

Two things about this stage shape what you can control. First, the model is synthesizing across multiple sources at once, blending claims from several passages into single sentences, which means your content competes for influence on the answer even when it made it into context. A passage that states a claim crisply, with a number or a direct statement, tends to shape the wording more than one that buries the same fact in hedged prose. The Princeton and IIT Delhi GEO study (Aggarwal et al., 2023) found exactly this pattern across roughly 10,000 queries: adding statistics, citations, and direct quotations raised a source's visibility in generated answers, with gains reaching around 40 percent on their metric, while keyword stuffing did nothing or slightly hurt. Lower-ranked sources gained the most, some by over 100 percent, so this stage is where a weaker page can still punch up.

Second, synthesis is where the model can quietly misattribute or blend, stating something drawn from your passage without connecting it to you. Which brings us to the last stage, and the one most people conflate with everything before it.

Stage seven: attribution is a separate step from retrieval

The citations you see, those little chips linking to sources, are attached during or after generation, and they follow their own logic. A source being in context does not guarantee it gets cited. The model can absorb a fact from your passage, use it in the answer, and hang the citation on a different source or none at all.

This is the distinction that trips up almost everyone measuring AI visibility: attribution is not the same as retrieval. Your content can clear every gate, reach the context window, materially shape the answer, and still go uncited. The reader sees a competitor's link under a sentence your page supplied. In the funnel, Citation is whether a source in context gets a visible reference, and Attribution is whether that reference correctly points to you rather than to an aggregator that happened to restate your claim. Third-party pages frequently contribute to brand mentions in AI answers, though the ratio varies by prompt set and engine: the page that gets the citation is often not the page that originated the fact. Your Citation Surface, the full set of third-party pages where an AI can find and credit you, matters as much as your own domain.

Reading the pipeline as a funnel

Line the stages up and the four-part funnel falls out of the machinery itself. Retrievability is whether you clear stage three, the index gate. Candidacy is whether a chunk of yours survives the stage-four rerank and the stage-five budget. Citation is whether a passage in context earns a visible reference during stage seven. Attribution is whether that reference lands on you. Each stage discards a different slice of content for a different reason, and a page can die at any one of them while looking perfectly optimized for the others.

The practical upshot is that a single "are we cited" number tells you almost nothing about why. A brand can have flawless pages and near-zero visibility because it never clears retrieval, and another can be cited constantly off a handful of third-party mentions it does not control. Measuring one stage in isolation hides the cause. This is the measurement problem shareof.ai exists to solve: it tracks Share of Retrieval and the full funnel across ChatGPT, Claude, Gemini, and Perplexity, so you can see which specific gate is losing you the answer instead of guessing from the final citation count.

A note on measurement

Everything above describes a mechanism, not a fixed product. Vendors rerank differently, tune their fan-out differently, and change their retrieval backends without announcing it. Treat any specific number here as directional and test it against your own category, because the one reliable finding across every study is that effects are domain-dependent. What holds is the shape of the pipeline: a decision to search, a fan-out into hidden sub-queries, a retrieval gate, a chunk-level rerank, a tight context budget, synthesis, and a separate attribution step that can leave you out. Optimize for the stage that is actually losing you the answer, and to know which one that is, you have to watch the whole sequence rather than the citation at the end of it.

Sources and further reading

• Aggarwal et al., "GEO: Generative Engine Optimization" (2023), arXiv:2311.09735. GEO-Bench methodology and the visibility effects of citations, statistics, and quotations.
• Background on retrieval-augmented generation, cross-encoder reranking, and the recall-versus-precision split between first-stage retrieval and reranking.
• shareof.ai for measuring Share of Retrieval and the full AI Visibility Funnel across ChatGPT, Claude, Gemini, and Perplexity.

Primary sources and evidence

• GEO: Generative Engine Optimization (Aggarwal et al.)↗ — the original paper and benchmark.
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks↗ — the foundational RAG paper.
• Web search in the OpenAI API↗ — current search-tool behavior and citations.
• OpenAI web crawlers↗ — official user-agent controls for search and training.

Continue through the system

• How AI Models Actually Choose Which Brands to Cite
• Measure Your Own Share of Retrieval With the Model APIs
• Reading Your AI Crawler Logs: GPTBot, ClaudeBot, PerplexityBot and Friends

Run the practical layer with the free AI visibility scan, explore the AI search library, or see the AI visibility benchmarks.