Cohere’s Rerank 5 can trim the text you send to your LLM

Cohere has released Rerank 5, a new generation of its reranking models for enterprise search. The headline feature is Context Focus, which returns the most relevant passages inside each document so applications can send the LLM less text.

Cohere describes the release as “sharper search, with fewer tokens sent to your LLM”. A reranker sits between a search system and a language model: it reads the query and a list of candidate documents, then orders them by relevance so only the best ones reach the model.

Two variants and a passage-level option

Rerank 5 comes in two versions. rerank-v5.0-pro targets the highest ranking quality, especially for reasoning-heavy queries and long or domain-specific documents. Cohere says its largest gains are in reasoning-intensive retrieval and in finance, healthcare and scientific research. rerank-v5.0-fast is tuned for low latency and high throughput, for example in high-volume search or agents that run many searches per task. Cohere says Fast outperforms its previous-generation Rerank 4 Pro model.

Both models handle more than 100 languages and a 32k token context window. They can also rank semi-structured data: JSON, tables and other records formatted as YAML strings. Scores are described as calibrated, meaning they stay consistent across queries and domains, so a team can set a single threshold to decide which results go on to the LLM.

Context Focus is off by default. When enabled through a relevant_spans setting in the request, the same Rerank call returns, for each document, the start and end character offsets of relevant passages along with a score for each. Cohere’s example request sets a mode, a threshold of 0.3 and a max_spans of 10. Developers can pass just those passages to the LLM or highlight them in the interface to show why a result matched.

Who can use it, and how to switch

Rerank 5 is available through the Rerank API, Microsoft Foundry and Amazon SageMaker, each with Pro and Fast listings. Cohere’s Model Vault offers it for single-tenant deployment, and companies that want it in their own VPC or on-premises are directed to Cohere’s sales team.

Upgrading is a one-line change. Because a reranker reads raw text at query time, moving from an earlier Rerank model means changing the model parameter to rerank-v5.0-pro or rerank-v5.0-fast. Unlike an embedding model swap, nothing needs to be re-indexed or re-embedded.

Passage extraction changes the cost equation

The announcement gives no prices for the Rerank API; it only points to Standard Vault pricing for single-tenant use. For context, third-party pricing guides such as Puter’s Cohere breakdown list the previous Rerank 4 generation at $2.00 per 1,000 searches for Fast and $2.50 for Pro. Those figures come from a reseller guide rather than Cohere, and Rerank 5 pricing may differ.

The same guide explains how billing works: one search is a single query ranked against up to 100 documents, and any document over 500 tokens, counting the query, is split into chunks that each count as a separate document. That per-search model means reranking is cheap next to LLM calls, which is where Context Focus matters. LLM providers bill per token, and our AI API pricing tracker shows what each major model costs per million tokens. If a retrieval pipeline passes a few relevant passages instead of whole documents, the saving lands on that larger line item, not on the reranking fee.

Whether the saving is real depends on details the announcement leaves out. Cohere publishes no benchmark figures on the page, so the claims of top quality, gains in finance and healthcare, and Fast beating Rerank 4 Pro are the company’s own and not independently verified. There is also no information on how much extra latency Context Focus adds, how accurate the character spans are on messy documents, or whether it carries a surcharge. Teams running agents that call search many times per task may find the Fast variant plus a calibrated threshold the more practical combination, but that is a question for their own evaluation data.

One structural point is worth noting: calibrated scores are only useful if the threshold holds up on a team’s own content. Cohere says scores are consistent across domains, yet any production system will still need to check where a cutoff of, say, 0.3 lets irrelevant text through or drops useful passages.