Salesforce’s research agents tuned an embedding model in 80 hours, not weeks

Salesforce AI Research has published SFR-AutoR&D, a set of agents that run the research loop on their own. Salesforce says one cycle took about 80 hours and raised an embedding model’s retrieval score by 6.13 points.

Given a goal, a system to work on and a compute budget, the agents form hypotheses, implement methods and run controlled experiments against baselines. According to Salesforce, a change to a codebase is kept only when the evidence supports it and it proves accurate and safe.

Three agents, one shared GPU backbone

The system has three parts. AutoR&D-Engineer looks for algorithmic and systems improvements in existing code. TrainForge explores new training methods and scores candidates on held-out benchmarks that the system never trains on. AutoInfra is the training backbone that runs both agents’ experiments across shared GPU capacity.

The first production result is SFR-Embedding V3, the retrieval model behind Salesforce’s Data Cloud. Salesforce then applied the agents to the open-source EmbeddingGemma 2 and produced SFR-Embedding v4, which it calls measurably better in quality and latency. On 10 public retrieval tasks in the MTEB benchmark, nDCG@10 rose by 6.13 points. That metric rewards placing relevant results near the top of the first ten.

A second session of roughly nine hours targeted speed and cost. Salesforce reports 36% higher throughput and 17% lower latency with almost no loss of quality. It also says the agents found further gains in three open-source projects: pandas, hnswlib and USearch, each packaged with benchmark evidence.

Salesforce says the full cycle took just over three days, where researchers traditionally needed several weeks. Shafiq Joty, senior research director at Salesforce AI Research, put it this way: “We used to count progress in weeks of human work. Now we count it in hours of compute.”

What the project site adds on open-source gains

The project site gives sharper figures than the announcement. For pandas it lists up to 1.84x faster reductions on nullable data types. For hnswlib it reports 4.1% more queries per second (median across three index builds), and for USearch 17.5% more on a single core with 1 million vectors. TrainForge gains on held-out benchmarks are listed as up to 11 points for Qwen3-4B on hard math, up to 12 for Qwen3.5-35B-A3B on code and up to 14 for GPT-OSS-20B on long-horizon research.

The project page does not describe the baselines in detail or state a license, so the size of these gains is hard to judge from outside.

Who this affects in practice

For Salesforce customers, the visible effect is indirect: a better retrieval model underneath Data Cloud, with no new product to buy. The announcement gives no pricing and does not say when, or whether, SFR-Embedding v4 reaches customers. It says only that any trained model must meet production requirements first, and that this work is ongoing. It also names no model sizes or licensing terms for v4.

For open-source maintainers, the numbers are more concrete. hnswlib, a header-only C++ library for approximate nearest-neighbor search, has about 5,400 stars on GitHub, according to its repository page, so it is a widely used target. A 4% speedup there is modest. The 17.5% on USearch is larger, though it was measured under one specific setup (single core, GCC 11, 1 million vectors) per the project site. Whether the changes were submitted upstream is not stated.

The speed claim rests on a narrow base. Both the 80-hour figure and the “several weeks” comparison are Salesforce’s own, and the article gives no breakdown of how much compute the cycle consumed. That makes the cost side of the trade impossible to check.

Salesforce frames humans as the ones who set the limits. Researchers define the goal, budget, evaluation method and data constraints, and the system explores within them. If a model does not beat its baseline, it stays in place. Models for product use are trained only on datasets that passed ethical and legal review, the company says. Silvio Savarese, Salesforce’s chief scientist, said that “For most of the history of machine learning, progress has moved at the speed of human experimentation.”

The release fits Salesforce’s wider agent push. It recently agreed to buy Cresta to bring voice agents into Service Cloud, as we reported in our coverage of that deal. This announcement points the same technology inward, at improving the models those products depend on. Salesforce also says a future post will cover CodeEvolve, a related system that uses evolutionary search with language models to improve Java and Apex code.