Mistral and Nvidia want open benchmarks to test AI agents on chip and CAD work

Mistral AI and Nvidia are teaming up on engineering AI, and the most concrete deliverable is a set of public benchmarks and open datasets for physics models and for AI agents in computer-aided engineering (CAE) and electronic design automation (EDA), Mistral said on October 21.

Benchmarks first, products later

The collaboration pairs Mistral Large 4 with Mistral’s own pipelines for producing training data and modeling physics, all running on Nvidia’s CUDA-X libraries and the Warp framework. The stated aim is faster engineering simulation and closed-loop workflows that link simulation, AI model development and design. A separate strand connects Mistral’s models to Nvidia’s Omniverse libraries, so GPUs can build 3D scenes and produce simulation-ready assets.

The benchmarks are the central piece. For physics models, they are meant to give researchers and industrial teams a common basis for comparing accuracy and performance, and for building and validating automated CAD-to-simulation workflows. For agents, the companies describe tests of whether an agent can operate engineering tools, finish multistep workflows and reach technically correct results, while recording execution time and compute cost. Target sectors are aerospace, automotive and semiconductors.

Neil Ashton, Mistral’s VP of Computational Engineering, said the aim is “faster simulation cycles, as well as open benchmarks rigorous enough to validate agents on real CAE and EDA work.” Nvidia’s Tim Costa, who runs industrial and computational engineering there, pointed to “shared industry benchmarks to build confidence in what AI can do.”

Mistral Large 4 and a compute target

The announcement also restates where the underlying model stands. Mistral Large 4, nicknamed Le Chonk, is in public preview. It has roughly 1 trillion total parameters and 49 billion active, and Mistral says it was trained on 3,800 Nvidia Grace Blackwell GPUs in Mistral-operated European data centers. Weights are planned for the end of October. Mistral adds that it is bringing more Nvidia-powered infrastructure online toward a goal of 1 GW of capacity by 2030.

The two companies previously co-developed Mistral NeMo and now work together in the Nvidia Nemotron Coalition on a frontier base model meant to push the open-weight frontier.

What engineers and developers can expect

Nothing in the announcement is a product engineers can use today. The page gives no release dates for the benchmarks or datasets, no names for the tasks or scoring methods, and no pricing. It also does not say which licenses the data will carry, or whether Mistral’s physics models will ship alongside Large 4’s weights. The performance claims, such as faster simulation cycles, come from the companies and are not independently verified.

The shared-benchmark approach does address a real gap. Nvidia’s own material on Warp argues that the cost of generating simulation data is often the limiting factor for CFD surrogate models, which is why GPU-native solvers matter. Open data already exists in pockets: Nvidia’s Kit-CAE tutorial, for instance, uses an open dataset of Ahmed body designs simulated across a range of Reynolds numbers, with geometry, pressure and wall shear stress fields, and Autodesk Research publishes the open source XLB fluid solver built on Warp. What is missing, and what this effort targets, is an agreed way to score agents and physics models against each other on whole CAD-to-simulation workflows.

For developers, the practical effect would be a common yardstick before choosing a model for simulation or chip-design automation. For manufacturers, it could mean fewer vendor-supplied demos and more comparable results. Whether it gets there depends on who else adopts the benchmarks; the announcement names no other participants.

The move also fits a broader Mistral push into physics. Mistral and CERN recently signaled plans for a single model trained on simulated LHC collisions, a project that is still a statement of intent with no stated size, budget or timeline. Together, the two efforts suggest Mistral sees scientific and industrial physics as a lane where an open-weight European lab can compete.

Open questions on timing and scope

Benchmarks only matter if they are hard to game and widely used. The companies promise repeatable, consistent ways to measure capability and track progress, but both have an interest in the results, since Mistral’s models run on Nvidia hardware. Details on governance, such as who maintains the tests and whether outside labs can submit results, would settle how neutral they are. The Large 4 weights release at the end of October will be the first checkable commitment.