GPU confidential computing: How AI workloads stay encrypted while a GPU runs them

GPU confidential computing is a way to run AI work on a cloud GPU while keeping the data, the prompts and the model hidden from the cloud operator. The GPU and the processor around it form a sealed zone. Everything that crosses into or out of that zone is encrypted, and the customer can check that the zone is genuine before sending anything.

The idea reached the news again when Google Cloud and Nvidia said Confidential Computing is now generally available on G4 virtual machines, as covered in our report on the A5X announcement.

Why data is exposed at the moment it is used

Think of a hospital sending patient scans to a cloud service for analysis. The scans are encrypted on the way there and encrypted again when stored. That covers data in motion and data at rest. But to analyze a scan, the machine must decrypt it and hold it in memory in plain form.

That moment, called data in use, is the gap. On a shared cloud machine, the operator’s software, a neighboring customer’s compromised program or a rogue administrator could in principle look at that memory. Nvidia’s own 2023 technical blog describes the goal as protecting data and code while they are being processed, not only before and after.

A sealed zone that now includes the GPU

Confidential computing already existed on processors. A virtual machine can run in a trusted execution environment, or TEE: a hardware-protected area where even the hypervisor, the software that manages the virtual machines, cannot read memory. Intel TDX and AMD SEV-SNP are two such technologies.

AI work mostly runs on GPUs, though, and a TEE that stops at the processor leaves a hole. Nvidia introduced GPU confidential computing with the H100 (Hopper) in 2023. According to Nvidia, the H100 was the first GPU to support it. It extends the processor’s TEE so the GPU sits inside the same protected zone.

The GPU cannot do this alone. A 2025 study from IBM Research and Ohio State University found it must be paired with a processor-side confidential virtual machine.

Encrypted hand-offs and a proof before any data moves

Two mechanisms do most of the work.

Encrypted transfers. The link between the processor and the GPU is treated as untrusted. Data passes through a staging area in ordinary system memory, called a bounce buffer. The sender encrypts it before it enters the buffer and the receiver decrypts it afterward. Nvidia says command buffers and GPU programs are also encrypted and signed before they cross the link. The host can see the buffer but only sees scrambled bytes.

Attestation. Before sending anything, the customer’s virtual machine asks the GPU to prove what it is. The GPU presents a certificate tied to a device-unique key, plus a signed report of its firmware and settings. Nvidia describes checking the certificate against its certificate authority and its revocation service, with its Remote Attestation Service as the main verifier. Only if the check passes does the customer start the job.

Nvidia says existing CUDA applications should run without code changes once the hardware, drivers and attestation are in place.

The cost: Slower transfers and a mode with fewer protections

Encryption is not free. Nvidia states that raw GPU compute and memory bandwidth match normal mode. The cost sits in the link between processor and GPU, which is limited by processor-side encryption. Nvidia’s 2023 post measured roughly 4 GB/s there and said workloads with little computation per byte of input see reduced throughput.

Other limits from the same post:

• Performance counters are switched off in the fully protected mode to limit side-channel attacks. A separate profiling mode, CC-DevTools, turns protections off so developers can debug.
• Only certain GPUs, processors, firmware and driver versions are supported.
• The GPU’s own high-bandwidth memory is not encrypted. Nvidia treats it as secure against everyday physical attack tools.

The 2025 IBM and Ohio State paper by Zhongshu Gu and colleagues probed the Hopper design from outside, since Nvidia’s specifications are proprietary. It reported open questions, including some control registers that still return non-zero values, unencrypted metadata in the driver’s message queues and transfer timings that vary with size and could leak information. The authors said they reported their findings to Nvidia. They also noted that physical attacks on the GPU package were out of scope. The study covered only Hopper and a single GPU, so it does not describe newer chips.

From one GPU to a whole rack, and who uses it today

Early GPU confidential computing covered one GPU at a time. The IBM and Ohio State paper noted that multi-GPU use was disabled in the Hopper release and cited an Nvidia presentation saying Hopper’s NVLink traffic, the fast links between GPUs, was unencrypted, with encryption expected on Blackwell.

That is why rack-scale support matters. Large models are spread over many GPUs, so protection has to cover the traffic between them. According to the announcement covered in our article, Google Cloud says it will be among the earliest clouds to run Nvidia’s confidential computing across a whole rack on bare-metal machines, with A4X Max, which uses GB300 NVL72 systems, as the example. That is described as coming, not available.

What is available is smaller. In April 2026, Nvidia’s Ian Buck wrote that Confidential G4 VMs with RTX PRO 6000 Blackwell GPUs were in preview, calling it the first confidential computing offering of Blackwell GPUs in the cloud. The October announcement says they are now generally available, with Vertebrae, Polaris Cloud and Confidential Core AI named as users. Google Cloud and Nvidia frame the use case as confidential inference and fine-tuning. These are vendor descriptions.

What the protection does and does not promise

Confidential computing narrows who must be trusted. It does not remove trust entirely. The customer still relies on the chip maker’s hardware, firmware and attestation service, and on the correctness of the verification policy they set up.

It also protects only the sealed zone. Data leaving through the application’s own outputs, leaks in application code or a poorly configured pipeline are outside its scope. One industry technology radar makes the same point: teams should validate the hardware trust chain, attestation evidence and key-release policy before treating it as a default requirement.

Three things are worth watching. First, whether rack-scale confidentiality ships on schedule and what it costs in speed, since no results are given yet. Second, independent security analysis of Blackwell and later designs, similar to the Hopper study. Third, whether regulated industries such as health care and finance accept hardware attestation as evidence in audits.