Nemotron 3.5 Super VL lands on SageMaker JumpStart, with a 75 GB NVFP4 build on G7

Amazon has made NVIDIA Nemotron 3.5 Super VL available in Amazon SageMaker JumpStart. The 120B-parameter vision-language model comes in an NVFP4 build that runs on G7 instances and a BF16 build on P5en.

Super VL takes images, video and text as input and returns text. Its hybrid Mixture-of-Experts backbone has 120B total parameters, of which 12B are active per token, paired with a C-RADIO vision encoder. According to Amazon, the model supports up to 1 million tokens of context, has a built-in Multi-Token Prediction head and lets developers switch reasoning on or off per request.

Two variants, three instance types

JumpStart lists two checkpoints. The NVFP4 version is quantized from BF16 with quantization-aware distillation and weighs about 75 GB. It defaults to ml.g7.24xlarge (four NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs, 128 GB of GPU memory) and also runs on ml.g7.48xlarge with eight. The BF16 version, about 249 GB, is meant as full-precision reference weights for fine-tuning, reinforcement learning, distillation or custom quantization. It runs on ml.p5en.48xlarge with eight H200 GPUs.

Deployment works from SageMaker Studio, the SageMaker Python SDK v3 (version 3.25 or later) or the Hugging Face model page. Amazon says deployment takes about 10 minutes on the default G7 instance. The post warns that ml.g7 quotas start at zero in new accounts, so a Service Quotas increase is needed first. The post gives no price, and lists no regions.

The endpoint exposes a vLLM-powered chat completions API. Because it runs with network isolation, images and video must be sent inline as base64 data URLs. The post says the model accepts images but not raw PDFs, so pages must be rendered to images first, and one image and one video are allowed per request.

Limits that shape real use

The 1M-token context is the model’s ceiling, not what JumpStart serves. The configurations use 520 to 15,600 tokens. The default generate setup serves 2,275 tokens, summarize 13,650 and modify 15,600. Amazon’s own examples use the modify configuration because image, video and reasoning requests need the room. A 1,536 x 1,024 page image costs about 1,536 tokens, according to the post.

The checkpoint includes the speculative-decoding head, but the JumpStart configuration does not enable it. LoRA adapters are not served either, so the BF16 weights are for customization elsewhere. For video with reasoning on, Amazon advises setting max_tokens to at least 8,192. Frame sampling is fixed at launch, and per-request media_io_kwargs are rejected.

Numbers in context: Speed is fine, long prompts are not

Amazon measured 250 input tokens, 1,500 output tokens and five concurrent streaming requests. NVFP4 on ml.g7.24xlarge produced a mean 63.07 tokens per second per request with 0.45 seconds to first token. On ml.g7.48xlarge it was 64.01 tokens per second and 0.51 seconds. BF16 on ml.p5en.48xlarge reached 140.85 tokens per second and 0.53 seconds. Amazon cautions that the runs use different GPU systems, so they are not a clean precision comparison.

That figure, from the summarize profile, is the main caveat for document work. On quality, Amazon cites NVIDIA’s model cards: NVFP4 ties BF16 on MMLongBench-Doc at 62.82, scores 68.34 against 68.12 on OCRBench v2 EN and 52.30 against 51.14 on OSWorld-Verified, and trails slightly on CharXiv reasoning (79.88 against 80.82) and Video-MME v2 (32.54 against 32.85). These are vendor numbers, not independently verified.

The G7 route matters because of hardware. In our earlier report on AWS’s Kubernetes guide, vLLM needed 16 H100s for BF16 and 8 for NVFP4. JumpStart now serves the NVFP4 build on four RTX PRO 4500 GPUs, and the BF16 build on one eight-GPU H200 node, which also appeared in the Kubernetes guide as a validated option. That guide aimed at self-managed clusters with Nvidia’s Dynamo, while JumpStart trades control for a managed endpoint.

The post leaves several things open: endpoint pricing, regional availability, behavior under heavy concurrency beyond five requests, and how the short serving contexts compare with workloads that need the full million tokens. The model ships under the OpenMDW License Agreement, version 1.1, according to the post.