Nvidia’s video search model finds events in footage, but only for research

Nvidia has published InternVideo2-CLIP-anomaly-detection on Hugging Face, a fine-tuned video-text model that ranks clips against written queries. It reaches 63.12% mAP on Nvidia’s event-retrieval test, but the release is for non-commercial research only.

A fine-tune that searches footage by description

The model encodes video clips and text into a shared 512-dimensional embedding space. A developer embeds clips once into an index, embeds each search query at run time, and ranks clips by cosine similarity. Nvidia stresses that the output is an embedding, not a caption, bounding box or calibrated probability, and that a similarity score is not proof that an event occurred.

It is a Nvidia TAO fine-tune of OpenGVLab’s InternVideo2-CLIP-L14, paired with a frozen Apple MobileCLIP-B text encoder. Training used vision LoRA on the last 12 vision blocks. Of roughly 373 million parameters, 589,824 are LoRA parameters. Nvidia says training used the train subset of its PhysicalAI-Event-Videos dataset: 2,743 source videos, 6,059 chunks and 85,621 caption entries.

The repository holds merged weights in model.safetensors, the TAO checkpoint with unmerged adapters in model.pth, a model config, and FP32 and FP16 ONNX versions of both towers. Inference runs through the video_clip command in TAO. Input is eight uniformly sampled frames at 224 x 224. Evaluation used English text only, and Nvidia says multilingual quality has not been established.

Retrieval gains of nearly 9 points, with a small general-domain cost

On the test subset, 393 clips and 3,040 English event queries, Nvidia reports the following for the base model, the fine-tune and two zero-shot Qwen3-VL-Embedding models:

• Upstream InternVideo2-CLIP-L14: 54.34% mAP, 50.69% Hit@1
• This model: 63.12% mAP, 59.11% Hit@1, 87.04% Hit@5
• Qwen3-VL-Embedding-2B: 59.67% mAP
• Qwen3-VL-Embedding-8B: 60.58% mAP

Results vary by query type. Location queries score 99.29% mAP, while precise-moment search, the largest group with 1,343 queries, manages 54.16% mAP and 39.02% Hit@1.

On Kinetics-400 zero-shot action classification, top-1 accuracy slips from 70.11% to 69.19%, a drop of 0.92 points, while top-5 is flat at about 89.6%. Nvidia’s model card lists 69.57% top-1 for the 8B Qwen model.

Nvidia gives its own caveat: the retrieval result is not an independent held-out estimate, because checkpoint selection used an earlier relevance revision of the same gallery. All numbers come from Nvidia and have not been independently checked.

On speed, a TensorRT 10.16 engine of about 0.76 GB processed 287.2 clips per second at batch 1 on a GB300 and 152.9 on an H100. Those figures run both towers in every call. The separate tower files were not benchmarked.

Who can use it, and the license limits

Anyone can download it, but commercial use is out. It falls under the NVIDIA Internal Scientific Research and Development Model License, and Nvidia says it is not for commercial products or services. The Apple text encoder carries a research-only license, so commercial deployment would need a separate Apple license or a replacement text encoder, according to the model card. Nvidia also says the model is not validated as an autonomous safety decision system, and that retrieval errors can produce false alarms or missed events. It says it is released on Hugging Face only, not on NGC.

The card lists evaluation on Linux, with accuracy verified on an A100. Nothing is stated on CPU throughput.

Another research-grade release from Nvidia’s open catalog

The model rests on a dataset Nvidia itself built. According to the PhysicalAI-Event-Videos dataset page, created June 15, 2026, it totals 11.8 GB, with 1,486 Nvidia-generated parent videos and 2,939 packaged clips. Some were generated with Google Veo 3 and remain subject to Google’s terms. The page describes a train-only split, which sits oddly with the model card’s references to a test subset; the card does not explain the difference.

The pattern of caveats echoes Nvidia’s other recent releases. Our report on the 4GB Nemotron Bolt model noted that Nvidia called it open without clearly stating license terms. Here the terms are explicit, and they are restrictive. For developers, the model is a reference for how far a small LoRA fine-tune can lift a general video-text model on surveillance-style data, not a product to ship. Nvidia has not said whether a commercially usable version will follow.