Hot AI News

Apple’s Depth Pro 2 gets a 23.5M-parameter focal length estimator

Apple has put the 23.5M-parameter focal length estimator behind Depth Pro 2 on Hugging Face as its own download, under a research-only license.

Apple’s Depth Pro 2 gets a 23.5M-parameter focal length estimator
Image: apple

Apple has posted DepthPro2-focal on Hugging Face, a 23.5M-parameter model that estimates the focal length of a photo or of video frames. It is the helper that lets Depth Pro 2 produce metric depth when a camera does not report its lens.

The weights fall under the Apple Machine Learning Research Model license, which limits use to non-commercial scientific research and academic development.

Apple's Depth Pro 2 gets a 23.5M-parameter focal length estimator
Image: apple

A small model that fixes the scale of depth

The model card describes a focal length estimator that works on a single photo or on the frames of a clip. In the example code, FOVEstimator.from_pretrained("apple/DepthPro2-focal") returns the focal length in pixels through focal_length_px. Frames can be passed as a tensor shaped (T, 3, H, W). The model is stored in F32 and, according to the page, is meant for photos and videos that carry no focal length.

It belongs to Depth Pro 2, a research project titled “Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth”. The authors listed are Peiyun Hu, Marko Mihajlovic, Lyndon Duong, Lahav Lipson, Wei Dong, Amaël Delaunoy, Stephan R. Richter and Vladlen Koltun. The citation still points to a placeholder arXiv entry, so no paper was readable when the model went up.

The page gives no speed figures and no accuracy results for the estimator. Its role is clear, but how good it is has to be taken from the paper once that appears.

Advertisements

How it fits with the depth models

The estimator does not produce depth itself. The companion DepthPro2-H page, which lists 872M parameters, says the estimator is bundled with the depth model and loads alongside it. The DepthPro2-focal card, by contrast, describes the estimator as left out of that repository and published separately. The two pages are not fully consistent on packaging, so developers should check which repository their install pulls from.

According to the project’s GitHub README, Depth Pro 2 comes in four sizes: S at 24M parameters, B at 97M, L at 334M and H+ at 872M. The field-of-view estimator is a 94 MB download on top of those. For photos, the code reads the focal length from EXIF data when it exists. Otherwise, and for video, it falls back on the estimator. For a video, the README says it takes the median estimate over the first window of frames. Passing a focal length by hand skips the estimator entirely.

Apple also published DepthPro2-coreml on 9 November. It holds Core ML packages for Apple silicon and macOS 15 or later, including a 91 MB field-of-view package that works at a single 896×512 size for 16:9 frames.

Who needs this piece, and who can use it

The angle here is practical: the estimator matters mostly to researchers and developers who run Depth Pro 2 on footage from unknown cameras.

The README explains why the focal length matters so much. Depth scales with it, so a focal length that is 10% too long places everything 10% too far away. A depth model that outputs meters is only as trustworthy as the lens value it is given. Scraped video, phone clips with stripped metadata and archive footage often lack that value, and this is the case the estimator targets.

The model card for DepthPro2-H gives a sense of the cost of the depth model it feeds. Running on an NVIDIA B200 in bfloat16 on a 32-frame clip, Apple lists 95.9 frames per second at VGA, 20.2 at 720p, 5.3 at 1080p and 0.4 at 4K, with peak memory from 8.3 GiB at VGA to 114.6 GiB at 4K. These are Apple’s own figures on a data-center GPU, and the focal length step is a small addition next to that load. For Mac users, the Core ML page says the S model needs about 1.5 GB at 854×480 and recommends the smaller sizes on a 16 GB machine.

For businesses, the license is the main limit. Research-only terms rule out putting these weights into a product, so the release serves labs, students and prototypes. Anyone building a commercial app would need to look at different models or wait for different terms. Apple has not said whether the terms will change.

The release follows Apple’s other recent model drops on Hugging Face. Our report on FastFrames-L covered a very different video model that Apple published days earlier under the same research-oriented approach.

Advertisements

What the pages leave open

Several things are not established. There is no paper to read yet, since the arXiv link is a placeholder, so Apple’s claims about sharpness and temporal consistency cannot be checked independently. The focal length estimator has no published error figures. The parameter count of the main model appears as 0.9B on Hugging Face and 872M on GitHub, which are probably the same number rounded differently, though Apple does not say so. The pages also do not say when the full set of weights and the paper will be complete.

For now, the practical step is simple: install the depth_pro_2 package from Apple’s GitHub repository, load the estimator for clips without metadata, and treat the resulting meters with the 10% sensitivity in mind.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like