embedl/dinov3-quantized-tensorrt

Optimized by Embedl Need to fine-tune, hit performance targets, or deploy on specific hardware? We've got you covered.

Learn more Get in touch →

Embedl Dinov3 Vitb16 (Quantized for TensorRT)

Deployable INT8-quantized version of facebook/dinov3-vitb16-pretrain-lvd1689m, optimized with embedl-deploy for low-latency NVIDIA TensorRT inference on edge GPUs.

Upstream Model

Highlights

• Mixed-precision INT8/FP16 quantization with hardware-aware optimizations from embedl-deploy.
• Drop-in replacement for facebook/dinov3-vitb16-pretrain-lvd1689m in TensorRT pipelines — same input shape (224×224), same output semantics.
• Validated accuracy within 0.48 pp of the FP32 baseline on imagenette (see Accuracy table below).
• Faster than trtexec --best on supported NVIDIA hardware (see Performance table below).
• Includes both ONNX (for TensorRT) and PT2 (torch.export-loadable) artifacts plus runnable inference scripts.

Quick Start

pip install huggingface_hub torch torchvision matplotlib pillow numpy
python -c "from huggingface_hub import snapshot_download; snapshot_download('embedl/dinov3-quantized-tensorrt', local_dir='.')"
python infer_trt.py --image path/to/image.jpg   # TensorRT
# or
python infer_pt2.py --image path/to/image.jpg   # pure PyTorch via torch.export

Files

File	Purpose
embedl_dinov3-vitb16_int8.onnx	INT8-quantized ONNX with Q/DQ nodes — feed to TensorRT.
embedl_dinov3-vitb16_int8.pt2	INT8-quantized torch.export ExportedProgram.
dinov3-vitb16_int8.onnx.data	External weight data for the ONNX — must sit next to the .onnx.
infer_trt.py	Build a TRT engine from the ONNX and render a patch cosine-similarity map (requires TensorRT).
infer_pt2.py	Load the .pt2 with torch.export.load and render a patch cosine-similarity map.

Example Inputs

DINOv3 is a general-purpose vision encoder — it produces rich 768-dim CLS-token embeddings that transfer across tasks (retrieval, segmentation, depth estimation) without task-specific heads.

Cats (COCO val)	English springer (imagenette val)	Church (imagenette val)

Each image is resized and center-cropped to 224×224, ImageNet-normalised, and fed through the encoder. The output is last_hidden_state of shape [1, 201, 768] — 1 CLS token + 4 register tokens + 196 patch tokens. Use output[:, 0, :] as the image-level embedding.

Patch Cosine-Similarity Maps

Move the anchor (×) and the similarity map reorganizes — identically in FP16 and Embedl INT8. Every frame is one already-computed forward pass; the encoder runs once per image.

DINOv3's key property: pick any anchor patch (red ×) and every other patch is coloured by cosine similarity to that anchor. Semantically related regions light up without any supervision — and the INT8 engines preserve this spatial structure (CLS cosine vs the FP16 reference ≈ 0.70–0.81 on the images below).

Each row: input (anchor = red ×) · TRT FP16 on Jetson AGX Orin · Embedl INT8 on Jetson AGX Orin · Embedl INT8 on NVIDIA L4, with per-engine GPU compute latency.

Emergent Semantic Features (PCA)

Projecting each patch feature onto the top-3 PCA components (as RGB) shows the encoder decomposing scenes into semantic regions with no supervision. The PCA basis is fitted on the FP16 reference and reused unchanged for the INT8 engines — matching colors mean the quantized features live in the same geometry.

Performance

Latency measured with TensorRT + trtexec, GPU compute time only (--noDataTransfers), CUDA Graph + Spin Wait enabled, clocks locked (nvpmodel -m 0 && jetson_clocks on Jetson).

NVIDIA L4 GPU

Configuration	Mean Latency	Speedup vs FP16
TensorRT FP16	1.50 ms	1.00x
TensorRT --best (unconstrained)	1.49 ms	1.01x
Embedl Deploy INT8	1.13 ms	1.34x

NVIDIA Jetson AGX Orin

Configuration	Mean Latency	Speedup vs FP16
TensorRT FP16	2.71 ms	1.00x
TensorRT --best (unconstrained)	2.66 ms	1.02x
Embedl Deploy INT8	2.25 ms	1.20x

Accuracy

Evaluated on the imagenette validation split. The quantized model retains nearly all of the FP32 accuracy with a small tolerance.

Model	knn_top1
facebook/dinov3-vitb16-pretrain-lvd1689m FP32 (ours)	99.75%
Embedl Dinov3 Vitb16 INT8	99.26%

Creating Your Own Optimized Models

This artifact was produced with embedl-deploy, Embedl's open-source PyTorch → TensorRT deployment library. You can apply the same workflow to your own models — see the documentation for installation and usage.

License

Component	License
Optimized model artifacts (this repo)	Embedl Models Community Licence v1.0 — no redistribution as a hosted service
Upstream architecture and weights	Dinov3 Vitb16 Pretrain Lvd1689M License

Contact

We offer engineering support for on-prem/edge deployments and partner co-marketing opportunities. Reach out at contact@embedl.com, or open an issue on GitHub.

Community & support Need help with this model? Chat with the Embedl team and other engineers on Discord. Quantization gotchas, hardware questions, fine-tuning tips — bring them all.

Join our Discord →