DINOv3 is Meta's newest vision foundation model — it looks at an image and turns every patch into a 384-dimensional "semantic feature." No labels, no training, no captions. It just knows what things are.
What it does
-
TOP in → TOP out. Wire any image/video TOP into it. Out comes a feature-map visualization (PCA → RGB, L2 norm, single-channel, raw).
-
CHOP out. A 384–4096 dimensional embedding vector you can pipe into anything — Math CHOPs, Lookup CHOPs, parameter bindings.
-
DAT out. Model metadata, dims, live inference timings.
-
One-click install button. Pulses pip install torch transformers timm into TD's bundled Python. No terminal required.
-
Ungated by default. Loads from the timm community mirrors, If you prefer you can register with huggingface and accept facebooks TOS to download the official weights.
What you can do with it
-
Saliency-driven generative scenes. L2-norm view → highlights wherever the model finds "interesting." Drive your particle spawn density off it.
-
Semantic segmentation, free. PCA mode on a multi-object scene gives you a painterly, semantically-coherent region map. No training, no annotation.
-
CLIP-style scene matching. The CHOP embedding is comparable across frames. Cosine-distance it against a stored reference, drive scene changes off similarity.
-
Audio-reactive visuals that actually understand what they're reacting to. Feed video → feed the embedding into your audio analysis chain.
-
Click-to-segment. Coming soon — point at a patch, get a heatmap of every other patch that's semantically similar.
What's in the box
-
12 model variants from the 21M-param ViT-S that runs on anything to the 6.7B-param ViT-7B for serious rigs
-
4 visualization modes
-
28 custom parameters so you can tweak everything without touching code
-
Cross-platform: Mac (MPS) and Windows (CUDA), both confirmed working
-
A "Cook every N frames" throttle so you can keep your timeline smooth even on slower hardware
Performance reality check
On my M-series Mac with the default ViT-S/16: ~17 ms per frame. That's roughly 60 fps headroom for the inference itself. The bigger variants are slower — ViT-L is ~150 ms, ViT-7B isn't really practical without a 24 GB GPU.
If your machine is older or you're on CPU only, drop the input resolution to 160 and crank "Cook every N frames" to 5. The feature map updates less often but TD stays smooth.If you think your machine cannot handle it, Stay tuned I am working on implementing a remote GPU renting function to offload compute while streaming the output in realtime
What you get
click-to-segment, semantic optical flow, feature-driven instancing, CLIP-like image retrieval scenes are built in,
Drop it into your project, wire a TOP in, click two buttons, and the model is running. That's it.
Big thanks to everyone supporting this — it's how I get to make and share weird tools like this.
Tech notes for the curious: loads via timm's DINOv3 mirrors on HuggingFace (ungated). Uses PyTorch with torch.inference_mode() and SDPA attention. Pre-warms on load to avoid first-frame stall. Reads input via numpyArray(delayed=True) to keep the GPU pipeline non-blocking. Cross-platform Python (Windows + macOS), CUDA/MPS/CPU autodetect. Released weights are under Meta's DINOv3 License — commercial use is fine with the standard conditions.
