TouchDesigner CoreML Syphon Bridge

Introducing CoreML-TDSyphon bridge. enabling any image based .mlmodel using apple CPU GPU & ANE (apples neural engine) to output video video TOPS with any... I repeat.. ANY .mlmodel.more models can be found here by apple, and here

CoreML Syphon Bridge is a standalone macOS app that bridges CoreML machine learning models with TouchDesigner via Syphon. Send any video from TD, get real-time depth maps, semantic segmentation, or any CoreML vision model output back — all running natively on Apple Silicon's Neural Engine.

What it does:

- Real-time monocular depth estimation (DepthAnythingV2) at 40+ FPS

- Semantic segmentation (DETR ResNet50) with per-pixel class colouring

- MiDaS depth estimation with normalized or raw output

- Load ANY CoreML .mlpackage or .mlmodel — if it takes an image input, it works

- Zero-copy GPU pipeline via Syphon for minimal latency

What's included:

- CoreMLSyphonBridge.app (signed, ready to run)

- 16 pre-tested CoreML models (depth, segmentation, style transfer, face parsing)

- Full source code for customisation

Requirements:

- macOS 14+ (Sonoma or later)

- Apple Silicon Mac (M1/M2/M3/M4)

- TouchDesigner 2024+ with Syphon Spout Out/In TOPs

syphon TD in out pipeline

Application Instructions

Quick Start

1. Unzip CoreMLSyphonBridge_v1.0.zip — contains the app and models folder

2. Open TouchDesigner and add a Syphon Spout Out TOP wired to your video source (webcam, movie file, etc.)

3. Launch CoreMLSyphonBridge.app

4. The app auto-connects to TouchDesigner and loads the default depth model

5. Add a Syphon Spout In TOP in TouchDesigner — set the server name to CoreML Syphon Bridge:CoreML Depth

6. You should see the depth map flowing into TD in real-time

Interface Guide

TouchDesigner Setup

Sending video to the bridge:

videodevinTOP (webcam) → [optional: resolutionTOP] → syphonSpoutOutTOP

Receiving model output:

syphonSpoutInTOP (server: "CoreML Syphon Bridge:CoreML Depth") → nullTOP → [your network]

Performance tip: Adding a Resolution TOP before the Syphon Out to downscale (e.g., 480x270) significantly improves FPS since the bridge processes fewer pixels. The model output resolution is independent of the input resolution.

Performance Optimization

1. Boost Mode ON — always recommended, sets real-time OS priority

2. Minimize the window — eliminates preview rendering overhead, auto-enables Boost

3. Memory slider to max — locks more memory pages, prevents page faults

4. Lower input resolution — add a Resolution TOP before Syphon Out in TD

5. All (Auto) compute unit — lets CoreML distribute work across CPU + GPU + Neural Engine

Technical Specs

• Pipeline: Syphon (IOSurface) → CIImage GPU resize → CoreML prediction → Metal texture → Syphon publish

• Zero-copy input: CIImage renders directly from the Syphon Metal texture to an IOSurface-backed CVPixelBuffer

• Pre-allocated buffers: 3 input + 3 output textures in a round-robin pool — zero per-frame allocation

• Synchronous inference: Runs on the Syphon callback thread to eliminate GCD scheduling overhead

• Code signed: Apple Development certificate with Neural Engine entitlements for full ANE access

• Latency: 1ms input prep + model inference time + 1ms output = model speed + 2ms overhead