Stream NDI depth video, ARKit face & body tracking, motion, audio, and speech from your iPhone (and Apple Watch) into TouchDesigner over Wi-Fi. No cables. No cloud. No accounts.
You can find it FREE on the App Store
If you've ever wanted to use your iPhone as a depth camera, a body tracker, an audio analyser, or a wearable motion controller for TouchDesigner — without buying a Kinect, an Azure SDK, or a clunky USB rig — TDLidar is built for exactly that.
It runs on iPhone (front TrueDepth or rear LiDAR on Pro models) with an optional Apple Watch companion. Everything streams over your local Wi-Fi: depth as NDI® video, all sensor and tracking data as OSC. TouchDesigner picks both up natively.
What it does:
TDLidar is a one-app capture suite that exposes everything an iPhone can sense as realtime media for TouchDesigner, OBS, Resolume, vMix, or any NDI / OSC receiver on your network:
- Depth NDI video — front TrueDepth or rear LiDAR, encoded with depth-aware colormaps
- RGB camera NDI — independent or combined side-by-side with depth
- Face tracking — 52 ARKit blend shapes + 4×4 head transform
- Body tracking — YOLO 17-keypoint pose from the rear camera
- Hand tracking — 21 landmarks per hand via Vision + ARKit world tracking
- Motion sensors — accelerometer, gyro, compass, barometer at 50 Hz
- Audio analysis — bass / mid / high, beat detection, 20-band FFT
- Speech transcription — on-device, live, word-level OSC events
- Watch streaming — wrist motion + heart rate from the optional watch companion
- On-device recording — capture the colormapped depth feed to .mp4 and export to Photos
It's all live, low latency, and stays on your local network. There's no account, no analytics, no telemetry, no third-party SDK. NDI® is the only third-party library and it doesn't phone home.
Why I built it
I make TouchDesigner installations and live visuals. The phone in my pocket has a depth sensor, a LiDAR, six high-quality cameras, eight microphones, and motion sensors precise enough for car telemetry — but nothing I owned could plug all of that into TouchDesigner cleanly.
Every existing solution made me pick: NDI but no depth, depth but no audio, body tracking but only on Android, or "just use the Kinect." I wanted one app that did all of it, didn't need a server, didn't need a login, and didn't need to be re-paired every time the Wi-Fi blinked.
TDLidar is that app.
What's new in 6.1
- Rear-LiDAR auto-profile. Switching to the back camera now automatically locks Mode to Environment, pins frame rate at 60 fps, and loads the highest depth resolution your iPhone supports. Switching back restores your previous front-camera tuning.
- Higher depth resolutions on Pro iPhones. The full set of LiDAR depth formats is now exposed in the Resolution picker. Non-Pro iPhones still see the front-camera options (rear-LiDAR-only modes are disabled in hardware).
- Loading state during camera switch. Switching cameras is a multi-step reconfigure (session restart + format change + FPS change); a clean loading overlay now bridges the gap so it never looks frozen.
- General polish: clearer captions, better explanations of locked controls, version-bumped to 6.1 (build 10).
Quick start
What you need
- An iPhone with a TrueDepth front camera (iPhone X or newer) or a Pro-model iPhone for rear LiDAR
- iOS 18.6 or newer
- A computer running TouchDesigner / OBS / Resolume on the same Wi-Fi network
- (Optional) An Apple Watch paired to the same iPhone
60-second setup
- Install TDLidar from the App Store.
- Allow Camera, Local Network, and Microphone when prompted on first launch. Speech and Motion prompts come up the first time you visit the Tracking page.
- Tap Start NDI. Your iPhone now publishes itself as TDLidar (and optionally TDLidar Camera) on your network.
- Open TouchDesigner, drop in an NDI In TOP, and pick TDLidar from the Source dropdown. You should see live colormapped depth.
- Open the Tracking page (swipe left), pick a mode (Body / Face / Hands / Motion / Audio / Speech), set your computer's IP and OSC port in Settings, and you'll start receiving messages on /tdlidar/... addresses.
That's it. No accounts, no logins, no cloud step.
The depth side
Modes
- Environment — full-scene depth, near = bright, far = dark. Pick a max range from 1 m to 8 m. This is the right mode for room-scale work, installations, and rear-LiDAR scans.
- Face Detail — a tight depth window centred on whatever the median depth says is your subject. ±20–200 mm window, sub-millimetre precision per step. Ideal for portraits and tight-shot face geometry.
Cameras
- Front TrueDepth — short range, very high precision, faces you. The default.
- Rear LiDAR — Pro-iPhone only. Longer range, wider FOV, room-scale. In 6.1, switching to rear LiDAR auto-applies a scene profile (Environment + 60 fps + max resolution).
Output adjustments
Brightness, contrast, gamma, near clip, far clip, depth invert, temporal smoothing, Apple's hole-fill on/off — all live, all on top of the NDI stream so what you tune on the phone is what TouchDesigner sees.
Colormaps
Black & White (the default and v1 output), Viridis, Inferno, Magma, Plasma, Turbo. Applied to the NDI side; raw depth is preserved internally for tracking.
Recording
Tap the record button on the depth page to write the colormapped NDI feed to .mp4 inside the app's sandbox. Tap the folder icon to browse, preview, save to Photos, or delete. Photos access is write-only — TDLidar can't read your existing library.
Side-by-side and dual streams
- Dual NDI streams — one source for depth (TDLidar), one for RGB (TDLidar Camera). Drop them into separate TOPs.
- Side-by-side mode — one combined source (TDLidar SBS) with camera left, depth right. Use when your receiver only supports one input.
60 fps mode
The TrueDepth and LiDAR sensors cap at 30 fps, but the NDI sender can advertise 60 fps with frame doubling. The result is smooth 60 Hz playback on receivers without the judder a naive duplicate would add.
The tracking side
Swipe left from the depth page to get to the Tracking screen. Pick one of six modes:
Body — /tdlidar/body/*
YOLO 17-keypoint pose from the rear camera. Each keypoint sends x y conf (3 floats) under its name, plus a flat /tdlidar/body/skeleton array of 34 floats (17 × x, y) normalised 0..1, plus /tdlidar/body/detected (0/1) and /tdlidar/body/conf (overall confidence).
Face — /tdlidar/face/*
ARKit's 52 blend-shape coefficients (camelCase: browInnerUp, eyeBlinkLeft, jawOpen, …) plus the 4×4 face transform and a look-at point. Front-camera only; uses the TrueDepth sensor.
Hands — /tdlidar/hand/*
Vision hand pose — 21 landmarks per hand, both hands tracked, anchored in ARKit world coordinates so positions are stable as the phone moves.
Motion — /tdlidar/motion/*
Accelerometer, gyroscope, magnetometer (compass), barometer / altitude, battery level. Streamed at 50 Hz.
Audio — /tdlidar/audio/*
Live mic analysis: bass, mid, high (0..1 each), drum-band onsets drums/low, drums/mid, drums/high, transient burst, and a 20-band FFT under fft/0 … fft/19.
Speech — /tdlidar/speech/*
On-device speech recognition (SpeechAnalyzer on iOS 26+, falling back to SFSpeechRecognizer). Emits partial, final, and per-word OSC. Audio never leaves the device.
The Apple Watch companion
Optional — users without a watch never see it. When installed and active, the watch app reads:
- Motion sensors via CMMotionManager — wrist orientation, accel, gyro
- Heart rate from HealthKit during a foreground workout session
Values stream to the iPhone over Apple's WCSession (Bluetooth / peer Wi-Fi) and the iPhone republishes them as OSC under /tdlidar/watch/*. The watch doesn't talk to the internet, doesn't store anything, and only opens a workout session because Apple requires one to keep the heart-rate stream live.
2. Face blend shapes → character rig
Drop an OSC In CHOP, set the Network Port to whatever you set in TDLidar's Settings. Filter to /tdlidar/face/*. Each ARKit blend shape comes through as its own channel, perfectly named — wire jawOpen straight into your character's jaw morph and you're done.
3. Audio reactive depth
Use the depth NDI as your subject and the OSC audio bands (/tdlidar/audio/bass, /mid, /high) to drive displacement, post FX, or geometry instancing. Beat detection on drums/low is great as a Trigger CHOP source.
4. Body tracking as world space points
/tdlidar/body/skeleton is 34 floats, normalised 0..1. Convert to world space with a Math CHOP, feed into a Trail CHOP for motion blur, render with a Skeleton SOP-style network. Works without depth — pure 2D pose, but stable.
5. Watch as a wand
Heart rate as a slow oscillator, wrist gyro as orientation, wrist accel as a "shake" trigger. The heart-rate field is live and updates ~once per second.
FAQ
Does TDLidar work without TouchDesigner?
Yes. NDI is consumed by OBS Studio (with the NDI plugin), Resolume, vMix, Wirecast, and a long list of streaming and AV tools. OSC is even more universal — Max/MSP, VVVV, Unity, Unreal, Resolume, QLab, and just about every interactive engine.
Does it work over cellular / mobile data?
No. NDI and OSC are local-network only. You and your computer must be on the same Wi-Fi (or wired-to-Wi-Fi) network, and that network has to allow multicast / Bonjour for NDI discovery. TDLidar will not stream to a remote machine.
Does it work on a non-Pro iPhone?
Yes — using the front TrueDepth sensor. Rear LiDAR features (long-range scanning, high-resolution depth modes, the back-camera auto-profile) require a Pro-model iPhone (12 Pro, 13 Pro, 14 Pro, 15 Pro, 16 Pro, 17 Pro, etc.). Non-Pro iPhones still get the full tracking suite (body, face, hands, motion, audio, speech) and front-camera depth.
Does TDLidar collect any data?
NO. There's no account, no analytics, no telemetry, no crash reporting, no advertising SDK. All processing is on-device. The full privacy policy is in the app and on the GitHub repo. Apple's TrueDepth API requirements specifically: TrueDepth and ARKit face data are never shared with third parties, never used for identification, and never leave your local network.
Can other people on my Wi-Fi see my stream?
Yes — that's how NDI works. Anyone running an NDI-capable app on your local network can discover the source and view the feed, the same way they'd see any other shared screen or printer. If you're on a public Wi-Fi this matters; on your home / studio network it's a feature.
How much CPU / battery does it use?
Depends on what you turn on. Depth-only NDI at 30 fps is light — the phone won't even warm up. Body tracking + audio + speech + 60 fps NDI is heavy and will warm a Pro iPhone after ~20 minutes; TDLidar watches the system thermal state and automatically caps frame rate at 15 fps if iOS reports serious thermal pressure, so the app doesn't get killed.
My computer can't see the NDI source.
Three things to check, in order:
- Same network. iPhone on Wi-Fi, computer on the same Wi-Fi (or Ethernet bridged to it). Hotspot networks usually don't pass NDI.
- Local Network permission. On first launch iOS shows a "TDLidar would like to find devices on your local network" prompt. If you tapped no, go to iOS Settings → TDLidar → Local Network and turn it on.
- Multicast. Some routers (especially in offices or rentals) block multicast / Bonjour. NDI Discovery Server (free from Vizrt) bypasses that.
My OSC isn't arriving in TouchDesigner.
In TDLidar Settings, set the Host IP to your computer's IP address (not localhost). In TouchDesigner, drop an OSC In CHOP and use the same port. Firewall on macOS can swallow the first packet — open System Settings → Network → Firewall → Options → make sure TouchDesigner is allowed incoming connections.
Why does the app keep asking for Microphone / Speech / Motion permission?
It doesn't — iOS does, the first time you visit the relevant tracking mode. After you grant once, it never asks again. TDLidar pre-warms the prompts in the background ~2 seconds after launch so they don't pop up mid-stream.
Does the rear LiDAR work in the dark?
Yes, that's the point. LiDAR is an active sensor — it emits its own infrared pulses and times their return. Pitch black isn't a problem. Direct sunlight on a glossy white surface might be (the floor will reflect the IR away), but that's a niche edge case.
Can I record everything, not just depth?
Right now the on-device recorder writes the depth NDI output (with whichever colormap is active). RGB capture and combined SBS recording are on the roadmap. For now, recording the RGB feed is best done in OBS or TouchDesigner on the receiving end.
What's the latency?
End-to-end depth → TouchDesigner: typically 50–80 ms on a clean Wi-Fi 5 network, ~30 ms on Wi-Fi 6. OSC is sub-frame. The bottleneck is Wi-Fi, not the iPhone.
Will it run on iPad?
Not yet. TDLidar is iPhone-only at launch. iPad Pro has the same LiDAR hardware so support is technically possible; it's on the longer-term list.
How do I support development?
If you're reading this on Patreon, you already are — thank you. Otherwise, you can Buy me a Coffee
Coming up
The 6.x line is about polish, performance, and device coverage — nothing flashy. Roadmap:
- RGB recording (record both streams together, or just camera, in addition to depth)
- iPad support
- Higher-resolution rear-LiDAR formats now exposed in the picker on Pro models (shipped in 6.1)
- A few more colormaps, including a custom-LUT loader for users who want their own
- Wider Watch sensor coverage (heading, altitude, optional HR continuity outside a workout)
If there's a sensor your installation needs and TDLidar isn't surfacing it yet, message me — most additions are small days-of-work, not weeks.
You can support my work on Patreon or by Buying me a Coffee
TDLidar is built and maintained by Aristides Lintzeris. NDI® is a registered trademark of Vizrt NDI AB.
