Stream LiDAR/TrueDepth video over NDI, capture Point Cloud Photos, and send body, hand, face, motion, audio, speech, AirPods, and Apple Watch data as OSC — straight into TouchDesigner over Wi-Fi. No cables. No cloud. No accounts.
FREE ON THE APP STORE. Pro features are a single $9.99 one-time lifetime purchase — no subscription.
If you've ever wanted to use your iPhone as a depth camera, a 3D scanner, a body tracker, an audio analyser, or a wearable motion controller for TouchDesigner — without buying a Kinect, an Azure SDK, or a clunky USB rig — TDLidar is built for exactly that.
It runs on iPhone & iPad (front TrueDepth on every modern model, rear LiDAR on Pro models) with an optional Apple Watch and AirPods companion. Everything streams over your local Wi-Fi: depth as NDI video, every sensor and tracking stream as OSC. TouchDesigner picks both up natively.
If you are learning Touchdesigner and want to use the app without the hassle of configuration - I have a Free TOX available on Patreon. With preconfigured Outputs, So you can get immediate results from the app.
What it does
TDLidar is a one-app capture suite that exposes everything an iPhone can sense as realtime media for TouchDesigner, OBS, Resolume, vMix, or any NDI / OSC receiver on your network:
- Depth NDI video — front TrueDepth or rear LiDAR, GPU-processed with depth-aware colormaps
- RGB camera NDI — independent, or combined side-by-side with depth
- Point Cloud Photos (Pro) — freeze the live cloud into a 3D "photo" you can orbit, record, and export as PLY
- Point Cloud OSC streaming (Pro) — send a real x y z r g b point cloud to TouchDesigner
- 3D body tracking — Apple's 3D body-pose model, 17 metric joints, with LiDAR depth fusion
- 3D hand tracking — 21 landmarks per hand, both hands, each landmark carrying a real Z
- Face tracking — 52 ARKit blend shapes + 4×4 head transform + look-at
- Motion sensors — accelerometer, gyro, compass, barometer at 50 Hz
- Audio analysis — bass / mid / high, beat detection, 20-band FFT
- Speech transcription — on-device, live, word-level OSC events
- AirPods head tracking (Pro) — pitch / roll / yaw + motion from AirPods' onboard sensors
- Apple Watch streaming — wrist motion + heart rate from the optional Watch companion
- On-device recording — capture the colormapped depth feed (or a Point Cloud orbit video) to .mp4 and export to Photos
It's all live, low latency, and stays on your local network. There's no account, no analytics, no telemetry, no third-party SDK that phones home. NDI is the only third-party library.
Videos
V7.12.1 Updates | v8.0.0 Updates
Why I built it
I make TouchDesigner installations and live visuals. The phone in my pocket has a depth sensor, a LiDAR, six high-quality cameras, eight microphones, and motion sensors precise enough for car telemetry — but nothing I owned could plug all of that into TouchDesigner cleanly.
Every existing solution made me pick: NDI but no depth, depth but no audio, body tracking but only on Android, or "just use the Kinect." I wanted one app that did all of it, didn't need a server, didn't need a login, and didn't need to be re-paired every time the Wi-Fi blinked, TDLidar is that app.
What's new in 8.0
This is the biggest TDLidar update yet. Your iPhone's LiDAR is now a 3D camera.
Point Cloud Photos — a brand-new mode (Pro)
Freeze the live LiDAR/TrueDepth cloud into an interactive 3D "photo" and explore it from any angle:
- One finger to orbit, two fingers to move, pinch to zoom — a 3D viewer in your pocket
- Tap the shutter to save a 3D model (PLY and USDZ)
- Hold the shutter (a red ring fills) to record an orbit video while you move around the cloud
- Re-open any capture in the built-in viewer, then Save to Files as a PLY for Blender, TouchDesigner, or any point-cloud tool
- Orbit videos save straight to your Photos library
- Works on the rear LiDAR scanner or the front TrueDepth camera
Raw mode topographic colour maps (Pro)
Raw mode now speaks in colour. Choose from 15 depth colour maps — each one paints the depth field and draws its own topographic contour lines, from bold neon bands to fine scanner lines. Dial the line thickness in, keep your colour wheel for a global grade, and the classic green look is still the default.
3D body + hand tracking with LiDAR depth fusion
The body tracker is rebuilt on Apple's 3D body-pose model with LiDAR depth fusion — your skeleton arrives with proper metric depth, not a flat projection. Every hand landmark now carries a real Z value too. One unified Vision pass on the Neural Engine handles both body and hands.
AirPods head tracking (Pro)
A new AirPods tracking mode reads the head-orientation sensors built into AirPods (Pro / 3rd-gen and newer) and streams pitch / roll / yaw, quaternion, acceleration, and gravity as OSC. Nod, tilt, and turn your head to drive visuals — no camera needed.
iPhone 16 Camera Control
On iPhone 16, the hardware Camera Control button scrolls live parameters without touching the screen — pick what it drives (colour map, line thickness, topographic density, hue, glitch, and more) for hands-free, tripod-friendly performance. In Point Cloud Photo mode it drives a smooth dolly-zoom.
Faster, cleaner, camera-agnostic depth
- GPU-direct depth pipeline — back-LiDAR is upsampled and colour-mapped entirely on Metal (joint-bilateral hole-fill + HD upscale), so the stream is sharper with lower CPU load.
- Camera-agnostic modes — every depth mode (Environment, Face Detail, Raw, Point Cloud) now runs on both the front TrueDepth and rear LiDAR cameras. Front-camera point clouds work on devices with no rear scanner.
- Lazy model loading — the body-pose model only loads when you actually enter Body mode, so launch and depth-only streaming stay light.
Quick start
What you need
- An iPhone with a TrueDepth front camera (iPhone X or newer) or a Pro-model iPhone for rear LiDAR
- iOS 18.6 or newer
- A computer running TouchDesigner / OBS / Resolume on the same Wi-Fi network
- (Optional) an Apple Watch paired to the same iPhone, and/or AirPods Pro
60-second setup
- Install TDLidar from the App Store.
- Allow Camera, Local Network, and Microphone when prompted on first launch. Speech and Motion prompts come up the first time you visit the Tracking page.
- Tap Start NDI. Your iPhone now publishes itself as TDLidar (and optionally TDLidar Camera) on your network.
- Open TouchDesigner, drop in an NDI In TOP, and pick TDLidar from the Source dropdown. You should see live colormapped depth.
- Open the Tracking page, pick a mode, set your computer's IP and OSC port in Settings, and you'll start receiving messages on /tdlidar/... addresses.
That's it. No accounts, no logins, no cloud step.
The depth side
Modes (all camera-agnostic in 8.0)
- Environment — full-scene depth, near = bright, far = dark. Pick a max range from 1 m to 8 m. The right mode for room-scale work, installations, and rear-LiDAR scans.
- Face Detail — a tight depth window centred on whatever the median depth says is your subject. Sub-millimetre precision per step. Ideal for portraits and tight-shot face geometry.
- Raw (Pro) — the artistic mode: 15 colour maps, topographic contour lines, posterisation, line thickness, a global hue wheel, and glitch. The classic green look is the default.
- Point Cloud (Pro) — a live, real 3D point cloud on-device, with an interactive viewer, PLY export, and OSC streaming.
Cameras
- Front TrueDepth — short range, very high precision, faces you. The default.
- Rear LiDAR — Pro-iPhone only. Longer range, wider FOV, room-scale. An active IR sensor, so it works in pitch black.
Switching cameras is seamless — every mode reframes itself for whichever sensor is live, and a clean loading overlay bridges the reconfigure so it never looks frozen.
Output adjustments
Brightness, contrast, gamma, near clip, far clip, depth invert, temporal smoothing, Apple's hole-fill — all live, all on top of the NDI stream, so what you tune on the phone is what TouchDesigner sees.
Colour maps (15)
Free: Black & White (the v1 default), Black Aqua White, Blue Red, Deep Sea, Color Spectrum, Incandescent, Heated Metal, Sunrise, Visible Spectrum.
Pro: TouchDesigner Heat, LiDAR Scan, Cyberpunk, Topographic, Depth Slice, Dual-tone.
Colour maps are applied to the NDI side; raw depth is preserved internally for tracking and point-cloud work.
Recording
Tap the record button on the depth page to write the colormapped NDI feed to .mp4 inside the app's sandbox. Tap the folder icon to browse, preview, save to Photos, or delete. In Point Cloud Photo mode, hold the shutter to record an orbit video instead. Photos access is write-only — TDLidar can't read your existing library.
Side-by-side and dual streams
- Dual NDI streams — one source for depth (TDLidar), one for RGB (TDLidar Camera). Drop them into separate TOPs.
- Side-by-side mode — one combined source (TDLidar SBS) with camera left, depth right. Use when your receiver only supports one input.
60 fps mode
The TrueDepth and LiDAR sensors cap at 30 fps, but the NDI sender can advertise 60 fps with smart frame doubling — smooth 60 Hz playback on receivers without the judder a naive duplicate would add.
The tracking side
Swipe to the Tracking screen and pick a mode. Every stream lands in a clean, predictable OSC namespace.
Body — /tdlidar/body/*
Apple's 3D human body-pose model, 17 metric joints, fused with LiDAR depth when available. Each joint sends x y z conf (metres, model-space) under its name, plus normalised image coords under /img, plus flat arrays for fast wiring:
/tdlidar/body/detected int 0|1
/tdlidar/body/conf float (best-joint confidence 0..1)
/tdlidar/body/abs_depth int 0|1 (1 = Z is real metric LiDAR depth)
/tdlidar/body/distance float (camera-relative distance, metres)
/tdlidar/body/<joint> x y z conf (metres, model-space)
/tdlidar/body/<joint>/img x y (normalised image coords 0..1)
/tdlidar/body/skeleton 51 floats (17 × x,y,z)
/tdlidar/body/skeleton/img 34 floats (17 × x,y)
Hands ride along in Body mode — 21 landmarks per hand, both hands, each with a Z:
/tdlidar/body/hands/count int 0|1|2
/tdlidar/body/hands/abs_depth int 0|1
/tdlidar/body/hands/distance float
/tdlidar/body/hands/<side>/wrist x y z
/tdlidar/body/hands/<side>/<finger> 12 floats (4 joints × x,y,z)
Smoothing is a 1€ filter (Casiez et al.) with optional velocity extrapolation to hide Vision's inference latency. Works on every modern iPhone; LiDAR adds true metric depth, otherwise Z is relative-scale.
Face — /tdlidar/face/*
ARKit's 52 blend-shape coefficients (camelCase: browInnerUp, eyeBlinkLeft, jawOpen, …) plus the 4×4 face transform and a look-at point. Front-camera only; uses the TrueDepth sensor.
Motion — /tdlidar/motion/*
Accelerometer, gyroscope, magnetometer (compass), barometer / altitude, battery level. Streamed at 50 Hz. No camera, no mic — pure IMU.
Audio — /tdlidar/audio/*
Live mic analysis: bass, mid, high (0..1 each), drum-band onsets drums/low, drums/mid, drums/high, a transient burst, and a 20-band FFT under fft/0 … fft/19.
Speech — /tdlidar/speech/*
On-device speech recognition (SpeechAnalyzer on iOS 26+, falling back to SFSpeechRecognizer). Emits partial, final, and per-word OSC. Audio never leaves the device.
AirPods — /tdlidar/airpods/* (Pro)
Head orientation from the motion sensors inside AirPods (Pro / 3rd-gen and newer), via CMHeadphoneMotionManager:
/tdlidar/airpods/pitch /roll /yaw radians
/tdlidar/airpods/pitch_deg /roll_deg /yaw_deg degrees
/tdlidar/airpods/quat w x y z
/tdlidar/airpods/accel x y z (g)
/tdlidar/airpods/gravity x y z (g)
/tdlidar/airpods/rotation x y z (rad/s)
/tdlidar/airpods/tilt x y (normalised, ±1)
/tdlidar/airpods/connected int 0|1
/tdlidar/airpods/calibrated int 0|1
A status row tells you when no AirPods are paired, and a one-tap recalibrate re-zeros your forward direction.
Point Cloud streaming — /tdlidar/cloud (Pro)
Send a real point cloud to TouchDesigner. A Metal compute kernel unprojects the depth map on the GPU; the result goes out as a single OSC message:
/tdlidar/cloud <count:int> then 6 floats per point: x y z r g b
Fixed-grid layout, adjustable density and point size, X/Y/Z flip toggles, and a send-colour on/off (colour off packs white and keeps the 6-float stride constant). It feeds the same cloud the on-device viewer draws, so what you see is what TD gets. The Blender add-on consumes the identical wire format.
Apple Watch companion — /tdlidar/watch/*
Optional — users without a watch never see it. When installed and active, the Watch app reads:
- Motion via CMMotionManager — wrist orientation, accel, gyro, gravity, rotation
- Heart rate from HealthKit during a foreground workout session
Values stream to the iPhone over Apple's WCSession (Bluetooth / peer Wi-Fi), and the iPhone republishes them as OSC under the watch namespace. The watch doesn't talk to the internet, doesn't store anything, and only opens a workout session because Apple requires one to keep the heart-rate stream live.
Patch recipes
1. Depth as a 3D camera Drop the TDLidar NDI source into an NDI In TOP. That's your depth/alpha matte. In Point Cloud or Raw mode you've got real 3D structure and topographic banding to displace or instance from.
2. Face blend shapes → character rig Drop an OSC In CHOP, set the Network Port to whatever you set in TDLidar's Settings. Filter to /tdlidar/face/*. Each ARKit blend shape comes through as its own perfectly-named channel — wire jawOpen straight into your character's jaw morph and you're done.
3. Audio-reactive depth Use the depth NDI as your subject and the OSC audio bands (/tdlidar/audio/bass, /mid, /high) to drive displacement, post FX, or instancing. Beat detection on drums/low is great as a Trigger CHOP source.
4. 3D body skeleton in world space /tdlidar/body/skeleton is 51 floats — 17 joints × x y z, in metres when LiDAR fusion is on. Feed it into a Trail CHOP for motion blur, or instance geometry onto the joints. Check /tdlidar/body/abs_depth to know whether Z is real metric depth or relative.
5. Point cloud straight into TouchDesigner Take /tdlidar/cloud into an OSC In DAT (or a custom parser), reshape count × 6 into a CHOP/TOP, and render the points. No PLY round-trip, fully live.
6. AirPods or Watch as a wand Head yaw/pitch from /tdlidar/airpods/*, or wrist gyro + heart rate from the Watch. Heart rate makes a lovely slow oscillator; wrist accel is a ready-made "shake" trigger.
FAQ
Does TDLidar work without TouchDesigner? Yes. NDI is consumed by OBS Studio (with the NDI plugin), Resolume, vMix, Wirecast, and a long list of AV tools. OSC is even more universal — Max/MSP, VVVV, Unity, Unreal, Resolume, QLab, Blender (there's a free TDLidar Blender add-on), and just about every interactive engine.
Does it work over cellular / mobile data? No. NDI and OSC are local-network only. You and your computer must be on the same Wi-Fi (or wired-to-Wi-Fi) network, and that network has to allow multicast / Bonjour for NDI discovery. TDLidar will not stream to a remote machine.
Does it work on a non-Pro iPhone? Yes — using the front TrueDepth sensor. Rear LiDAR features (long-range scanning, room-scale point clouds, high-resolution depth) require a Pro-model iPhone (12 Pro through 17 Pro). Non-Pro iPhones still get the full tracking suite (body, face, hands, motion, audio, speech, AirPods) and front-camera depth and front-camera Point Cloud Photos.
What's free and what's Pro? NDI depth + RGB streaming, all OSC tracking (body, face, hands, motion, audio, speech), the Apple Watch companion, recording, and 9 colour maps are free. Pro ($29.99 lifetime, no subscription) adds Point Cloud Photos, the Raw topographic colour maps, the 6 premium colour maps, HD upscale, extended LiDAR range, Point Cloud OSC streaming, and AirPods tracking.
Does TDLidar collect any data? No. There's no account, no analytics, no telemetry, no crash reporting, no advertising SDK. All processing is on-device. TrueDepth and ARKit face data are never shared with third parties, never used for identification, and never leave your local network.
Can other people on my Wi-Fi see my stream? Yes — that's how NDI works. Anyone running an NDI-capable app on your local network can discover the source and view the feed, the same way they'd see a shared screen or printer. On a public Wi-Fi that matters; on your home / studio network it's a feature.
Can I record everything, not just depth? The depth recorder writes the colormapped NDI output (with whichever colour map is active). In Point Cloud Photo mode you can also record an orbit video of the 3D cloud and export PLY/USDZ models. RGB-stream recording is still best done in OBS or TouchDesigner on the receiving end.
How much CPU / battery does it use? Depends what you turn on. Depth-only NDI at 30 fps is light. 3D body tracking + audio + speech + 60 fps NDI is heavy and will warm a Pro iPhone after ~20 minutes; TDLidar watches the system thermal state and caps frame rate automatically under serious thermal pressure so the app doesn't get killed.
My computer can't see the NDI source. Check, in order: (1) same network — hotspot networks usually don't pass NDI; (2) Local Network permission — iOS Settings → TDLidar → Local Network → on; (3) multicast — some office/rental routers block Bonjour, in which case the free NDI Discovery Server (Vizrt) bypasses it.
My OSC isn't arriving in TouchDesigner. In TDLidar Settings, set the Host IP to your computer's IP (not localhost). In TouchDesigner, drop an OSC In CHOP on the same port. The macOS firewall can swallow the first packet — System Settings → Network → Firewall → Options → allow TouchDesigner incoming connections.
Does the rear LiDAR work in the dark? Yes — that's the point. LiDAR is an active sensor: it emits its own infrared pulses and times their return. Pitch black is no problem.
What's the latency? End-to-end depth → TouchDesigner: typically 50–80 ms on Wi-Fi 5, ~30 ms on Wi-Fi 6. OSC is sub-frame. The bottleneck is Wi-Fi, not the iPhone.
Will it run on iPad? Not yet. TDLidar is iPhone-only. iPad Pro has the same LiDAR hardware, so it's on the longer-term list.
Coming up
The 8.x line keeps building on the 3D-camera foundation:
- In-cloud measure / tap-to-measure in Point Cloud Photos
- Auto-orbit export presets
- RGB-stream recording on-device
- A custom-LUT loader so you can bring your own colour maps
- Wider Watch sensor coverage (heading, altitude, HR continuity outside a workout)
If there's a sensor your installation needs and TDLidar isn't surfacing it yet, message me — most additions are days of work, not weeks.
TDLidar is built and maintained by Aristides Lintzeris. NDI is a registered trademark of Vizrt NDI AB.
You can support development on Patreon or by Buying me a Coffee.
