The last time I posted here, TDLidar was v8.0.0: an iPhone that streamed depth over NDI and tracking over OSC. Two months later it is a ten-mode instrument that a lighting desk can drive end to end, it estimates depth on iPhones that have no LiDAR at all, and it now ships with a proper TouchDesigner operator family so you never have to build an OSC In CHOP by hand again.
Everything below is new since that post. Same rules as always: no cables, no cloud, no accounts, nothing leaves your local network.
Show Control — the phone becomes a fixture
This is the biggest change, and it inverts the whole app. Instead of only sending data out, TDLidar now listens. Anything in your rig that already speaks OSC or MIDI — a lighting desk, QLab, Bitfocus Companion or a Stream Deck, MIDI Show Control, a DAW's clock — can drive the phone without anyone touching the screen.
It is not a mode you select. It runs underneath whatever mode is active, so a remote mode-switch works even from the main menu.
Transport. UDP port 9200 by default, deliberately separate from the sensor OSC bus on 9000 so the two never collide. MIDI arrives over Network MIDI (no pairing, it just connects) or Bluetooth MIDI through Apple's own picker. MTC is received and assembled.
Commands. /tdlidar/show/* covers mode switching, record start/stop, the persistent-NDI master switch, output resolution, the alpha mask, Look Preset recall, Mesh Cloud scan finish/rescan, Point Cloud stream start/stop, and a full factory reset.
Every setting is a parameter. All 177 user-facing controls across the four live modes are remote-addressable — 48 in LiDAR, 16 in Monocular Depth, 28 in LiDAR + Monocular Depth, 85 in Point Cloud. Each one takes a normalized 0–1 value, which means it maps one-to-one onto a MIDI CC's 0–127 without any translation layer. Toggles use a 0.5 threshold, enums map 0 to the first option and 1 to the last. Colormap, gamma, near/far clip, point size, trail decay, orbit mode — all of it, live, from a fader.
Show blackout. The one I use most. /tdlidar/show/screen takes the screen to zero brightness, blocks all touch and suspends preview rendering — while NDI, OSC and recording keep running untouched. For a phone rigged on truss for a six-hour install this is a genuine battery and thermal saver, not a cosmetic dim. Triple-tap the dark screen to wake it, or send a 0.
Status heartbeat. Turn it on and the phone reports its own state back out once a second, so your patch can show what the device actually thinks it is doing rather than what you last told it.
Three modes became ten
At v8 there were essentially three things the app did. Now there are ten, and seven of them are new.
Monocular Depth — depth on any iPhone
Live depth estimation from a single ordinary camera. No LiDAR, no TrueDepth, no Pro model — a CoreML model infers per-pixel depth from the image and the app colour-maps and streams it over NDI exactly like the LiDAR feed. This is the one that removes the hardware barrier: if you own an iPhone, you can stream depth into TouchDesigner.
Three model tiers — Small, Medium, High — trading detail against frame rate, with Small comfortably holding 60 fps. It cycles Front, 1x and 0.5x cameras. Raw monocular depth normally flickers badly frame to frame, so there is a temporal filter on top: a stabilized normalization range, a 3-tap median and an edge-aware smoothing pass, which is what makes it steady enough to composite live rather than just look impressive in a screenshot.
Auto-adjust rides brightness, contrast and gamma off a fixed lattice of sample points — 17 on the rear cameras, 33 on the front, denser toward the middle where a face sits in a selfie framing. Each point is weighted by how tightly its neighbourhood agrees with itself, so a point sitting on a depth edge devalues itself instead of dragging the average into noise.
LiDAR + Monocular Depth — dense depth in real metres
The mode I am most pleased with. The rear LiDAR knows true distances but only samples them on a coarse 256x192 grid, so its edges are blobby and it drops out on dark, glossy and distant surfaces. A monocular network resolves fine structure beautifully but has no idea how far away anything actually is.
Here the LiDAR's own depth is fed into the network every frame as a prompt. The sensor supplies real distance; the network supplies the sharp, image-aligned edges the sensor cannot resolve. What comes out is a dense depth image in actual metres.
That matters more than it sounds. Every other depth mode normalizes each frame against whatever happens to be in view, so a given grey is a different distance from frame to frame — the image quietly breathes as things enter and leave the shot. This mode does not. You set a fixed Metric Near and Far, and a subject two metres away produces the same grey in your studio as it does on stage. The NDI feed becomes genuinely measurable rather than merely pretty, and an alpha threshold becomes a real distance instead of an arbitrary number.
Two fusion sliders sit between the sources: LiDAR Weight decides whose geometry wins when they disagree, RGB Detail scales the fine structure only the network can supply. Push them to the extremes and you get free inspection views — sensor alone, or network verbatim — so you can judge whether the fusion is earning its compute in a given room. There is also Auto Range driven off a confidence-weighted point lattice, a self-consistency trust map that suppresses flicker in the LiDAR prompt, and a frame-level health gate that fades the sensor out entirely and runs pure RGB when the room is too dark for it, with an honest on-screen note explaining the change rather than leaving you to wonder why the look shifted.
Requires a Pro iPhone and a Pro purchase.
ARKit Point Cloud — person-segmented silhouettes (beta)
ARKit person segmentation crossed with the LiDAR depth. Every human figure in view is rebuilt as a point-cloud silhouette that is world-locked, not attached to the camera, and every new scan fully replaces the last rather than accumulating — so what you see is this instant's figures, not a build-up of every pass.
Pause freezes the last captured frame in place while the camera keeps running, so you can physically walk around a frozen cloud and look at it from behind. Roughly 60 parameters across 13 groups: scan gating with turn- and shake-reject, point budget and 13 point shapes, spectrum/static/RGB colour sampling, an eleven-step cleanup stack, and the effects — trails, echo, light paint, sculpt, riptide, vanish and gravity fall.
Capture-only, to Photos. No NDI or OSC out of this one. It is in beta and I would genuinely like bug reports.
Mesh Cloud, Scene Build, Cue Deck, Align
Four smaller modes aimed squarely at TouchDesigner work:
- Mesh Cloud — scan a space with the LiDAR mesh, free-fly the cleaned-up result, edit it, then send it to TD over TCP or export a PLY.
- Scene Build — Apple's RoomPlan room scanner. Walls, floor, doors, windows, openings and major furniture come back as clean structured geometry rather than a point cloud, and the Scene Build operator rebuilds it in TD with each category on its own colour so you can style them independently.
- Cue Deck — a bank of edge-to-edge on-screen OSC trigger pads with a full editor and haptics. Manual cues without a control surface.
- Align — tap a real surface and the app streams its corners as OSC, feeding a projection-mapping warp network in TD. Live camera preview, drag-to-edit corners.
Point Cloud, rebuilt
The point cloud got a ground-up pass. The renderer moved from SceneKit to a full Metal MTKView pipeline, and streaming moved off OSC entirely.
Lossless over TCP. Up to 50,000 points, bit-exact XYZ plus RGB, straight into TouchDesigner POP geometry. OSC capped out around 2k points and quantized them; TCP does not, so what leaves the phone is what arrives. There is a matching receiver on the TD side.
A real cleanup stack. Apple depth filtering toggle, bilateral surface smooth, grazing-angle cull, temporal accumulate, hole fill, despeckle, voxel downsample, and RGB-guided joint bilateral upsample for detail recovery.
Motion effects driven by what actually moves in the scene: Riptide, Vanishing, Gravity Fall, Light Paint and Echo/Strobe, plus trails that shed off silhouette edges.
Show-safe controls. A padlock freezes orbit, pan, pinch and reset so a framed shot cannot be nudged mid-show. Whole-cloud tilt on X/Y/Z. An auto-orbit picker with Off, continuous Spin, or a ±90-degree Sway pendulum.
Not a TouchDesigner user? The same lossless TCP cloud also streams into Arkestra.
The TDLiDAR Operator Family
This did not exist when I last posted, and it is now half the product.
TDLidar is an iOS app plus a TouchDesigner operator family: 48 operators, one per sensor, tool or output. You drag one .tox into a project and they appear in the TAB / OP Create menu under a blue TDLiDAR family. Drop any of them and it listens on OSC port 9000, previews its live output right on the node tile, and only connects to sensible targets.
No Python. No networking knowledge. No hand-built OSC In CHOP with a Select and a Rename in front of it.
- Motion — Acceleration, Gravity, Gyro, Quaternion/Euler, Magnetometer, Barometer, Motion Activity
- Device — Battery, Thermal State, Low Power, Screen Brightness
- Body & Vision — Body, Hand, Pinch, Gesture, Face, AR Body, Device Pose
- Scene & Detect — AR Planes, AR Mesh, QR/Barcode, Text OCR, Saliency, Front/Back Distance, Ambient Light, Camera Exposure, Animal, Scene Room
- Audio — Mic Level, Audio FFT, Speech, Sound ID
- Touch & Input — Touch, Apple Pencil, Proximity, NFC, Volume/Remote, Cue
- External — AirPods head pose, Apple Watch
- Output / Utility — NDI, Point Cloud, Scene Build, Depth, Rectangle, Align, and tdlidar_show
tdlidar_show is the odd one out: it is the only operator in the family that sends rather than receives. It is the Show Control surface as a node — three parameter tabs covering all 177 settings, plus dropdown menus for the discrete ones and pulses for reset and capture commands.
Get the operator family: https://www.patreon.com/aristideslab/posts/tdlidar-operator-164655143
There are also receivers for Blender, Unity and Unreal if your pipeline runs somewhere else.
Smaller things that changed the daily feel
- Shutter. Tap for a photo, hold 0.37s to start a recording, tap to stop. In LiDAR and both Monocular modes. A photo saves the on-screen frame plus a plain RGB still.
- Live camera / depth switch. One button flips the entire output — preview, NDI stream and any active recording — between the colour-mapped depth and the plain RGB camera, instantly, mid-stream.
- Hardware buttons. The volume buttons and the iPhone 16 Camera Control click each run an action of your choice: Photo, Record, RGB/Depth switch, Camera Flip or Adjust. The Camera Control slide scrolls a setting you pick.
- 25 colormaps plus a custom colormap creator, and a switch for what the map is interpreting — depth, confidence, or depth lines.
- Draggable distance crosshair with optional burn-in to the NDI feed.
- LAN auto-discovery. mDNS plus a TCP sweep replaced every manual IP field with a dropdown.
- Persistent NDI across mode switches, with a unified alpha mask shared by LiDAR, Monocular and Point Cloud.
- What you see is what records. Preview, recording and NDI are now fed from one processed frame, so Sharpen and your chosen Resolution actually show up in the file.
Documentation
There is now a full docs site: https://tdlidar.github.io/DOC/
App Guide with a page per mode, exhaustive references for every LiDAR setting and every Point Cloud control, a page for each of the 48 operators, the complete OSC reference, an FAQ, and a feedback form that reaches me directly.
Availability
TDLidar is on the App Store, free, iOS 18.6 or later. Depth and scan modes need a Pro iPhone with the LiDAR scanner; Monocular Depth works on any recent iPhone; Face needs TrueDepth.
Pro is a one-time $29.99 unlock, or a $2.99 3-Day Pass if you want to try the Pro features for a specific gig first — once per account, non-renewing.
Still no account, no analytics, no telemetry. Everything runs on-device and stays on your local network.
App Store: https://apps.apple.com/us/app/tdlidar/id6760954732
Docs: https://tdlidar.github.io/DOC/
Operator family: https://www.patreon.com/aristideslab/posts/tdlidar-operator-164655143
Questions, bugs and feature requests are all welcome — either here or through the feedback form on the docs home page.
