This tool merges the point clouds of several depth cameras into one coordinate space. You place the cameras, pulse Calibrate, and the component works out how they sit relative to each other. No chessboard, no charuco, no markers.
This is the old Azure Pointcloud Merger, rebuilt. Version 0.4.0, built and tested on TouchDesigner 2025.33070. The new name is because it is no longer Kinect only.
What is new
- More cameras. Kinect Azure, Orbbec, ZED, or any pair of TOPs you supply yourself (one point cloud, one colour). That last option covers recorded clouds, synthetic data, and sensors I have never even heard of. Adding a new camera type is as simple as adding a row in a table.
- Open3D now runs outside TouchDesigner. The old method was running Open3D inside TouchDesigner's own Python. That caused a lot of headaches, such as crashes on current builds and uncertainty on what versions of the library to use and how to install it. To bypass all that, now the component never imports Open3D. It writes the clouds to a temp folder, runs a separate Python as a normal program, and reads back a 4x4 matrix. TouchDesigner itself only uses numpy.
- You point the Python exe parameter at any Python 3.11+ that has Open3D, pulse Check worker, and it tells you whether it works. Your TouchDesigner install stays clean, and the component no longer cares which Python version TouchDesigner ships.
- The output is a POP. One out_pop carrying every camera's cloud in the reference camera's frame, with P and Color attributes. Invalid sensor pixels are dropped per device before the merge. The old out_points and out_colors TOPs are gone. Use a POP to TOP if you need the old shape.
- It now tells you whether the calibration is any good. After each run you get status, fitness, RMSE, correspondences, and overlap. Overlap is the one to read: how much of what one camera sees the other one also sees. At 0.55 and above, 86 to 100 percent of pairs came out correct in my testing, to a few millimetres. Below it, 27 percent, and the failures are metres out, not small errors. No parameter fixes a low overlap. Moving a camera does.
- It flags unstable calibrations. Every Calibrate runs the rough search four times and compares the answers. The search is random, so it only disagrees with itself when the scene is ambiguous. A disagreement raises a warning, which means this pair is unstable and the next attempt may land somewhere wrong. That check took confidently wrong results from 3 or 4 in 40 down to 0 or 1.
- Better refinement. The refine stage is Generalized ICP now. Over 18 pairs, median error went from 0.0032 to 0.0014 m and the worst case from 0.122 to 0.031 m.
- Chaining works properly. Calibrate 1 to 2, then 2 to 3, and everything composes into the reference camera's frame. Camera 3 lands correctly even though it never saw what camera 1 saw. On a ring of cameras this is the difference between a few millimetres and metres of error.
- Also new: viewport preview of the merged clouds and each camera's frustum, per pair results kept in a table so you can see which link in a chain is weak, coloured ICP, a max range cut, a repeatable seed, presetting a matrix from a table DAT, refine, rebuild chain, and reset.
Try it with no camera
The repo ships DemoTDXDepthCamMerger.toe. Three fake depth cameras looking at one scene from different places, coloured red, green and blue, starting misaligned. Set Python exe, pulse Gather devices, calibrate 1 and 2, then 2 and 3, and watch them come together.
How to use it
- Drop TDXDepthCamMerger.tox into your project.
- Make one camera TOP per physical camera in a single base COMP, named kinectazure1, kinectazure2, orbbec1, zed1 and so on. The names matter.
- Set Device type and Inputs op, pulse Gather devices. The component builds the rest.
- Set Specify pair, pulse Calibrate, repeat for each neighbouring pair.
- Read out_pop.
Point each camera at its neighbour rather than always at camera 1. Cameras far apart barely see the same things.
Limitations
Cameras need real overlap, seen from broadly similar angles. Beyond about 90 degrees apart, or facing each other, there is not enough in common to match. Chaining is the way around it.
A large flat floor with little else cannot be calibrated reliably. A floor looks the same after you turn or slide it, so a wrong answer fits as well as the right one. Give the cameras something with shape to look at.
A good status means the solver settled, not that the answer is right. Always look at the merged cloud in the viewport. That is what confirms a calibration.
Coming from the old version
0.4.0 is a breaking change, and the install changed most. Read the install section of the README before upgrading. Several parameters were renamed: Gather kinects is now Gather devices, Use player for calibration is now Use mask for calibration, and Current pair, Number of pairs and As intermediary are gone, replaced by Reference device and the calibration table. The 2021 walkthrough video still shows the old interface. The idea is the same, the names are not.
Tested with Open3D 0.19.0.
How to use it?
Support
Instagram: https://www.instagram.com/darien.brito/
Patreon: https://www.patreon.com/c/darienbrito
All the best,
Darien
