The electric XYZ actuators are based on the DMX Linear Actuators from my studio KASUGA, we develop them since ca. 2019 as a result for my work Hypergradient (2016). We already had designed the LED fixture before and hence a 3-wire cable through the tube with the WS2811 protocol on the PCB firmware. With a little hack I can extract the 8bit data from WS2811, read them with a ESP32 and control from there the pan / tilt servos and the round LED ring on top of the tilt servo inside a custom build fixture. In total 12 DMX channels are used to control one XYZ actuator (1+2 16bit Position, 3 Speed, 4 Ramp mode, 5 Reset, 6 Dimmer, 7 Pan, 8, Tilt, 9 Empty, 10 R LED, 11 G LED, 12 B LED.
To capture the visitors, a standard 1080p RGB cam is used on top of the installation and then the MediaPipe TD integration to extract the 3D vectors from the face (418 individual points). From there, 24 points are selected (like the amount of the actuators) and tracked in 3D space with the Pan/Tilt CHOP based on the great example from the OP snippets (see image). To store the capture of every visitor and use it for the resemble phase, the CHOP data is transformed into a 32bit TIFF and saved with 30fps, and then replayed with the MOV IN TOP and transformed again into CHOPS. Then, all data is formatted accordingly for DMX output and sent with DMX OUT CHOPs to Art-Net nodes, which connect with the XYZ actuators. The interface screen on the back shows real time data from the capture and processing phases.
