2020 Official Update

We are delighted to kick off this new year (and a new decade!) with an Official update that is the culmination of our development over the past 6 months. There are dozens of new features and hundreds of improvements, we are sure there is something for everyone to get excited about! These new features have been maturing in experimental 2019.30000 builds since last summer and are now stable for production work in our 2020.20000 Official Update.

We outline the incredible highlights of this release here, new features and workflows that we hope inspire you with an abundance of new ideas. This release has three primary areas of focus, Point Cloud workflows, Web Server and Client integration, and new devices supported. For more details on everything that has changed, read through the Release Notes. 

A growing availability of point cloud data and recent GPUs' power to handle it in real-time all pointed to the need for a better point cloud workflow in TouchDesigner. 

More broadly, the ability to process general data (other than image pixels) on the GPU is something we see people doing often. Particle systems, simulations, machine learning and all sorts of parallel processing tasks benefit from leveraging the GPU. This led us to design new features to make it easier to do this in TouchDesigner using just built-in nodes.

Loading the data

Starting with the Point File In TOP, you can read data from a large number of supported file formats (.xyz .obj .exr .pts .ply .FITS/.FIT .csv .txt) into this multi-buffer 32-bit float TOP. Supporting multiple buffers is key because point cloud files can contain more than the 4 channels that a single RGBA buffer offers in TOPs. In the Point File In you can select which channels from the file you want to assign to the Red, Green, Blue and Alpha channels of the TOP. To access the additional channels from the same file, use a Point File Select TOP which references the original Point File In which loaded the file. You can find out what channels are available using an Info DAT pointing to the Point File In TOP.

Making this data easier to inspect and visualize two new TOP viewer modes have been added to give more informative views. The first mode called ‘Normalized Split’ shows all channels available in separate ‘split’ views, giving at-a-glance view of each channel’s contents. The data is normalized between 0-1 for viewing so that floating point data with wide-ranging values can be visualized in full black to white pixel ranges. The second mode ‘View as Points’ gives a 3D point view of the data using the assumption that the TOP’s Red, Green, and Blue channels hold XYZ positional data.

Processing the data

Enabling 3D positional data and general data usage in TOPs signalled the need for more general data processing in TOPs as well. With that in mind we have introduced the new Function TOP to apply mathematical functions to your TOP channels (works similar to the Function CHOP). In addition a new Limit TOP is very handy to clamp, normalize, and quantize values in TOPs. The Math TOP has also been improved to offer range controls for individual color channels. Lastly the CHOP to TOP has new features to give you better control over the resolution and layout of pixels that are created when converting your CHOP channels into TOP data.

Saving out point cloud data presents new challenges in dealing with an arbitrary number of channels. To fully support this TouchDesigner’s OpenEXR file support has been overhauled. EXR files support up to 32-bit float data and any number of channels required for these workflows. The Movie File Out TOP has been augmented with an EXR parameter page to handle exporting these multi-channel EXR files. You can also now add custom header information to your EXR files and set a flag to mark them as ‘Point Cloud’ files so in TouchDesigner they will automatically load into Point File In TOPs instead of Movie File In TOPs. 

Furthermore, besides loading individual EXR files TouchDesigner can also read EXR file sequences, offering a complete pipeline for animated point cloud recording and playback.

For a quick introduction to a point cloud setup using many of these new features, check out the Palette’s PointClouds folder and examine the network inside the pointRender component. There are also 5 other components in that folder that can help when working with point clouds.

2.Instancing with TOPs

GPU instancing of geometry has been a core feature of the Geometry COMP for years now, enabling 1000s of geos to be rendered easily by your GPU. Previously the instances needed to get their per-instance attributes from a SOP, CHOP, or DAT, all of which are CPU based operators which could become a bottleneck as the number of instances increased. To keep your data on the GPU the instancing of geometry can now be fed directly from TOPs! This complete GPU pipeline for instancing offers a significant increase in performance. 

Additionally, a more flexible instancing setup now allows you to specify different OPs for each instance attribute group (ie. Translate, Rotate, Scale, Color, etc.). Often a requirement when using TOPs since they only have 4 channels, it is also convenient to mix data from any OP source at any time. Instancing also now supports up to 12 custom attributes for those really complex setups that need additional information passed in. 

3.Web Client and Server and SocketIO

Whether you are trying to access a database, social network, CMS, or other web service, the new Web Client DAT improves on the old Web DAT to access all these. The Web Client DAT supports various authentication methods including oAuth. TouchDesigner can now easily receive input from any web-based interface allowing remote control from any network connected device.

For the first time, you can now run your very own webserver inside TouchDesigner using the new Web Server DAT. This opens a world of new possibilities in making TouchDesigner the central hub of operations and a primary distributor of information for any project. It also includes a websocket server to create bi-directional and multi-endpoint communication between TouchDesigner and other applications using the websocket protocol. You can run lightweight web-based interfaces served from TouchDesigner to thin clients, or monitor your project with a custom website fed from the Web Server DAT. We’re very excited to see what systems are imagined with these new features.

The new SocketIO DAT completes this trio of amazing tools to more deeply integrate your projects with web-based solutions and technologies. Supporting the most recent version of socket.io, connecting to node.js applications is straightforward.

Over the past decade the Microsoft Kinect has been a favorite sensor to provide depth camera, point cloud and body tracking data. In 2015, Microsoft’s decision to discontinue the Kinect Version 2 left us wondering and looking for what would be next. Fortunately, the newly released Kinect Azure has answered that question and we are very excited to now support it in TouchDesigner!

The new Kinect Azure device comes in a much smaller and more robust package than before, with a fully aluminum construction and multiple mounting points, the build quality is excellent. It now sports a 12 megapixel color camera and 1 megapixel depth camera, both giving a massive improvement in the quality of data from the sensor. A new Kinect Azure TOP gives you access to the Kinect’s settings and one of the camera feeds, you can then use the Kinect Azure Select TOP to pick one of the other camera feeds such as Color, Depth, IR, Player Index, or Point Cloud.

Body tracking (skeleton position tracking) is handled by the new Kinect Azure CHOP. Similar to previously versions of Kinect, you can get bone positions in worldspace, bone lengths and bone rotations, and more. New to Kinect Azure is that the body tracking can now be done on your system’s Nvidia GPU using the Nvidia CUDA Deep Neural Network (cuDNN) for best performance. There is also a CPU mode for systems without a Nvidia GPU, but unfortunately this is a slow option so we recommend Nvidia GPUs for those interested in using the Kinect Azure CHOP.

Other exceptional new features of Kinect Azure are the ability to run multiple Kinect Azure sensors on the same computer, something not possible with the older Kinect version 2. Furthermore, sync between these multiple Kinect sensors is supported and the Info CHOP reports color camera timestamps and body tracking timestamps giving you the tools to use multiple Kinect Azures together to create seamless 3D virtual point cloud environments.

Ouster makes high-resolution LIDAR sensors for scanning 3D environments with lasers. The new Ouster TOP supports real-time point cloud capture and recording of their OS1 series sensors. Additionally the new Ouster Select TOP works with the primary Ouster TOP allowing you to capture additional channels at the same time (more than the default RGBA channels). Of course all the other point cloud workflow features mentioned above apply to working with the Ouster point cloud data streams.

6.Pangolin Laser Support

Pangolin Laser Systems are an industry standard for laser shows and you will find these lasers in use worldwide by the ProAV community. With the addition of a new Pangolin CHOP, TouchDesigner can now talk directly to the Pangolin Beyond software that driving these systems. Feeding the Pangolin CHOP input from a SOP or CHOP will send laser frames to Beyond software to be processed and sent to the Pangolin lasers.

7.Binding

Binding was introduced in 2019 to provide a bi-directional link between multiple parameters, letting you adjust any bound parameter value while having the other parameters stay in sync. The new parameter based workflows are now a preferred method for connecting a component’s Custom Parameters to interfaces, operator parameters, and anyplace you need to send a value. 

To bring binding to CHOPs we have introduced the Bind CHOP. This let’s you bind parameters to channels and maintain bi-directional sync. In such a setup, a changing CHOP channel will update the bound parameter and a changing parameter value will update the CHOP channel value by overriding its input (until the next value change is detected, similar to the deprecated Override CHOP).

Binding to DAT cells is also now supported by simply pointing to a DAT cell in a parameter’s bind expression.

NotchLC is a high quality, GPU powered video codec that is great for high resolution playback. The Movie File In TOP now supports this codec and it is an excellent alternative when maintaining color and image quality is your top priority.

TouchDesigner has long supported Nvidia GPU hardware encoding of H264 and H265, but now we have added a Lossless H264/H265 preset for hardware encoding these codecs giving you more options when exporting your final movies.    

NDI is being used by many of you to route your content easily to other machines and devices on your network. To keep up with the latest NDI developments TouchDesigner now supports the latest NDI 4.1 SDK which offers 16-bit support and improved performance. 

If you are using SDI devices there are a number of great improvements. TouchDesigner now supports Blackmagic Design SDK 11.x to get the latest features out of those devices, such as the Decklink 8K Pro’s ability to use its four 12G-SDI for quad link 8K. Great improvements if you use AJA include extended support for the AJA io4k as well as support for Fill and Key pairs and the ability to sync to their reference inputs.

A major new feature for SDI users is proper audio over SDI support for Blackmagic Design and AJA. The Video Device Out TOP lets you send audio out over SDI and the Audio Device In CHOP now has native driver support for Blackmagic and AJA. This will give you the lowest latency possible and perfect sync of audio and video signals over SDI.

9.Audio Recording

If you are working with audio there are a number of things to get excited about. First is audio recording. 

A new Audio File Out CHOP allows for saving out multi-channel audio files to .wav .ogg .mp3 or .aif formats. 

When working with movies the Movie File Out TOP now allows you to record more than 8 channels on supported codecs. Speaking of audio in movies, a new addition to the Audio Movie CHOP lets you select which audio track to use when the file contains multiple tracks, nice!

10.Compatibility

For anyone using older versions of macOS, it is important to note a new macOS minimum requirement is macOS 10.12 Sierra.

Python 3.7.2 is now the built-in version of python in TouchDesigner. Take note version 3.7.2 has 2 additional reserved keywords, 'async' and 'await', adjust your code accordingly.

Backward Compatibility

While we try to maintain backward compatibility between builds as much as possible, a number of changes were made this year that are not compatible with older versions of TouchDesigner. It is important to review all BACKWARD COMPATIBILITY notes when moving projects forward from previous builds to this new release.

The addition of Nvidia’s CUDA Deep Neural Network for Kinect Azure (and other upcoming features) has doubled the size of the Windows installer to around 800MB. This full download is available on the primary download page. If you do not have an Nvidia GPU or do not need Kinect Azure support, we now also offer an installer without cuDNN that is only ~480MB, you can download it here.

The features mentioned above highlight some of the things we are most excited about in this release, but there are 100s of new features and improvements to discover in this Official. You can further explore the new additions by opening Operator Snippets and clicking the 'Newest' button to see the ones recently added. Check out the complete changelog in the Release Notes and let us know what you are happy or excited about in the comments below!