We are very pleased to announce TouchDesigner's 2021 Official Update and tell you about all the exciting features of this release.
This year's update touches (pun intended) so many parts of TouchDesigner. We spent a lot of time improving workflows and optimizing features people are already using daily, while also thinking about how to open new doors for experimentation. This is also our first full year with the new website, where the community feed
has given us great insight to what everyone is working on and the new forum
has offered a better experience for feedback and working together. We think you'll find many of the additions to TouchDesigner this year the result of our communications with you and the great connection we enjoy with our community, we thank you for your support.
On to the major highlights of this release! For the complete list of everything that has changed, check out the Official Release Notes.
Operators are the building blocks of any TouchDesigner project, so let's begin with the new operators in this Official Update.
In the past year, there has been a rapid increase in streaming events, XR environments, and virtual productions where TouchDesigner has been proven a flexible and invaluable tool on set. In addition to last year's Redspy Stype OPs for camera tracking, we have added the FreeD CHOP. FreeD is another camera tracking protocol that sends camera position, zoom, and focus values into Touch, allowing virtual cameras to sync to the actions of live cameras.
All real world camera lenses introduce some distortion to the image they capture. The new Lens Distort TOP takes camera lens intrinsics and distorts the image to match your lens distortion. This helps properly align images from camera live feeds with your 3D renders when compositing the real world with the virtual world.
At the TouchDesigner Summit in 2019, David Braun wowed us in his presentation on generative animation based on easing functions and phasing techniques to create 'overlapping action' in animations. We were so impressed by his work we collaborated with David to bring this functionality natively into TouchDesigner via the new Phaser CHOP. You can blend images, morph particle systems, move UI elements and more with this powerful CHOP without the need for any keyframe animation.
For those of you who love pushing GPU instancing to the limit, the Geometry COMP can now use 2D Texture Arrays and 3D Texture for instancing opening new ways of packing instancing data into TOPs.
The Sprinkle SOP is a helpful new operator to generate an abundance of points in SOPs. It can generate points on the surface of a 3D object and also can fill an object's volume or bounding box with points.
Data Processing OPs
Two new OPs join the TOP family. First the Point Transform TOP is designed to be used with point cloud workflows allowing you to more easily translate, rotate, and scale 3D positions stored in a TOP's RGB channels.
Second the Spectrum TOP performs a Discrete Fourier Transform (DFT) on the GPU to output the magnitude and phase of the image, its ‘Fourier’ or frequency domain. This can be used for image analysis, filtering, reconstruction and compression. Note that this operation runs in Nvidia's CUDA and therefore requires a Nvidia GPU on a Windows system.
Finally, the JSON DAT converts and filters JSON text using JSONPath syntax and outputs the filtered results, reducing the need to write scripts to parse JSON and keeping the data flow procedural.
The increased demand for streaming found us adding a number of streaming features at the beginning of 2020 to help users create new workflows for online events. New additions included RTMP output, supporting hardware decode, upgrading to NDI 4.5 and making NDI In and NDI Out available to Non-Commercial licenses as well. The streaming toolset in TouchDesigner has helped create incredible live shows such as The Interactive and Immersive Championship, the first live TouchDesigner esports event, streamed using TouchDesigner.
This official build brings us a number of new streaming options, starting with support for HLS/DASH input in the Video Stream In TOP by simply providing a URL that points to m3u8 files. Support for HLS is widespread and it is considered one of the most popular streaming formats.
Support for the SRT (Secure Reliable Transport) protocol has been added to the Video Stream In and Out TOPs. SRT offers low latency and high quality, with options for not only H.264 but also H.265/HEVC encoding.
SRT sent from the Video Stream Out TOP can include per-frame metadata making it easy to send and receive CHOP/DAT data in sync with video. This joins NDI as another great option embedding per-frame metadata which can be used in many mediaserver workflows.
Audio coming into TouchDesigner from any of these video streams can now have its sync fine-tuned with the 'Audio Sync Offset' parameter in the Audio Stream In CHOP.
Other nice improvements that make TouchDesigner an even more powerful mediaserver include hardware decoding for NDI|Hx input, and full support for AJA 8K SDI hardware.
NotchLC codec is proving to be great for high-resolution video playback when image quality is a top priority. TouchDesigner now supports a complete real-time NotchLC workflow as we've added NotchLC encoding to the Movie Out TOP to compliment the playback added last year.
“There's no better combination than TouchDesigner and NotchLC which works seamlessly and it's only gotten better now that we can perform our NotchLC encoding pipeline directly in TouchDesigner!"
-Elburz Sorkhabi, The Interactive & Immersive HQ
Two new options for skeleton tracking join the Kinect to give you a number of solutions to choose from.
The wrnchAI CHOP offers GPU accelerated skeleton and face tracking from any image source in TouchDesigner. Using wrnchAI's computer vision and advanced AI, multiple targets can be accurately tracked simultaneously with low latency and at high frame rates. The feature uses Nvidia's cuDNN (CUDA Deep Neural Network) and requires an Nvidia GPU on Windows.
Intel RealSense released a new skeleton tracking SDK (by Cubemos) this year to finally offer a skeleton tracking solution for current RealSense sensors. The RealSense CHOP was overhauled, removing the old tracking SDK (this was no longer supported and only worked on early F200 and SR300 models making it not very useful) and adding Cubemos tracking which works with all current RealSense models. There’s also generally improved RealSense sensor support you can read about in the next section.
These two integrations are both licensed 3rd-party services, and they require licenses available from wrnchAI and RealSense purchased separately. Both offer a 30-day free trial so you can try them first.
OpenVR in TouchDesigner connects to SteamVR and offers support for many VR headsets including the new Valve Index, HTC Vive, and most Windows Mixed Reality headsets (for Oculus VR see Oculus Rift TOP and Oculus Rift CHOP). The OpenVR CHOP has been overhauled to be more inline with SteamVR configurations. Custom action manifests can now be selected to create CHOP channels with values filled in from arbitrary SteamVR controller bindings. In addition these changes added support for the Valve Index headset, Valve Index Controllers, and Hand Skeleton and Finger Curl tracking.
The RealSense TOP has been updated to support D455, SR305, and their new LiDAR 515 sensors. Numerous performance optimizations have been added such as multi-thread frame processing and better filtering controls and behavior. The RealSense CHOP is also completely new as mentioned in the section above, skeleton tracking now works with all current RealSense sensors.
Flir cameras offer a wide range of cameras for specific uses and have long been respected for high quality and performance. The Video Device In TOP’s Flir support has been updated to the latest SDK and we added support for adjusting image bit depth, controlling camera options, manual capture triggering, and loading presets.
The ZED TOP also gets an update and now supports Stereolabs ZED2 camera, offering higher resolution and improved tracking.
5.OpenCV and Extending TouchDesigner with Code
TouchDesigner’s extensibility through python and C++ is one of the aspects that make it so powerful and flexible, giving you tools to extend its functionality to meet the needs of almost any project.
OpenCV is an open source library of functions for real-time computer vision, TouchDesigner can directly access these OpenCV functions to extend functionality in areas like object tracking, image analysis, camera and projector calibration and much more. TouchDesigner now comes with the full OpenCV 4.5 toolkit pre-installed, and for the first time now includes all CUDA and opencv_contrib modules. The extra modules give you access to functionality not packaged with the default OpenCV distribution. Examples of such modules include Aruco Marker Detection, 3D Object Recognition and Pose Estimation, Custom Calibration Patterns, Structured Light as well as various CUDA accelerated libraries. See the full list OpenCV Modules.
You will also find improved workflows for Python Script OPs. The Script TOP has many new functions such as loadbytearray() to load compressed data stored in bytearrays, copyCUDAMemory() to load data from CUDA directly, and new options for working with copyNumpyArray(), all of which make it easier to work with openCV and get more out of the Script TOP.
To simplify creating Custom Parameters for Script OPs, you can use the same Customize Component dialog as components by dragging the Script OP onto the dialog’s ‘Path’ or right-clicking on the OP and selecting ‘Customize Operator…’. Additionally, for both Script OPs and Custom OP custom parameters, the labels, page names, and defaults now support unicode, letting you use any language you desire.
There are also new Script OP examples in OP Snippets, be sure to check them out!
The release of the Web Server DAT, Web Client DAT, and SocketIO DAT last year have opened a whole new genre of 'connected applications' being built with TouchDesigner. Amazing new tools like Markus Heckmann’s Rabbitcontrol - Remote Control for Everything and Alphamoonbase’s Stream Fileserver can be found in the community assets section of our website. We continue to develop and improve these tools to open even more possibilities. Recent additions include the Web Server DAT working with password protected certificates, having improved start-up error reporting, and can now receive larger websocket frames. The WebSocket DAT has improved callback functions for managing connections.
Finally the Web Render TOP can now access the mic and camera for web pages it loads. It also includes an option for user-specified 'User Cache Directory' and has had its Chromium CEF SDK updated.
This addition might have flown under the radar so we wanted to highlight it here. This is a massive upgrade for all those who care about typography and font rendering in TouchDesigner, the addition of Slug Library. Slug Library is a dynamic GPU font rendering and layout engine for professional high-quality, resolution-independent text. You can now render fonts with the precision required to match industry-standard typography tools and ensure your font and text designs can be faithfully recreated in TouchDesigner.
Supporting all unicode characters, it is fully scalable from small to huge sizes with sharp outlines and no artifacts at any resolutions, this type is beautiful!. This font rendering is available in the Text TOP using the Display Method called 'Scalable' and is now the default rendering method used when the Text TOP is set to 'Automatic' and the font size is 16 or larger.
We have also overhauled the font loading in TouchDesigner so now all font weights within font families are available to use.
8.3D Viewer Camera Controls
3D geometry viewers have a slew of upgrades that make them much more pleasant to work with. Starting with navigation of the 3D space, tumble-pivot placement has been overhauled and there are now 4 navigation modes to choose from in the right-click menu. These include a simple center viewport mode, a cursor-based pivot, object-based pivot, and finally a way to navigate from the perspective of the camera. There should be a mode for you no matter what you are used to in other 3D modelers.
In addition, those who use any of the 3Dconnexion SpaceMouse devices will be excited to hear we have added support for them in 3D viewers, again with a few different navigation modes.
Finally, a new geometry outline shader for selected geometry makes it easier to see your geometry when selected, and there is a new camera component in the Palette (Tools > camera) that brings all these new navigation modes to your camera used for 3D scene rendering.
9.Drag-Drop and Python improvements
A new python based drag-drop system for building control panels and UIs with Panel COMPs has been developed using python callbacks for all ‘When Dragging’ and ‘On Dropping’ events. There are usage details and examples of this new workflow here.
There's a long list of python improvements (see Release Notes - Python) but these are particularly noteworthy.
If you use TDFunctions and TDJSON, they now work with normal python import statements making them easier to use everywhere.
The COMP Class has new saveByteArray() and loadByteArray() methods allowing you to save and load .tox components directly to and from memory.
The Matrix Class adds getInverse() and getTranspose() methods to make your life with matrix math that much easier.
A new floating license option allows any internet connected computer to access a pool of licenses from a single account. We call these new licenses Floating Cloud Licenses as they are distributed on demand by Codemeter's Cloud licensing service. Open TouchDesigner, it grabs a key; close TouchDesigner, the key is returned to the pool.
There are a number of scenarios where these licenses are very convenient, here are three examples.
First for distributed teams and when remotely working from multiple locations, local computers can be keyed on demand from a shared pool of licenses available to anyone on your team, at offices, studios, or installations located anywhere in the world. A second common usage case is for lab computers at schools and universities where the computers are often wiped and do not have admin privileges to install a key. Using floating cloud licenses here allows the system administrator to install the credentials onto lab computers and the keys will be distributed automatically on demand. Another usage case that is rapidly gaining popularity is running TouchDesigner and TouchPlayer on cloud platforms. Often these cloud servers are not persistent which requires new key codes, or you may be spinning up instances dynamically and need a solution to auto-key them, both issues are conveniently handled by floating cloud licenses that distribute keys automatically.
Available now in store in the shopping cart's 'Select Product' list.
There are hundreds of other improvements in this release, too many to review here so head to the Release Notes where you'll find all the details on every little change.
We recommend reviewing the BACKWARD COMPATIBILITY changes and the SDK/API updates to make sure any drivers you use are up to date. Windows users should update Windows as version 1803 from 2018 and earlier is not supported.
Don’t forget to check out the new Operator Snippets of which there are dozens. To explore them simply go to Help > Operator Snippets and then click the “Newest” button a numner of times to preview the new additions.
We would also like to point you to the 2020 Official Update Announcement
if you did not catch it last year. It covers a great showcase of recent features in TouchDesigner you may not be aware of.
Thinking back to our 2020 Release last February, it can be surprising how much can happen in one year (the length of the release notes is testament to this!). All of us at Derivative are amazed and motivated by the work and perseverence of the TouchDesigner community who continues to inspire us. We are truly grateful for your continued support and encouragement.
We would like to thank the following members of the community who contributed their stories, images, and assets to make this article possible. Check out their work or say hello when you get the chance.
- *Article teaser image
- Lens Distort TOP
- Phaser CHOP
- Texture 3D instancing
- Beautiful painterly effects in TOPs
Volvox - Auto-calibration tools for XR LED stages